Evaluating the Admission Process
Gaining insights for future improvements
Did your admissions process help you achieve your desired goals? Periodic review and reflection can lead to insights regarding improvements you can plan to implement in the next cycle.
EVALUATE: At a glance
- Evaluate and measure outcomes data against admissions information periodically to ensure desired outcomes for the program, institution, and student body are being achieved.
- Reflect on past students who have been successful, and those who were not, to gain insight into the effectiveness of the information collected.
- Understand why students typically drop out of graduate programs to ensure inputs are being analyzed appropriately.
Periodically evaluating and measuring outcomes data against admissions information can help ensure that the admissions policies and practices are producing the desired outcomes regarding institutional and program goals, as well as the composition of the student body. Any observed patterns between admissions data and important student outcomes may be useful in refining the admissions practices in subsequent cycles.
If you find that one of your program’s enrollment goals has not been met, such as to increase diversity, reviewing the strategies for recruiting and nurturing relationships can lead to active exploration of new ways to attract applicants.
Reflect on successful and unsuccessful students
Reflecting upon past students who have been successful in the program, as well as those who were not, can provide insight into the information collected. It can reveal how effective pieces of information were in helping determine which applicants have the potential to make it through the program or other desired outcomes. Doing this can also determine if there is any information not currently collected that would have been useful.
A regression analysis may help a program determine which inputs contribute toward the prediction of the outcomes it values. Generally, a sample size of 75 is a reasonable threshold for making any meaningful conclusions, and the more variables considered, the larger the sample size should be. Programs can combine data collected over several years to achieve a sufficient sample size for this type of analysis. It’s important to use caution in relying upon study results with smaller sample sizes, as they can often lead to incorrect conclusions. Someone in the research or statistics department at your institution may be able to advise on parameters for such a study. ETS researchers are also available to advise regarding factors to consider; email firstname.lastname@example.org to connect with someone who has experience in this area.
It’s important to exercise care when comparing outcomes with inputs as many factors contribute toward whether a student is successful in a particular program. In a study for the National Center for Education Statistics (NCES) exploring the reasons students did not complete graduate school programs, researchers identified “change in family status,” “job/military conflict” and “dissatisfaction” as the top three of 15 reasons. While there may not be any way for programs to determine who will drop out due to competing life priorities, conducting exit surveys could help your institution make changes to avoid dissatisfaction. While it can be time intensive, some programs take the extra step to conduct exit interviews using a standard set of questions to solicit richer feedback.
IMPORTANT NOTE: Some programs have stopped requiring GRE® General Test scores because they don’t believe the test predicts program completion. The NCES study can be used to understand why. The GRE test could never have predicted the impact that competing life priorities and other external influences could have on degree completion. Only 1% of interviewees indicated they were dropping out of their program due to academic problems. By using the GRE test in combination with other measures, however, the majority of graduate schools have been very successful in admitting only students with the reasoning, critical thinking and analytical writing skills needed for success. Therefore, very few students drop out because they lack these skills.
Retention is personal
The issue of retention in graduate programs is generally misunderstood and understudied. Explore the most common reasons why students leave graduate school.