Evaluating the Admission Process

Gaining insights for future improvements

Did your admissions process help you achieve your desired goals?  Periodic review and reflection can lead to insights regarding improvements you can plan to implement in the next cycle.

Measure outcomes

Periodically evaluating and measuring outcomes data against admissions information can help ensure that the admissions policies and practices are producing the desired outcomes regarding institutional and program goals, as well as the composition of the student body. Any observed patterns between admissions data and important student outcomes may be useful in refining the admissions practices in subsequent cycles.

If you find that one of your program’s enrollment goals has not been met, such as to increase diversity, reviewing the strategies for recruiting and nurturing relationships can lead to active exploration of new ways to attract applicants.

Reflect on successful and unsuccessful students

Reflecting upon past students who have been successful in the program, as well as those who were not, can provide insight into the information collected. It can reveal how effective pieces of information were in helping determine which applicants have the potential to make it through the program or other desired outcomes. Doing this can also determine if there is any information not currently collected that would have been useful.

Analyze inputs

A regression analysis may help a program determine which inputs contribute toward the prediction of the outcomes it values. Generally, a sample size of 75 is a reasonable threshold for making any meaningful conclusions, and the more variables considered, the larger the sample size should be. Programs can combine data collected over several years to achieve a sufficient sample size for this type of analysis. It’s important to use caution in relying upon study results with smaller sample sizes, as they can often lead to incorrect conclusions. Someone in the research or statistics department at your institution may be able to advise on parameters for such a study. ETS researchers are also available to advise regarding factors to consider; email gretests@ets.org to connect with someone who has experience in this area.

It’s important to exercise care when comparing outcomes with inputs as many factors contribute toward whether a student is successful in a particular program. In a study for the National Center for Education Statistics (NCES) exploring the reasons students did not complete graduate school programs, researchers identified “change in family status,”  “job/military conflict” and “dissatisfaction” as the top three of 15 reasons. While there may not be any way for programs to determine who will drop out due to competing life priorities, conducting exit surveys could help your institution make changes to avoid dissatisfaction. While it can be time intensive, some programs take the extra step to conduct exit interviews using a standard set of questions to solicit richer feedback.   

IMPORTANT NOTE:   Some programs have stopped requiring GRE® General Test scores because they don’t believe the test predicts program completion. The NCES study can be used to understand why. The GRE test could never have predicted the impact that competing life priorities and other external influences could have on degree completion. Only 1% of interviewees indicated they were dropping out of their program due to academic problems. By using the GRE test in combination with other measures, however, the majority of graduate schools have been very successful in admitting only students with the reasoning, critical thinking and analytical writing skills needed for success. Therefore, very few students drop out because they lack these skills.