I have recently done a series of posts on the use of Baysian approaches to improve peer review:
- Peer review in ex ante and ex post evaluation: probability based improvements in vote tallying.
- Probability Analysis to Improve Scientific Peer Review
- Subjective Probabilities, Ratings, Handicappers, and Open Access Online Scientific Literature
- From Estimating Probabilities of Ratings to Figures of Merit and Rankings
The same approach can be used in university admissions. I discussed the complexity of such decisions in a post yesterday.
One might consider the admission decision in U.S. medical schools to be based on undergraduate grade point average and completion of an undergraduate pre-med curriculum, grades on the Medical College Admission Test (MCAT), and ratings of the applicant by several interviewers. Typically the applicant pool at that point would be divided into three groups: the highly qualified who will be admitted without further discussion, the unqualified who will be rejected without further discussion, and a group of acceptably but not outstandingly qualified applicants. Further discussion would focus on this third group, choosing those who complement the first group to produce a well balanced class meeting as many of the objectives of the college as possible.
The previous posts have described a Baysian approach to pool the information from individual reviewers which has the side benefit of weighing the judgments of the reviewers who have the best correlations with each other more heavily than those who are less in agreement with their peers. The approach as described can be used to combine the information from the interviews of medical school applicants into a distribution of the probability that the applicant would be admitted.
The approach can easily be extended to utilize information from the undergraduate GPA and the MCAT.