Don't Get Fooled That You Are Measuring What Is Most Important.

Source: "How a Self-Fulfilling Stereotype Can Drag Down Performance," Shankar Vedantam, The Washington Post, February 2, 2009.

In yesterday's WP, Shankar Vedantam summarizes recent research which indicates how sensitive test performance can be to the expectation of prejudice. Black test takers performance falls by a small but statistically significant amount when whites give the tests as compared with the situation when they are tested by other blacks. Similar effects can occur to girls taking math tests are informed that boys outperform girls on mathematics, or when ethnic groups are told before taking tests that other ethnic groups outperform theirs on such tests. Psychologists

came up with the term "stereotype threat" for the phenomenon: When people are threatened by a negative stereotype they think applies to them, they can be subtly biased to live out that stereotype.

The threats do not have to take place at a conscious level: When volunteers in experimental studies that have found huge stereotype-threat differences in performance are told about the phenomenon afterward, they invariably tell researchers that the theory is interesting but does not apply to them.

The article also states:

SAT scores are typically seen as measures of aptitude and predictors of a student's performance. Colleges have long known, however, that women and minorities typically underperform relative to their SAT scores.

"Let's say on the SAT you have a score of 1200," Spencer said. "What would that predict about your GPA at university? It would predict a 3.2 GPA. What has been observed in high-stereotype-threat environments is that that 1200 does not predict a 3.2 GPA but a 3."

In a pure meritocracy, the college admissions officer ought to pick the man, since his score predicts he will do better in college than a woman with the same score.

But in two meta-analyses involving nearly 19,000 students, Walton and Spencer found that when schools and colleges go out of their way to ameliorate stereotype threats, the performance of women and minorities soars -- it's as if these students are athletes who have been running against a headwind. Without the headwind, Walton and Spencer found that minorities, and women in math and science, do not just do as well as whites and men with the same SAT scores -- they outperform them.

Comment: There is always a problem in believing that what you measure with a test is what you intend to measure. This research suggests to me that the problem is worse, in that simply conducting a measurement may influence the subsequent performance of an ethnic group.

The problem is worse, of course, in that schools should not be selecting students on the basis of probable performance in the school. I think they should be trying to find the students for whom the university education will contribute most to overall social objectives.

A doctor who will devote his/her life to providing much needed medical services to the underserved populations in the inner city might provide a lot more social benefit as a result of a medical education than would still another Beverly Hills cosmetic surgeon, even if the latter got better grades in med school. JAD