Standardized Educational Testing Should be Kept in its Place!

Read "Just Whose Idea Was All This Testing? Fueled by Technology, Nation's Attempt to Create a Level Playing Field Has Had a Rocky History" by Jay Mathews, The Washington Post, November 14, 2006.

"I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely in your thoughts advanced to the state of Science, whatever the matter may be."
Sir William Thomson, Lord Kelvin, Popular Lectures and Addresses, vol. 1, "Electrical Units of Measurement", 1883-05-03

Mathews' article considers the history of educational testing in the United States, and has a questioning tone.

Standardized tests are obviously useful, but I suspect that they are taken much too seriously. I once was responsible for calculating the scores for university entrance test and ranking the applicants by test score. I am sure that there had been no effort to assure the validity of the test as a predictor of success at the school, much less of career success or likelihood of the student to achieve the university's objectives. The main advantage of the test was that it gave a numerical score that had some verisimilitude to objectivity which people could point to and defend decisions to include some kids and exclude others.

Standardized tests can be useful for monitoring. When teaching I find frequent quizzes helpful in telling me how well I have gotten the content to the students.

We should be interested in test results that indicate that kids in some schools, or of some socio-economic groups are not performing as well as others -- there may be a problem we can work to resolve.

But tests are just tools. Like any tools, some are better than others. Using the wrong test for the wrong purpose is like using a hammer to cook a omelet -- it can be of limited help, but only if used gently and with understanding, and is more likely to get in the way.

The numbers that we use to evaluate universities are even more suspect than those the schools use to evaluate their applicants or their students. A kid should not feel bad when a university turns down his admission because of his test scores. First, it may not be the right school for him in the first place, and second, the university doesn't really have a clue which students it can serve best, or which students will most help it achieve its long term goals. (I know that if anyone reads this, they will say that university officials know that kids from rich families will be more likely to help them achieve their goals of keeping the university well supplied with money.)

I have been involved in a lot of processes in which groups of people make choices, and I have come to the conclusion that they are very bad at the task. As Churchill said of democracy, panel review is the worst form of decision making "except all the others that have been tried." But I know deep down that there is no process that can accurately draw the line between the worst applicant to be accepted and the best to be rejected. If a university is selecting 1,000 to offer admission from 5,000 applicants, there is no way to convince me that the those ranking 951 to 1000 are superior to those ranking 1001 to 1050.

I suggest that the real issue is helping people to learn and guiding them towards learning important and useful things. I suspect that the best educational system is a gifted and motivated educator sitting at one keyboard connected to the Internet and a smart, motivated kid at a keyboard also connected to the Internet, assisted by really good educational software, and both in the same room. I suspect that our understanding of this fundamental interaction between learner and facilitator of learning is still of a meagre and unsatisfactory kind; quantification via standardized tests may make information on the process appear more precise, but will not yet make it accurate nor satisfactory.