In general, of course, one wants a process of lifelong learning that produces a strong, competitive workforce, strong families capable of effective management of the household, good citizens, etc.
The problem is that government education programs are mostly about schooling, and the relationship between schools and the learning outcomes of the population is so complex that overall impact analysis is never practical. Moreover, in practice one is more interested in the impact of changes in schools than on the impact of the existing system -- the huge sunk investment in schooling can only be changed slowly.
So we do partial impact analysis: Do girls who have more schooling have smaller families and are their children healthier than girls who have less schooling? How do lifetime earnings vary with years of schooling? Are the schools producing enough people with required skills to fill the available jobs and fuel the economy?
We also do outcome analysis. How many kids go how far in school. How do they do on examinations. We hope that the kids who go to school longer and do better on exams also do better in life.
And we do process analysis -- how good is the teaching and school management, how many students in a classroom, how many hours of instruction per day, how well designed is the curriculum? And we do input analysis -- how we trained are the teachers, how good are the teaching materials, how good are the school buildings? Again we depend on the chain -- good inputs lead to good processes, good outcomes and good impacts.
So how does UNESCO fit into this picture. UNESCO helped gather people together and did much of the policy analysis for the creation of the Education for All goals and the Millennium Development Goal for education. It has developed standards for educational indicators, and collects and publishes comparative information from governments on education. It produces the annual Educational for All Monitoring Report. It provides forums for Ministers of Education, education sector leaders and scholars to discuss progress in school capacity building, educational reform and schooling. And of course, it has produced thousands of publications over the years on education for educators.
In theory, all of this encourages educational leaders to develop and implement better policies, leading to the betting of school inputs, school processes, school availability, school accessibility, school outcomes, and the schooling of the population of their countries. However, the proof of the pudding is long in coming. The EFA program was conceptualized for 25 years 22 years ago, and a new program is already in planning to replace it. Kids entering school now will continue for years, many finishing secondary school, some getting university degrees and a few completing graduate school. They will not complete their working lives for some six decades. 100 years from now we may have a better idea of how well the EFA program worked.
How much of the success or failure of a country's efforts to improve its educational system can we attribute to UNESCO's inputs? Such an assessment is extremely difficult and likely to be anecdotal if undertaken at all. Obtaining a view of the global impact of UNESCO on education in its 195 member states is many times more difficult. Relating the benefits to school kids in those 195 nations per dollar spent on UNESCO is but a dream.
We can of course look at outcome indicators. How well done are the EFA Monitoring Reports? Are the educational statistics promoted by UNESCO cost-effective? Do UNESCO's proposed educational planning methods work well? Do the participants in UNESCO sponsored meetings of educators feel that the meetings were effective? These data are important, but can not really answer the question of the donors as to whether the money spent on UNESCO was a good investment in improving education.
CERN -- An Example of Why UNESCO Can't Be Evaluated
I used the example of CERN to illustrate my concern for the difficulty of evaluating UNESCO. The U.S. investment in creating the atom bomb not only demonstrated the practical use of science, but was an introduction to big science. Thousands of scientists, engineers and soldiers worked on the program, and the efforts to produce enough enriched Uranium and Plutonium to test the bomb and demonstrate it to the world. To do that work, the government built the worlds largest building, used tons of silver from Fort Knox for wire for the separator magnets, and used a significant fraction of the electrical output of the United States to power those magnets. War torn European nations could not aspire to finance such investments on their own but realized that big science was the wave of future nuclear research. Those nations (and their scientists) called upon UNESCO to catalyze the creation of a shared facility, the European Center for Nuclear Research -- CERN. CERN came into existence in 1954.
Scientists at CERN published the description of the World Wide Web (WWW) in 1990, putting the software that led to today's popularity of the Internet into public domain. I hold that the WWW alone probably justifies not only the creation of CERN, but also the creation of UNESCO and its budget over its lifetime,. However, the WWW was not invented until 36 years after the creation of CERN (and the value of the WWW was not realized until some years later, until after browsers were created to utilize its power, until connection to the Internet got sufficiently large, and until a very large amount of information was posted on the Internet using the WWW). How many evaluations look back 40 or 50 years?
This year, CERN has announced that a huge experiment had found evidence for the existence of the Higgs boson, in the process clarifying a final piece of evidence about the basic workings of the universe. The Higgs boson is the fundamental particle related to mass and thus to gravity. Will knowledge of its existence and properties prove useful in the future? Almost certainly, but in ways we can not yet imagine. Thus we have a discovery made because of Cern, 58 years after CERN was created with UNESCO's support, that is of huge intellectual importance and is likely to have a very great yield of practical application. What kind of evaluation waits a couple of lifetimes to quantify the impact of an organization's efforts?
No comments:
Post a Comment