Thursday, October 13, 2011

Thoughts on the Logical Framework

Log Frame Matrix
The Logical Framework, the logframe, has been used for decades for planning, monitoring and evaluation of development projects. I have used it as an employee of USAID and the World Bank, both in documenting project designs and in project evaluation. The approach involves specification of the inputs to a project, the outputs expected from those inputs, the specific purposes or objectives of the project, and the more fundamental goal that the project is intended to pursue. As the matrix above indicates, a key element of the approach is the specification of indicators at each of these levels and how measurements were to be made with respect to those indicators.

Last night I heard a talk in which BHAG was recommended for development agencies -- Big, Hairy Audacious Goals. The point was that sometimes more is accomplished by promulgating audacious goals that we don't now know how to achieve and that may not be achieved than by using goals that are most likely estimates of what will be accomplished.

More generally, should goals and objectives be set to maximize performance of the project or as benchmarks on which to judge staff performance? I prefer the former. But if goals and objectives are not intended to be taken literally as a basis for ex post evaluation of projects, what should be used for that purpose?

Monitoring seems to me most useful in providing information for mid course modifications of project/program implementation. What is the most useful kind of information for the monitoring, that is, what approaches to monitoring will produce the most useful modifications of project activities while protecting those activities that are working well? In my experience with the management of research programs, the most useful information for monitoring in this sense involved participant discussions by those implementing the project with well qualified consultants -- hardly an objectively verifiable indicator. Similarly, I had good experience with qualitative reviews of project performance by outside experts, again hardly what I would call an objectively verifiable indicator since different panels of experts give different results.

It seems to me that one purpose of project evaluation is to motivate project personnel to design projects well and to implement them well.  Thus the questions I would ask on ex post project evaluation are:
  • Was it a good project?
  • Was it managed well?
To answer those questions, I think the emphasis of ex post evaluation should be on what actually happened. Were the project results worth the resources devoted to the project. Were the resources used in ways appropriate to their sources?

It seems to me that different stakeholders in a project usually have different objectives. I don't see how a logical framework can do a really good job in documenting the different perspectives of the project stakeholders. On the other hand, the logframe has been used by donor agencies to protect their own interests. In this respect the ex post evaluation can address the questions:

  • Did the project do enough to forward the funding agency's goals and objectives to justify that agency's funding,
  • Was it managed well?
  • Was the agency performance (management, inputs) good?
In the best of cases, there is limited information and limited rationality at the time of project design -- often five or more years before project completion. Formal changes can be made in the logframe in mid course, but those changes in my experience tend to be conservative, and they too are based on limited information and limited rationality. In those circumstances, unforeseen results -- either positive or negative -- are of great importance in ex post project evaluation. Yet, by their very nature, they do not appear in the pre-project logframe nor often in the modified logframe. Thus good ex post evaluation seems to be more about what actually happened rather than about what was once thought would happen.

The logical framework would seem to be more useful for some classes of projects, less for other classes. If a project is primarily engineering or construction, the logical framework would seem likely to be very useful. Engineers these days have a very good idea of how much it should cost to build a road and very good understanding of the quality of built roads. It should be relatively straight forward to develop a logical framework matrix for a road construction project.

Very complex projects would seem less suited for the logical framework approach. Support for the democratization of countries that have had successful revolutions during Arab Spring would seem be very important, but (it seems to me) that it is very hard to see how the political institutions will evolve in these countries and how effective support and resources will be in promoting changes toward more representative and more responsive government.

As in this case, timely help may be more important than well thought out assistance projects. Think about the earthquake in Haiti that left hundreds of thousands dead, that left many orphans and many families grieving for their dead members. Millions of people needed housing, and housing that would withstand hurricanes, food and medical attention including public health services that would deal with epidemics that might well occur. Aid was needed in desperately and quickly in early 2010; there was not time for a lot of project planning. Moreover, it would have been all but impossible to foresee the evolution of the situation in Haiti at the time;

I am in fact a fan of the Logical Framework approach, but as a tool that should not be thoughtlessly applied. It is useful for documenting and describing the reasoning behind a project and can contribute to ex ante project evaluation by donor agencies.  Revision of a project logical framework during a project may be useful as an aid to thought and as an aid to communicating about experience to date with a project. Indeed, I have found the logframe useful in ex post evaluation, when I paid due attention to unforeseen aspects of the project and was willing to reconsider indicators that could be used at the time of the evaluation. However, one should use a grain of salt in the application of the logical framework in the real world of complex projects, difficult conditions, limited rationality, limited information, and limited time for planning. It is a tool, but in the hands of mortals it does not produce divine revelations.

No comments: