The Economist Special Report on Econometric Models

Read the full article in the July 13, 2006 edition of The Economist. (Subscription required.)

The article notes,

Economic models fall into two broad genres. Macroeconomic models, the distant descendants of Phillips's machine, belong mostly in central banks. They capture the economy's ups and downs, providing a compass for the folks with their hands on the monetary tiller. The second species, known as computable general equilibrium (CGE) models, largely ignore the vagaries of the business cycle. They concentrate instead on the underlying structure of production, shedding light on the long-term repercussions of such things as the Doha trade round, a big tax reform or climate change.

It casts some light on the Doha Round (see my last posting):

Two years into the round, as trade ministers gathered for a summit in Mexico, the World Bank was pushing another extravagant simulation. It argued that an ambitious Doha agreement could raise global incomes by $290 billion-520 billion and lift 144m people out of poverty by 2015. Those figures found a ready place in almost every news report about the Doha round that autumn.

Such extravagance did not last. The World Bank has since cut these figures drastically, in part because the ambitions of the Doha negotiators have fallen short of the bank's expectations. One estimate made last year had cut the increase in global incomes to $95 billion and projected 6.2m people might instead move out of poverty. But even as they curb their enthusiasm for Doha, proponents of freer trade argue that CGE models do not show their cause to its best advantage.

Given the apparent failure of the round, I would stress than saving more than six million people from poverty is not a small thing, nor is 95 million dollars a small amount of benefit, especially when more reasonable negotiators might have gotten even more.

I suggest that the key lesson from the article however, comes from these two paragraphs:

In a recent article, Roberta Piermartini and Robert Teh, two economists at the WTO, urge modellers to "?demystify" their creations, making it clear to their audience what makes their models tick. A failure to do this, they argue, "?risks bringing a useful analytical tool into disrepute and may even induce unwarranted cynicism about the economic case for open trade.".......

Shantayanan Devarajan, of the World Bank, and Sherman Robinson, of the International Food Policy Research Institute, point out that policymakers need not grasp exactly how a model works, any more than "?a pilot needs to understand the insides of a flight simulator."? This may be true. But too many policymakers never even "fly" their models. They just want to know where they will land. If they were instead prepared to work through the simulations they might find inconsistencies in their thought, unforeseen implications of their policies, or new reasons for their actions. The big number that sums up a model's story "?$520 billion, 1.5% of world GDP, $4.4 trillion" ?is often the least interesting thing about it.

I suggest large scale mathematical models are most useful in helping us to extrapolate the behavior of very complex systems that are described by large quantities of data. They are especially useful to illuminate unexpected, emergent properties of such systems. However, properties can emerge from the models due not to the behavior of the real system, but from elements of the model that do not accurately represent the real system. I suggest that it is especially important when a model yields unexpected behavior that we go back and understand just where and how that behavior arises from the model. In that way we have a better chance of understanding whether it is a characteristic of the real system, or an artifact of the model.

Unfortunately, in my experience, those who are uncomfortable with quantitative approaches are too likely to accept the predictions of a model without understanding the model itself. Those who best use models, distrust their predictions, and spend time to understand the basis for those predictions. The models themselves are very useful tools in this exploration.

An anecdote:

Many years ago, when computers were young and models were simple, a colleague and I did a computer model to optimize the business of a small company. We walked into the company president's office and began the meeting with the comment,

We have good news. You can increase production by one-third.

The production manager, at the right of the president, jumped to his feet and yelled, "we can't sell that much". Simultaneously, the sales manager leaped to his feet and yelled, "we can't produce that much." The president looked to the right and to the left, and said, "thank you gentlemen, I now understand the problem". That was a successful model!