Monday, September 01, 2014

The Ebola Epidemic

Thanks to Calestous Juma for sharing. Read his article on Al Jazeera.

Research on forecasting.

Tetlock also found that specialists are not significantly more reliable than non-specialists in guessing what is going to happen in the region they study. Knowing a little might make someone a more reliable forecaster, but Tetlock found that knowing a lot can actually make a person less reliable. “We reach the point of diminishing marginal predictive returns for knowledge disconcertingly quickly,” he reports. “In this age of academic hyperspecialization, there is no reason for supposing that contributors to top journals—distinguished political scientists, area study specialists, economists, and so on—are any better than journalists or attentive readers of the New York Times in ‘reading’ emerging situations.” And the more famous the forecaster the more overblown the forecasts. “Experts in demand,” Tetlock says, “were more overconfident than their colleagues who eked out existences far from the limelight.”
The quotation is from a review of Philip Tetlock's book,  Expert Political Judgment: How Good Is It? How Can We Know?, from the New Yorker magazine.

Some years ago I compared the forecasts of GDP made by World Bank economists with simple linear extrapolations of GDP. On average, the linear extrapolations were slightly more accurate. Now economists realize that there is a great deal of inertia in national economic systems, so they tend to be relatively conservative in forecasting deviations from the trend. My data seemed to show that when they did forecast a significant change from the trend, they tended to overestimate the actual deviation. I have known and worked with several World Bank economists, and I have the greatest respect for their competence. I think my findings were consistent with Tetlock's view that experts are frequently not as good at forecasts as we might expect from their frequent appearances in televised news programs.

Another quote from the New Yorker:
Low scorers look like hedgehogs: thinkers who “know one big thing,” aggressively extend the explanatory reach of that one big thing into new domains, display bristly impatience with those who “do not get it,” and express considerable confidence that they are already pretty proficient forecasters, at least in the long term. High scorers look like foxes: thinkers who know many small things (tricks of their trade), are skeptical of grand schemes, see explanation and prediction not as deductive exercises but rather as exercises in flexible “ad hocery” that require stitching together diverse sources of information, and are rather diffident about their own forecasting prowess.
Foreign affairs are complicated. There seem to be high probabilities of making mistakes in predictions on what is going to happen in Ukraine, Syria, Iraq and other hot spots. Perhaps it is best to make small decisions and not work from a grand plan (one big thing) in foreign policy. Certainly it seems wise to look for the "black swan", the event or piece of information that doesn't fit with forecasts and assumptions.
Tetlock also has an unscientific point to make, which is that “we as a society would be better off if participants in policy debates stated their beliefs in testable forms”—that is, as probabilities—“monitored their forecasting performance, and honored their reputational bets.” He thinks that we’re suffering from our primitive attraction to deterministic, overconfident hedgehogs.
I wonder whether Tetlock is a hedgehog or a fox? How good is his intuition as to what would improve our society? 

Sunday, August 31, 2014

With hunger a huge problem, don't spread false information on farming.

I am really annoyed by this article in The Guardian. I quote:
Kumar, a shy young farmer in Nalanda district of India's poorest state Bihar, had – using only farmyard manure and without any herbicides – grown an astonishing 22.4 tonnes of rice on one hectare of land. This was a world record and with rice the staple food of more than half the world's population of seven billion, big news 
It beat not just the 19.4 tonnes achieved by the "father of rice", the Chinese agricultural scientist Yuan Longping, but the World Bank-funded scientists at the International Rice Research Institute in the Philippines, and anything achieved by the biggest European and American seed and GM companies. And it was not just Sumant Kumar. Krishna, Nitish, Sanjay and Bijay, his friends and rivals in Darveshpura, all recorded over 17 tonnes, and many others in the villages around claimed to have more than doubled their usual yields........ 
That might have been the end of the story had Sumant's friend Nitish not smashed the world record for growing potatoes six months later. Shortly after Ravindra Kumar, a small farmer from a nearby Bihari village, broke the Indian record for growing wheat.
I suggest you watch this video from The Guardian which shows one of the farmers described in the article. It shows what seems to be a simple man proud of his accomplishments as a farmer.

This story got picked up by others. Here is a quote from one such follower:
Another Bihar farmer broke India’s wheat-growing record the same year. They accomplished all this without GMOs or advanced seed hybrids, artificial fertilizer or herbicide. Instead, they used a technique called System of Rice [or root] Intensification (SRI). It’s a technique developed in Madagascar in the 1980s by a French Jesuit and then identified and promulgated by Cornell political scientist and international development specialist Norman Uphoff.
Do you really believe that application of the farming system developed for growing rice could be used to grow wheat, and the result would be record yields? The article even suggests that it improved potato yields for another farmer.

Here is a report from a world leader in agricultural research, Achim Dobermann. I quote:
The Guardian article suggests that Mr. Kumar and other farmers achieved something beyond what can be achieved by scientists and GMO companies, and that they used only farmyard manure and no herbicides. The much more detailed article in Agriculture Today shows almost the opposite. No GM rice is grown anywhere yet. However, all five record farmers grew commercial rice hybrids from Bayer or Syngenta; their seed was fungicide-treated (carbendazim); they used intensive tillage (two deep plowings plus two puddling operations); they applied green manure, farmyard manure, mineral fertilizer (N, P, K), and other nutrient input products (poultry manure, vermicompost, phosphorus-solubilizing bacteria, micronutrient foliar spray of zinc sulfate); and some also used herbicide (2,4-D) for additional weed control. These are in fact very intensive input management practices, perhaps not something that many poor, small farmers could afford.
Here is the link to the Agriculture Today article mentioned.

I believe Dobermann and not the Guardian -- that the farmers used hybrid seed, both artificial and processed natural fertilizers, and chemical inputs.

Scientists from the International Rice Research Institute and from China doubted the accuracy of the yield reports.  Here is what Dr. Dobermann says in the article cited:
In summary, given what we know about the upper limits of nutrient supply from soils in that region, I conclude again that the actual yield may have been in the 10–12 t/ha range at best. Japanese researchers studying SRI fields in Madagascar have shown a clear linear relationship between nutrient supply (enhanced by deep plowing as also done in Bihar) and rice yield, irrespective of other SRI management practices, but also not exceeding yields of about 10 t/ha.
And this from an article quoting a distinguished Chinese scientist:
China’s most renowned agricultural scientist has described as “120 per cent fake” claims that farmers in Bihar harvested a world record 22.4 tonnes of rice from one hectare of land without using herbicides or genetically modified (GM) seeds last year. A national icon, Yuan Longping is known here as “the father of hybrid rice” for developing varieties that enabled China to transform its grain output. His rice varieties were subsequently introduced widely in the world, and marked a record yield of 19.4 tonnes a hectare.
OK, a few farmers in Bihar claimed huge yields in various crops. The claims were picked up and shared by politicians, newspapers, and various factions opposed to modern agricultural techniques. The farmers got famous, and at least one of them got prize money from the government. Clearly most of the people who picked up on the story spread false information.

This is not a critique of Oxfam and its Indian collaborating organization that provided extension services to the farmers promoting good farming practice. It is important in growing rice when you transplant, what density you plant at, how you irrigate your crop, how you treat the soil, how well you control weeds, what fertilizer you use, and how you deal with pests and diseases.

People should also recognize that improved crop varieties have been responsible for huge increases in crop yields. Among other things, varieties have been developed to provide resistance to diseases and pests and to utilize water and nutrients more effectively.

As the Indian farmers and extension workers recognize, there is an appropriate place for water and chemical inputs. As all recognize, good farming practice counts. Smart farmers use all the tools that they can get to efficiently increase the yield of their crops.

Scientific breeding has produced improved cultivars and they are important. Since the first domestication of crops, crop variety improvements have been sought, and they always come from genetic changes in the crops grown. Ancient practice was random and very slow. For most of the 20th century, scientific breeding was more directed by knowledge and faster than in the distant past; the development of hybrid varieties for some crops greatly improved the effectiveness of breeding of those crops. We have been dependent on hybrid corn (and chickens) for many decades. New techniques of biotechnology utilize even more knowledge, are still faster, and make fewer genetic changes to improve a variety; they are safer in part because the new varieties are so extensively safety tested.

As the world population moves to 11 billion by the end of the century, and as climate change increases and increases the challenges faced by farmers, lets not spread false information and fear about the science we need to feed people.

What you believe doesn't make is so!

Saturday, August 30, 2014

A thought about quality in painting

Jan van Eyck's Arnolfini Wedding Portrait

Detail from the same painting

I found myself in a discussion about art a few days ago. I was facing someone who felt there was no valid standard other than whether one liked a picture. She maintained that the paintings in her house were of equal quality of those in the National Gallery of Art, since she had purchased only those paintings she liked very much.

I buy pictures based on a number of criteria. Of course, I would not buy a picture that I did not find interesting and well done; that would be a combination of the craft of the artist, the colors, the composition, the subject, and the degree that the artist achieved his/her purpose. I also buy pictures by artists whose body of work I admire (at least from that which I have seen). These days; I also tend to buy because I also want to support an artist who seems to merit and need encouragement or a venue (such as a craft fair) that I want to support.

However, I appreciate many pictures that I will not buy, indeed many that are not for sale and that I could never afford to buy even if they were for sale. The painting shown above, by Jan van Eyck in 1434, is one that I like very much. In part, I like it because I know something about it. (Check out the Wikipedia entry on the painting.)

Van Eyck's painting is clearly crafted with great skill and care. However, van Eyck was inventing new techniques when he made the painting. I admire that ability to make inventions (such as oil painting), mastering the technique in the process, and setting a standard for the craft that has influenced painters for six centuries.

The picture tells a story, and tells it well. Since that was clearly an intent of the artist, his success in that effort is worthy of at least respect and I would say admiration. Clearly the picture documents a relationship between husband and wife. The contemporaries of the artist would have understood more from the persons, the pose, the clothing, etc. It also demonstrates the status of a well off merchant couple, based on the quality of the clothing and furnishings. That must have been a purpose of the artist demanded by the patron for the picture. The picture is also clearly located in time and place by the light, the fruit on the tree, the war the room is furnished and the couple is dressed. That took a lot of thought and skill, and was done with mastery.

The artist has included complex symbolism -- the dog, the placement of the hands, the candle lit in the candelabra, the scenes in the mirror frame. Van Eyck has composed the picture both using the symbols to convey information and to decorate the scene.

The mirror is I suppose the thing that brings the picture even higher in my estimation. It appears realistic, but of course it is not. It shows not only the two subjects of the picture in another view, but a third figure -- a witness to the scene. Who he is and what his presence means have interested art historians for generations, and I suppose that is part of van Eyck's intent. The idea of adding the mirror and painting the scene was brilliant. The skill to do it well, breathtaking.

Of course, the picture -- which is in the National Gallery in London -- is pretty to look at. It is historically interesting in that it has interested so many other artists over the centuries and since so many students of art and art history have devoted time and effort to its study. It is also interesting in the innovations created by van Eyck and their impact on the artists who followed. It rewards the viewer for looking at it and trying better to understand what is says and how it makes its statement. These are all aspects of the "quality" of the picture.

A question for media talking heads and politicians

What is wrong with this syllogism:
  • Anything I don't like in the international situation is a problem
  • All problems can be solved.
  • America can solve problems.
  • America should fix any international situation I don't like.

The History of Knowledge Loss

I was working on the analysis of a huge survey in the 1970s when an earthquake hit the country where the data was kept (on punched cards). The cabinets in which the cards were stored fell over and many of the drawers fell open and spilled out their contents. The storage room was on the ground floor and the floor above was devoted to a chemical laboratory. During the earthquake, many of the bottles of chemicals and solvents broke in the lab, spilling their contents. Of course, the building caught on fire (lots of gas in the labs). The fire department came and put the fires out (eventually) with their hoses. The water not only soaked the punched cards, but it washed the chemicals from the labs above into the cards. Eventually, in the clean up, the cards were bulldozed into the institution's courtyard and burned. Of course, the data was not backup up. Years of work was lost with only the most partial results published.

I remember watching a bitter discussion in another country after another earthquake. It seems that the government computer center had been badly damaged in the quake. The income tax records for the country had been destroyed, and it was discovered that a major portion of them had not been backed up off line.

It is not only through the destruction of libraries that knowledge is destroyed.

I believe that knowledge depreciates. The value of the knowledge of how to make buggy whips is not nearly as valuable now as it was when horses were the advanced transportation technology. But not all knowledge is outdated, even when it is old.

Is your computer backed up in the cloud?

Friday, August 29, 2014

A Map of Every Device in the World That's Connected to the Internet

Not surprising that Internet connectivity is concentrated in the USA and Europe, with secondary centers in Brazil/Argentina, Australia, and parts of Asia.

Thursday, August 28, 2014

On institutions and their sclerosis

There is a long, but interesting article in Foreign Affairs by Francis Fukuyama titled "America in Decay: The Sources of Political Dysfunction". I want to quote some sections:
Institutions are “stable, valued, recurring patterns of behavior,” as Huntington put it, the most important function of which is to facilitate collective action. Without some set of clear and relatively stable rules, human beings would have to renegotiate their interactions at every turn. Such rules are often culturally determined and vary across different societies and eras, but the capacity to create and adhere to them is genetically hard-wired into the human brain. A natural tendency to conformism helps give institutions inertia and is what has allowed human societies to achieve levels of social cooperation unmatched by any other animal species....... 
Institutions are created to meet the demands of specific circumstances, but then circumstances change and institutions fail to adapt. One reason is cognitive: people develop mental models of how the world works and tend to stick to them, even in the face of contradictory evidence. Another reason is group interest: institutions create favored classes of insiders who develop a stake in the status quo and resist pressures to reform........ 
Political decay thus occurs when institutions fail to adapt to changing external circumstances, either out of intellectual rigidities or because of the power of incumbent elites to protect their positions and block change. Decay can afflict any type of political system, authoritarian or democratic.
Of course, some institutions are formal, and indeed some have structures and rules defined by law, not by tradition nor by managerial decisions. I am not sure which are harder to change -- those which are explicit but defined by outside power, or those that are implicit and often unconscious.

Fukuyama goes on to consider the state (as the executive institution in government), the legislature (as the democratic institution in government) and the judiciary (as the institution responsible for assuring the rule of law in government).

The article begins citing the creation of the Forest Service, an agency of the executive branch that represented a pioneering reform of the Progressive Era:
Prior to the passage of the Pendleton Act in 1883, public offices in the United States had been allocated by political parties on the basis of patronage. The Forest Service, in contrast, was the prototype of a new model of merit-based bureaucracy. It was staffed with university-educated agronomists and foresters chosen on the basis of competence and technical expertise, and its defining struggle was the successful effort by its initial leader, Gifford Pinchot, to secure bureaucratic autonomy and escape routine interference by Congress. At the time, the idea that forestry professionals, rather than politicians, should manage public lands and handle the department’s staffing was revolutionary, but it was vindicated by the service’s impressive performance. Several major academic studies have treated its early decades as a classic case of successful public administration.
Institutions change over time. Taking the Forest Service as an example:

  • The real world changes, as the American forests changed over the last 100 years. Over time, while the forest service successfully suppressed fires, dead wood built up in the forests and the composition of the forest changed.
  • Society changes, as the American lumber industry changed over the last 100 years. Lumber production became much less important for the U.S. economy.
  • Our understanding changes. Scientists learned about forest ecology and realized that forests had evolved with occasional forest fires, and that some trees even needed fire for their seed to germinate; thus eliminating forest fires changed the forest ecosystem.
  • There is mission creep. The Forest Service was asked to focus on environmental conservation more than on providing a sustainable resource for the lumber industry; it was called upon to provide employment for demobilized soldiers.
Moreover, institutions don't function in isolation. Changes in other institutions can induce changes in an institution of interest. As the map at the head of this post shows, there were few people in the west at the time that the Forest Service was created (fewer still in Alaska, not shown); the Congress was dominated by representatives of the populous areas in the east. Today the representatives of the western states with extensive forests seem likely to have more power. Fukuyama also suggests that some areas the courts have become responsible for making policy in areas that were once under legislative control. Such changes can change the charter of the Forest Service.

Returning to Fukuyama's own words:

The Forest Service’s performance deteriorated, in short, because it lost the autonomy it had gained under Pinchot. The problem began with the displacement of a single departmental mission by multiple and potentially conflicting ones. In the middle decades of the twentieth century, firefighting began to displace timber exploitation, but then firefighting itself became controversial and was displaced by conservation. None of the old missions was discarded, however, and each attracted outside interest groups that supported different departmental factions: consumers of timber, homeowners, real estate developers, environmentalists, aspiring firefighters, and so forth. Congress, meanwhile, which had been excluded from the micromanagement of land sales under Pinchot, reinserted itself by issuing various legislative mandates, forcing the Forest Service to pursue several different goals, some of them at odds with one another. 
Thus, the small, cohesive agency created by Pinchot and celebrated by scholars slowly evolved into a large, Balkanized one. It became subject to many of the maladies affecting government agencies more generally: its officials came to be more interested in protecting their budgets and jobs than in the efficient performance of their mission. And they clung to old mandates even when both science and the society around them were changing. 
The story of the U.S. Forest Service is not an isolated case but representative of a broader trend of political decay; public administration specialists have documented a steady deterioration in the overall quality of American government for more than a generation. In many ways, the U.S. bureaucracy has moved away from the Weberian ideal of an energetic and efficient organization staffed by people chosen for their ability and technical knowledge. The system as a whole is less merit-based: rather than coming from top schools, 45 percent of recent new hires to the federal service are veterans, as mandated by Congress. And a number of surveys of the federal work force paint a depressing picture. According to the scholar Paul Light, “Federal employees appear to be more motivated by compensation than mission, ensnared in careers that cannot compete with business and nonprofits, troubled by the lack of resources to do their jobs, dissatisfied with the rewards for a job well done and the lack of consequences for a job done poorly, and unwilling to trust their own organizations.”
The scenario appears to be a frightening one in that it may be generalized to many other institutions in many other societies. A new institution is created to solve a specific problem. Over the years, the problem itself changes and new problems are added to the institution's responsibilities. In spite of institutional rigidities, the institution changes; some of the changes are adaptive, some less so. Eventually the institution is seen to be dysfunctional and must be revolutionized or replaced. Or perhaps, if one has confidence on the power to revitalize institutions by revolution or to create new and better institutions, the scenario is optimistic.

"whenever you do a thing, act as if all the world were watching"

Very interesting statistic from an article in the Wall Street Journal:
(I)n Rialto, Calif., where an entire police force is wearing so-called body-mounted cameras, no bigger than pagers, that record everything that transpires between officers and citizens. In the first year after the cameras' introduction, the use of force by officers declined 60%, and citizen complaints against police fell 88%. 
It isn't known how many police departments are making regular use of cameras, though it is being considered as a way of perhaps altering the course of events in places such as Ferguson, Mo., where an officer shot and killed an unarmed black teenager. 
What happens when police wear cameras isn't simply that tamper-proof recording devices provide an objective record of an encounter—though some of the reduction in complaints is apparently because of citizens declining to contest video evidence of their behavior—but a modification of the psychology of everyone involved. 
The effect of third-party observers on behavior has long been known: Thomas Jefferson once advised that "whenever you do a thing, act as if all the world were watching." Psychologists have confirmed this intuition, showing that something as primitive as a poster with a pair of glaring eyes can make test subjects behave better, and even reduce theft in an area.
This technology seems to be used in some Maryland police departments.