Facts can be misleading

I noted a couple of posts on Facebook yesterday that reminded me that the way a fact is presented can be misleading. Here are examples of facts and my reason for believing their presentation is misleading.

World Bank Reports

A report recently presented data on the frequency with which reports posted as PDF files on the World Bank website are downloaded. A blog post, based on the report is titled "The solutions to all our problems may be buried in PDFs that nobody reads". The post states:

The World Bank recently decided to ask an important question: Is anyone actually reading these things? They dug into their Web site traffic data and came to the following conclusions: Nearly one-third of their PDF reports had never been downloaded, not even once. Another 40 percent of their reports had been downloaded fewer than 100 times. Only 13 percent had seen more than 250 downloads in their lifetimes. Since most World Bank reports have a stated objective of informing public debate or government policy, this seems like a pretty lousy track record.

I assume that the data on the number of downloads for the various reports is correct.  It is slightly misleading in that the original study looked at reports posted over a five year period. As one might expect, the most recently posted reports were downloaded less often than earlier posted reports, since those reports which draw interest tend to do so over a number of years. It looks to me on reading the original report that on average the posted reports are downloaded 175 to 200 times, with many never downloaded and some downloaded thousands of times.

I assume that these reports are written for a specific purpose and distributed in paper or digital form to the intended audience. Then they are posted to the Internet in case others might benefit. I think it would be interesting to know how widely they are distributed in their original forms, how widely they are read by their original intended audience, and how influential they were through the original formal distribution. None of those questions is addressed by looking at the frequency by which the archival PDF document is downloaded. The blog post (and its blurb on Facebook) might unfortunately have readers infer that the reports are not useful based on data that have little or nothing to do with their quality, total distribution, nor impact.

In the 1970s, a team of World Bank staff members wrote a health policy paper. A colleague and I, then working at the Office for International Health in the U.S. Department of Health, Education and Welfare were invited to participate in the preparation of that policy paper. I have no idea how many people read it, but one of the results of the paper was the creation of a World Bank Office of Health, Nutrition and Population, which then went about increasing Bank lending in those areas supporting human capital. The impact of the paper was probably not in how many people read it, but in that key senior officials in the Bank read it and it influenced them to modify Bank policy and make an organizational change to assure the program changed in accord with the new policy direction.

Why does the Bank make these reports available on the Internet? One reason is transparency -- so that anyone can see the analysis put forward by the Bank. That certainly is a good idea. Moreover, many of these reports are valuable, as we must infer from the fact that some are downloaded thousands of times. The cost of posting an already prepared report on the World Bank website and making it available to the public is minimal. I for one am glad that the Bank does this.

Moreover, it probably is not a bad idea for a specific audience to see how often the posted reports are downloaded. But that data should not be misinterpreted, nor thought to be more important than it really is.

The Maternal Mortality Rate in the USA is Increasing

The Lancet has recently published an article titled "Global, regional, and national levels and causes of maternal mortality during 1990—2013: a systematic analysis for the Global Burden of Disease Study 2013". A man I greatly respect, Robert Reich, posted the following on Facebook:

According to a report released last week in the The Lancet, the U.S. now ranks 60th out of 180 countries for deaths in childbirth. For every 100,000 births in America, 18.5 mothers die. By comparison, 13.5 women die in Iran, 6.1 in the United Kingdom, and only 2.4 in Iceland. Why? The rate is directly related to poverty and lack of healthcare. U.S. states with high poverty rates have maternal death rates 77 percent higher than states with lower levels of poverty, and women with no health insurance are four times more likely to die in childbirth than insured women. 

Yet many of these high-poverty states are among the 21 that have refused to expand Medicaid, even though the federal government will cover 100% of the cost for the first 3 years and at least 90 percent thereafter. Several of these states have also cut family planning, restricted abortions, and shuttered women’s health clinics. Their policies are literally killing women. This is (or should be) a national outrage. 

Why aren't more women (and men) protesting in the capitals of these states? Why aren't the rest of us making more of a ruckus?

The Lancet is a very highly respected journal, known for the care it takes in peer review before accepting articles for publication. The article in question was the work of a very large number of authors, and has special importance as an input to the understanding of the global burden of disease. It is likely to be used not only by the World Health Organization and the United Nations.

The Lancet team reported 796 deaths in childbirth in the USA in 2013, 752 in 2003 and 527 in 1990. The U.S, government expects about 650 such deaths per year among nearly 4 million live births. Thus the report's figures have face validity.

The report also notes that the rate increased from 12.4/100,000 in 1990 to 18.5/100,000 in 2013. The increase in rate reflects both an increase in the number of women who died, and a decrease in the denominator -- the number of births per year in the United States since 1990.

Birth rates are at an all time low, and women are having their children at an older age. 40 percent of births are to unmarried women today, much higher than in the past. Birth rates are higher in low income households. These factors influence maternal mortality rates.

The actual number of deaths in childbirth increased by 269 in 2013 over 1990. In a country of 330 million, that is not a large number. Some 13,000 per year are murdered in the USA. More than 30,000 people die per year in traffic accidents. 150 to 200 people per year die from food allergies.

Of course, a single death that can be avoided is tragic. That is especially true in a young woman and a mother. Reich is right to focus also on the reduced access to medical care for the poor as a factor in this mortality. Prenatal care and timely hospital care during delivery  probably could reduce this mortality. He is also right that there is a significant problem in poor young women not having health insurance, and not having access to maternal and child health services. Indeed, the unnecessarily high maternal mortality rate in the United States may well be symptomatic of the more general problems of low life expectancy, lack of health insurance for many low income people, and their consequent lack of access to medical services.

Still, I find the fact of the maternal mortality rate as cited can be misleading. It is in fact relatively low. Maternal mortality in Developing Countries, according to the study, was 232.8/100,000.  So too saying that the U.S. rate was 60th out of 180 countries neglects the fact that the number of deaths varies randomly from year to year, and next year the random nature of the occurrence of these rare events may change the ranking significantly.