Predicting citation rates and predicting success in Vietnam.

There is an interesting article in Science magazine last month providing a model of the long term impact of articles in scientific journals. The model recognizes that the more an article has already been cited, the more likely it is to be cited again. It also recognizes that eventually the citation rates for most articles trails off, so that the time since initial publication is a parameter. Finally, the model posits a "fitness" parameter which the authors believe captures the perceived novelty and scientific importance of the content of the article.

This fitness parameter is similar to a parameter that I have described in previous posts as the probability that a paper submitted for publication will be recommended for publication by peer reviewers.

Note, however, these statements by the authors:

Paradigm-changing discoveries have notoriously limited early impact, precisely because the more a discovery deviates from the current paradigm, the longer it takes to be appreciated by the community. Indeed, although for most papers their early- and long-term citations correlate, this correlation breaks down for discoveries with the most long-term citations. Hence, publications with exceptional long-term impact appear to be the hardest to recognize on the basis of their early citation patterns.

And:

The proposed model has obvious limitations: It cannot account for exogenous “second acts,” like the citation bump observed for superconductivity papers after the discovery of high-temperature superconductivity in the 1980s, or delayed impact, like the explosion of citations to Erdős and Rényi’s work 4 decades after their publication, following the emergence of network science.

Of course, that is a major problem with peer review. The ideas that are most likely to be revolutionary or to assume major future importance are not likely to be recognized for their importance when first submitted.

 I thought about this while watching a program on TV on President Kennedy's decision making in 1963 with respect to the Vietnam War. That summer, with some 14,000 military advisers in South Vietnam, Kennedy sent high level teams to the country to complement the regular reporting from the military and the Ambassador as to whether the South Vietnamese government was being successful or not in fighting the insurgency. One of the issues under discussion was whether 1,000 advisers should be withdrawn at the end of 1963 to strongly signal that the government of South Vietnam did not have an open check for U.S. military support.

The problem with this approach seemed to me to be similar to estimating the long term impact of a scientific paper from its early citations. If the situation changes, the biggest future impacts can be missed by projecting current situation data ignoring the risk of those changes. The success or lack of success of local tactics in Vietnam in 1963 did not help in predicting the long long term strategy of the insurgents and their supporters in North Vietnam and the Communist block countries. Short term success of the Southern government troops was met by increased opposition forces and would be until the United States withdrew its troops and until the government of South Vietnam fell.