The decreasing number of studies with negative results

February 8, 2012 Filed under Blog, Featured, Publishing

When I lived in Oklahoma, Chuck Doswell used to lament to me that it was difficult to publish null cases in meteorology (for example, when something was forecast to happen, but didn’t). Later, when talking to Roseanne McNamee at the University of Manchester, she lamented the same.

There are even several journals for negative results:

Presumably, the last journal exists because the other journals present uninteresting negative results.

Now, this effect has been quantified in the literature: Daniele Fanelli’s “Negative results are disappearing from most disciplines and countries”. A portion of the abstract follows:

A system that disfavours negative results not only distorts the scientific literature directly, but might also discourage high-risk projects and pressure scientists to fabricate and falsify their data. This study analysed over 4,600 papers published in all disciplines between 1990 and 2007, measuring the frequency of papers that, having declared to have ‘‘tested’’ a hypothesis, reported a positive support for it. The overall frequency of positive supports has grown by over 22% between 1990 and 2007, with significant differences between disciplines and countries. The increase was stronger in the social and some biomedical disciplines. The United States had published, over the years, significantly fewer positive results than Asian countries (and particularly Japan) but more than European countries (and in par- ticular the United Kingdom). Methodological artefacts cannot explain away these patterns, which support the hypotheses that research is becoming less pioneering and/or that the objectivity with which results are produced and published is decreasing.

Fanelli, D., 2012: Negative results are disappearing from most disciplines and countries. Scientometrics, 90, 891–904. [PDF]

Image: http://www.kk.org

Comments

One Response to “The decreasing number of studies with negative results”

Chuck Doswell says:

Wednesday, 8 February 2012 at 16:03

Part of the problem, at least in atmospheric science, is it’s difficult to offer explanations for something that didn’t happen. You’d have to be able to go back and show that if it were possible to change something in the past, then the event that didn’t happen would indeed eventuate. That’s pretty tough to do.

Another problem is that a negative result seems like a failure, and most people (a) don’t want to reveal their failures, and (b) may not care much about someone else’s failures.

Negative results can help others avoid going down blind alleys, but might also stimulate others to try to fix what might have gone wrong to produce a negative result. Done properly, a paper presenting negative results could be a positive contribution!