Wednesday, June 28, 2017

News Feed Comments

Accepted at Atmospheric Chemistry and Physics after nearly two and half years

February 22, 2013 Filed under Blog, Featured, Publishing 

On 25 January 2013, 904 days from the date it was submitted (5 August 2010), a manuscript was finally published at Atmospheric Chemistry and Physics. Not only was this manuscript noteworthy for how long it remained in limbo before a final thumbs up or thumbs down from the Editor, but it was noteworthy for other reasons as well.

First, the impressive title: “Where do winds come from? A new theory on how water vapor condensation influences atmospheric pressure and dynamics”. Could we atmospheric scientists have been wrong all along? The article suggests that numerical models of the atmosphere are flawed and our conceptual understanding of how condensation affects dynamics is is wrong, as well. That’s a pretty bold claim.

(I note here that channel experiments of growing baroclinic waves in the absence of moisture still produce cyclones of comparable size and intensity to real ones. Thus, the origin of winds are most certainly not condensation.)

Second, the paper was published despite many reviewers, including one of the formal reviewers (Isaac Held), not recommending publication of the article. To illustrate the type of argument made against the paper, I present the following from Held’s review. Held opens his review with the following:

Recommendation: Reject

The authors make an extraordinary claim that a term that is traditionally considered to be small, to the point that it is sometimes neglected in atmospheric models and, even when not neglected, rarely commented on, is in fact dominant in driving atmospheric circulations. The effect concerned is that of the mass sink associated with condensation. This term is of first-order importance in some planetary atmospheres, such as Mars, where the total mass of the atmosphere has a substantial seasonal cycle, but for Earth the standard perspective is that the heat release associated with condensation dominates over the effect of the mass loss. A claim of this sort naturally has to pass a high bar to be publishable, given the accumulated evidence, implicit as well as explicit, that argues against it. I am afraid that this paper does not approach the level required. I have done my best to keep an open mind, but do not see any cogent arguments that overturn the conventional wisdom. I do applaud the authors for questioning the foundations of our understanding of the atmosphere and provide some unsolicited advice on how the authors might proceed to clarify some of these issues. There is a need for some clarification.

Third, the Editor A. Nenes accepted the article in January 2013, accompanied by the following Editorial, from which I quote below.

The authors have presented an entirely new view of what may be driving dynamics in the atmosphere. This new theory has been subject to considerable criticism which any reader can see in the public review and interactive discussion of the manuscript in ACPD (http://www.atmos-chem-phys-discuss.net/10/24015/2010/ acpd-10-24015-2010-discussion.html). Normally, the negative reviewer comments would not lead to final acceptance and publication of a manuscript in ACP. After extensive deliberation however, the editor concluded that the revised manuscript still should be published – despite the strong criticism from the esteemed reviewers – to promote continuation of the scientific dialogue on the controversial theory. This is not an endorsement or confirmation of the theory, but rather a call for further development of the arguments presented in the paper that shall lead to conclusive disproof or validation by the scientific community.

This paper and how it was handled raise a few questions about how peer review works at ACP, how peer review works in general, and how peer review should work.

1. I think we can all agree that no paper should be left in limbo for as long as it was (the last comments were uploaded in April 2011). Submissions should be handled promptly, and the authors should receive an “accept”, “revise”, or “reject” decision as quickly as possible. This is important because awards and promotions depend upon publications. If ACP does not want to publish the paper, the authors should be told as soon as possible so that the paper can be revised and submitted to a different journal.

2. My colleague Doug Parker reminded me of one of my favorite quotes by the eminent fluid dynamicist G. K. Batchelor (1981):

Papers of poor quality do more than waste printing and publishing resources; they mislead and confuse inexperienced readers, they waste and distract the attention of experienced scientists, and by their existence they lead future authors to be content with second-rate work.

By publishing this paper and admitting that they do not necessarily support the science behind it, ACP risks wasting the time of the community. Science moves forward because the burden of proof lies with the author, not the reviewers.

3. The Editor notes that the paper would otherwise not normally be published. Yet, the authors argue that they should not be held to the higher bar noted by Held, just a single bar for all of science. Taking this argument at face value, however, means that the authors did not even reach that single bar.

In reality, the decision by the Editor indicates that the authors received a lower bar to publication. According to the reviewers, the authors have not presented sufficient evidence for the reviewers to recommend publication.

(On the other hand, answering all the comments from readers on the ACPD site was arguably a lot more work than many papers go through to get published.)

4. In my experience, well-reasoned rejections should be taken seriously. As I have discussed on Roger Pielke Sr.’s blog, I usually don’t encounter multiple reviewers recommending rejection very often. Consensus in the peer-reviewing community happens less than you think it does, for various reasons discussed in “Are three heads better than two? How the number of reviewers and editor behavior affect the rejection rate” (Schultz 2010). Therefore, ACP’s decision to publish this paper to “to promote continuation of the scientific dialogue” goes against the valid reviewer’s concerns. (One of those concerns is that previously published work shows that the term in question is small, in general.)

5. I wonder what the disposition of this paper would have been if it had been submitted to a more traditional atmospheric dynamics journal such as Journal of Atmospheric Science, Quarterly Journal of the Royal Meteorological Society, or Tellus. Would it have been published? I have seen papers on basic meteorological concepts get rejected (or that would have been rejected) from traditional meteorological journals get published by AGU journals. I’m not sure why. Is it because they are less likely to identify the most relevant reviewers? Is it because the editors and reviewers are unaware of standard meteorological practice? Or is it because their papers get a fresh hearing that traditional meteorological journals would otherwise be too biased against?

Given the high status of the reviewers and commenters, it is hard to argue in this case that the manuscript ended up in the hands of people who didn’t know what the issues were.

6. The Editor’s decision to publish this paper didn’t need to happen. In fact, the very nature of Atmospheric Chemistry and Physics (Discussion) and the tremendous discussions within the blogosphere (e.g., here) means that the work was already available for comment. One thing is for certain: the authors are happy to engage those with criticisms against the paper.

7. Finally, I note that this paper is not a “theory”, it is a “hypothesis.”

The chapter isn’t completely written on this topic yet. I suppose the authors will keep it going anyway. Read the article for yourself, read the comments and the Editor’s decision, and make up your own mind.

Makarieva, A. M., Gorshkov, V. G., Sheil, D., Nobre, A. D., and Li, B.-L.: Where do winds come from? A new theory on how water vapor condensation influences atmospheric pressure and dynamics, Atmos. Chem. Phys., 13, 1039-1056, doi:10.5194/acp-13-1039-2013, 2013.

order at Amazon.com

Comments

5 Responses to “Accepted at Atmospheric Chemistry and Physics after nearly two and half years”
  1. James Allan says:

    I agree that it shouldn’t have taken that long to make a decision and furthermore, I find it regrettable that there was a bit of cross-contamination between ACPD and the general bloggosphere as part of the process.

    However, I do think there should be a place in the world for people putting forward mathematically-based hypotheses without burdening them with the requirement to prove their importance, because there is healthy debate that can occur on a level not offered by the traditional journal formats. If this had happened in another discipline, such as particle physics, astronomy, economics or pure maths, the forum for that discussion would have been arXiv. For better or for worse, ACPD is probably the closest thing the mainstream atmospheric science literature has to that. Whether it is doing itself any favours by inadvertently adopting that role is another matter entirely, as is whether this paper had the sufficient scientific merit to progress to ACP (I can’t really comment on that because I’ve not read it in any detail and wouldn’t pretend to understand most of it anyway).

  2. Prof. David M. Schultz says:

    Agreed, James. ACPD does already fill that niche, and it gives a formality to the open peer-review debate. It is to be commended for that. But, publishing a paper in ACP that the reviewers agree is flawed weakens the merit of those papers ACP does publish that have been sufficiently vetted.

  3. Jay says:

    I have fewer problems with potentially flawed theories/hypotheses being published in this fashion than I do with an apparently flawed presentation of a theory/hypothesis being published in this fashion. An effectively and succinctly presented study can be thoroughly scrutinized and criticized. A poorly presented study that includes imprecise terminology, incomplete derivations, and insufficiently supported statements, assertions, and conclusions is not, I feel, helpful to the process. Furthermore, if the authors choose to be combative (or are simply more energetic than their critics), a poor presentation can be used to mask inadequacies in the work by creating a framework from which any given comment can be claimed to have missed the point of the work. If knowledgeable readers are missing the point of your work in a way that causes them to reject your theory/hypothesis then you have not presented your work with sufficient clarity and, in my opinion, it should not be published.

  4. Your last sentence hits the nail on the head of this whole problem with this paper. Thanks, Jay!

  5. Douglas in Australia says:

    I am an author. Thanks for the interest. Allow me to offer some clarifications:

    1) One referee Dr Judith Curry accepted that the mechanism we describe is real as did several commentators.

    2) Dr Held’s comments were indeed negative (we expected that as we criticised a perspective that he is known for — it is normal when choosing referees to choose one such and one more “objective” view). We also responded in detail to each point (you can see our detailed reply at the APCD site http://www.atmos-chem-phys-discuss.net/10/24015/2010/acpd-10-24015-2010-discussion.html. We specifically made a review of the study by Spengler et al. (2011) that he holds as a comparison to ours and explained flaws in its physics (see http://www.atmos-chem-phys-discuss.net/10/C14894/2011/acpd-10-C14894-2011.pdf).
    We indeed argue why our ideas should not be given a “higher bar” than conventional ideas. See http://www.atmos-chem-phys-discuss.net/10/C15085/2011/acpd-10-C15085-2011.pdf.
    You don’t need to accept the ideas in our paper to accept the general principles for objective appraisal.

    3) Most papers receive criticisms, especially from referees whose work they may undermine, and the editors then weigh and judge the exchange and the authors’ ability to address these in the final text — that is how the system works. A recommendation to reject a submitted draft does not mean the reviewer might not accept the later revision. In this case the journal looked at all the comments, replies and revisions and decided we had made an adequate case for publication. We all know that the paper was likely to be controversial, but rejection also needs to be based on clear and compelling arguments – right? We all know that there are many cases where ideas in science gain a very negative response but later become accepted — that doesn’t mean we are right, but it does call for certain standards of argument from BOTH sides.

    4) The physical principles being derided by Dr Held et al. have a more respectable pedigree than is being realised. These concepts have been published in the physics literature — e.g. see this physics article: Phys. Lett. A, 373, 2801–2804, 2009a. Our challenge here was to get it into the climate literature.

    5) Personally I find the idea that the condensation term has “traditionally” been considered small a worry — where is the theory and literature that shows that it is small? Doesn’t that matter? Indeed there is some recent work by other authors suggesting that this “tradition” is incorrect. See for example: Lackmann, G. M., and R. M. Yablonsky, 2004: The importance of the precipitation
    mass sink in tropical cyclones and other heavily precipitating systems. J. Atmos. Sci. 61, 1674-1692.

    6) By publishing the theory the journal doesn’t say we are right or wrong — only that there is a genuine case to answer. We make a number of specific predictions. What the journal is agreeing is that these ideas seem coherent (at least to some if not to all) and deserve to be properly evaluated. Once a certain standard has been attained evidence (not reviewers) should decide.

Speak Your Mind

Tell us what you're thinking...
and oh, if you want a pic to show with your comment, go get a gravatar!