The Baloney Detection Kit: A 10-Point Checklist for Science Literacy

In fact, brainpickings.org has a lot of good posts about books, creativity, life, and writing. Enjoy surfing!

Put yourself in the reviewer’s shoes. It may be hard to do so, but it is often the best way to understand what the reviewer is trying to communicate. If the reviewer misunderstands something you wrote, could you write it more clearly and avoid the misunderstanding?

View the reviewer as a representative member of your community. Suppose your manuscript had received three reviews, and one reviewer recommended rejection. It might be tempting or easy to treat that reviewer as a crank, but resist. If you consider the reviewer as a representative of your community, then a third of the readers of your manuscript may arrive at the same conclusion. Are you content to have a third of your potential readers disenchanted with your manuscript? Wouldn’t it make sense to try and address these issues, and thus gain greater credibility once the manuscript is published? If you view the three reviewers each representing a statistically representative sampling of the meteorology community, then you move away from “satisfying the individual reviewer’s comments” to “improving your manuscript to gain greater acceptance by the community”.

How do you tell if the reviewer is biased? Look at the comments. Is the reviewer pushing a certain agenda? Or, is the reviewer asking for concessions that do not pertain to a certain agenda: additional supporting evidence, refuting contradictory hypotheses, improved text, more legible figures? In my experience, most reviewers are not expressing personal bias in reviews. So, try to look more objectively at the reviews.

(Image from pirateplush.com)

You can take a stand by signing up to publish elsewhere, avoid serving on their editorial boards, and avoid doing reviews for Elsevier-published journals.

Images by Giulia Forsythe

Nancy Rothwell

I had some bad news on the research publication front, my research group had a great paper rejected by completely unreasonable referees – they were obviously biased – or at least we think so. More work to do!

Schultz, D. M., 2011: Rejection rates for multiple-part manuscripts. Scientometrics, 86, 251-259. [PDF]

I found that although the rejection rates for multiple-part manuscripts were not that different from the rejection rates for manuscripts as a whole, the reviewer comments about multiple-part manuscripts suggested some ways that authors could avoid those problems in the first place:

“Write manuscripts that are sensibly independent of each other, make minimal reference to unsubmitted manuscripts, and have sufficient and substantiated scientific content within each manuscript.”

Since doing that research, I’ve had discussions with several authors before submission, and I am proud to say that I’ve convinced them to follow a different course: writing sensible manuscripts that are independent of each other. We still get cases of multiple-part manuscripts coming in, though. They are a pain to handle.

Recently, I became aware of a case where Part 1 was submitted to one journal, and Part 2 was submitted to another journal. Part 2 was accepted, yet Part 1 is still in review. This kind of outcome just complicates matters for everyone: authors, editors, reviewers, publishers and readers. Who is the winner here? Is labeling your papers as part of a multiple-part series really worth all this hassle at all stages: peer-review, publication, and post-publication?

What are the indications that you are going to struggle with peer review with your multiple-part paper? (With apologies to Jeff Foxworthy.)

If your two manuscripts are so intertwined that Part I refers to Part II as much as Part II refers back to Part I, you may have a problem in peer review.

If you started writing a paper, only to realize that you have more than the required word-count that you want to say and so start writing a second paper, you may have a problem in peer review.

Simply put, writing a multiple-part manuscript is not easy. Balancing material between the two parts, yet providing enough information so that the reviewers believe they have all the evidence is not easy.

Since publishing that article, I have thought more about what the results from my article in Scientometrics mean. My impression is that high-quality authors could navigate the potential minefield pretty well. They were clever enough to avoid the pitfalls. Mediocre authors struggled to get published because of the particular challenges that writing multiple-part papers required. And, low-quality authors who had trouble getting their single-submission papers published in the first place had little hope of getting a multiple-part paper published.

Personally, I can’t see the benefit in writing multiple-part papers for myself, but I see why some people do it. (Frankly, I think a lot of authors just do it because they have seen it in the literature and do the same thing. In fact, a lot of the literature has that flavor…it worked for Author A, must work for me the same way!)

Some journals ban multiple-part papers. Is it time for the AMS journals to abolish multiple-part papers?

If the results from a recently published article on the factors affecting judges making parole decisions are analogous to that of a journal editor making accept/revise/reject decisions on manuscripts, then send your editor some food and encourage them to take a break.

Danziger, S., J. Levav, and L. Avnaim-Pesso, 2011: Extraneous factors in judicial decisions. Proc. Nat. Acad. Sci., doi: 10.1073/pnas.1018033108.

The authors found that “the percentage of favorable rulings [parole granted] drops gradually from about 65% to nearly zero within each decision session and returns abruptly to about 65% after a break.” The figure from the article below tells the whole story.

The authors argue that such external factors affect the presumed judicial impartiality based only upon the facts of the case and the relevant laws.

Do the same outside influences affect editor decisions at journals? Hard to say. Rarely do I have a day where I have to make a large number of decisions in a row. That being said, if I do have several decisions to make, I usually handle the easiest ones first (those tend to be the accept or revise decisions first). Rejections generally involve more weighing of the reviews and the manuscript, require more thought, and take more time to write the decision letter. So, if the same effect happens in journals, it is likely because I ordered my workload that way.

The journals of the American Meteorological Society now require line numbers in submitted manuscripts. How do you add such line numbers to your manuscript? From p. 374 of Eloquent Science: A Practical Guide to Becoming a Better Writer, Speaker and Scientist:

“Final Checks of Your Manuscript,” “Lines numbered in margin”: You may wish to add line numbers for manuscripts that you submit whether or not the journal requires it, making the reviewers’ task easier. In Microsoft Word for Windows, enabling line numbers is under the layout tab. For Microsoft Word for Mac, enabling line numbers is under Format, Document, Layout. For LaTeX, usepackage{lineno} and linenumbers*.

(Image from Elsevier)

This response has always bothered me, but I didn’t know why. After thinking about it recently, now I know why.

This comment is wrong for two reasons. The first is a practical one, the second represents a misunderstanding of how the peer-review process works.

First is the practical issue. Simply, the author has not addressed the reviewer’s comment. By pitting one reviewer against the others rather than providing a sensible response to the reviewer, the author has not provided what is usually requested from the editor: a response to all the reviewers’ concerns. As an editor, I want to hear what the author thinks of all of the reviewers’ comments. I don’t want to hear that because the other two reviewers didn’t raise that issue means that it is irrelevant.

Which leads to me to the second issue: a misunderstanding of the peer-review process. Peer review is not a democracy! Not all reviewers are created equal, whether it be by expertise, experience, or even quality. Thus, I don’t expect the comments provided by one reviewer to overlap the comments provided by another reviewer. (If the comments did overlap all the time, then I wouldn’t need a second reviewer!) For example, Reviewer A might be an expert in statistics, and Reviewers B and C might be experts in synoptic meteorology. If Reviewer A raised an issue about how the statistics were calculated in the manuscript, I wouldn’t necessarily expect Reviewers B and C to pick up on exactly the same point. Yet, that doesn’t mean that the author is absolved from addressing the issue. The editor makes decisions, and I want to ensure that the author has addressed all the reviewer concerns.

There is the issue of what happens if, for example, Reviewers A and B explicitly say that they like section 5 of the manuscript and Reviewer C wants to see it eliminated. In that case, I think an author has more to stand on in siding with the majority, although I still would hope to see a spirited rebuttal of why the section should remain (not just “Reviewers A and B both like this section”) and perhaps some efforts to improve the section to address Reviewer C’s concerns.

What do you think? Do you think it is an acceptable excuse for not addressing reviewers’ comments?

I offered a different perspective. Maybe the author wanted to leave the text so obvious that the meaning to readers would be “look at the foolishness that I have to address in my manuscript just to get the paper past the peer reviewers.”

Regardless of the author’s intent, this is defensive writing, responding to what the reviewers want.

Of course, another perspective is that the author is so insightful and careful a writer that the author addresses potential reviewer concerns before submission. That’s offensive writing.

Interestingly, this discussion with John happened right after I had been meaning to write this blog post about offensive and defensive writing styles. Getting your manuscript published requires submitting high-quality science that is written well. In my view, one of the secrets to receiving good reviews is to spot reviewers’ potential concerns and address them upfront before submission.

Of course, being an offensive writer isn’t easy because you have to anticipate what might reviewers have to say about your own work. Self-inspection does not come easy for some of us, but the papers that I see sail through the peer-review process are those that anticipate their greatest weaknesses and address them before submission. Reviewers have much less to say about these papers.

So, be offensive!