The weak in review

As scientists, we live or die by what we publish. Unsurprisingly, a heightened sensitivity and anxiety are associated with all aspects of publication, and particularly with the reviewing process. Most scientists participate on both sides of the peer-review process, and many hold the following two opinions. First, I am the most fair, thoughtful, and even-handed of reviewers. Second, the reviewer of my last paper is the most unreasonable, ignorant, and vicious of reviewers.Comparing these two statements is like comparing the answers of male and female respondents to sex surveys. They never quite match. Surely some interesting truth about human nature lies in the middle, along with the correct answer to the question itself.What do you see as a reviewer? Some wonderful papers. But also papers that are not quite as good as they could be. Experiments that almost answer the question, but stop short of proving it. Tunnel vision with respect to favorite models, especially those that make the authors’ work seem very important. And sometimes, intellectual or technical gaps that cast the whole idea into doubt.The worst thing about reviewing is the complete lack of respect you get from the authors of the paper. Nothing is more depressing than writing a long, well-reasoned review of a paper, only to receive the same paper a month later from a different journal without a single change. And the authors’ rebuttals to reviewers can be amazing. When a paper is revised after review, the editor usually sends it back to the same reviewers that saw it in the first place, together with the authors’ comments. If the authors thought about this fact carefully, they would probably not write rebuttals in which they savaged the reviewer's intelligence and parentage.What do you see as an author? Sometimes the reviewer is correct. I consider this possibility, at least for a few minutes. But some reviewers need to demonstrate that they are smart enough to think of the next experiment by asking that it be included in a paper that already represents five years work. Other reviewers are so attached to one point of view that they generate ridiculously complex alternative explanations of your results. Yet others ask you to do incredibly difficult controls for unimportant points. And, inevitably, there are the reviewers who ask you to cite five marginally relevant papers by a certain person…maybe the reviewer himself? Of course, nothing is more depressing than getting a ludicrous review from one journal, only to send the paper to a second journal and get the same ludicrous review again.Psychologically, I suspect that reviewer anonymity itself is a problem. The argument for anonymity boils down to fear. Your fellow scientists are children. They cannot be trusted to react to the reviewer's comments in a professional fashion. If they find out that you are the negative reviewer, they will trash your grants and poison your dog. Having read some of those authors’ rebuttals, this argument might have some basis in fact. On the other hand, the anonymity of the reviewer turns him into a nameless, abstract opposing force. I can project the worst possible motives on him: arrogance, malignance, satanism. Any real person is bound to fall short of this ideal of evil.Where is the common ground here? As I think of my own experience, one possibility is that I am the most superior of persons, unique in the level to which I sacrifice myself as a reviewer and am mistreated by the world as an author. Noble as it makes me feel, I have to admit that this explanation is unlikely. A more likely, but less gratifying, explanation lies in bad scientific writing. If a reviewer who knows something doesn't think the paper is interesting, a naive reader is even less likely to be captivated. If a reviewer thinks the problem is already solved, the background isn't explained well.On the other side, it's hard to write a review that conveys the correct level of enthusiasm or concern. Sometimes an author is greatly upset by a minor comment, or misses the point of a criticism. Furthermore, the editor may think that the reviewer and the author are much further apart than they are.For most of the papers I review, what I want most is to talk with the author for half an hour about the results. Why didn't you try this experiment? How many times was this control done? What is that ugly blob in the upper right panel? Keeping the editor out of that conversation would encourage open answers. I hope for a new technology that will allow me this luxury while preserving anonymity, and perhaps it exists. I recently received a right-wing-anarchist catalogue that offered me The Voice Scrambler for the telephone. I was entranced. I could just program my Scrambler to make me sound like an elderly man from New England, and I could ask those simple questions without fear for my future. And as an author, I would probably prefer any human voice to another anonymous nasty review. Or how about communicating by e-mail, using the Anonymous Remailer in Finland, that lets you send e-mail stripped of all identifying information? Either seem worth a try.