Reviewing Peer Review

peer: (pîr) n.
A person who has equal standing with another or others, as in rank, class, or age: children who are easily influenced by their peers.

peer: (pîr) intr.v.
peered, peer·ing, peers
1. To look intently, searchingly, or with difficulty. See Synonyms at gaze.
2. To be partially visible; show: The moon peered from behind dark clouds.

- definitions from dictionary.com


Is the system of scientific peer review ready for a paradigm shift? Peer review has long relied on the usage of the noun in common parlance, but perhaps that has been too limiting. Will scientists of my generation and beyond accept a broader definition? In a surprising move, one of the most respected names in scientific publishing is willing to permit open public comment on articles submitted for publication.

The Editor-in-Chief of Nature, Philip Campbell, writes on the Nature website that,

Nature's peer review process has been maintained, unchanged, for decades. We, the editors, believe that the process functions well, by and large. But, in the spirit of being open to considering alternative approaches, we are taking two initiatives: a web debate and a trial of a particular type of open peer review.

The trial will not displace Nature's traditional confidential peer review process, but will complement it. From 5 June 2006, authors may opt to have their submitted manuscripts posted publicly for comment.

Any scientist may then post comments, provided they identify themselves. Once the usual confidential peer review process is complete, the public 'open peer review' process will be closed. Editors will then read all comments on the manuscript and invite authors to respond. At the end of the process, as part of the trial, editors will assess the value of the public comments.

This is intriguing. When presented with this news, many scientists I have spoken with do not believe it. It just seems so different than what we are used to doing. From my perspective, there are several potential benefits to this, and some detractors. Let’s start with the detractors.

Peering into Peer Review is Poor because:

• It’s bad for the social structure of science. At large scientific meetings, the most powerful individuals in the field – the guys and gals who review grants and receive awards – are not as criticized on their science as much as junior colleagues. When a comment is made that could be perceived as a personal attack, there is a real potential of retribution. On the flip side of this, the more democratic the review process gets, the greater the potential for reviews to become popularity contests. Any scientist can tell you the last time she heard a speaker legitimize a finding because the researcher was from Boston or San Francisco.
• More than 90% of Nature’s submissions are rejected. That means there could be a large body of evidence presented to the public arena that is not formally addressed by citation. If ideas for experiments, and dare we say, marketable products, are submitted to Nature but rejected, is that information in the public domain?
• Messing with a well-established system will throw the socio-economic foundations of the practice of science out of order to the extent that the current system of funding science will be unstable and scientists will not be able to keep their labs up and running. Maybe this seems a bit of a doomsday scenario, but this is just a blog. What’s the harm in a little doom?
• The system works, and it works well.

Peering into Peer Review has Potential because:

• Peer Review is a taxing system on researchers. It takes half a day to properly review an article. Scientists are bombarded with requests to review papers, and after the top scientists refuse (due to over-commitment), reviewers can end up reading papers on the periphery of their expertise. Permitting the scientists who are most concerned with a topic to self-select themselves as reviewers will increase the quality of reviews.
• Petty requests and personal attacks in reviews have no place in science. We might expect that a named reviewer will not make comments such as “more information could be drawn from slides dipped in weak grape juice” than the hematoxylin and eosin stained tissue sections in the submission’s figures. I will concede a bit of sour grapes here – this was taken from one of my own reviews – but the point is still valid. Public statements are typically less subjective than anonymous ones.

However this turns out, I am interested in the potential for improving the practice of science. In the least, the activity of reviewing peer review may result in a better mechanism of how science is communicated.