Peer-review, what is it for?

An interesting discussion started at Retraction Watch, in the comments of the post Brian Deer’s modest proposal for post-publication peer review. Let me repeat the part which I find interesting: post-publication peer review.

The previous post “Peer-reviews don’t protect against plagiarism and articles retraction. Why?”  starts with the following question:

After reading one more post from the excellent blog Retraction Watch, this question dawned on me: if the classical peer-review is such a good thing, then why is it rather inefficient when it comes to detecting flaws or plagiarism cases which later are exposed by the net?

and then I claimed that retractions of articles which already passed the traditional peer-review process are the best argument for an open, perpetual peer-review.

Which brings me to the subject of this post, namely what is peer-review for?

Context. Peer-review is one of the pillars of the actual publication of research practice. Or, the whole machine of traditional publication is going to suffer major modifications, most of them triggered by its perceived inadequacy with respect to the needs of researchers in this era of massive, cheap, abundant means of communication and organization. In particular, peer-review is going to suffer transformations of the same magnitude.

We are living interesting times, we are all aware that internet is changing our lives at least as much as the invention of the printing press changed the world in the past. With a difference: only much faster. We have an unique chance to be part of this change for the better, in particular  concerning  the practices of communication of research. In front of such a fast evolution of  behaviours, a traditionalistic attitude is natural to appear, based on the argument that slower we react, a better solution we may find. This is however, in my opinion at least, an attitude better to be left to institutions, to big, inadequate organizations, than to individuals. Big institutions need big reaction times because the information flows slowly through them, due to their principle of pyramidal organization, which is based on the creation of bottlenecks for information/decision, acting as filters. Individuals are different in the sense that for them, for us, the massive, open, not hierarchically organized access to communication is a plus.

The bottleneck hypothesis. Peer-review is one of those bottlenecks, traditionally. It’s purpose is to separate the professional  from the unprofessional.  The hypothesis that peer-review is a bottleneck explains several facts:

  • peer-review gives a stamp of authority to published research. Indeed, those articles which pass the bottleneck are professional, therefore more suitable for using them without questioning their content, or even without reading them in detail,
  • the unpublished research is assumed to be unprofessional, because it has not yet passed the peer-review bottleneck,
  • peer-reviewed publications give a professional status to authors of those. Obviously, if you are the author of a publication which passed the peer-review bottleneck then you are a professional. More professional publications you have, more of a professional you are,
  • it is the fault of the author of the article if it does not pass the peer-review bottleneck. As in many other fields of life, recipes for success and lore appear, concerning means to write a professional article, how to enhance your chances to be accepted in the small community of professionals, as well as feelings of guilt caused by rejection,
  • the peer-review is anonymous by default, as a superior instance which extends gifts of authority or punishments of guilt upon the challengers,
  • once an article passes the bottleneck, it becomes much harder to contest it’s value. In the past it was almost impossible because any professional communication had to pass through the filter. In the past, the infallibility of the bottleneck was a kind of self-fulfilling prophecy, with very few counterexamples, themselves known only to a small community of enlightened professionals.

This hypothesis explains as well the fact that lately peer-review is subjected to critical scrutiny by professionals. Indeed, in particular, the wave of detected plagiarisms in the class of peer-reviewed articles lead to the questioning of the infallibility of the process. This is shattering the trust into the stamp of authority which is traditionally associated with it.  It makes us suppose that the steep rise of retractions is a manifestation of an old problem which is now revealed by the increased visibility of the articles.

From a cooler point of view, if we see the peer-review as designed to be a bottleneck in a traditionally pyramidal organization,  is therefore questionable if the peer-review as a bottleneck will survive.

Social role of peer-review. There are two other uses of peer-review, which are going to survive and moreover, they are going to be the main reasons for it’s existence:

  • as a binder for communities of peers,
  • as a time-saver for the researchers.

I shall take them one-by-one. What is strange about the traditional peer-review is that although any professional is a peer, there is no community of peers. Each researcher does peer-reviewing, but the process is organized in such a manner that we are all alone. To see this, think about the way things work: you receive a demand to review an article, from an editor, based on your publication history, usually, which qualifies you as a peer. You do your job, anonymously, which has the advantage of letting you be openly critical with the work of your peer, the author. All communication flows through the editor, therefore the process is designed to be unfriendly with communications between peers. Hence, no community of peers.

However, most of the researchers who ever lived on Earth are alive today. The main barrier for the spread of ideas is a poor mean of communication. If the peer-review becomes open, it could foster then the appearance of dynamical communities of peers, dedicated to the same research subject. As it is today, the traditional peer-review favours the contrary, namely the fragmentation of the community of researchers which are interested in the same subject into small clubs, which compete on scarce resources, instead of collaborating. (As an example, think about a very specialized research subject which is taken hostage by one, or few, such clubs which peer-reviews favourably only the members of the same club.)

As for the time-saver role of peer-review, it is obvious. From the sea of old and new articles, I cannot read all of them. I have to filter them somehow in order to narrow the quantity of data which I am going to process for doing my research. The traditional way was to rely on the peer-review bottleneck, which is a kind of pre-defined, one size for all solution. With the advent of communities of peers dedicated to narrow subjects, I can choose the filter which serves best my research interests. That is why, again, an open peer-review has obvious advantages. Moreover, such a peer-review should be perpetual, in the sense that, for example, reasons for questioning an article should be made public, even after the “publication” (whatever such a word will mean in the future). Say, another researcher finds that an older article, which passed once the peer-review, is flawed for reasons the researcher presents. I could benefit from this information and use it as a filter, a custom, continually upgrading filter of my own, as a member of one of the communities of peers I am a member of.

Advertisements

One thought on “Peer-review, what is it for?”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s