People tend to associate peer review with science. As an example, even today there are still many scientists who believe that an arXiv.org article is not a true article, unless it has been peer reviewed. They can’t trust the article, without reading it first, unless it passed the peer review, as a part of the publishing process.
Just because a researcher puts a latex file in the arXiv.org (I continue with the example), it does not mean that the content of the file has been independently validated, as the scientific method demands.
The part which slips from the attention is that peer review is not independent validation.
Which means that a peer reviewed article is not necessarily one which passes the scientific method filter.
This simple observation is, to me, the key for understanding why so many research results communicated in peer reviewed articles can not be reproduced, or validated, independently. The scale of this peer reviewed article rot is amazing. And well known!
Peer review is a part of the publishing process. By itself, it is only a social validation. Here is why: the reviewers don’t try to validate the results from the article because they don’t have the means to do it in the first place. They do have access only to a story told by the authors. All the reviewers can do is to read the article and to express an opinion about it’s credibility, based on the reviewers experience, competence (and biases).
From the point of view of legacy publishers, peer review makes sense. It is the equivalent of the criteria used by a journalist in order to decide to publish something or not. Not more!
That is why it is very important for science to pass from peer review to validation. This is possible only in an Open Science frame. Once more (in this Open(x) fight) the medical science editors lead. From “Journal Editors To Researchers: Show Everyone Your Clinical Data” by Harlan Krumholz, a quote:
“[…] last Wednesday, the editors of the leading medical journals around the world made a proposal that could change medical science forever. They said that researchers would have to publicly share the data gathered in their clinical studies as a condition of publishing the results in the journals. This idea is now out for public comment.
As it stands now, medical scientists can publish their findings without ever making available the data upon which their conclusions were based.
Only some of the top journals, such as The BMJ, have tried to make data sharing a condition of publication. But authors who didn’t want to comply could just go elsewhere.”
This is much more than simply saying “peer review is bad” (because is not, only that it is not a part of the scientific method, it is a part of the habits of publishers). It is a right step towards Open Science. I repeat here my opinion about OS, in the shortest way I can:
There are 2 parts involved in a research communication: A (author, creator, the one which has something to disseminate) and R (reader). The legacy publishing process introduces a B (reviewer). A puts something in a public place, B expresses a public opinion about this and R uses B’s opinion as a proxy for the value of A’s thing, in order to decide if A’s thing is worthy of R’s attention or not. Open Access is about the direct interaction of A with R, Open Peer-Review is about transparent interaction of A with B, as seen by R and Validation (as I see it) is improving the format of A’s communication so that R could make a better decision than the social one of counting on B’s opinion.
That’s it! The reader is king and the author should provide everything to the reader, for the reader to be able to independently validate the work. This is the scientific method at work.