Adoption of open peer review is increasing

By Lilian Nassi-Calò

Photo: SciELO.

Peer review, one of the pillars of scholarly communication, was first proposed in 1831 by William Whewell to the Royal Society of London, suggesting that a committee of eminent scholars produced reports on articles submitted for publication to the Philosophical Transactions journal. Whewell argued that these reports “could often be more interesting than the papers themselves” and in addition, authors would be grateful that their papers would be carefully reviewed by at least two or three specialists. The forerunner of the peer review system, however, did not aim at preventing the publication of poor quality articles or to propose the creation of a system to support editorial decisions. He was actually seeking to increase the visibility of science in society and to give an identity to the scientific enterprise in the United Kingdom, which faced shortage of resources and lack of public recognition.

The receptivity of the scientific community at the time was not the best, initially. It took about a decade for the arbitration system to be accepted and recognized, not before facing the wrath of the authors: “full of envy, hatred, malice, and all uncharitableness, hidden away in some secret chamber, this scientific judiciary, used the cover of anonymity to advance their personal interests — perhaps through undetectable acts of piracy — at the expense of helpless authors.”¹ Although exacerbated, this reaction may not be uncommon among authors who have had their articles rejected by harsh referees.

It was only at the end of the 19^th century that journal editors adopted assessment practices and reviewers were incorporated into the process of scientific publishing as a way of ensuring the integrity of academic literature. Despite frequent attempts to eliminate the “veritable sewage thrown into the pure stream of science,” UK scientific societies came to consider unifying their scientific apparatus and creating a standardized system to oversee the entire scientific publishing. The plan, however, did not go forward, as it would have been necessary to bring together publishers of independent journals as well. Nonetheless, reviewers have, over the years, acquired the character of “gatekeepers,” or guardians of good science.

It did not escape the analysis of many, however, that the entire arbitration system was intrinsically flawed and could even prevent innovative ideas from being published and that, ultimately, it should be abolished. However, we all know that the system has survived to the present, but not without objections, criticisms and caveats.

Peer review is currently facing a transition moment, and many believe that it is necessary to redefine its principles and practices so as not to delay or hinder the progress of science. In this sense, alternatives to the traditional assessment model have recently emerged, which is usually closed and keeps confidential the referees’ identity (blind system) and often the authors’ identity as well (double-blind system). It is interesting to note that since 1833 closed and anonymous reports prevailed, since it was considered inappropriate or impolite to sign an explicit criticism to a colleague, and moreover, the referee speaks not in his own name, but ex cathedra.

Journals such as Nature Communications, Peer J, BMJ and F1000Research currently offer authors the option of open peer review, i.e. reviews are published along the article and with the reviewers’ consent, their name and institutional affiliation is also published. A recent article in Nature evaluates the increasing adoption of this peer review modality, comparing it to a “frigid swimming pool: they are dipping their toes in the water, but are reluctant to plunge in”².

In the survey carried out by Nature Communications, 60% of the authors in 2016 authorized the publication of reviews on their articles, which led the journal to continue offering this option this year, without, however, making it mandatory. Confirming this trend is an unpublished survey by the European Commission, in which more than half of the more than 3,000 respondents among authors, editors and publishers stated that open peer review should be adopted as a routine. When asked whether certain aspects would improve, worsen or have no effect on peer review, respondents provided the following opinions: 65% stated that open discussion between authors and reviewers would make peer review better/much better; 55% stated that publishing review reports would make peer review better/much better; 47% think that allowing open comments on final paper would make peer review make peer review better/much better, but only 35% believe that posting papers online before going through formal peer review would make peer review better/much better and no more than 30% feel that revealing the referees identity would make peer review better/much better, however, almost 50% believe that this procedure would make peer review worse/much worse.

Survey results, however, are largely influenced by the fact that open peer review has different meanings for different people, says Anthony Ross-Hellauer, an information scientist at Göttingen State University, in Germany and coordinator of the OpenAIRE project funded by the European Commission. While some believe that it means not disclosing the name of the referees but only their reports, others think that anonymous reports should be published, while some interviewees believe that the process should be totally open, including the contribution of anyone interested. Different disciplines also show particular behaviors. In Nature Communications‘ experience, 70% of the authors on ecology and evolution, molecular biology and Earth sciences opted for open reviews, while physics article authors showed the smallest percentage. For a community familiar with using arXiv, however, this is an unexpected behavior.

The practice of making public referees reports has been consolidated in recent years. Initiatives such as Publons, and Hypothes.is, created in 2014 as start-ups, allow publication of reports on published articles, with the consent of editors and authors. The multinational publisher Elsevier has published in the last two years open reports from five of its journals, and intends to include new titles this year.

Advocates of open peer review argue that assigning accountability to reviewers by posting their reports and, when possible, their identity makes the process fairer and more transparent. Others make reservations about the possible publication of reviews without the proper knowledge of the referees, and whether the reports would reach the desired readers’ attention. Joseph Tennant, a paleontologist at Imperial College London says it is the first session of the article he reads, after the abstract, but the extent of researchers who do the same is unknown.

Csiszar, in his article in Nature, informs that the term peer-review comes from the procedures that public funding agencies used to define who would be awarded research grants. When the referee systems became ‘peer review’, it has acquired a strong public symbol that science has mechanisms of self-regulation and generate consensus, even if it is not certain whether peer reviewers are up to undertake such a calling. He thinks that the current debate of the scholarly community in rethinking peer review considers a number of factors, such as psychology of bias, objectivity, and the ability to assess reliability and importance, but rarely considers the history of the peer review institution, which “did not develop simply out of scientists’ need to trust one another’s research. It was also a response to political demands for public accountability. Understanding that other practices of scientific judgement were once in place ought to be a part of any responsible attempt to chart a future path.”¹

Notes

¹ CSISZAR, A. Peer review: Troubled from the start. Nature. 2016, vol. 532, pp. 306–308. DOI: 10.1038/532306a

² CALLAWAY, E. Open peer review finds more takers. Nature. 2016, vol. 539, pp. 343. DOI: 10.1038/nature.2016.20969

References

CSISZAR, A. Peer review: Troubled from the start. Nature. 2016, vol. 532, pp. 306–308. DOI: 10.1038/532306a

CALLAWAY, E. Open peer review finds more takers. Nature. 2016, vol. 539, pp. 343. DOI: 10.1038/nature.2016.20969

NASSI-CALÒ, L. Article analyses saturation of peer reviewers. SciELO in Perspective. [viewed 06 January 2017]. Available from: http://blog.scielo.org/en/2015/01/22/article-analyses-saturation-of-peer-reviewers/

NASSI-CALÒ, L. Annotating the scholarly literature online. SciELO in Perspective. [viewed 06 January 2017]. Available from: http://blog.scielo.org/en/2015/12/08/annotating-the-scholarly-literature-online/

External links

Publons – http://publons.com/

Hipothes.is – https://hypothes.is/

Projeto OpenAIRE – https://www.openaire.eu/

About Lilian Nassi-Calò

Lilian Nassi-Calò studied chemistry at Instituto de Química – USP, holds a doctorate in Biochemistry by the same institution and a post-doctorate as an Alexander von Humboldt fellow in Wuerzburg, Germany. After her studies, she was a professor and researcher at IQ-USP. She also worked as an industrial chemist and presently she is Coordinator of Scientific Communication at BIREME/PAHO/WHO and a collaborator of SciELO.

Translated from the original in Portuguese by Lilian Nassi-Calò.