By Lilian Nassi-Calò
Adapted photo from the original: Ian Hampton.
A recently published post in this blog marks the 350th anniversary of the creation of the first scientific journals that disseminated the ideas of philosophers, mathematicians, astronomers, physicists, physicians, lawyers and other intellectuals of the 17th Century. The author reports that “from the beginning there was a consensus among researchers and scholars that findings should be disseminated and presented openly to the evaluation by their peers. The ‘peer review’ is as old as scientific journals, and moreover, it is worth noting that at that time, arbitration was not an anonymous or blind process, but open to all colleagues, a kind of peer review and Open Access”.
The validation of scientific reports before publication, therefore, it is well established practice whose effectiveness and importance is recognized by authors, publishers, funding agencies and scientific societies around the world, in order to ensure the originality, quality, reliability, integrity and consistency of scholarly literature.
The main publishers associations around the world, such as WAME, Council of Science Editors, ICMJE, COPE, EASE, ALPSP, and publishers as BioMed Central, Nature Publishing Group, Elsevier, Wiley-Blackwell, and many others offer on their websites manuals and good practice guides on peer review, in order to orient researchers, editors and reviewers on how to conduct correctly, reputably and transparently the most important step of the publication of research outcomes.
Despite the widespread availability of resources, peer review still raises doubts and questions to the scientific community, research institutions, and civil society, which funds much of the research through tax paying. What should be recognized as guardian of good science, based on solid concepts and appropriate methodology, can be misinterpreted as a way through which personal or corporate interests can be preserved, lack of transparency, bias against breakthroughs or negative results, or reasons other than the common good.
According to the biologist and Chemistry Nobel Prize laureate in 2008, Martin Chalfie, of Columbia University, peer review is important to qualify the scientific output, however, it is necessary to avoid that good works are not retained in the scrutiny of peer review and don’t get published. Chalfie was the discoverer, along with two other scientists, of the Green Fluorescent Protein (GFP)1. The GFP protein of Aequorea victoria jellyfish, discovered in 1994, can be used as a marker of gene expression in prokaryotes and eukaryotes, making it a very simple and sensitive assay.
Chalfie gave an exclusive interview to a Brazilian weekly magazine reporting the difficulties he had to publish in Science the article that earned him the Nobel Prize. Chalfie believes that peer review should undergo adjustments to make the process more efficient and fair. He considers that there are currently reviewers that do not evaluate the submitted paper, but rather their ideal vision of that paper. The lack of consensus between reviewers can be problematic because a negative opinion, even among many other positive ones can make the editor decline its publication. There are journals such as eLife that require reviewers to reach a consensus among themselves on the points that the author should review to get his work published. The editor of Genetics, on the other hand, believes that editors should be leading scientists in the area, so they can assess the true importance of a paper and even criticize the review of a renowned scientist.
Chalfie also mentions the need to assess the quality of a researcher by examining his work and not the prestige of the journals in which he published, and cites the Declaration of Research Assessment (DORA)2, as a movement against the overvaluation of the Impact Factor in research assessment.
This negative impression of peer review takes place, most likely, due to the lack of knowledge about it and its operation rules. A detailed study undertaken by a Working Group of the British agency Sense About Science published in 20043 shows that due to lack of knowledge on the principles of peer review by people outside the scientific community in that country, there is a mistaken judgment about scientific publication practices. The term ‘scientific evidence’ is often confused with ‘science policy’. In most cases exaggeration and concern about scientific discoveries are related to results considered of low quality or fabricated by researchers and specialists.
The Working Group considers, therefore, that understanding peer review can help people when reading about science in a scientific journal, in a newspaper or on the Internet, to argue about how this discovery was made, and to which scrutiny it passed through publication.
To name a few examples of scientific discoveries that cause great concern in the general population, there is the claim that the mumps, measles and rubella vaccine (MMR) could cause autism in children; that non-ionizing electromagnetic radiation emitted by cell phones could cause from sleeping disturbance to brain tumors; that food from genetically modified seeds would be harmful to health; concern about the harmful effects of pesticides found in food; and the risks that outweigh the benefits of hormone replacement therapy in women.
In the British researchers’ view, it is a missed opportunity for the scientific community to explicitly disclose how scientific discoveries advance and are shared, especially considering the hours of specialized and thorough work involved, and because many of the resulting research discoveries has serious implications ranging from family health to global policy.
In Latin America, moreover, the question may be even more dramatic. A recent study on the public perception of science conducted in Argentina and presented at FAPESP Week Buenos Aires4, shows that despite efforts to boost science communication in the region, the levels of knowledge about science in the general population are quite low. The survey was conducted by the Consejo Nacional de Investigaciones Scientific and Technical (Conicet) in partnership with FAPESP, and is confirmed by studies carried out in Brazil by the Ministério da Ciência, Tecnologia e Inovação (MCTI) and FAPESP, and in other Ibero-American countries by Red Iberoamericana de Indicadores de Ciencia y Tecnología (RICYT).
In Marcelo Knobel’s opinion, professor at Instituto de Física Gleb Wataghin, Unicamp, one of the coordinators of the event, “The populations of Latin America countries do not even know if science is done, let alone in which institutions such activity takes place in their respective countries”. In fact, recent research on the public perception of science in Brazil indicates that only 14% of respondents can name a research institution in the country.
RICYT concluded that the level of knowledge varies depending on the education level of respondents. Only 20% of respondents in Argentina indicating they knew a research institution in the country, however, among those with higher education levels, this percentage reached 70%. This perception is linked to the fact that only 2-5% of the populations of the countries of the region visit museums and science centers or read about science in newspapers, magazines and on the Internet.
The populations of developed countries have provided in the daily newspapers, magazines and on the Web many contents about science written by science journalists who translate into simple terms the recent scientific discoveries. This activity is far from trivial, and requires specialized training, including graduate courses in journalistic practice and science. Moreover, the major scientific journals maintain pages on social networks and blogs where they share digested scientific information for non-specialists consumers. In Latin America, this occurs in a much lesser degree, and the general population considers it difficult to understand, thus losing interest. Of course, the responsibility here lies with the communication vehicle and not on the reader, who does not rely on adequately prepared professionals to ‘translate’ scientific language into something simple, but not just trivial.
But what does all this has to do with peer review? The results presented by the British Working Group pointed out that if scientists drew attention to the assessment process through which their work goes before it is published, it should encourage ordinary people to be more demanding about what they read, to seek reliable and confident sources about their topics of interest, rather than believing in the first information conveyed in a magazine or on the Internet. It may seem that we are far from such awareness in our geographic region, but it is important to start this process.
Much has been said and written about peer review, but nowadays there are many more organizations dedicated to understand and improve the process, which has long been the prerogative of publishers. The aforementioned Sense about Science has produced research, training and peer review policy material, such as the document written by the Working Group3 and the program for young scientists that includes workshops and guides for early career researchers5.
Young researchers have questions about their involvement in the peer review process, such as how to evaluate their performance as referee and what to expect about it as authors and as reviewers. Despite these issues, their involvement is very productive because while assessing, they understand how peer review works, its limitations and the role of peer review in society, becoming, thus, better authors.
In fact, 91% of researchers dedicate to peer review because they believe their latest work was improved by suggestions of reviewers, 85% of them claim appreciate having access to other peoples’ work and being able to improve them, and 90% take part in the process because it makes them feel members of the academic community, acting as gatekeepers for the sake of the publication quality in their area. On the other hand, 56% of respondents at the Sense about Science survey state that they did not have any training on how to be a reviewer.
In view of this fact and other evidences, in 2014 the European Cooperation Agency on Science and Technology (COST), funded by the European Union, formed a multinational group to strengthen the efficiency, transparency and reliability of peer review through a multidisciplinary and intersectoral approach. The initiative, called New Frontiers of Peer Review (PEERE), scheduled to last four years, aims to examine peer review in different areas of knowledge through qualitative and quantitative research; assess implications of different types of peer review and explore new approaches; enroll researchers and stakeholders in the data sharing and test results; and finally collaboratively define a joint agenda to propose an evidence based reform of peer review. Those responsible for the initiative believe that peer review as a regulatory process of science can be improved and strengthened, as its results will benefit society as a whole. PEERE organizes events and workshops and provides documents and good practice manuals on peer review on its web portal.
The network Enhancing the Quality and Transparency of Health Research (EQUATOR) supports editors and young researchers facing peer review by publishing good practice guides and checklists. Specialized in clinical trials and health literature, EQUATOR manuals are also available in Spanish, translated with the support of the Pan American Health Organization, and seek to meet the demand for information in languages other than English.
The commitment and involvement of other stakeholders in strengthening peer review besides publishers and editors is very beneficial and should bring significant improvements to the process. It is a collective and continuous learning for the sake of reliability, transparency and ethics in science.
Notes
1 CHALFIE, M. et al. Green fluorescent protein as a marker for gene expression. Science. 1994, vol. 263, nº 5148, pp. 802-805. DOI: http://dx.doi.org/10.1126/science.8303295.
2 Declaration recommends eliminate the use of Impact factor for research evaluation. SciELO in Perspective. [viewed 26 April 2015]. Available from: http://blog.scielo.org/en/2013/07/16/declaration-recommends-eliminate-the-use-of-impact-factor-for-research-evaluation/
3 Sense about Science. Peer Review and the Acceptance of New Scientific Ideas. London: Sense About Science, 2004. Available from: http://www.senseaboutscience.org/data/files/resources/17/peerReview.pdf
4 ALISSON, E. El nivel de conocimiento sobre ciencia en la sociedad latinoamericana es dramáticamente bajo. In: FAPESP Week Buenos Aires, Buenos Aires, 2015. Available from: http://www.fapesp.br/week2015/buenosaires/el-nivel-de-conocimiento-sobre-ciencia-en-la-sociedad-latinoamericana-es-dramaticamente-bajo/
5 Peer Review, The nuts and bolts. A guide for early career researchers. Sense about Science 2012, London. Available from: http://www.elsevier.com/__data/assets/pdf_file/0006/228597/Peer-review_The-nuts-and-bolts.pdf
References
350 years of scientific publication: from the “Journal des Sçavans” and Philosophical Transactions to SciELO. SciELO in Perspective. [viewed 13 April 2015]. Available from: http://blog.scielo.org/en/2015/03/05/350-years-of-scientific-publication-from-the-journal-des-scavans-and-philosophical-transactions-to-scielo/
ALISSON, E. El nivel de conocimiento sobre ciencia en la sociedad latinoamericana es dramáticamente bajo. In: FAPESP Week Buenos Aires, Buenos Aires, 2015. Available from: http://www.fapesp.br/week2015/buenosaires/el-nivel-de-conocimiento-sobre-ciencia-en-la-sociedad-latinoamericana-es-dramaticamente-bajo/
Authors & Referees: Peer-review policy. Nature Publishing Group. Available from: http://www.nature.com/authors/policies/peer_review.html
BEER, R. “É preciso uma revisão mais justa de artigos científicos”. Veja.com. 2015. Available from: http://veja.abril.com.br/noticia/ciencia/e-preciso-uma-revisao-mais-justa-de-artigos-cientificos
Best Practices for Peer Reviewer Selection and Contact to Prevent Peer Review Manipulation by Authors. World Association of Medical Editors (WAME). 2015. Available from: http://www.wame.org/about/policy-statements#Best Practices for Peer Reviewer Selection
CHALFIE, M. et al. Green fluorescent protein as a marker for gene expression. Science. 1994, vol. 263, nº 5148, pp. 802-805. DOI: http://dx.doi.org/10.1126/science.8303295.
COPE Ethical Guidelines for Peer Reviewers. Committee on Publication Ethics (COPE). Available from: http://publicationethics.org/files/u7140/Peer%20review%20guidelines.pdf
Declaration recommends eliminate the use of Impact factor for research evaluation. SciELO in Perspective. [viewed 26 April 2015]. Available from: http://blog.scielo.org/en/2013/07/16/declaration-recommends-eliminate-the-use-of-impact-factor-for-research-evaluation/
For Reviewers. Willey Blackwell. Available from: http://www.nurseauthoreditor.com/forreviewers.asp
How to serve as an effective referee in the peer review process. European Association of Science Editors (EASE). Available from: http://www.ease.org.uk/publications/ease-toolkit-authors/how-serve-effective-referee-peer-review-process
Peer Review, The nuts and bolts. A guide for early career researchers. Sense about Science 2012, London. Available from: http://www.elsevier.com/__data/assets/pdf_file/0006/228597/Peer-review_The-nuts-and-bolts.pdf
Peer review. Elsevier. Available from: http://www.elsevier.com/reviewers/peer-review
Peer-review policy. BioMed Central. Available from: http://www.biomedcentral.com/editors/peerreview
Responsibilities in the Submission and Peer-Review Process. International Committee of Medical Journal Editors (ICMJE). Available from: http://www.icmje.org/recommendations/browse/roles-and-responsibilities/responsibilities-in-the-submission-and-peer-peview-process.html
Sense about Science. Peer Review and the Acceptance of New Scientific Ideas. London: Sense About Science, 2004. Available from: http://www.senseaboutscience.org/data/files/resources/17/peerReview.pdf
TRUDGETT, A. Reviewer Roles and Responsibilities. Council of Science Editors. 2012. Available from: http://www.councilscienceeditors.org/resource-library/editorial-policies/white-paper-on-publication-ethics/2-3-reviewer-roles-and-responsibilities/
External links
FAPESP Week Buenos Aires – http://www.fapesp.br/week2015/buenosaires
New Frontiers of Peer Review (PEERE) – http://www.peere.org/
Rede EQUATOR – http://www.equator-network.org/
Sense about Science – http://www.senseaboutscience.org
The Association of Learned & Professional Society Publishers (ALPSP) – http://www.alpsp.org/
About Lilian Nassi-Calò
Lilian Nassi-Calò studied chemistry at Instituto de Química – USP, holds a doctorate in Biochemistry by the same institution and a post-doctorate as an Alexander von Humboldt fellow in Wuerzburg, Germany. After her studies, she was a professor and researcher at IQ-USP. She also worked as an industrial chemist and presently she is Coordinator of Scientific Communication at BIREME/PAHO/WHO and a collaborator of SciELO.
Translated from the original in portuguese by Lilian Nassi-Calò.
Related Posts:
Como citar este post [ISO 690/2010]:
Recent Comments