Open peer review perspectives: a thought-provoking question mark

By Maria das Graças Targino and Joana Coeli Ribeiro Garcia

Image: David Trawin

As a phenomenon and/or dynamic process, science goes through stages that are not necessarily exclusive and vary according to the theorists’ point of view. For example, Le Coadic¹ lists five steps. The first has as maximum representative the scientist without institutional protection, but with clawing and stubbornness, giving way to the stereotype of the “crazy scientist, solitary and eccentric”: the beloved comic books’ Gyro Gearloose… In a second moment, isolated efforts give rise to the first attempts of a collective work around the leader researcher – it is scientific amateurism. The third stage is marked by the expansion of universities as knowledge academies, shaping academic science, while the so-called organized science gives scope for the concentration of research, favoring official programs aimed at the development of science and technology (S&T) through generation of theories towards generalizations for apprehension of the phenomena. The last stage – mega science – refers to the recognized value of the researchers involved, in national and international terms, working in laboratories with the latest equipment and relying on ample financial resources.

For David and Spence², besides the first phase, markedly moved by trial-and-error, there is a second model of science focused on the formulation of laws and theories (generalizations) and a third moment, with the intervention of technological advance in scientific practices. From then on, the fourth paradigm emerges. This is e-science or enhanced science, an expression attributed in 1999 to Britain’s John Taylor, Director General of the UK Research Councils to define the broader and wider collaboration between different areas of knowledge through the investment in computational infrastructure for science, such as that highlighted by Hey and Trefethen³. This is an optimized science that resides in “[…] a collaborative, borderless type of research, made possible by the vast amount of data shared via data network and the Internet”⁴.

For the effectiveness of e-science, three indispensable elements are admittedly interdependent: (1) cyberinfrastructure, which names the technological architecture capable of favoring e-science practices; (2) open science, which acts in the delineation of policies and mechanisms with the purpose of defining parameters with a view to broad access to scientific findings; (3) collaboration, which concerns the participation of researchers and civil society in general, besides the support of the State, aiming at the construction of new scientific knowledge, and, above all, its sharing.

The concepts of e-science and open science are, therefore, intrinsically interconnected. As a result, the evaluation of the newly generated knowledge for validation by the scientific community, and, a posteriori, by society in general, inevitably follows the trend of greater transparency, as proposed in the eighteenth century by the philosopher, chemist and Irish physicist Robert Boyle, and evidenced by norms established by the American sociologist Robert K. Merton at first in 1937, until with further detail in 1973. The evaluation system to which are submitted the original manuscripts for publication in the scientific literature, the assessment of grants to funding agencies, the creation of examination boards for graduate studies, etc., are clearly linked to the Mertonian prescriptions, which remain in vogue until today – universality; sharing; material detachment and systematic skepticism – underpinning any form of arbitration of science, its producers and its products.

Therefore, based on the premise that evaluation is intrinsic to the advancement of S&T, it is in this context of open science, greater accessibility to new knowledge and greater collaboration among areas that there is an increasing (although slowly) tendency to replace blind peer review. In other words, blind assessment, either single blind (when only the referee knows the author’s identity), or double blind (when authors’ and referees’ identities are concealed from each other’s), is replaced by open peer review (OPR). This is because blind peer review, over time, has aroused controversy and friction in a continuum. Among its negative aspects, Spinak⁵ cites how unreliable it tends to be, coupled with the fact that it takes a long time; allows distortions that affect authors and journals; and does not properly retributes or give the due credit to reviewers for their work.

In this line of thought, even though the evaluation process is decisively controversial and diffuse, with multiple and varied possibilities, with orthodox or heterodox methods, in such a way that no evaluative resource is capable of fulfilling everyone’s expectations, especially in the scientific community, which includes an intricate network of relationships and conflicts, the authors have carried out studies to evaluate the viability of the adoption of the OPR by Information Science journals. Although most of the journals (as in other areas) adopt blind peer review, there is no evidence that it is more efficient than open peer review, in the same way as the open review, whose models have emerged since the 1980s, also maintains obscure points from its very conception and (dis)advantages.

Regarding the concept, due to the variations that the open system provides, there are many recent studies that contemplate it, such as the one by Ross-Hellauer⁶, which brings up 122 definitions and Spinak’s⁵ 22. All this proves how complex open science is and its unfolding, since the concept of the term – open – the case of the OPR covers different strands that merge and/or complement each other. For example: opening of the original manuscripts in preprint servers, such as F1000Research, Figshare and PeerJPreprints; authors disclosure; referee’s disclosure; open peer review to the general public to make suggestions /criticisms /observations.

Furthermore, it is necessary to reinforce, as Spinak⁵ did, that OPR favors a series of variations, such as: (1) open identities – authors and referees are aware of each other’s identities; (2) Open Reports – review reports are published alongside the relevant article; (3) Open participation – allows the wider community to contribute to the review process; (4) Open interaction – allows and encourages direct reciprocal discussion between reviewers, and/or between authors and reviewers; (5) Provision of originals prior to any formal evaluation on preprints servers; (6) Open final version for commenting; (7) Open platforms – the review process does not remain linked to the publisher, since it may come from different entities, such as independent peer review services.

As for the advantages of the OPR, its biggest bet is the perception of e-science as open science, and therefore, on a larger scale, thanks to global collaborations, which extend to arbitration, it is now possible to increase the visibility, use and reuse of scientific and academic output. On the other hand, among the disadvantages, there is the suspicion that the OPR does not allow faster evaluation overall, nor does it reduce costs.

In view of the above, in the field of Information Sciences, in a first stage, completed and duly presented and published during the XVIII Encontro Nacional de Pesquisa em Ciência da Informação (Enancib), the main event in the area, which is under the responsibility of the Associação Nacional de Pesquisa e Pós-Graduação em Ciência da Informação (Ancib), Garcia and Targino⁷ analyzed the feasibility of open peer review by Information Science (IS) journal editors, as mentioned before. It is a pioneering initiative within the framework of IS, therefore, open to contributions and complement. When using the survey as a research method, the authors, a priori, incorporate the editors of the 37 IS journals ranked A and B by the Qualis-CAPES system, reaching the sample of 15 editors (40,54%). Information was collected through questionnaires sent through Google Docs to sample units. Among the results, it is worth noting that the majority (67%) of editors were willing to adopt open peer review, while 60% believed that it can improve the quality of journals in the area. However, there was flagrant disagreement regarding the mention of positive and negative points, which reveals a lack of knowledge about the subject, and, therefore, the need to invest in the diffusion of the theoretical and methodological possibilities of this assessment mode.

Adopting similar methodological procedures in the same universe of previous titles, at present, the authors are currently in the discussion phase of the results obtained from the sample of 189 reviewers, i.e., 26.6% out of 709 interviewed, since there was the intention to cover the different segments of the publishing chain, in order to reach a comprehensive view of the possibility of adopting OPR in the area. In any case, it is evident until then that there is at least intense intellectual curiosity around the subject, and therefore, interest in innovating the evaluation processes inherent to scientific output in the universe of IS, given that e-science and open science produce technological, socio-cultural and legal impacts, according to detailed description by Hey and Trefethen³. This corresponds, of course, to “open” perspectives surrounded by instigating question marks around open peer review in contemporary times.

Notes

1. LE COADIC, Y.F. A ciência da informação. Brasília: Briquet de Lemos Livros, 1996.

2. DAVID, P.A. and SPENCE, M. Towards institutional infrastructures for e-science: the scope of the challenge. Oxford: The University of Oxford, 2003.

3. HEY, T. and TREFETHEN, A. e-Science and its implications. Philosophical Transactions of the Royal Society of London. 2003, no. 361, pp. 1809-1825.

4. CURTY, R.G. eScience: diferentes vieses, fontes e iniciativas. In: TOMAÉL, M.I. and ALCARÁ, A.R. (org.). Fontes de informação digital. Londrina: UEL, 2016.

5. SPINAK, E. Sobre as vinte e duas definições de revisão por pares aberta… e mais [online]. SciELO em Perspectiva, 2018 [viewed 14 May 2018]. Available from: http://blog.scielo.org/blog/2018/02/28/sobre-as-vinte-e-duas-definicoes-de-revisao-por-pares-aberta-e-mais

6. ROSS-HELLAUER, T. What is open peer review? A systematic review. F1000Research [online]. 2017, vol. 6, no. 588, 2017 [viewed 14 May 2018]. DOI: 10.12688/f1000research.11369.2. Available from: https://f1000research.com/articles/6-588/v2

7. TARGINO, M.G. and GARCIA, J.C.R. Open peer review sob a ótica de editores das revistas brasileiras da ciência da informação. In: Encontro Nacional de Pesquisa em Ciência da Informação (Enancib), Marília, 2017.

References

CURTY, R.G. eScience: diferentes vieses, fontes e iniciativas. In: TOMAÉL, M.I. and ALCARÁ, A.R. (org.). Fontes de informação digital. Londrina: UEL, 2016.

DAVID, P.A. and SPENCE, M. Towards institutional infrastructures for e-science: the scope of the challenge. Oxford: The University of Oxford, 2003.

HEY, T. and TREFETHEN, A. e-Science and its implications. Philosophical Transactions of the Royal Society of London. 2003, no. 361, pp. 1809-1825.

LE COADIC, Y.F. A ciência da informação. Brasília: Briquet de Lemos Livros, 1996.

MERTON, R.K. The sociology of science: theoretical and empirical investigations. Chicago: The University of Chicago, 1973.

ROSS-HELLAUER, T. What is open peer review? A systematic review. F1000Research [online]. 2017, vol. 6, no. 588, 2017 [viewed 14 May 2018]. DOI: 10.12688/f1000research.11369.2. Available from: https://f1000research.com/articles/6-588/v2

SPINAK, E. Sobre as vinte e duas definições de revisão por pares aberta… e mais [online]. SciELO em Perspectiva, 2018 [viewed 14 May 2018]. Available from: http://blog.scielo.org/blog/2018/02/28/sobre-as-vinte-e-duas-definicoes-de-revisao-por-pares-aberta-e-mais

TARGINO, M.G. and GARCIA, J.C.R. Open peer review sob a ótica de editores das revistas brasileiras da ciência da informação. In: Encontro Nacional de Pesquisa em Ciência da Informação (Enancib), Marília, 2017.

About Maria das Graças Targino

Maria das Graças Targino holds a doctorate in Information Sciences (Universidade de Brasília) and a postdoctoral fellowship in journalism from the Instituto Interuniversitario de Iberoamérica of the Universidad de Salamanca and a Master’s in Communication and Education from Universidade Autónoma de Barcelona (Spain). Currently, she is linked to the Centro de Educação Aberta e a Distância of Universidade Federal do Piauí and to the Graduate Program in Information Sciences of Universidade Federal do Paraíba (PPGCI).

About Joana Coeli Ribeiro Garcia

Joana Coeli Ribeiro Garcia holds a doctorate in Information Sciences (Universidade Federal do Rio de Janeiro / Instituto Brasileiro de Informação em Ciência e Tecnologia); and Coordinator of the Research Group: “From information to knowledge”, registered at the Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq). She was the president of the Associação Nacional de Pesquisa e Pós-Graduação em Ciência da Informação (ANCIB) in 2009-2010. Currently, she is a full professor of the Information Science Department and Graduate Program in Information Science of the Universidade Federal da Paraíba (PPGCI) and collaborator in a program of the same name at Universidade Federal de Pernambuco.

Translated from the original in Portuguese by Lilian Nassi-Calò.