Annotating the scholarly literature online

By Lilian Nassi-Calò

The Internet changed totally and irreversibly the way the scholarly literature is published, assessed, disseminated, read, shared and cited. The peer review process, particularly, has gained, with the support of automated systems, greater efficiency, transparency and agility across the Web, even taking into account the disproportionate increase in submissions that saturates journals and peer reviewers.

As a result, online journals such as PeerJ, PLoS,e Life and F1000 have changed the way of conducting peer review, assessing validity, originality and methodology soundness, but leaving the evaluation of the results and conclusions to post-publication review. In some journals, reviews and their authors are published along with the paper, promoting transparency and quality of the reviews, besides acknowledging due credit to the referees. The article level alternative metrics or altmetrics, which gathers comments in social networks, blogs and media, emerge as a measure of scientific impact.

Thus, quality opinions and comments made by researchers have become extremely valuable, and rather than being discarded after peer review, there already are initiatives to encourage them and increase their dissemination. PaperCritic, for example, provides feedback to the authors about their article in the form of comments posted on its website and PeerEvaluation publishes suggestions for improvement of the papers and open reviews about them. Publons, on the other hand, compiles contributions from peer reviewers and encourages researchers to post their experiences as referees. The National Library of Medicine (NLM) launched in 2013 PubMed Commons, the open feedback system on scientific articles published in the largest biomedical literature database in the world. The platform – still in the pilot version – receives comments from any researcher who has at least one publication in PubMed.

Meeting this trend, the start-up Hypothes.is created an online open source platform that allows users to highlight text and annotate Web pages or PDF files of scholarly articles. Its founder, Dan Whaley, believes that technology can simplify the act of annotating, and thus, stimulate scientists to do so. The initiative, whose mission is “to enable conversations over the world’s knowledge” was founded in 2011 and currently announces a coalition of over 40 scholarly organizations such as scientific publishers, technology companies and portals of academic institutions, including the PLoS, arXiv, CrossRef, PKP, eLife and the Biodiversity Heritage Library. A paper from Jeffrey Perkel in Nature this week reports the development of the initiative¹.

Hypothes.is users when opening an article’s Web page have access to any annotations made in the document. Users can add their own comments, which are open by default, but can be made private and shared only with certain researchers. Annotations made in HTML and PDF versions of the same article are visible in both interfaces of the document, either in open access or subscription papers. There is an example available of an article showing the comments and passages marked in the text². The tool finds application in education, where professors can select a paper and invite comments from their students. In this case, only the restricted group has access to the notes that are discussed in class. One can further imagine that the tool may also serve to train graduate students to act as peer reviewers in the future.

The success of the initiative led the publisher eLife in the UK to consider replacing its open referee comment system with Hypothes.is. Another partner of the initiative, the repository arXiv at Cornell University, considers adopting Hypothes.is to convert posts in external blogs on their articles in order to make them visible in the abstracts page.

In order to validate the identity of commenters, Hypothes.is intends to incorporate researcher’s ORCID profile in the notes. According to the founder of arXiv, Paul Ginsparg, this may increase the credibility of comments and encourage scientists to annotate, aware that their peers participate in the same initiative.

Cameron Neylon, Open Access advocate, former director of PLoS and researcher at the Centre for Culture and Technology at Curtin University in Perth, Australia, believes that the practice of online annotation, if successful, could change scientific communication as we know it today. Featuring an open source platform to comment on scholarly literature brings the document to the spotlight, and allows control at different contribution levels, in order to improve the article.

According to Ginsparg, the success of Hypothes.is depends, however, on the willingness of researchers to annotate on published articles, even if they share their notes with restricted audiences. Write thoughtful comments on publications takes time and effort, and it is an activity for which scientists generally do not receive credits. It is unknown whether researchers will devote to this practice, if only because there is not yet an incentive structure to reward them. Anyway, he believes, it is an experiment worth doing.

Notes

1. PERKEL, J. M. Annotating the scholarly web. Nature. 2015, vol. 528, nº 7580, pp. 153-154. DOI: 10.1038/528153a

2. Available example of an article showing the comments and passages marked in the text: http://via.hypothes.is/http:/www.nature.com/news/annotating-the-scholarly-web-1.18900

References

PERKEL, J. M. Annotating the scholarly web. Nature. 2015, vol. 528, nº 7580, pp. 153-154. DOI: 10.1038/528153a

SCIENTIFIC ELECTRONIC LIBRARY ONLINE. Article analyses saturation of peer reviewers. SciELO in Perspective. [viewed 06 December 2015]. Available from: http://blog.scielo.org/en/2015/01/22/article-analyses-saturation-of-peer-reviewers/

SCIENTIFIC ELECTRONIC LIBRARY ONLINE. eLife: an example of improved peer review. SciELO in Perspective. [viewed 06 December 2015]. Available from: http://blog.scielo.org/en/2015/05/06/elife-an-example-of-improved-peer-review/

SCIENTIFIC ELECTRONIC LIBRARY ONLINE. Paper proposes four pillars for scholarly communication to favor the speed and the quality of science. SciELO in Perspective. [viewed 06 December 2015]. Available from: http://blog.scielo.org/en/2013/07/31/paper-proposes-four-pillars-for-scholarly-communication-to-favor-the-speed-and-the-quality-of-science/

SCIENTIFIC ELECTRONIC LIBRARY ONLINE. Peer review modalities, pros and cons. SciELO in Perspective. [viewed 06 December 2015]. Available from: http://blog.scielo.org/en/2015/03/27/peer-review-modalities-pros-and-cons/

SCIENTIFIC ELECTRONIC LIBRARY ONLINE. Peer-review as a research topic in its own right. SciELO in Perspective. [viewed 06 December 2015]. Available from: http://blog.scielo.org/en/2015/04/24/peer-review-as-a-research-topic-in-its-own-right/

SCIENTIFIC ELECTRONIC LIBRARY ONLINE. PubMed Commons: NLM launches pilot version of open comments on articles. SciELO in Perspective. [viewed 06 December 2015]. Available from: http://blog.scielo.org/en/2013/12/20/pubmed-commons-nlm-launches-pilot-version-of-open-comments-on-articles/

SCIENTIFIC ELECTRONIC LIBRARY ONLINE. Scientometrics of peer-reviewers – will they be finally recognized?. SciELO in Perspective. [viewed 06 December 2015]. Available from: http://blog.scielo.org/en/2014/05/14/scientometrics-of-peer-reviewers-will-they-be-finally-recognized/

External links

Altmetrics – <http://altmetrics.org/>

Hypothes.is – <http://hypothes.is/>

ORCID – <http://orcid.org/>

PaperCritic – <http://www.papercritic.com/>

PeerEvaluation – <http://www.peerevaluation.org/>

Publons – <http://publons.com/>

PubMed Commons – <http://www.ncbi.nlm.nih.gov/pubmedcommons/>

About Lilian Nassi-Calò

Lilian Nassi-Calò studied chemistry at Instituto de Química – USP, holds a doctorate in Biochemistry by the same institution and a post-doctorate as an Alexander von Humboldt fellow in Wuerzburg, Germany. After her studies, she was a professor and researcher at IQ-USP. She also worked as an industrial chemist and presently she is Coordinator of Scientific Communication at BIREME/PAHO/WHO and a collaborator of SciELO.

Translated from the original in portuguese by Lilian Nassi-Calò.