The Open Scientometric Data Infrastructures Workshop took place at CWTS on 28 February and 1 March 2019. Over the course of two days, 14 researchers from CWTS and other research institutes and universities came together to discuss current projects and initiatives regarding open scientometric data infrastructures. This blog post provides a summary on the presentations and discussions held at the event.
Day 1
The workshop started with an introduction on the historical trajectory of open scientometric data by Ludo Waltman (CWTS). The OpenCitations project can be regarded as a starting point, which was followed by the Initiative for Open Citations (I4OC). Whereas OpenCitations provides a technical infrastructure for open citation data, I4OC is a lobby group for promoting openness of citation data. In December 2017, a number of scientometricians published an open citations letter to support I4OC. In September 2018, a Workshop on Open Citations was held at the University of Bologna. One of the reasons to establish the journal Quantitative Science Studies (QSS) in January 2019 was that Elsevier, the publisher of Journal of Informetrics, the predecessor of QSS, was unwilling to make citation data openly available. As of March 2019, almost 50% of the citation data in Crossref still needs to be opened. In addition, some publishers do not deposit any reference data in Crossref. Opening up citation data also benefits scientometric software, such as VOSviewer, which provides functionality to query Crossref for bibliometric visualisations.
Signatories of open citations letter, as of April 2018 (source: Sugimoto, Cassidy R.; Murray, Dakota S. and Larivière, Vincent (2018). Open citations to open science. Retrieved from http://issi-society.org/blog/posts/2018/april/open-citations-to-open-science/)
Nees Jan van Eck (CWTS) and Ludo Waltman (CWTS) presented their work on comparing bibliometric data sources. The Dimensions database, established in 2018 by Digital Science, is mainly based on Crossref data but also benefits from additional data made available by publishers. Web of Science (WoS) and Scopus have the advantage of providing document types, while Crossref and Dimensions are unable to distinguish between different types of documents published in scientific journals. Comparisons of the different data sources also revealed differences in citation links. Crossref has fewer citation links than WoS and Scopus because of publishers that do not make citation data openly available. Dimensions enriches Crossref data through agreements with publishers and therefore provides more citation links than Crossref. Only a limited number of abstracts are indexed in Crossref. Even open access publishers do not always make abstracts available in Crossref. Other metadata elements, such as affiliations, are also often missing. There was broad support among the workshop participants for the idea that metadata of scientific publications should be made openly available.
Jochen Gläser (Technical University of Berlin) presented the COPSSH (Communication Patterns in the Social Sciences and Humanities) project (a summary in German is available here), which investigates communication patterns in the Social Sciences and Humanities (SSH). The project is part of the funding line quantitative research on the science sector of the German Federal Ministry of Education and Research (BMBF) and is carried out in collaboration with CWTS. The way in which SSH researchers publish research differs from Science, Technology, Engineering and Mathematics (STEM). Furthermore, SSH are not well represented in traditional bibliographic databases, and within SSH publications there is a higher proportion of negative citations compared to STEM publications. The main research question to be investigated in the project is: What can we learn from communication practices in SSH by overcoming the coverage problem and combining citation analysis with citation context analysis? The project will create manually a near-complete publication database which includes publication lists, citation databases or national databases, including citing and cited literature from Google Scholar. Art history and international relations in Germany will be compared to the Netherlands. Finally, the project includes interviews with researchers to validate the findings. COPSSH is a challenging project. For example, for some publications, there may be no PDF files available or these files may not have a clear structure (e.g. side notes instead of footnotes and endnotes in certain articles from art history). The project is currently testing citation grabbers and their machine learning capabilities. The research data is to be published as an open dataset including citation context analysis.
David Shotton (University of Oxford) and Silvio Peroni (University of Bologna) are the Directors of OpenCitations. They presented recent developments regarding OpenCitations. Open Citation Identifier (OCI) equals DOI for citations. The EU-funded FREYA project recognizes OCIs as persistent identifiers for citations, and the identifiers are used for citations in Wikipedia articles with data from Wikidata, and for open DOI-to-DOI citations defined by open references in Crossref. This has made it possible to publish COCI, the OpenCitations Index of Crossref open DOI-to-DOI citations. The COCI Index employs live calls to the Crossref Application Programming Interface (API) to pull publication metadata not stored in the Index. WOCI, the OpenCitations Index of Wikidata citations, will be published soon. In addition, CROCI, the Crowdsourced Open Citations Index, has been released. The community is able to submit citation data to CROCI, but CROCI has yet to develop significant content following the call for crowdsourcing data to CROCI made in February 2019. All three of these indexes are accessible through a unified OpenCitations API.
In the future, OpenCitations hopes to harvest references extracted from the arXiv corpus as part of the EXCITE project that is carried out at GESIS – Leibniz Institute for the Social Sciences and WeST – Institute for Web Science and Technologies. Furthermore, a collaborative initiative with OpenAIRE is planned. During the workshop discussion, it was mentioned that it is almost impossible for one organisation to host all the information, and that data needs to be enriched from other databases, for example via live API calls, or by database federation.
Following the discussion of the developments relating to OpenCitations, Silvio Peroni presented the Open Biomedical Citations in Context Corpus funded by the Wellcome Trust as part of the Open Research Fund programme, which started in July 2019. The project is about harvesting the textual context of individual in-text reference pointers in the full text of publications in the biomedical literature. The data will be derived from an open access subset of Europe PMC (Pubmed Central) by using the EPMC API to harvest XML (Extensible Markup Language) documents. Europe PMC is a comprehensive database of life sciences and biomedical research. Finally, the project will also provide a description of the ingestion workflow.
Gianmarco Spinaci (University of Bologna) presented an ongoing research project that aims to analyse Arts and Humanities (A&H) publications in major bibliographic databases, such as WoS, Scopus, Crossref, Dimensions and Microsoft Academic Graph. One of the goals is to identify, count and cluster A&H publications in the different databases. Furthermore, all A&H fields of studies were retrieved from Microsoft Academic Graph. The clustering will be visualised with VOSviewer. One preliminary result is the relatively small amount of books and book series within WoS. Potential further use cases are still being explored.
Giovanni Colavizza (University of Amsterdam and CWTS) presented the Scholar Index, which includes a citation index for the Arts and Humanities from the Arts and Humanities. Information retrieval in A&H is challenging. Scholar Index is to be integrated in the OpenCitations corpus as well as to Europeana, which “provides access to over 50 million digitised items – books, music, artworks and more”. Currently, a prototype is being developed to improve information retrieval by connecting several systems. The prototype is focused on the History of Venice, and it is also possible to add tools. The coordinators of Scholar Index are currently looking for pilot partners around Europe (e.g. libraries and archives).
Thomas Franssen (CWTS) presented an overview on the RISIS2 project funded by Horizon 2020, which is the follow-up of the recently concluded RISIS (Research Infrastructure for Science and Innovation Policy Studies) project. Compared to the first RISIS project, the frontend (e.g. core facility) is to be developed further. Several research infrastructures are available to researchers as part of the RISIS2 project, and the consortium partners include various research institutes from all over Europe.
Day 2
The second day started with a presentation by Rodrigo Costas (CWTS) on the use of Mendeley readership statistics to develop an open classification scheme for Crossref. Crossref has certain limitations that this project aims to tackle, such as lack of metadata on affiliations, funding acknowledgements and particularly a homogeneous classification for journal publications. This lacking of metadata hinders research efforts based on Crossref data, including for example the monitoring of the disciplinary uptake of open citations. Mendeley is a reference manager and an academic social network by Elsevier that provides free access to its data for research purposes (which can be freely queried using the Mendeley API. For the study, a free global classification of Crossref based on all available DOIs in Crossref was carried out. This classification is based on the 28 academic fields as defined by Mendeley. Mendeley users classify themselves in these subject areas when they create their profile on Mendeley. The main idea of the project is to classify Crossref publications in the field(s) of the Mendeley users that are saving them in their individual libraries. Thus, it is possible to develop a sort of ‘crowdsourced’ classification of Crossref publications, independently from their indexing in other databases (e.g. Scopus or WoS). The study first investigated journal classifications, leaving classifications of publications as a next step. The potential of the Mendeley dataset is to provide a global free classification for all Crossref publications. During the workshop discussion it was suggested to explore the open reference manager Zotero as an alternative, while Microsoft Academic Graph could also be tested to develop open classifications of publications.
Grischa Fraumann (Leibniz Information Centre for Science and Technology) provided a summary on the ROSI (Reference Implementation for Open Scientometric Indicators) project which is carried out at the Leibniz Information Centre for Science and Technology. The project aims to develop a prototype that visualises open scientometric indicators, for example in an online dashboard. This prototype will be tested with researchers in interviews and workshops. The ROSI project is also part of the funding line quantitative research on the science sector of the BMBF. The workshop discussion focused on the public registry of data sources as part of the project, and on the sustainability of research infrastructures.
Nees Jan van Eck, Ludo Waltman, David Shotton and Silvio Peroni discussed recent developments regarding the VOSviewer software and open data sources that can be queried via APIs. Historically, VOSviewer supported WoS and Scopus, which both require a subscription. More recent data sources include Dimensions, Crossref, Wikidata and the OpenCitations Corpus (OCC). Dimensions has a limited edition that is freely accessible. Crossref is open but about half of its citation links are closed. OCC is open but currently provides only a very limited coverage of the scientific literature. Crossref, OCC, Europe PMC, Wikidata, and Semantic Scholar, a data source created by the Allen Institute for Artificial Intelligence, can all be queried via APIs. Support for these APIs was recently added to VOSviewer. Compared to downloads from WoS or Scopus, working with APIs is more convenient. However, the APIs of the different data sources all have limitations, for instance in their speed and flexibility. In the end, the discussion focused on the best way for VOSviewer and other scientometric tools to support data sources via APIs. Ideas were developed for improved interoperability between scientometric data sources and scientometric tools.
Example of a visualisation created by VOSviewer for the query ‘zika’ via the Europe PMC API
Public presentations and outlook
The workshop concluded with short presentations of the above-mentioned initiatives and projects. These presentations were open to the public and a significant number of researchers from CWTS and other research institutes attended. The discussions led to useful feedback to develop the projects and initiatives further.
The workshop provided a summary on recent developments in open scientometric data infrastructures from some selected initiatives and projects. It will be interesting to see the next steps in these important developments. For those interested to learn more about these developments, at the ISSI Conference 2019 at Sapienza University Rome in September 2019, a workshop titled Open Citations: Opportunities and Ongoing Developments will be organised.
