Radiography of open access academic publishing and its bibliometric indicators

Adapted photo from the original: Daz Danks.

By Lilian Nassi-Calò

A detailed report of a study¹ performed by Science-Metrix² funded by the US National Science Foundation, analyzes the situation of Open Access in the second half of 2016. The study¹ published in January 2018 used as a universe of articles those indexed in the WoS and Scopus databases, plus the 1science database to identify their openness. The study identified that among the countries with the highest scientific output, more than 50% of papers published between 2010 and 2014 were available in open access, with Brazil taking the lead with 75% of articles. The study also presents the citation patterns of open access articles in relation to paywalled articles.

The Open Access movement, initiated in the late 1990s through the initiatives of numerous advocates, institutions, funding agencies and publishers, was consolidated at the Budapest Meeting in February 2002 with the publication of the Budapest Open Access Initiative – BOAI – which defined the terms Green Route and Gold Route, among other concepts. The Declaration that defined Open Access eliminates, besides the availability barriers, also the copyright barriers that hinder the free reproduction, interchange, copy, etc. for legal purposes of the contents published in open access, the only restriction on reproduction and distribution being to grant recognition to the authors and proper citation. It is worth mentioning the pioneering nature of SciELO, which formally began publication in open access in 1998, i.e., four years before BOAI.

The study

Those responsible for the study warn that the impact measures of open access articles are more indicative than conclusive, since open access is particularly difficult to measure as compared to subscription journals articles. When considering the green route, for example, one must take into account hundreds of thousands of repositories in which authors can make available their final manuscripts or copies of documents published by journals, if authorized by the publishers. This diversity of sites adds an additional challenge to the measurement of open access bibliometric indexes, which is not the case with paywalled publications.

The Science-Metrix study aimed to assess the evolution of open-access publication in peer reviewed journals indexed in reference databases such as the Web of Science and Scopus; the open access situation in the United States as compared to other countries; and the citation indexes and the impact of open access articles as compared to subscription journal articles.

As previously mentioned, the databases Web of Science (Clarivate Analytics) and Scopus (Elsevier) were used in the analyses, besides the open access 1science database (developed by 1science, a Science-Metrix sister company).

The data collection for the study considers the two forms of open access mentioned above, the gold route – articles made available in fully open access or hybrid journals on its own website or in aggregators portals such as SciELO and PubMed Central; and the green route – articles openly shared by the authors themselves, librarians or other actors in post-print repositories such as ResearchGate, institutional and thematic repositories, or preprint servers such as arXiv, BioRxiv, and countless others recently launched in specific areas. The report stresses, however, that a particular article may be available both the gold route and on the green route, that is, both categories are not mutually exclusive.

Another important fact to consider is that the Web of Science (WoS) and Scopus concentrate mainly subscription journals published in English language from western countries, of disciplines of natural sciences and health sciences. The report estimates that less than 30% of peer reviewed journals published in the world are open access (OA). However, out of some 27,000 journals in Scopus, only 3,000 are open access (10%) and in WoS, of the 18,000 journals, there are less than 1,200 in OA, reaching an even lower percentage (7%).

The report took into account the characteristics of the journals present in the WoS and Scopus databases when calculating the availability of OA as compared to the articles found in the 1science database, which includes only OA literature, either gold or green route. The adjustment was made to provide a more accurate result of the actual situation. Analogously, adjustments and tests were made to quantify the percentage of open access in the countries, taking as standard the two largest producers of academic literature, the United States and China, which together hold more than 40% of the world scientific output measured in 2014 in WoS and Scopus databases. These two countries also allowed us to characterize 1science data for an English-speaking country and one non-English-speaking country, and in the case of China, one which does not use the Roman alphabet. Finally, in view of the non-uniform coverage of WoS and Scopus in the categories Natural Sciences, Health Sciences, Economics and Social Sciences, Arts and Humanities and Applied Sciences, calibrations were also made to validate the data collected.

Availability of open access

The collection of data on the availability of OA articles as a function of time is influenced by the delays and embargo periods to which many OA publications are subject. Delays are inherent in the activity of self-archiving in postprint repositories (green route), but the main cause is the embargo period imposed by publishers for self-archiving or for making articles available in OA after a certain period as paywalled articles. The percentage of OA articles located in WoS and present simultaneously in 1science database between 2006 and 2015, measured in 2016, ranges between 50 and almost 60% and shows an inflexion point in 2011, that is, this year reaches a maximum value, for later decline slowly. This decline, as already mentioned, is due to the embargo imposed by the publishers and delay in self-archiving by the authors themselves. This result confirms previous Science-Metrix studies that also reach values around 50% regarding the availability of articles in OA worldwide.

The OA distribution according to the country of publication shows interesting numbers. The United States and China dominate the academic publishing landscape in absolute numbers. In addition, 68% of articles with at least one American author and 45% of articles with at least one Chinese author indexed in WoS are available in some form of OA (as measured by 1science). However, it is Brazil that leads the percentage of OA articles worldwide with 77%. These figures refer to the years 2011-2012, the peak point of the curve, after which there is a slight decline due to delays and embargo, as discussed above. The report emphasizes that this position of Brazil “is certainly supported by the SciELO repository”. After Brazil are the Netherlands (74%), Switzerland (68%) and the United Kingdom (64%).

Gold route and green route

Data collected by the Science-Metrix report show that the green route (self-archiving) is the preferred route to make OA articles available worldwide. This route represents around 35% of the contents in OA, whereas the gold route (OA journals) represents almost 25%. The report also highlights the role of the national SciELO collections – “in Roman-speaking countries” – and J-Stage’s in Japan, as well as PubMed Central’s. It should be noted that about 10% of the OA papers available in 1science could not, however, be characterized as a green or gold route, but are accounted for in the total OA index, which, as mentioned earlier, amounts to 50-60% at its maximum value.

The authors of the study speculate that observing the increasing tendency of gold OA over time – also remembering that this modality is not influenced by delay or embargo – it is possible that this form of publication is actually increasing and could overcome the green route in the near future. In fact, there is growing interest from commercial publishers in launching OA journals whose business model involves article processing charges (APC).

When analyzing the type of OA per country of publication, Brazil again stands out due to the presence of SciELO, “which is an extremely efficient platform used to diffuse scholarly papers published in Brazil and many other countries where Romance languages are spoken.” Brazil appears with 74% of total OA, 42% of green and 42% of gold, and 11% of non-determined forms of OA. As the table that shows this data was built with the 20 countries with the largest number of papers in WoS in 2014, unfortunately, no other country that integrates the SciELO Network is represented. In this table, Brazil ranks 13^th in terms of article production, with the United States, China, the United Kingdom, Germany and Japan in the top five, respectively.

Citation indexes and open access

The potential positive relationship between open access and citations has often been suggested and there are numerous articles that have sought to measure it. In 2016, the scholarly social network Academia.edu reported the results of a previous year’s survey³ showing that articles deposited in its repository had 83% more citations in a five year interval. It is beyond our purpose to evaluate or question these results, but the notion that OA articles attract more citations is even intuitive, since they would be easier to obtain, read, and, therefore, cite. On the other hand, it is known that the reasons for citing one or another article are complex and follow a very particular taxonomy, and often the best articles are not the most highly cited.

Considering the wide variety of indexes based on citations to measure impact, Science-Metrix, in its report, used the average of relative citations (ARC), a normalized index of scientific impact. Its calculation involves counting the citations of each article normalized against the average citation level of all articles in the same subarea, year, and document type, to obtain a relative citation index. The average of the relative citations, therefore, is simply the average of all relative citations related to a specific entity (country or discipline).

The average relative citations (ARC) of articles published between 2006 and 2013 measured in 2016 was, then, compared for four articles categories: total OA, gold OA, green OA and non-OA. When establishing the ARC baseline at 1.00, values higher than 1 indicate above-average citation, and values lower than 1, below-average citations. The interpretation of the results is not simple and direct, since it is known that the release of OA documents occurs progressively, especially for the green route. In any case, there is a significant advantage of OA citations in relation to paywalled articles throughout the analyzed period. Articles from subscription journals have ARC ranging from 0.80 to 0.85, whereas for green OA articles it ranges between 1.20 and 1.30.

The behavior of the gold OA citations is analyzed in detail in the report, because the gold OA data originally collected contained both green and gold articles and, thus, gave the impression that the ARC diminished over time. In fact, the ARC of the gold OA continues to increase: it was 0.75 in 2006 and reached 1.10 in 2011. Given these inaccuracies, and like the studies cited in this post, the report mentions that using other approaches and methodologies to measuring and analyzing the impact of OA is critical for decision makers and for future mandates for the publication of research results in OA.

To analyze the impact of OA by areas of knowledge, the authors calculated a relationship between ARC for OA publications and non-OA publications. The higher this ratio, the greater the citation advantage of OA in relation to paywalled articles. Among the disciplines, the highest relation was observed for Arts and Humanities (ARC 1.99), followed by Economics and Social Sciences (1.75), Health Sciences (1.48), Natural Sciences (1.39) and Applied Sciences (1.30). Also among the disciplines, green OA gets more citations than the gold OA.

Finally, the ARC at country level was measured for the top 20 producers of academic literature in the WoS in 2010. The report presents the total ARC (all articles) as well as the indices for total OA and total non-OA. In addition, the ARC index for green OA and gold OA was also calculated, as well as the OA/non-OA ratio in each case. The world ARC was normalized to 1.00. The country with the highest OA/non-OA ratio is Russia (ARC 2.65), followed by Poland (1.70) and France (1.66). Brazil presented ARC 1.06, that is, a positive value (above 1.00) but below the United States (1.46). The OA/non-OA ARC ratio for Brazil to green OA is 1.46, but that of gold OA is only 0.65. The report mentions that in future studies it may be of interest to examine how the level of OA publications in the countries contributes to strengthening their scientific communication systems, as in the case of Brazil, “and other countries that use SciELO, compared to countries who made little efforts to make their contents openly available”.

The data presented, according to the authors of the report, “confirm that open access publishing continues to grow. Some countries have three-quarters of their articles freely available for download on the Internet and most of the research leading countries have more than 50% of their articles freely available.” Further studies of this nature must be pursued to support open access policies and mandates.

As it has been widely debated on this blog, the academic community and society as a whole would greatly benefit from the transition from the subscription model to open access funded by article processing fees, since there is more than enough resources for it. This may be possible in the near future.

Notes

1. Open access availability of scientific publications [online]. 2018 [viewed 9 February 2018]. Available from: http://www.science-metrix.com/sites/default/files/science-metrix/publications/science-metrix_open_access_availability_scientific_publications_report.pdf

2. Independent research evaluation firm specializing in the assessment of science and technology activities based in Canada.

3. SPINAK, E. Will your paper be more cited if published in Open Access? [online]. SciELO in Perspective, 2016 [viewed 9 February 2018]. Available from: http://blog.scielo.org/en/2016/01/21/will-your-paper-be-more-cited-if-published-in-open-access/

References

ARCHAMBAULT, E., et al. Proportion of Open Access Peer-Reviewed Papers at the European and World Levels — 2004–2011 [online]. Science-Metrix, 2013. [viewed 9 February 2018]. Available from: http://www.science-metrix.com/pdf/SM_EC_OA_Availability_2004-2011.pdf

NASSI-CALÒ, L. Paper investigates: is your most cited work your best work? [online]. SciELO in Perspective, 2014 [viewed 9 February 2018]. Available from: http://blog.scielo.org/en/2014/11/24/paper-investigates-is-your-most-cited-work-your-best-work/

NASSI-CALÒ, L. Study proposes a taxonomy of motives to cite articles in scientific publications [online]. SciELO in Perspective, 2014 [viewed 9 February 2018]. Available from: http://blog.scielo.org/en/2014/11/07/study-proposes-a-taxonomy-of-motives-to-cite-articles-in-scientific-publications/

Open access availability of scientific publications [online]. 2018 [viewed 9 February 2018]. Available from: http://www.science-metrix.com/sites/default/files/science-metrix/publications/science-metrix_open_access_availability_scientific_publications_report.pdf

PACKER, A.L., et al. SciELO: uma metodologia para publicação eletrônica. Ciência da Informação [online]. 1998, vol. 2, no. 2, pp. 109-212, ISSN: 1518-8353 [viewed 9 February 2018]. DOI: 10.1590/S0100-19651998000200002. Available from: http://ref.scielo.org/435gm9

SPINAK, E. Open access articles are here to stay: in less than 10 years nearly 50% of articles worldwide can be accessed this way [online]. SciELO in Perspective, 2013 [viewed 9 February 2018]. Available from: http://blog.scielo.org/en/2013/08/28/open-access-articles-are-here-to-stay-in-less-than-10-years-nearly-50-of-articles-worldwide-can-be-accessed-this-way/

SPINAK, E. What’s the deal with preprints? [online]. SciELO in Perspective, 2016 [viewed 9 February 2018]. Available from: http://blog.scielo.org/en/2016/11/22/whats-the-deal-with-preprints/

SPINAK, E. Will your paper be more cited if published in Open Access? [online]. SciELO in Perspective, 2016 [viewed 9 February 2018]. Available from: http://blog.scielo.org/en/2016/01/21/will-your-paper-be-more-cited-if-published-in-open-access/

VELTEROP, J. The BOAI (Budapest Open Access Initiative) celebrates its 15 year anniversary [online]. SciELO in Perspective, 2016 [viewed 9 February 2018]. Available from: http://blog.scielo.org/en/2016/12/21/the-boai-budapest-open-access-initiative-celebrates-its-15-year-anniversary

VELTEROP, J. What is holding back the transition to open access if it does not cost more? [online]. SciELO in Perspective, 2015 [viewed 9 February 2018]. Available from: http://blog.scielo.org/en/2015/09/10/what-is-holding-back-the-transition-to-open-access-if-it-does-not-cost-more/

External links

1science <http://1science.com>

BOAI –The Budapest Open Access Initiative <http://www.budapestopenaccessinitiative.org/read>

Science-Metrix <http://www.science-metrix.com>

Scopus <https://www.elsevier.com/solutions/scopus>

WoS – Web of Science <https://clarivate.com/products/web-of-science>

About Lilian Nassi-Calò

Lilian Nassi-Calò studied chemistry at Instituto de Química – USP, holds a doctorate in Biochemistry by the same institution and a post-doctorate as an Alexander von Humboldt fellow in Wuerzburg, Germany. After her studies, she was a professor and researcher at IQ-USP. She also worked as an industrial chemist and presently she is Coordinator of Scientific Communication at BIREME/PAHO/WHO and a collaborator of SciELO.

Translated from the original in portuguese by Lilian Nassi-Calò.