By Lilian Nassi-Calò
Image: William Warby.
Readers of this blog are familiar with the limitations and criticisms of using citation-based bibliometric indicators to evaluate science journals. Above all, the impact, relevance, quality, and rigor attributed to research results published in journals with high citation rates solely based on these indicators.
Movements such as the San Francisco Declaration on Research Assessment (DORA) and the Leiden Manifesto recommend using indices such as the Impact Factor only to rank journals, not researchers, institutions, graduate programs, or to grant research subsidy. Added to this is the fact that estimates indicate that about 50% of published articles are not formally cited by other authors, and their impact on the scientific community is not null. These articles are read, downloaded, shared, and cited through social networks, blogs, news, public policies, and other forms of online presence, gathered, and measured in indices such as Altmetrics – alternative metrics.
The speed with which newly published articles are shared across the web is one of the strengths of altmetrics compared to citation-based metrics, which are counted two or three years after publication. In addition, studies indicate that articles with a high presence on social networks are more disseminated and receive more citations.
In the last decade, numerous journals, including those in the SciELO Brazil collection, began to incorporate the Altmetric index into their articles, alongside other citation-based indices, such as Dimensions, ScienceOpen and others, which show readers the “alternative impact” with daily updates, that is, how much that article was mentioned on social networks (Twitter, Facebook, Instagram, LinkedIn etc.), blogs, news, patents, Wikipedia pages and other mentions.
Spatti, Cintra, Bin, and Ferreira Araújo carried out an altmetric study1 to assess the presence of scientific research output in Latin America on the social web using the collection of journals and articles indexed in SciELO. The cut made by the authors in the collection comprises the following countries: Argentina, Bolivia, Brazil, Chile, Colombia, Costa Rica, Cuba, Ecuador, Mexico, Paraguay, Peru, Uruguay, and Venezuela. Data were extracted from Digital Science’s altmetric.com platform for articles published between 2015 and 2018. This subset includes 1,211 journals and 140,158 articles.
The research resulted that of the 1,211 journals, 707 (58%) had articles shared by social networks and mentions covered by the Altmetric platform. Of the 141,158 articles, however, a smaller fraction, 18,737 (13%) were mentioned on the social web.
Of the total of these mentions, 58,994 (average of 3.14 mentions/article), the vast majority, 94.3%, were in social networks (Twitter, 73.8%; Facebook, 20.0%; and Google+, 0, 4%;); 3.7% on news sites and blogs; 0.7% in policy documents and patents; and 1.3% from other sources (Wikipedia, videos, Q&A posts, and academic peer review sites).
The fact that Twitter ranks in first place (73.8%) as the most used platform to share research articles – followed by far by Facebook (20.0%) in second place was, in a way, expected, due to the fact of being the fastest growing platform among academics to share and recommend publications, according to previous studies. It is necessary, however, to make a reservation about Facebook data, as it is possible that the data on this social network are underestimated, since the altmetric search does not account for posts in closed groups or restricted access profiles, unlike Twitter, whose data are widely captured.
An interesting piece of data obtained by the authors refers to the origin of tweets about Latin American publications. More than a third of them originate outside Latin America. This percentage is 100% for articles from Cuba and Bolivia, 63% for Argentina, 57% for Venezuela, 43% for Brazil, 36% for Chile, 29% for Mexico and 24% for Costa Rica. The other countries show percentages of the order of 10-15%. The countries where most tweets come from are the United Kingdom, Spain, the United States and Malaysia.
These results immediately call attention to the publication language, a variable that may influence the articles’ altmetric performance. Of the sample studied by the authors, 57% of the mentions are to articles in English, 24% in Spanish and 18% in Portuguese, indicating an internationalization of Latin American publications. It is worth mentioning the case of Brazil, where 70% of the mentions are to articles in English, 27% to articles in Portuguese and 3% in Spanish. These data are in line with SciELO’s internationalization policy,2 which came into force in 2014.
The authors also emphasize that most of the web developments and applications, especially those of academic interest, are developed by researchers for publications in the English language, and for this reason, in non-English-speaking nations, biases may occur in tracking data on the web.
The study also evaluated the average number of articles mentions per journal in each Latin American country. Chile ranks first in the percentage of journals with 81% of its journals present in altmetric, followed by Brazil (76%), Colombia (75%), Costa Rica (61%), Peru (60%), and Mexico (50%). Brazil leads the ranking of the country with the highest percentage of articles with mentions (20%), and the highest average of articles with mentions per journal (44%). The good result in Brazil is possibly due to SciELO’s policy of encouraging the dissemination of journals on social networks, which as of 2014 includes in the Selection and Permanence Criteria of the SciELO Brazil Collection3 a dissemination plan that provides for the creation of a journal profile on social networks such as Facebook and/or Twitter.
To assess the influence of the area of knowledge on the web presence of articles, the study by Spatti, et al., indicated that it is necessary to identify a pattern of behavior according to the area of knowledge, especially for countries with greater participation. Brazil, as well as Chile, Colombia, and Ecuador, have strong publications in biology and health sciences and applied human and social sciences, and it is in this area that concentrate most posts in social networks, blogs and news sites, documents from policies and patents and academic sources. On the other hand, in Chile, Venezuela, Argentina and Costa Rica, there is a predominance of Health and Biological Sciences journals, as well as mentions on the social web. In Peru, however, there is no difference between areas for altmetric mentions.
The altmetric behavior of the country profiles analyzed here makes it possible to identify three indicators for the characterization of profiles: penetration (altmetric coverage in terms of journals and publications); insertion (percentage of self-mention and Latin American mention) and internationalization (percentage of non-Latin American mention and articles mentioned in English).
Based on these indicators, it was possible to identify three groups. The first group is formed only by Brazil. It has high insertion evidenced by the number of journals and number of articles located in altmetric; high insertion, with a percentage of 48% of self-mention; and high internationalization, with 43% of non-Latin American mentions and 70% of publications mentioned in English.
The second group is formed by Chile, Colombia and Costa Rica, countries whose altmetric behavior shows: medium penetration (percentage of journals with mentions greater than the median of the region and percentage of articles with mentions close to the region median); high insertion, with a significant percentage of self-mention and average internationalization, characterized by percentages between 10-36% of non-Latin American mentions and 23-35% of articles mentioned in English.
The third group, formed by Peru and Mexico, has a rather moderate profile, with medium penetration, characterized by the percentage of articles and journals with mentions close to the countries median; medium insertion, with a percentage of self-mention of 27-44% and moderate international profile, 14%-29% of non-Latin American mentions and 25-27% of mentions of articles in English.
A fourth group is formed by Argentina only, characterized by low penetration and insertion, but high internationalization, with 63% of non-Latin American mentions. Finally, a fifth group includes Ecuador, Venezuela, Uruguay, Cuba, Bolivia and Paraguay, countries with low penetration and absolute values of publications with mentions below 100, making further analysis unfeasible.
It is important to highlight that the article by Spatti, et al.1 was published in 2021, with data from 2015 to 2018. It is to be expected that after three years, there have been changes both in the profile of published articles, in terms of the percentage of internationalization of publications, and in the dissemination of these articles through social networks, which would have increased penetration, insertion and internationalization.
Alternative metrics, as the authors point out, are better interpreted when compared to citation-based and other metrics such as usage and download measures. In this sense, the evaluation of science produced and published in Latin American journals benefits greatly from further such analyzes.
Notes
1 SPATTI, A.C., et al. Métricas alternativas para avaliação da produção científica latino-americana: um estudo da rede SciELO. Inf. Inf. [online]. 2021, vol. 26, no. 2, pp. 596-624 [viewed 09 February 2022]. http://doi.org/10.5433/1981-8920.2021v26n2p596. Available from: https://www.uel.br/revistas/uel/index.php/informacao/article/view/41474
2. PACKER, A. Internationalization of journals was the central topic of the 4th Annual SciELO Meeting [online]. SciELO in Perspective, 2014 [viewed 04 February 2022]. Available from: https://blog.scielo.org/en/2014/12/16/internationalization-of-journals-was-the-central-topic-of-the-4th-annual-scielo-meeting/
References
ENKHBAYAR, A. and ALPERIN, J. P. Challenges of capturing engagement on Facebook for Altmetrics [online]. arXiv. 2018 [viewed 09 February 2022]. Available from: https://arxiv.org/abs/1809.01194
ORTEGA, J. S. Academic journals with a presence on Twitter are more widely disseminated and receive a higher number of citations. LSE Impact of Social Sciences, 2017 [viewed 09 February 2022]. Available from: https://blogs.lse.ac.uk/impactofsocialsciences/2017/12/04/academic-journals-with-a-presence-on-twitter-are-more-widely-disseminated-and-receive-a-higher-number-of-citations/
ORTEGA, J. S. The presence of academic journals on Twitter and its relationship with dissemination (tweets) and research impact (citations). Aslib Journal of Information Management [online]. 2017, vol. 69, no. 6, pp. 674-687, ISSN: 2050-3806 [viewed 09 February 2022]. https://doi.org/10.1108/AJIM-02-2017-0055. Available from: http://www.emeraldinsight.com/doi/abs/10.1108/AJIM-02-2017-0055
PACKER, A. Internationalization of journals was the central topic of the 4th Annual SciELO Meeting [online]. SciELO in Perspective, 2014 [viewed 04 February 2022]. Available from: https://blog.scielo.org/en/2014/12/16/internationalization-of-journals-was-the-central-topic-of-the-4th-annual-scielo-meeting/
SCIENTIFIC ELECTRONIC LIBRARY ONLINE. Interview with Atila Iamarino [online]. SciELO in Perspective, 2013 [viewed 09 February 2022]. Available from: https://blog.scielo.org/en/2013/11/29/interview-with-atila-iamarino/
SCIENTIFIC ELECTRONIC LIBRARY ONLINE. Interview with Euan Adie, CEO of altmetric.com [online]. SciELO in Perspective, 2013 [viewed 09 February 2022]. Available from: https://blog.scielo.org/en/2013/08/29/interview-with-euan-adie-ceo-of-altmetric-com/
SPATTI, A.C., et al. Métricas alternativas para avaliação da produção científica latino-americana: um estudo da rede SciELO. Inf. Inf. [online]. 2021, vol. 26, no. 2, pp. 596-624 [viewed 09 February 2022]. http://doi.org/10.5433/1981-8920.2021v26n2p596. Available from: https://www.uel.br/revistas/uel/index.php/informacao/article/view/41474
The donut and Altmetric Attention Score [online]. Altmetric [viewed 09 February 2022]. Available from: https://www.altmetric.com/about-our-data/the-donut-and-score/
External links
The Leiden Manifesto for Research Metrics
The San Francisco Declaration on Research Assessment (DORA)
About Lilian Nassi-Calò
Lilian Nassi-Calò studied chemistry at Instituto de Química – USP, holds a doctorate in Biochemistry by the same institution and a post-doctorate as an Alexander von Humboldt fellow in Wuerzburg, Germany. After her studies, she was a professor and researcher at IQ-USP. She also worked as an industrial chemist and presently she is Coordinator of Scientific Communication at BIREME/PAHO/WHO and a collaborator of SciELO.
Translated from the original in Portuguese by Lilian Nassi-Calò
Related Posts:
Como citar este post [ISO 690/2010]:
Recent Comments