{"id":5670,"date":"2025-09-24T11:00:03","date_gmt":"2025-09-24T14:00:03","guid":{"rendered":"https:\/\/blog.scielo.org\/en\/?p=5670"},"modified":"2025-09-24T11:15:56","modified_gmt":"2025-09-24T14:15:56","slug":"impact-beyond-academia","status":"publish","type":"post","link":"https:\/\/blog.scielo.org\/en\/2025\/09\/24\/impact-beyond-academia\/","title":{"rendered":"Impact beyond academia: SciELO’s presence in public policy"},"content":{"rendered":"

By\u00a0Evandro Coggo Cristofoletti, Sergio Salles-Filho, Yohanna Juk, Bernardo Cabral, Karen Esteves, Fernandes Pinto, Carlos Graziani, and C\u00e9sar Antonio Pereira<\/strong><\/p>\n

Introduction<\/h3>\n
\"Photograph<\/a>

Image: Ed Us via Unsplash<\/a><\/em><\/p><\/div>\n

Funding agencies, universities, and Science, Technology, and Innovation (ST&I) policymakers have been discussing ways to strengthen the development of indicators, metrics, and assessments of research\u2019s social relevance, including its impact on public policy. For the SciELO Program, measuring the impact of research on public policy is a keyway to demonstrate the social value of the research results published in SciELO-indexed journals. As one of the leading scientific publishing platforms globally, SciELO has the opportunity to highlight how indexed research transcends the academic environment and contributes effectively to the formulation of policies that impact society, offering metrics that complement traditional bibliometric indices<\/p>\n

Methodology<\/h3>\n

Assessing the impact of research on public policy is no trivial task. Besides involving various methodological and practical challenges, there is also the difficulty related to the availability of data that allows for the production of consistent metrics and indicators. Qualitative or mixed approaches, based on case studies of policies influenced by research\u2014through questionnaires, interviews, ethnographies or documentary research\u2014continue to be widely used and recognized in literature. At the same time, there is a growing number of initiatives that seek to identify elements of research (mentions or use of articles, authors, institutions and funding agencies, among others) in public policy documents. Although still exploratory and incipient, this approach has the advantage of allowing work with large volumes of data and the application of bibliometric and scientometric techniques for the development of impact metrics. Among the tools used in this context, a highlight is Overton, a database that aggregates public policy documents from different countries, allowing these documents to be linked to academic publications cited or used in their preparation.<\/p>\n

Considering this approach, we carried out a data collection and processing approach based on XML files made available by SciELO<\/a>, totaling approximately 6 GB, organized by country and subdivided into multiple folders. Initially, we extracted the \u201cpublisher-id\u201d (id_scielo) identifier from all XML files and, when available, the articles\u2019 DOI, covering the folders referring to Chile, Ecuador, Paraguay, SciELO Brazil, and SciELO Public Health. The other folders contained only the id_scielo. Then, using the Xylose<\/a> library, we obtained additional DOIs through the ArticleMeta API. In total, we identified 620,411 DOIs (from a universe of about 1.2 million records), which were used as input for searching Overton for policy documents that mentioned or cited these publications; Overton tracks and indexes documents produced by political organizations or those that aim to influence public policy, notably government sources, multilateral organizations, and think tanks<\/p>\n

Based on this procedure, we downloaded the data returned by Overton and organized it into a consolidated spreadsheet. This spreadsheet brings together, for each policy document identified, the articles from the SciELO database mentioned, as well as metadata about the policy itself\u2014including the source, country of origin, and organization responsible for its development. Based on this dataset, we were able to generate initial analyses, which we present below, offering a preliminary overview of the scope and connections between research indexed in SciELO and public policy documents mapped by Overton.<\/p>\n

Results<\/h3>\n

General results<\/h4>\n

Using the initial input of 620,411 DOIs, a total of 47,231 unique DOIs (i.e., excluding any citation repetition) were identified in policy documents. However, when we consider the data with repetition (since a single article can be cited in more than one policy document and\/or a policy document can cite more than one article from the identified sample), we have a total of 100,401 mentions to articles. Excluding repetitions of policy documents from this total number of mentions, we have 47,796 unique policy documents that mention an article from the database. This means that many articles are mentioned more than once, which is an indication of a positive impact on policies. We were able to identify that 4,435 articles were present in more than one policy document.<\/p>\n

Countries that cite articles indexed in SciELO<\/h4>\n

Most of the political organizations that mention articles indexed in SciELO are International Governmental Organizations (IGOs), such as the World Health Organization (WHO) and UNESCO (United Nations Educational, Scientific and Cultural Organization), among others (see next item), totaling 14,565 policy documents (about 34%). The second largest share comes from Brazil (considering government, think tanks, and other organizations), with 6,599 policy documents (15.42%), followed by the US, with 4,247 (9.92%). This is a great finding, given that the database indexes more than 4 million policy documents from the US and about 300,000 from Brazil. Meaning there\u2019s a good impact on national documents. Following, Table 1. lists the 20 countries with the most policy documents (totaling 39,508 documents in total, about 92% of the total).<\/p>\n

\u00a0<\/strong><\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Country<\/strong><\/td>\nCount<\/strong><\/td>\nTotal documents<\/strong><\/td>\nPercentage of total documents<\/strong><\/td>\n<\/tr>\n
IGO<\/td>\n14565<\/td>\n42,796<\/td>\n34.03<\/td>\n<\/tr>\n
Brazil<\/td>\n\u00a0 6599<\/td>\n42,796<\/td>\n15.42<\/td>\n<\/tr>\n
United States<\/td>\n\u00a0 4247<\/td>\n42,796<\/td>\n9.92<\/td>\n<\/tr>\n
Spain<\/td>\n\u00a0 3035<\/td>\n42,796<\/td>\n7.09<\/td>\n<\/tr>\n
United Kingdom<\/td>\n\u00a0 1502<\/td>\n42,796<\/td>\n3.51<\/td>\n<\/tr>\n
European Union<\/td>\n\u00a0 1487<\/td>\n42,796<\/td>\n3.47<\/td>\n<\/tr>\n
Canada<\/td>\n\u00a0 1254<\/td>\n42,796<\/td>\n2.93<\/td>\n<\/tr>\n
Germany<\/td>\n\u00a0\u00a0 833<\/td>\n42,796<\/td>\n1.95<\/td>\n<\/tr>\n
Chile<\/td>\n\u00a0\u00a0 775<\/td>\n42,796<\/td>\n1.81<\/td>\n<\/tr>\n
France<\/td>\n\u00a0\u00a0 745<\/td>\n42,796<\/td>\n1.74<\/td>\n<\/tr>\n
Peru<\/td>\n\u00a0\u00a0 681<\/td>\n42,796<\/td>\n1.59<\/td>\n<\/tr>\n
Argentina<\/td>\n\u00a0\u00a0 667<\/td>\n42,796<\/td>\n1.56<\/td>\n<\/tr>\n
Colombia<\/td>\n\u00a0\u00a0 575<\/td>\n42,796<\/td>\n1.34<\/td>\n<\/tr>\n
Portugal<\/td>\n\u00a0\u00a0 566<\/td>\n42,796<\/td>\n1.32<\/td>\n<\/tr>\n
Australia<\/td>\n\u00a0\u00a0 526<\/td>\n42,796<\/td>\n1.23<\/td>\n<\/tr>\n
Uruguay<\/td>\n\u00a0\u00a0 461<\/td>\n42,796<\/td>\n1.08<\/td>\n<\/tr>\n
South Africa<\/td>\n\u00a0\u00a0 358<\/td>\n42,796<\/td>\n0.84<\/td>\n<\/tr>\n
Mexico<\/td>\n\u00a0\u00a0 341<\/td>\n42,796<\/td>\n0.80<\/td>\n<\/tr>\n
Sweden<\/td>\n\u00a0\u00a0 291<\/td>\n42,796<\/td>\n0.68<\/td>\n<\/tr>\n
Switzerland<\/td>\n\u00a0\u00a0 224<\/td>\n42,796<\/td>\n0.52<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

Table 1. Countries of origin of policy documents<\/strong><\/p>\n

 <\/p>\n

Organiza\u00e7\u00f5es que citam DOIs do SciELO<\/h4>\n

When we look at organizations, we see that the WHO leads in terms of policy documents citing articles indexed in SciELO (3,588 policy documents), followed by the Institute of Applied Economic Research (IPEA) (3,503), Guidelines in PubMed (1,892 policy documents), and the Brazilian Government (with 1,809 policy documents). This indicates a significant presence in the area of health, but also in economics and public policy.<\/p>\n

 <\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Organization<\/strong><\/td>\nCount<\/strong><\/td>\n<\/tr>\n
World Health Organization<\/td>\n3,588<\/td>\n<\/tr>\n
Instituto de Pesquisa Econ\u00f4mica Aplicada<\/td>\n3,503<\/td>\n<\/tr>\n
Guidelines in PubMed Central<\/td>\n1,892<\/td>\n<\/tr>\n
Government of Brazil<\/td>\n1,809<\/td>\n<\/tr>\n
World Bank<\/td>\n1,670<\/td>\n<\/tr>\n
UNESCO<\/td>\n1,544<\/td>\n<\/tr>\n
United Nations CEPAL<\/td>\n1,458<\/td>\n<\/tr>\n
Inter-American Development Bank<\/td>\n1,404<\/td>\n<\/tr>\n
Publications Office of the European Union<\/td>\n\u00a0\u00a0 831<\/td>\n<\/tr>\n
Food and Agriculture Organization of the United Nations<\/td>\n\u00a0\u00a0 821<\/td>\n<\/tr>\n
Empresa Brasileira de Pesquisa Agropecu\u00e1ria (EMBRAPA)<\/td>\n\u00a0\u00a0 818<\/td>\n<\/tr>\n
Pan American Health Organization (PAHO)<\/td>\n\u00a0\u00a0 798<\/td>\n<\/tr>\n
United Nations<\/td>\n\u00a0\u00a0 718<\/td>\n<\/tr>\n
Government of Peru<\/td>\n\u00a0\u00a0 549<\/td>\n<\/tr>\n
Government of Chile<\/td>\n\u00a0\u00a0 531<\/td>\n<\/tr>\n
Government of Spain<\/td>\n\u00a0\u00a0 530<\/td>\n<\/tr>\n
OECD<\/td>\n\u00a0\u00a0 505<\/td>\n<\/tr>\n
Government of Portugal<\/td>\n\u00a0\u00a0 504<\/td>\n<\/tr>\n
Generalitat de Catalunya<\/td>\n\u00a0\u00a0 475<\/td>\n<\/tr>\n
Euskal Autonomia Erkidegoa<\/td>\n\u00a0\u00a0 439<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

Table 2. The 20 organizations with the most policy documents<\/strong><\/p>\n

 <\/p>\n

Sources in Brazil<\/h4>\n

When we break down Brazilian sources, we have IPEA, the Brazilian Government, and Empresa Brasileira de Pesquisa Agropecu\u00e1ria<\/em> (Brazilian Agricultural Research Corporation, Embrapa) as the most prominent.<\/p>\n

 <\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Organization<\/strong><\/td>\nCount (n\u00ba of policy documents<\/strong><\/td>\nType of organization<\/strong><\/td>\n<\/tr>\n
Instituto de Pesquisa Econ\u00f4mica Aplicada<\/td>\n3,503<\/td>\nGovernmental think tank<\/td>\n<\/tr>\n
Government of Brazil<\/td>\n1,809<\/td>\nGovernment<\/td>\n<\/tr>\n
Empresa Brasileira de Pesquisa Agropecu\u00e1ria (EMBRAPA)<\/td>\n\u00a0\u00a0 818<\/td>\nPublic company<\/td>\n<\/tr>\n
Banco Central do Brasil<\/td>\n\u00a0\u00a0\u00a0\u00a0 79<\/td>\nFederal agency linked to the Ministry of Economics<\/td>\n<\/tr>\n
Anvisa<\/td>\n\u00a0\u00a0\u00a0\u00a0 65<\/td>\nRegulatory agency<\/td>\n<\/tr>\n
Centro de Desenvolvimento e Planejamento (CEDEPLAR)<\/td>\n\u00a0\u00a0\u00a0\u00a0 48<\/td>\nThink tank<\/td>\n<\/tr>\n
Senado Federal do Brasil<\/td>\n\u00a0\u00a0\u00a0\u00a0 46<\/td>\nLegislative branch<\/td>\n<\/tr>\n
Di\u00e1rio Oficial do Mato Grosso<\/td>\n\u00a0\u00a0\u00a0\u00a0 29<\/td>\nLegislative branch<\/td>\n<\/tr>\n
Centro Brasileiro de Rela\u00e7\u00f5es Internacionais<\/td>\n\u00a0\u00a0\u00a0\u00a0 25<\/td>\nGovernment<\/td>\n<\/tr>\n
Instituto Igarap\u00e9<\/td>\n\u00a0\u00a0\u00a0\u00a0 24<\/td>\nThink tank<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

However, there is a diversity of institutions, such as Banco do Brasil<\/em> (Bank of Brazil) and Ag\u00eancia Nacional de Vigil\u00e2ncia Sanit\u00e1ria<\/em> (Brazilian Health Regulatory Agency, Anvisa), reflecting the thematic scope of the articles and the indexing scope of SciELO, and its ability to cater to different areas and specializations of knowledge, which is also reflected in the diversity of use in various arenas and policy segments.<\/p>\n

Conclusion<\/h4>\n

The results presented show that research published in SciELO has achieved significant relevance in public policy documents, with emphasis on a significant number of Brazilian documents. Despite the wide variety of public policy areas citing research published in SciELO, initial analyses reveal an impact on policies in areas such as health and economics. However, it is important to recognize some limitations of the study, such as its dependence on Overton database coverage, which favors countries in the Global North (with issues such as indexing for each country and language), as well as the challenge of identifying how these articles are actually being used. Future investigations may explore complementary approaches, locate specific journals or areas, and gain a deeper understanding of the profile of policy documents compared to the articles mentioned in these documents.<\/p>\n

External links<\/h3>\n

SciELO Article Meta<\/a><\/p>\n

SciELO ARTICLEMETA Restful API – GitHub<\/a><\/p>\n

 <\/p>\n

About Evandro Coggo Cristofoletti\"Evandro<\/a><\/h3>\n

Postdoctoral researcher in Research Management at Unicamp. PhD in Science and Technology Policy from Unicamp. Member of the Innovation Policy Analysis Group (GAPI-DPCT) and the Laboratory for Studies on the Organization of Research and Innovation at Unicamp (Lab-GEOPI). Conducts research on higher education and science and technology policy, focusing on the relationship between universities and society. He is interested in the following topics: social and political impact of research, universities and sustainable development, the relationship between universities and the public sector, university extension programs, neoliberalism, and universities.<\/p>\n

 <\/p>\n

About Sergio Salles-Filho\"Sergio<\/a><\/h3>\n

Agricultural Engineer (UFRRJ, 1981), holds a master in Agricultural Sciences (Unesp, 1985), and PhD in Economics (Unicamp, 1993). Full Professor in the Department of Science and Technology Policy at IG\/Unicamp, he was director of FCA (2010-2013) and IG (2017-2021), as well as superintendent of FINEP (2001-2003). Visiting Researcher at MIoIR\/UK (2013-2014), he has received the Zeferino Vaz Award three times. He was deputy evaluation coordinator at FAPESP (2010-2020). He works in economics, planning, and management of science, technology, and innovation, with more than 110 publications, 50 supervisions, and coordination of projects on policy evaluation and strategic planning in ST&I.<\/p>\n

 <\/p>\n

About Yohanna Juk\"Yohanna<\/a><\/h3>\n

Postdoctoral researcher at the InSySPo (Innovation Systems, Strategies and Policies), a Project of the Department of Science and Technology Policy at Unicamp, funded by FAPESP (SPEC). She holds a PhD and master’s degree in public policy and a bachelor\u2019s degree in economics from UFPR. She conducts research on the design, implementation, and evaluation of science, technology, and innovation policies. She is interested in the following topics: gender inequality in science, with a focus on equity, diversity, and inclusion in research funding; the use of evidence in public policy; and the evaluation of ST&I programs and institutions through bibliometrics, altmetrics, and impact measurement methodologies.<\/p>\n

 <\/p>\n

About Bernardo Cabral\"Bernardo<\/a><\/h3>\n

Adjunct professor at the Faculty of Economics of the Universidade Federal da Bahia (UFBA), currently on secondment to the Funda\u00e7\u00e3o Oswaldo Cruz (Fiocruz). He holds a PhD in Industrial and Technological Economics from the Universidade Federal do Rio de Janeiro (UFRJ) (2018) and a bachelor’s degree (2010) and master’s degree (2012) in Economics from the Universidade Federal da Bahia. He also completed postdoctoral studies at the Department of Science and Technology Policy at the Universidade Estadual de Campinas (DPCT\/UNICAMP) (2024). He is a researcher and deputy leader of the Foresight Studies Center (NEF) and a collaborator at the Laboratory for Studies on the Organization of Research and Innovation at Unicamp (Lab-GEOPI).<\/p>\n

 <\/p>\n

About Karen Esteves Fernandes Pinto\"Karen<\/a><\/h3>\n

Postdoctoral researcher at the Genomics and Bioenergy Laboratory at Unicamp. She works on two research fronts: (1) Research and Innovation Research, focusing on the economic and socio-environmental impacts of scientific production, especially financed by public funds; and (2) Innovation and Entrepreneurship Ecosystems, analyzing the activities of Unicamp’s Innovation Agency (INOVA) and its role in mapping business activities linked to the university. Her work seeks to understand how academic research and university-business interactions contribute to sustainable development and the strengthening of innovation in Brazil.<\/p>\n

 <\/p>\n

About Carlos Graziani\"Carlos<\/a><\/h3>\n

Carlos Graziani holds a degree in Systems Analysis and Development from the Faculdade de Ci\u00eancias Sociais Aplicadas de Extrema (2014). He has experience in network design, configuration, and implementation; web development with PHP, MySQL, and jQuery; and is support analyst.<\/p>\n

 <\/p>\n

 <\/p>\n

Translated from the original in Portuguese<\/a> by Lilian Nassi-Cal\u00f2.<\/p>\n","protected":false},"excerpt":{"rendered":"

Research published in SciELO transcends academic boundaries and influences public policy on a national and global scale. With thousands of citations in documents from organizations such as WHO, IPEA, and national governments, the study reveals SciELO’s strategic role in social valorization of science. …<\/span> Read More →<\/span><\/a><\/span><\/p>\n","protected":false},"author":134,"featured_media":5671,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_links_to":"","_links_to_target":""},"categories":[3],"tags":[29,62,41,7],"class_list":["post-5670","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-analysis","tag-bibliometrics","tag-preprint","tag-public-policies","tag-scholarly-communication"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t