Measuring and comparing Brazilian and Latin-American universities in terms of academic and industry related knowledge production

By Antonio Prado and André Luiz Sica de Campos

Image: Ümit Yıldırım.

The so called “third mission” of the Universities is the most recent, but very important task to be performed by modern Universities. It is related to activities developed in partnership with society, including University-Industry links made with the goal of developing new technologies, constituting a possible result from applied research. Those links can help increase the quality of the technological development in a given country, which is very important to support industry to compete in a globalized knowledge economy.

Therefore, it is very important to stimulate those links, because they increase the number of academic institutions working under the so called “triple helix” context, where Universities, Industry and the Government share efforts to develop the economy of the country. It means the creation of “entrepreneurial universities”, which are academic institutions with a strong focus in performing research that are almost immediately applied to existing or new industrial products needed by the society and developed by the “entrepreneurial universities”.

In that sense, to help increase those practical activities, it is very important to develop metrics that can be used to evaluate the quantity and quality of university outputs in terms of academic and industry-related products. Using those metrics, we can verify the evolution of those outputs and compare them among different countries. In this way, we can verify the success of innovation policies applied by different countries and institutions in a relative comparison between academic and industry-related products.

The reason to make comparative measurements is to consider the intensification of knowledge application by the academic sector and to measure the evolution of the focus in pure and in applied research, rather than to indicate a general increase in Universities’ output.

Considering those aspects, the paper Using the “U-Multirank” to Compare the Performances of Brazilian Universities with Latin American Countries in Academic and Industry Products,¹ available in SciELO Preprints, compares the performances of Brazilian Universities, in terms of academic and industry related productions, with other Latin American countries, selected due to shared economical, historical, and cultural aspects. The selected countries are Argentina, Chile, Colombia, Equator and Peru, representing South America, and Mexico and Costa Rica, representing Latin American countries in North and Central America.

Those measurements were based in the grades generated by the U-Multirank from 2017 to 2020. The U-Multirank is an academic multidimensional ranking implemented in Europe in2014. It evaluates academic institutions in five Dimensions (Teaching and Learning, Research, Knowledge Transfer, International Orientation and Regional Engagement). Each of these Dimensions is divided into several Indicators, going from four to eleven depending on the Dimension.

The main goal of the ranking is to provide information to help future students choose an international higher education institution that most closely matches their interests. University candidates can evaluate each Indicator, separately or in groups, in order to check the institutional orientation before reaching a final decision. Further references that can be used to learn more about the ranking are the book “Multidimensional Ranking: The Design and Development of U-Multirank”² and the articles Multidimensional Academic Evaluation using the ‘U-Multirank’³, Performances of the Brazilian Universities in the “U-MULTIRANK” in the Period 2017-2020⁴, and Study of the multidimensional academic ranking “U-Multirank”⁵.

These characteristics not only make this ranking very adequate for the comparisons proposed here, but they also render this ranking a suitable model for the evaluations of the graduate Brazilian Programs performed by CAPES (Coordination for the Improvement of Higher Education Personnel, Coordenação de Aperfeiçoamento de Pessoal de Nível Superior). It has many of the features desired by the multidimensional evaluation proposed by CAPES. This new evaluation is a “hot topic” for debate in Brazil, so learning about multidimensional rankings is currently relevant.

To implement this study, we calculated at the average performances of all the Brazilian Institutions listed in the U-Multirank during the years of 2017 to 2020 in which 35 Indicators that were used. Studying these Indicators, we identified that four of them can be considered related to academic production: Citation rate, Research publications (absolute numbers), Research publications (size-normalized) and Top cited publications.

There are also four Indicators that can be used to measure the output of “University-Industry” relations: co-publications with industrial partners, patents awarded (absolute numbers), patents awarded (size-normalized), publications cited in patents and co-publications with industrial partners. The present study concentrates in these eight Indicators. The definition of these indicators can be found in the U-Multirank Indicator Book (2021)⁶ (accessed by the authors in February 11^th2022).

The results presented here show that Brazilian Universities present robust indicators in terms of publication quantity when compared to quality, with results above world averages in quantity, but below world averages in the Indicators “Citation Rate” and “Top-cited” publications. In spite of this, there is a positive point here, because there’s been an improvement in recent years for the “Citation Rate” Indicator. Looking at South American countries, Chile achieved the best results regarding citations in both Indicators.

In terms of patent numbers, Brazilian Universities are lagging behind world averages in terms of absolute and size normalized numbers, but the differences are much larger in absolute numbers. This Indicator showed an important increase in 2020, when Brazil reached a number just above world average for the first time in the years studied.

The “Publications cited in patents” Indicator has numbers that are steady and below world averages every year, but these are close to the results obtained in the “Citation Rate” Indicator, meaning that the importance of Brazilian science regarding academic and industry related products are similar. Chile is the best performer in all the Indicators regarding Industry-related products.

Of course, this is a preliminary study and further observations should be made in this topic, looking at more countries, longer timeframes, and other aspects related to innovation products. Anyway, it offers some guidelines to follow and which aspects seem to be more important to look at.

Notes

1. PRADO, A. and CAMPOS, A.L.S. Using the “U-Multirank” to Compare the Performances of Brazilian Universities with Latin American Countries in Academic and Industry Products. SciELO Preprints. 2022 [viewed 09 March 2022]. https://doi.org/10.1590/SciELOPreprints.3568. Available from: https://preprints.scielo.org/index.php/scielo/preprint/view/3568/version/3779

2. VAN VUGHT, F.A. and ZIEGELE, F. (ed.). Multidimensional Ranking: The Design and Development of U-Multirank. New York: Springer, 2012.

3. PRADO, A. Multidimensional Academic Evaluation using the ‘U-Multirank’. SciELO Preprints. 2021 [viewed 09 March 2022]. https://doi.org/10.1590/SciELOPreprints.2661. Available from: https://preprints.scielo.org/index.php/scielo/preprint/view/2661/version/2813

4. PRADO, A. Performances of the Brazilian Universities in the “U-MULTIRANK” in the Period 2017-2020. SciELO Preprints. 2021 [viewed 09 March 2022]. https://doi.org/10.1590/SciELOPreprints.2351. Available from: https://preprints.scielo.org/index.php/scielo/preprint/view/2351/version/2490

5. PRADO, A. Study of the multidimensional academic ranking “U-Multirank”. SciELO Preprints. 2021 [viewed 09 March 2022]. https://doi.org/10.1590/SciELOPreprints.3063. Available from: https://preprints.scielo.org/index.php/scielo/preprint/view/3063/version/3234

6. U-MULTIRANK. U-Multirank Indicator Book [online]. U-Multirank. 2021 [viewed 09 March 2022]. Available from: https://www.umultirank.org/export/sites/default/press-media/documents/Indicator-Book-2021.pdf

References

CLARK, B.R. Creating entrepreneurial universities: organizational pathways of transformation. Oxford: International Association of Universities/Pergamon Press Inc., 1998.

DE CAMPOS, A. and DA COSTA, J. P. Governança de redes e formalização da pesquisa e desenvolvimento (P&D): novos fatores para a análise da relação universidade empresa. In: VESSURI, H., et al. (ed.) Perspectivas latinoamericanas em el estudio social de la ciencia, la tecnología y el conocimiento. México, D.F.: Editorial SigloVeintiuno, 2014.

DE CAMPOS, A.L. A review of the influence of long-term patterns in research and technological development (R&D) formalization on university-industry links. Revista Brasileira de Inovação [online]. 2010, vol.9, pp.379-410 [viewed 09 March 2022]. https://doi.org/10.20396/rbi.v9i2.8649006. Available from: https://periodicos.sbu.unicamp.br/ojs/index.php/rbi/article/view/8649006

LAREDO, P. Revisiting the Third Mission of Universities: Toward a Renewed Categorization of University Activities? High Educational Policy [online]. 2007, vol. 20, pp.441–456 [viewed 09 March 2022]. https://doi.org/10.1057/palgrave.hep.8300169. Available from: https://link.springer.com/article/10.1057/palgrave.hep.8300169

PRADO, A. and CAMPOS, A.L.S. Using the “U-Multirank” to Compare the Performances of Brazilian Universities with Latin American Countries in Academic and Industry Products. SciELO Preprints. 2022 [viewed 09 March 2022]. https://doi.org/10.1590/SciELOPreprints.3568. Available from: https://preprints.scielo.org/index.php/scielo/preprint/view/3568/version/3779

PRADO, A. Multidimensional Academic Evaluation using the ‘U-Multirank’. SciELO Preprints. 2021 [viewed 09 March 2022]. https://doi.org/10.1590/SciELOPreprints.2661. Available from: https://preprints.scielo.org/index.php/scielo/preprint/view/2661/version/2813

PRADO, A. Performances of the Brazilian Universities in the “U-MULTIRANK” in the Period 2017-2020. SciELO Preprints. 2021 [viewed 09 March 2022]. https://doi.org/10.1590/SciELOPreprints.2351. Available from: https://preprints.scielo.org/index.php/scielo/preprint/view/2351/version/2490

PRADO, A. Study of the multidimensional academic ranking “U-Multirank”. SciELO Preprints. 2021 [viewed 09 March 2022]. https://doi.org/10.1590/SciELOPreprints.3063. Available from: https://preprints.scielo.org/index.php/scielo/preprint/view/3063/version/3234

THUNE, T. The training of Triple Helix workers: doctoral students in university-industry government collaborations. Minerva [online]. 2010, vol.48, pp.463-483 [viewed 09 March 2022]. http://dx.doi.org/10.1007/s11024-010-9158-7. Available from: https://link.springer.com/article/10.1007/s11024-010-9158-7

U-MULTIRANK. U-Multirank Indicator Book [online]. U-Multirank. 2021 [viewed 09 March 2022]. Available from: https://www.umultirank.org/export/sites/default/press-media/documents/Indicator-Book-2021.pdf

VAN VUGHT, F.A. and ZIEGELE, F. (ed.). Multidimensional Ranking: The Design and Development of U-Multirank. New York: Springer, 2012.

About Antonio F B A Prado

Graduated in Physics and Chemical Engineering from USP, Master in Space Sciences from INPE, Master and PhD in Aerospace Engineering from University of Texas at Austin. Member of the “International Academy of Astronautics” and “Associate Fellow” of the American Institute of Aeronautics and Astronautics. CNPq researcher PQ1A. Recognized as one of the most cited researchers (2% higher) in the years 2019 and 2020 by Elsevier and Stanford University. Past activities: Member of the Astrodynamics Committee of the “International Astronautical Federation”, Chairman of the CNPq Advisory Committee on Mechanical, Naval and Oceanic Engineering and Aerospace Engineering, Pro-Rector of INPE.

About André Luiz Sica de Campos

Dr. André Campos undertook undergraduate and master degree studies at Unicamp State in Brazil in Economics and Science and Technology Policy respectively. In the latter field, he obtained his Doctor of Philosophy degree at Sussex University. Following a period of postdoctoral activity in England, he joined Unicamp State, where he teaches public management undergraduates and post-graduates in management and Science and Technology Policy. He published over 35 peer-reviewed articles and book chapters in journals such as Human Relations and Social Studies of Science. His research has been funded by CNPq, ESRC and Capes.