The Qualis system: a perspective from a multidisciplinary journal [Originally published as the editorial in Anais da Academia Brasileira de Ciências vol. 89 no. 3]

By Alexander W.A. Kellner, Laboratório de Sistemática e Tafonomia de Vertebrados Fósseis, Departamento de Geologia e Paleontologia, Museu Nacional/UFRJ, Rio de Janeiro, RJ, Brazil

There is quite an amount of literature out there regarding the Qualis system and one wonders what someone can write about it that has not been discussed already. In this short essay, I will show the perspective of a multidisciplinary journal that has to live (or survive) in the very complex environment created by Qualis and the generated deleterious side effects for periodicals with broad scopes like that of the Annals of the Brazilian Academy of Sciences (AABC).

The Qualis system was introduced in 1998 by CAPES (Coordenação de Aperfeiçoamento de Pessoal de Nível Superior), an agency of the Brazilian government whose purpose is to evaluate and foster the Brazilian graduate programs. Right from the beginning, it was clear that one of the major aspects of a graduate program are publications by both, researchers and students. Since the amount of produced papers is high, a system had to be developed to discriminate this production. This is how Qualis got into the picture. Essentially, this system classifies publications in strata that are further used to evaluate a particular graduate program. This system has been changed along the years, and presently there are eight levels: A1 and A2, the highest; B1 to B5; and C, the lowest that does not add anything for the evaluation process. So far so good, if it were not for a small detail: the percentage of A1 + A2 cannot be more than 25% of all periodicals evaluated. Moreover, the percentage of A1 + A2 + B1 cannot be over half the periodicals that are taken into consideration in a given year. I will come back to this later.

As known, the AABC is the sole periodical produced in Brazil that can be regarded as truly multidisciplinary, publishing articles from physics and mathematics, to paleontology and scientometrics, passing through the description of new species (biodiversity), research of new chemical compounds, studies that could lead to the cure of diseases, and new methodologies applied to engineering, just to cite a few. No wonder our journal appears in several different areas evaluated by CAPES, that are now slightly less than 50. Since 2010, the AABC is listed in 28 (2011) up to 39 (2016) areas with quite some variation regarding the strata. For example, in the last Qualis list (2016), the classification of the AABC fluctuates from A2 to B5. Just a reminder: B5 is close to nothing in the present evaluation system.

When the Qualis classification started to focus on the Impact Factor (IF) to rank journals in the respective strata, there was an immediate concern of Brazilian periodicals that had little chances to compete with others abroad. The fear was that the contributions with a higher impact potential produced by Brazilian scientists would fade away from the editorial surface of journals produced in the country. Aware of that, CAPES allowed each area to choose a small selected number of national periodicals considered relevant for the respective area to be classified in higher strata. For some time, this was also followed by a substantial increase in funding, helping several periodicals to rise. This inductive method was very welcome and attenuated the problems created by the Qualis system. But not for the AABC.

Along the years and with different editors-in-chief, the AABC has made great efforts to improve the journal. Since 2008, the main goal was to create the necessary conditions to increase the amount of peer-reviewed published manuscripts of good quality, that changed from 61 in 2008 to 200 (not including editorials) last year (2016; Fig. 1a). This number is still far too low when compared with the more than 3000 published by PNAS (Proceedings of the National Academy of Sciences of the United States of America) and we are presently aiming to reach 500 in the near future. Despite difficult economic times, we managed to introduce several improvements, such as the possibility to publish color pictures in the printed version for low cost (free for the online version), color cover, publishing papers ahead of print, newsletters, special volumes and, for all, a new online system. The latter was made available by SciELO in November of 2012, resulting in a dramatic increase of submissions (Fig. 1a).

Figure 1 – Graphs showing the annual balance between 2008 and 2016 for papers received, rejected and published in the AABC. a – total values for all AABC sections; b – total values for Physical Sciences; c – total values for Biological Sciences; d – total values for Agrarian Sciences.

The submission rate varies from field to field. By far, the lowest number of articles received by this journal comes from Physical Sciences. In the last eight years, the AABC has received from one to a maximum of five manuscripts per year (Fig. 1b). In 2011 and 2013 we did not publish a single paper of this basic scientific field. Screening through the Qualis system, the best classification of the AABC in Physics (actually Astronomy and Physics) was B3, but most of the times this journal was not listed, even when manuscripts were published. Honestly, I keep wondering when this important area of science is going to become extinct in our journal.

It is well known that the Qualis system affects submission rates, but not many empiric data was made available by the journals. The AABC has also felt this effect, what can be exemplified in two areas: Biological and Agrarian Sciences. Most of the manuscripts we receive in Biological Sciences fall in what CAPES regards as Biodiversity, which unites areas like ecology, zoology and botany. This area was always the one with the highest submission rate. Not anymore. In 2013 the AABC was downgraded from A2 to B2. The effect on the submission rate was not immediate, particularly since we had a special volume in production. But in 2016, the number of submitted manuscripts went down from 355 (received the year before) to 274 (Fig. 1c). There is indication that another drop should be expected for this year.

On the other hand, Agrarian Sciences was always well classified in the Qualis system. It is in the higher strata (A2) and therefore submissions are increasing to a point that last year it surpassed Biological Sciences (Fig. 1d). Considering the submission rate history of the AABC, this is a highly unexpected result and there seems little doubt that the Qualis system is behind it.

Lets discuss the downgrade the AABC suffered in Biodiversity in 2013, from which it never recovered. What was the reason for that? The main criteria for classifying periodicals in a certain strata was the IF (despite the known problems of this number, see discussion in Casadevall, et al. 2016¹). With the customary ups-and-downs of this value, the AABC would have fallen from the strata A2 to B1 – which is still quite attractive to authors in Brazil! But, since A1 + A2 + B1 cannot be over 50% of the total periodicals evaluated in that year, the evaluation committee of the Biodiversity area had to make some “tough choices”. Therefore, in order to produce less “harm”, periodicals where less researchers of the area have published papers, were artificially downgraded to comply with the 50% rule of CAPES. In other words, although the AABC had the numbers for a better classification, this periodical has published less papers in Biodiversity that year compared to others, and was “sacrificed” for the “overall good” of the area. Of course, the AABC, being a multidisciplinary journal, can never publish the same amount of papers of journals with a more specific focus! In another situation I have witnessed something similar happening when I was on the CAPES evaluation committee – needless to say the bad feeling that this has caused.

The 25% and 50% rules of CAPES have been seriously questioned for some time, but still apply. The attempts to mitigate the unwanted problems caused by the Qualis system have never reached the AABC. Understandable, each time an area has the possibility to select periodicals for any benefit, it will always choose the ones that publish more articles in that specific area. No way the AABC can compete.

In the review published by Barata (2016)², some criticisms of the misuse of the Qualis system were presented. However, it must be regarded as naive not to have foreseen that this would happen in several circumstances, including promotions. For example, Barata (2016)² criticizes editors for using the Qualis ranking to get more funding for their journals. However, the last grant announcement for funding possibilities for journals of both Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) (Chamada MCTIC/CNPq Nº 25/2016) and CAPES (Edital N° 13/2016) clearly state that, to be able to apply, the periodical must be at least classified at the B2 level of the Qualis list. Such contradictions show the complexity of the situation.

Although there is a general recognition of the difficulties imposed by the Qualis system to multidisciplinary journals, nothing was done so far. Due to its nature and size, the AABC does not offer competition to any journal published in Brazil, since we would never be able to publish all the good work that one specific area produces. Therefore, an affirmative action should be taken in the case of the AABC regarding the Qualis system. This is the only way by which the country, that suffers from “Qualisitis” (= obsession in using the Qualis system for several evaluations, inducing the decision for authors where to submit their manuscripts) might aspire to publish a stronger multidisciplinary periodical. If such an action is not done to foster the journal of the Brazilian Academy of Sciences, then for which other?

Notes

1. CASADEVALL, A., et al. ASM journals eliminate Impact Factor information from journal websites. J Clin Microbiol [online]. 2016, vol. 54, no. 9, pp. 2216-2217 [viewed 30 September 2017]. DOI: 10.1128/JCM.01418-16. Available from: http://jcm.asm.org/content/54/9/2216.full

2. BARATA, R.C.B. Dez coisas que você deveria saber sobre o Qualis. Revista Brasileira de Pós-Graduação [online]. 2016, vol. 13, no. 30, pp. 13-40, ISSN: 2358-2332 [viewed 30 September 2017]. DOI: 10.21713/2358-2332.2016.v13.947. Available from: http://ojs.rbpg.capes.gov.br/index.php/rbpg/article/view/947

References

ANDRADE, J.B. and GALEMBECK, F. Qualis: Quo Vadis? Quím. Nova [online]. 2009, vol. 32, no. 1, pp. 5. ISSN: 1678-7064 [viewed 30 September 2017]. DOI: 10.1590/S0100-40422009000100001. Available from: http://ref.scielo.org/syf44r

BARATA, R.C.B. Dez coisas que você deveria saber sobre o Qualis. Revista Brasileira de Pós-Graduação [online]. 2016, vol. 13, no. 30, pp. 13-40, ISSN: 2358-2332 [viewed 30 September 2017]. DOI: 10.21713/2358-2332.2016.v13.947. Available from: http://ojs.rbpg.capes.gov.br/index.php/rbpg/article/view/947

CABRAL FILHO, J.E. The CAPES Qualis and beyond. Rev. Bras. Saúde Matern. Infant. [online]. 2010, vol. 10, no. 4, pp. 403-406, ISSN: 1806-9304 [viewed 30 September 2017]. DOI: 10.1590/S1519-38292010000400001. Available from: http://ref.scielo.org/j5y4mc

CASADEVALL, A., et al. ASM journals eliminate Impact Factor information from journal websites. J Clin Microbiol [online]. 2016, vol. 54, no. 9, pp. 2216-2217 [viewed 30 September 2017]. DOI: 10.1128/JCM.01418-16. Available from: http://jcm.asm.org/content/54/9/2216.full

FERREIRA, R.C., ANTONELI, F. and BRIONES, M.R.S. The hidden factors in impact factors: a perspective from Brazilian science. Frontiers in Genetics [online]. 2013, vol. 4, ISSN: 1664-8021 [viewed 30 September 2017]. DOI: 10.3389/fgene.2013.00130. Available from: https://www.frontiersin.org/articles/10.3389/fgene.2013.00130/full

ROCHA-E-SILVA, M. O novo Qualis, que não tem nada a ver com a ciência do Brasil. Carta aberta ao presidente da CAPES. Clinics [online]. 2009, vol. 64, no. 8, pp. 721-724, ISSN: 1980-5322 [viewed 30 September 2017]. DOI: 10.1590/S1807-59322009000800002. Available from: http://ref.scielo.org/2wq6qy

SANTOS, M.A. Paidéia consagra-se como revista A1 no Qualis Periódico CAPES. Paidéia [online]. 2012, vol. 22, no. 51, pp. 1-4, ISSN: 0103-863X [viewed 30 September 2017]. DOI: 10.1590/S0103-863X2012000100001. Available from: http://ref.scielo.org/cktm5g

To read the editorial, go to:

KELLNER, A.W.A. The Qualis system: a perspective from a multidisciplinary journal. An. Acad. Bras. Ciênc. [online]. 2017, vol. 89, no. 3, pp. 1339-1342, ISSN: 1678-2690. DOI: 10.1590/0001-37652017893. Available from: http://ref.scielo.org/p5h8w5

About Alexander W.A. Kellner

Full Professor and member of the Academia Brasileira de Ciências, he is Editor-in-Chief of AABC. He holds a doctorate from Columbia University (New York) and works at the Museu Nacional-UFRJ, for which he was elected director (2018-2022). His research interests include Paleontology, especially the evolution and diversification of fossil reptiles, mainly dinosaurs and pterosaurs (winged vertebrates). Besides several Brazilian regions, he carried out field activities in countries such as Antarctica, Iran, China, Argentina and Chile.