Tag: Research Evaluation

Scientific rigor and open science: ethical and methodological challenges in qualitative research

The literature demonstrates growing criticism of the reliability of qualitative research, including claims that it lacks rigor and methodological clarity. In the publication system, several actions reflect this increased attention to rigor. Initiatives by major research funding agencies also emphasize rigor. The U.S. National Science Foundation (NSF), for example, has fostered efforts to promote strategies to increase rigor and transparency in the reporting of results of qualitative research. Here, we offer a brief panorama, permeated by transformations that include increasing initiatives to promote open science. We explore some questions about the current discussion of scientific rigor, not only in publications, but also in proposing qualitative research projects. Read More →

Towards a more open Soil Science

Most of the data resulting from research conducted in Brazil is not yet available in open access repositories. Here, we urge soil scientists to adopt a more open stance towards research data in the area, aiming to increase science sustainability and foster scientific collaboration. Read More →

Bibliometrics: a new threat to zoological taxonomy?

One of the backbones of the life sciences, taxonomy —the science of biodiversity—suffers from multiples impediments, including the use of bibliometric indexes by organizations. These days such indexes play a big role in the scientific decisions. However, what for-profit companies, which own these bibliometric platforms, have to do with science? Read More →

The path to reproducibility tests is through Registered Reports

The need to reproduce research results for the sake of science transparency and credibility goes through numerous challenges. An article published in Nature indicates that, in order to obtain better results from reproducibility tests, it is important to establish protocols in agreement with the authors of the original study and to align expectations. Registered Reports, submitted to peer review before the experimental stage of the study, present themselves as a promising solution for successful reproducibility tests. Read More →

JCR, Kafka, and MAI!

For the second time, the Anais da Academia Brasileira de Ciências was surprised to find out that about one quarter of the papers of an issue were gone missing from the Jornal Citation Reports platform, a problem that should come to the attention of the publishing and evaluation industry. Some objective ways to appraise the performance of specific volumes or areas of a journal are presented. Read More →

Why does reform of scientific communication seem so difficult and slow?

The world faces global problems for which science is needed as part of the solution. Yet the scientific communication system is not nearly as quick and open as necessary for efficient worldwide collaboration. The powers that are in a position to reform the system are too timid. Will crises like COVID-19 shake them awake? Read More →

Peer review is not just quality control, it is part of the social infrastructure of research [Originally published in LSE Impact of Social Sciences blog in June/2019]

The purpose of peer review is often portrayed as being a simple ‘objective’ test of the soundness or quality of a research paper. However, it also performs other functions primarily through linking and developing relationships between networks of researchers. In this post, Flaminio Squazzoni explores these interconnections and argues that to understand peer review as simply an exercise in quality control is to be blind to the historical, political and social dimensions of peer review. Read More →

Working to the rule – How bibliometric targets distorted Italian research [Originally published in LSE Impact of Social Sciences blog in November/2019]

As Goodhart’s law states: “when a measure becomes a target, it ceases to be a good measure”. Using bibliometrics to measure and assess researchers has become increasingly common, but does implementing these policies therefore devalue the metrics they are based on? In this post Alberto Baccini, Giuseppe De Nicolao and Eugenio Petrovich, present evidence from a study of Italian researchers revealing how the introduction of bibliometric targets into the Italian academy has changed the way Italian academics cite and use the work of their colleagues. Read More →

A look at peer review of grant proposals

The scholarly peer review tracking platform Publons has launched the Grant Review in Focus on project assessment and identification for funding. Four thousand and seven hundred researchers were interviewed as well as data extracted from Web of Science. Read More →

Google Scholar, Web of Science or Scopus, which gives us better indexing coverage?

Recent analysis of the indexing coverage of the Google Scholar, Web of Science, and Scopus databases shows that higher values do not always mean higher quality or better indexing, as the inclusion of more lower-quality or lower-impact documents may reflect in other aspects of the analysis and, depending on the type of assessment, it should be necessary to eliminate certain types of documents from the citation count. Read More →

The 2019 Workshop on Open Scientometric Data Infrastructures at Leiden University [Originally published in the CWTS blog in August/2019]

The Open Scientometric Data Infrastructures Workshop took place at CWTS on 28 February and 1 March 2019. Over the course of two days, 14 researchers from CWTS and other research institutes and universities came together to discuss current projects and initiatives regarding open scientometric data infrastructures. This blog post provides a summary on the presentations and discussions held at the event. Read More →

Mapping the impact of UN Sustainable Development Goals on global research [Originally published in LSE Impact of Social Sciences blog in May/2019]

The UN Sustainable Development Goals (SDGs) represent one of the largest and most sustained influences on global research to date. However, charting the effect of these 17 goals on the global research community is a complex task. In this post, Martin Szomszor draws on the findings of a recent bibliometric study to produce a ‘citation map’ of sustainability research, which highlights how the UN SDGs have enabled the development of new areas of transdisciplinary and international collaboration in research. Read More →

The role of non-Brazilian contribution in the publishing performance of psychology journals in Brazil

An examination of publishing performance among psychology journals in Brazil finds higher publishing performance associated with non-Brazilian contribution, in terms of: authors and editorial board members from English-speaking countries; as well as collaboration with authors from English-speaking countries. Implications are discussed for editors and publishers, as well as arbiters of public policy. Read More →

The absurdity of the same requirement for law that the rest of the scientific publications

Bibliometric indexes (e.g., WoS/Scopus), normally used for hard sciences and even social sciences, should not be used as a parameter for law research in the same way, as it does not respond to the same extent to measure quality or productivity of research in this field. Text available only in Spanish. Read More →

Implications of SciELO in the history of science coverage in LA&C

Historical data on the presence of America/Latin America in bibliographic sources were used as references to review the implications of SciELO in the recent coverage of scientific journals in the region. Three scenarios were mentioned. Expansion of modern science (journals and catalog of the Royal Society, XVII-XIX centuries); hegemony of the Anglo-Saxon citation indexes (20th century); diversification of coverage and type of indexes (21st century). Through the citation geography visible in SciELO, signs of changes can be seen in the specialization of regional journals as cited sources; we believe that this trend will continue in journals covering regional issues. Text available only in Spanish. Read More →