The SciELO journals are being improved by the adoption of classic workflows relating to the online management of article submissions

By Abel L. Packer, Alex Mendonça and Fábio Almeida

The publication of research outcomes is the raison d’être of scientific research. Drafted in texts known as manuscripts (alternately known as papers, and article submissions once received by journals), the description and discussion of research outcomes undergo evaluation by the editorial boards of scientific journals to determine their worthiness for publication. This process of evaluating and refereeing the relevance of article submissions presupposes a process which is carried out by the journal editorial boards in a transparent and ethical manner, with the support of researchers who are qualified in the particular discipline concerned. Being the most important operation which editorial boards carry out in the complex process of communicating research outcomes, it has not been free from flaws and defects and, consequently, criticisms and controversies as well as proposals and innovations for fine-tuning this process. Notwithstanding the criticisms, conflicts of interest and the challenges that this process faces, it is peer-review with its ongoing refinements which has prevailed historically as the system of self-regulation, which has been adopted by the scientific community to attest to the credibility of research outcomes. The Editor-in-Chief and his team of associate editors is responsible for ensuring that this process of self-regulation adheres to the required standards of quality and contributes to the advancement of research in a particular discipline.

Brazilian journals face various kinds of difficulties when it comes to operating the processes for the evaluation of article submissions in accordance with the international state of the art in this area. Amongst theseare the growth in the number of article submissions, the limited availability of high-level peer-reviewers, the length of time taken by these people to carry out their evaluations and the relative importance attached by many to this activity and, even more so, the repeated questioning of the quality of the article submissions as well as their evaluation. In addition to these difficulties which are part and parcel of research and communication systems, the evaluation of article submissions requires the combination of automated management systems which organize the roles of all parties involved and facilitate the processes required for the identification of peer-reviewers, the monitoring of workflows and the production of statistics for these processes with a view to their systematic improvement.

The SciELO Program is fostering the improvement of the management of article submissions for the journals it indexes and publishes by the means of recommendations, solutions and measures which are being translated into criteria and indicators for the approval of the admission and retention of journals in the collection. The objective is to contribute to the maximization of professionalism and as a consequence of this, make a similar contribution to the maximization of the quality of the evaluation of article submissions, eliminate the lack of transparency and minimize errors, conflicts of interest, failures and amateurism. Among these basic measures are: the requirement that journals make available online the structure and composition of their editorial body, and those who are responsible for authenticating the credibility of the research outcomes they publish; that the instructions to authors provide a detailed description of the evaluation process for article submissions; and, that published articles bear the date of submission and approval. By the end of 2015, all journals indexed by the SciELO Program will be required to operate automated systems which have been certified by SciELO for the management of article submissions. This will make it possible for authors and editors to follow-up the evaluation process, and to keep track of the stages in the evaluation process of article submissions, the participation of editors and peer-reviewers and the relevant decisions and recommendations with the appropriate levels of confidentiality. The systems will also be required to optionally permit the levying of author processing charges for those journals which are adopting this payment methodology. Keeping track of these processes will provide editors and the SciELO Advisory Committee with an indication of the performance of the journals as far as the management of article submissions is concerned. At the present time, SciELO has certified the Open Journal System (OJS) and ScholarOne, with the provision that any new systems will have to be certified in the future. Journal editors are free to adopt the system which best responds to their policies and procedures for the evaluation of article submissions. These developments are an integral part of the SciELO action lines which are directed towards the professionalization, internationalization and strengthening of the financial sustainability of the journals themselves.

As an example of the advancements made in improving the management of article submissions, we analyze three classic workflows in the automated management of submissions that have been adopted by the publishers of the journals of the SciELO Brazil collection, and that we will label here as workflows A, B, and C. All the workflows have the Editor-in-Chief (EIC) as the position whose prime responsibility is the management of the evaluation of the submissions, a role which can also be shared with Associate Editors (AE), with greater or lesser responsibility, and supported by one or more colleagues that carry out the function of administrative or executive secretary (Admin). The management of submissions by Editors-in-Chief and Associate Editors can result in immediate rejection for a variety of reasons, or in the collective opinion of peer-reviewers whose recommendations support the final decision on publishing or rejecting the submission.

The following table describes three types of workflows for the processing of article submissions, indicating those who are responsible and the roles they carry out at each stage of the process.

Workflow A generally applies to journals with a small number of submissions whose evaluation process can be managed by the Editor-in-Chief alone. Workflow B delegates the responsibility of managing the submissions to Associate Editors in their corresponding area of expertise.This workflow is recommended for journals that receive a large number of submissions, cover a wide range of topics, and which have proactive Associate Editors. In workflow C, the Editor-in-Chief, as in workflow B, shares the management of the submissions with Associate Editors, but reserves the right to make the final decision.

The set of 57 journals that adopted the ScholarOne system in the last two and a half years as of January 2012 is considered for this analysis. Workflow A was adopted by 11 journals (19%), B by 13 (23%) and C by 33 (58%). This set is dominated by journals in the health sciences and biology, which make up 70% of the total number of journals. This fact may influence the preference for workflow C.

Considering only those journals that have processed at least 50 submissions, we arrive at a set of 54 journals that have received 47,000 submissions since January 2012, with annual averages of 360, 600 and 800 submissions in journals with workflows A, B and C, respectively. The Editors-in-Chief of the journals that have adopted workflow B or C have the support of 19 and 26 Associate Editors respectively registered in the system. Regarding the average distribution of the number of days taken by each journal between submission and the final decision, the median was 53 days and the maximum was 151 days for workflow A, 81 and 168 for B ,and 87 to 184 for C. Workflow C is the most complex and the most utilized by journals that have adopted ScholarOne. Workflow C also stands out by processing a greater number of submissions per year, by operating with a greater number of associate editors and by having a higher average processing time. This predominance of workflow C may change in the future as the online management of submissions becomes widely adopted by the SciELO journals from additional fields of knowledge.

Overall, the results show promise, and support the feasibility of the decision by SciELO to require the journals it indexes to adopt article submission management systems that contribute decisively to organizing the makeup and functions of the editorial board of the journals, and to making the management of the processes of evaluating article submissions more efficient and effective.

Translated from the original in Portuguese by Nicholas Cop Consulting.