How to reduce the processing time of submitted articles: the experience of Revista Brasileira de Ciências Sociais

By Bruno Wilhelm Speck, Department of Political Sciences, Universidade de São Paulo, São Paulo,SP, and Research fellow, CNPq, Brazil

The delay in favor of quality

Image: Katerina Limpitsouni via unDraw.

When we submit an article to a journal, we are eager to receive some feedback, preferably positive, but at least an answer. Because the worst of all worlds is to have an article submitted that never leaves the evaluation phase.

When the author becomes an editor, the perspective changes. Suddenly there are many reasons why manuscripts are delayed. Good reasons, in most cases. Sometimes the initial submission is incomplete. Often authors also do not submit manuscripts in anonymous format. Everything causes delays. The search for reviewers willing to evaluate the article can be time-consuming, also because the texts often deal with rather specific topics. Reviewers often find it difficult to deliver their assessment, because the review tends to be overlooked in the midst of many other duties. Moreover, because reviewers are the anonymous heroes who guarantee the quality of the evaluation. Often the reviews are too short or do not converge with each other, which results in the need of including another colleague for an additional review. Other weeks go by. When the article returns to the author, it takes time for her to present a reformulated text. There are plenty of reasons, all fair. The semester is ending, another project needs attention, the revisions demand more work than expected. The ping-pong between author and reviewer can also be prolonged due to disagreements on basic questions. Editors need to come up with a solution. Finally, the articles eventually approved enter the production line. Now they are proofread by paid professionals or, once again, volunteers, return to the authors for approval, go through diagramming, to return again to the authors for approval and finally are indexed and published. And always in this exchange between authors, reviewers, reviewers and production the manuscript passes through the hands of the journal editors, who makes other quality assessment.

When delay turns into procrastination

What I wish to emphasize is that in the odyssey from submission to rejection or publication of the article, there are many actors and many good reasons to justify why the process sometimes is dragged on for months or years. But there are also factors in favor of procrastination that can be eliminated. Here I deal with what lays in the hand of the journal, how editors and the secretariat of the journal can contribute to ensure the an evaluation process without delay, without losing sight of the most important criterion, which is to publish articles of high quality.

Uncomfortable with the evaluation deadlines, we took four steps to accelerate the evaluation process in the Revista Brasileira de Ciências Sociais (RBCSoc), one of the most important journals in the field of social sciences in Brazil.

The four measures are:

1. The expanded editorial committee which now directly operates Open Journal Systems (OJS).
2. We increased the number of external reviews requested initially.
3. We changed the way of following-up on the articles under evaluation.
4. We committed to maximum evaluation deadlines.

The editorial committee operates the OJS

The incorporation of area editors was essential to ensure close monitoring of a relatively large number of articles. We receive approximately 20-25 new submissions monthly. The incorporation of 4 colleagues as area editors allowed for important changes. The editorial committee, composed of the editor-in-chief and area editors, manages the entire processing of articles in the OJS system directly, without intermediation by the journal’s secretary. The impact was immediate. We freed up time that the secretariat now devotes to other matters, linked to the production of approved articles. We use the OJS system for collegiate decisions. Each text in process has an area editor as the main responsible person who makes suggestions of editorial decisions to the editorial committee. And we saved time by eliminating the step of inserting referrals (including reviewers, contacting authors, checking forwarded documents) in the OJS system by the secretariat.

Speed up external evaluation without losing reviewers

We changed the procedure of requesting external reviews. The journal works with double-blind peer review, using at least two external reviews for each text, once it has passed the desk review. The previous policy was to initially request two reviews and, once the external reviewer failed to deliver, eventually include other reviewers. This caused long delays.

Although every decision to approve (or not) an article continues to be made based on at least two reviews, we have changed the management of reviews. We increased the initial number of reviews to four or five, which guaranteed us celerity and also enabled us to work around problems in case of delays, the quality of the review, or divergent reviews.

Still in the interface with the reviewers, we try to make the work less heavy for these external collaborators. Without the clear identification of changes in the revised manuscripts by the authors, a second and third reading by the reviewer become painful exercises. To make the job easier, we changed the focus of the reevaluation to the response letter to the reviewers. It became the first tool to reevaluate the changes made, thus reducing the workload and time spent with reevaluation. Additionally, the reformulated texts must contain revision marks, making it easier to identify the changes made.

Generate up-to-date information on errors and successes in the processing deadlines

We introduced and improved a monitoring tool to follow up on the articles, external to the OJS management system. The management of the articles tends to be considered a secondary issue, and in fact does not deserve more attention when there are only one or two dozen texts being processed at the same time. As this volume increases, it becomes difficult to set priorities and stick to them. With a hundred or more articles running simultaneously and at various stages of evaluation, some are almost certain to be overlooked. The submission systems (OJS and ScholarOne) do not offer satisfactory solutions to this problem.

We have developed a tool (using Excel) that allows just that: to monitor the processing of articles daily and over time. In this file, the basic data of the articles are initially recorded. Then the data are updated in four stages: after the desk review, after the first decision based on external reviews, after the last decision on the publication of the text and after publication. The dates entered in the system refer to the moment when the author receives the feedback from the journal (and in the last case the date of the effective publication).

This tool allows us to monitor the processing stage of each manuscript at any time and takes stock of the development of the journal’s processing deadlines over the years. This helps a great deal to identify and monitor the most problematic cases, in addition to providing a detailed diagnosis whether the journal is going in the right direction in terms of processing deadlines.

Commitment to desirable, acceptable and problematic deadlines

Without being clear to ourselves what we consider desirable, acceptable or going over the limits, it would be difficult to face any problems. To better define what we consider problematic, we have identified four benchmarks for publication deadlines that should not be exceeded. The first is not to spend more than one month for the desk review, the second does not exceed three months until the first answer based on external reviews, the third is to comply with a maximum limit of six months until the final decision on any article and the fourth is to publish articles approved in a maximum of nine months. All deadlines count from the date of manuscript original submission. Initially, these benchmarks were a commitment made internally. In mid-2020 we included the benchmarks in the journal’s editorial policies, making public the commitment.

Photography: the current situation of articles submitted

Now we are going to present some results of the management of the processing deadlines of the texts, using graphics extracted from the tool mentioned above. The tool also allows you to see the journal’s evolution over time. The Revista Brasileira de Ciências Sociais (RBCSoc) processed 503 articles in 2 years (our observation window here). Virtually one submission per business day.

Graph 1 shows the number of articles per submission month. There is an average of 20-25 articles submitted each month. The oscillations are large. The graph also shows the current stage of the articles (date of extraction: September 3, 2020). Most of the texts are either published (black) or rejected (gray) at some point in the process. Articles depicted in green have been approved but are still in production. The articles shown in red and orange represent manuscripts that are in the first or second round of evaluation by external reviewers, respectively. There are no articles in desk review here, because on September 3 texts submitted until June 2020 had already gone through desk review.

Graph 1. Current stage of submitted texts, by month of submission

The graph shows that there are still problematic cases. If we had met the goal in the past, there would be no articles in production (green) from September 2019 back, nor would there be any text in the second evaluation from December 2019 back and so wouldn’t there be any text in the first evaluation before April 2020. But we introduced the changes just to solve a gap. Did we get closer to the desired situation? We will answer this question in the following paragraph.

Movie: the evolution of the average evaluation time

The tool also allows to assess the evolution of processing time over time, which can be seen in the following graphs. In general, we can see that there was a significant improvement in relation to the processing time in all stages. We must warn that for some periods the balance is not yet consolidated, as there are still articles in process, as can be seen in Graph 1. Mainly, the data from the last two stages (final approval and publication) must undergo changes for the first two quarters of 2020. The graphs correspond again to the moment of extraction (September 3, 2020).

Graph 2 shows that the average desk review time has dropped significantly. Maybe it wasn’t even the biggest problem. Today we managed to process the manuscripts in approximately 10-15 days.

Graph 2. Time elapsed until response on the result of the desk review (benchmark: 30 days)

Graph 3 allows us to identify the drop in the elapsed time, until we can give the first answer based on external reviews. This celerity is the result of an increase in the number of reviews requested.

Graph 3. Time elapsed until the first response based on external review (benchmark: 90 days)

Graph 4 illustrates that the most difficult phase remains the second and, eventually, the third round of evaluation. The willingness of reviewers to dedicate themselves to a second or third reading naturally decreases. The emphasis on the response letter to the reviewers may have helped to reduce processing time. Here we come close to the evaluation target, but there is still room to shorten the deadlines until the final evaluation.

Graph 4. Time elapsed until final answer based on external review (benchmark: 180 days)

Graph 5 on the time from submission to publication shows that the journal is still only approaching its goal. Only the latest publications managed to get close to the nine months of time that we aim for as acceptable.

Graph 5. Time elapsed until publication (benchmark: 270 days)

We must add that the problems in our count are cumulative. Any delays in the first stages will have an effect on the entire process of processing the article. But the purpose is to create more predictability for authors as to the deadlines that they must consider when submitting articles in the journal. After all, we demand authors exclusivity of submission.

One of the challenges of managing scientific journals with a large volume of submissions is to monitor the processing of manuscripts. In addition to day-to-day management, editors need an instrument to evaluate their performance from a the long-term perspective. Based on the OJS platform, editors can hardly answer questions about the average processing time of articles in their different phases. Consequently, without answers to these questions, they will hardly get involved in initiatives to reduce the processing time of the texts.

RBCSoc’s experience to face this challenge can serve as a guideline for other journals.

In addition to the policies indicated above, we share the Excel tool, which was essential for us to identify the most recurring problems at each stage and allowed us to monitor and manage the journal with more focus. Only through this detailed monitoring we managed to to face the journal’s main problems more objectively and efficiently.

Below, you can find a series of shor tutorial videos of the article processing monitoring tool, available in Portuguese only.

External links

ORCID – Bruno Wilhelm Speck <http://orcid.org/0000-0001-5085-6825>

Template of the article processing monitoring tool <https://drive.google.com/file/d/1dbVxy3NgNoByGFT0QAoSRToRzdbmQ29K/view?usp=sharing>

Translated from the original in Portuguese by Lillian Nassi-Calò.