{"id":1755,"date":"2015-03-20T10:20:10","date_gmt":"2015-03-20T13:20:10","guid":{"rendered":"http:\/\/blog.scielo.org\/en\/?p=1755"},"modified":"2016-01-11T15:26:22","modified_gmt":"2016-01-11T17:26:22","slug":"could-grant-proposal-reviews-be-made-available-openly","status":"publish","type":"post","link":"https:\/\/blog.scielo.org\/en\/2015\/03\/20\/could-grant-proposal-reviews-be-made-available-openly\/","title":{"rendered":"Could grant proposal reviews be made available openly?"},"content":{"rendered":"

By Lilian Nassi-Cal\u00f2<\/strong><\/p>\n

\"Photo:<\/a>

Photo: AJC1<\/a>.<\/p><\/div>\n

Recently there have been many discussions, including in this blog, about the peer review process, the most important step in the process of scientific journal publishing, which gives articles quality and credibility.<\/p>\n

Many authors advocate the publication of reviews openly as comments following the article, claiming that this practice could help controlling fraud in the evaluation process, while allowing public recognition of referees.<\/p>\n

More recently, researchers have discussed, also through social networks, what would be the impact of making the peer review process more open and transparent. Furthermore, the discussion was expanded to include the review of research grant proposals. This topic raises the attention of both the scientific community and civil society. For researchers, to ensure fairness and objectivity on the process and to the general public (taxpayers) to ensure that projects are approved on the merits and not for political reasons or to favor third parties, increasing society’s trust in scientific research.<\/p>\n

A paper by Chris Woolston in Nature1<\/sup> analyzes the impact of two recently published articles on the opening of grant reviews and more, the possibility of changing the evaluation of proposals after publication of the results.<\/p>\n

Scholars have difficulties to understand why some research funding proposals are approved, while others are not. Studies point out several inconsistencies and low reproducibility in the grant review process. Referees assign different weights to aspects of the proposals such as methodology, originality and feasibility of the projects.<\/p>\n

To address these disparities, Daniel Mietchen, an evolutionary biologist at the Museum of Natural History in Berlin, Germany, sustained in a publication in PLoS Biology2<\/sup>, among other things that approved grant applications should be made publicly available. Comments on Twitter following the publication either supported or discouraged the initiative, for several reasons. The author of the study supports his hypothesis that opening the reviews can transform science as a whole based on the fact that researchers can compare their proposals to their colleagues\u2019, besides opening prospects for cooperation.<\/p>\n

Mietchen\u2019s study results from a transparency survey on financing of scientific projects designed by David Gurwitz and coworkers, also published in PLoS Biology3<\/sup> performed with 27 funding agencies in five continents. The authors concluded that none of them publish the approved proposals final results and 18 of them publish only brief summaries. Generally, agencies publish extensive manuals on how to obtain financing, but refrains from reporting on the proposals \u2013 approved or not \u2013 or decisions on funding.<\/p>\n

Just as in the evaluation process of manuscripts, the careful work of grant assessment is discarded upon the proposal approval or declining. The time employed by the referees in a barely visible and recognized work causes the so-called “reviewers fatigue”, who are increasingly demanded in view of the growing number of projects to analyze. The open availability of reviews would allow the public recognition of their work and help researchers to better prepare their proposals.<\/p>\n

According to the article already mentioned on transparency in project evaluation, there is no need to disclose all the details of the approved projects. It would suffice to report the summary, the impact of the research, the reviewer\u2019s names, summary of the assessments, value of the granted proposal, the agency approval rate, and a final report. However, even disclosing only part of the project data, there is some degree of concern among researchers that it may, somehow, compromise the legitimacy of the evaluation process. The authors consider, however, that peer review will soon require a new dimension in its legitimacy, which might actually be the disclosure of the reviews.<\/p>\n

Another concern relates to the confidentiality of the proposal contents. Gurwitz and coworkers state that the funding agencies consulted in their study prefer to keep confidential certain information from the proposals until the research results are published, to prevent unscrupulous researchers to plagiarize ideas and projects, in such a competitive environment, as scientific research.<\/p>\n

Welcome Trust, a leading biomedical research funding agency currently publishes brief summaries of many of their approved research projects. According to scientific director Kevin Moses, there is no intention to expand that information, in order to respect the confidentiality of researchers and their projects.<\/p>\n

The availability of reviews could pave the way for other types of evaluation, for example the one that occurs after publication, such as non-anonymous post-publication review articles, as suggested by Hilda Bastian, scientific editor of PLoS Medicine and PubMed Commons, an initiative that allows to post comments on PubMed papers after publication. In her opinion, open comments and criticism can improve the quality of research by efficiently exposing mistakes and weaknesses. Lisa Langsetmo, an osteoporosis researcher at McGill University in Montreal, Canada, shares Bastian\u2019s ideas, suggesting a two-step review, “the first being blind and anonymous and the second, post-publication, open and permanent.”<\/p>\n

In Brazil it still prevails the blind (double or single) peer review system for journal articles and research funding proposals, and the idea of making available openly the reviewers comments has not yet raised\u00a0 interest among the scientific community.<\/p>\n

Notes<\/h3>\n

1<\/sup> WOOLSTON, C. What would happen if grant reviews were made public? Nature.<\/em> 2015, vol. 517, n\u00ba 247. DOI: 10.1038\/517247f<\/a><\/span><\/p>\n

2<\/sup> MIETCHEN, D. The transformative nature of transparency in research funding. PLoS Biol<\/em>. 2014, vol. 12, n\u00ba 12. DOI: 10.1371\/journal.pbio.1002027.<\/span><\/p>\n

3<\/sup> GURWITZ, D., MILANESI, E., KOENIG T. Grant application review: The case of transparency. PLoS Biol<\/em>. 2014, vol. 12, n\u00ba 12. DOI: 10.1371\/journal.pbio.1002010.<\/span><\/p>\n

References<\/h3>\n

Editorial ethics \u2013 good and bad scientific practices<\/em>. SciELO in Perspective. [viewed\u00a014 January 2015]. Available from: http:\/\/blog.scielo.org\/en\/2014\/09\/10\/editorial-ethics-good-and-bad-scientific-practices\/<\/a><\/p>\n

Editorial ethics: fraudulent arbitration<\/em>. SciELO in Perspective. [viewed\u00a014 January 2015]. Available from: http:\/\/blog.scielo.org\/en\/2015\/02\/20\/editorial-ethics-fraudulent-arbitration\/<\/a><\/p>\n

GURWITZ, D., MILANESI, E., KOENIG T. Grant application review: The case of transparency. PLoS Biol<\/em>. 2014, vol. 12, n\u00ba 12. DOI: 10.1371\/journal.pbio.1002010.<\/span><\/p>\n

MIETCHEN, D. The transformative nature of transparency in research funding. PLoS Biol<\/em>. 2014, vol. 12, n\u00ba 12. DOI: 10.1371\/journal.pbio.1002027.<\/span><\/p>\n

Paper proposes four pillars for scholarly communication to favor the speed and the quality of science<\/em>. SciELO in Perspective. [viewed\u00a014 January 2015]. Available from: http:\/\/blog.scielo.org\/en\/2013\/07\/31\/paper-proposes-four-pillars-for-scholarly-communication-to-favor-the-speed-and-the-quality-of-science\/<\/a><\/p>\n

PubMed Commons: NLM launches pilot version of open comments on articles<\/em>. SciELO in Perspective. [viewed\u00a015 January 2015]. Available from: http:\/\/blog.scielo.org\/en\/2013\/12\/20\/pubmed-commons-nlm-launches-pilot-version-of-open-comments-on-articles\/<\/a><\/p>\n

Scientometrics of peer-reviewers \u2013 will they be finally recognized?<\/em>. SciELO in Perspective. [viewed 14 January 2015]. Available from: http:\/\/blog.scielo.org\/en\/2014\/05\/14\/scientometrics-of-peer-reviewers-will-they-be-finally-recognized\/<\/a><\/p>\n

WOOLSTON, C. What would happen if grant reviews were made public? Nature.<\/em> 2015, vol. 517, n\u00ba 247. DOI: 10.1038\/517247f<\/a><\/span><\/p>\n

 <\/p>\n

\"lilian\"<\/a>About\u00a0Lilian Nassi-Cal\u00f2<\/h3>\n

Lilian Nassi-Cal\u00f2 studied chemistry at Instituto de Qu\u00edmica <\/em>\u2013 USP, holds a doctorate in Biochemistry by the same institution and a post-doctorate as an Alexander von Humboldt fellow in Wuerzburg, Germany. After her studies, she was a professor and researcher at IQ-USP. She also worked as an industrial chemist and presently she is Coordinator of Scientific Communication at BIREME\/PAHO\/WHO and a collaborator of SciELO.<\/p>\n

 <\/p>\n

Translated from the\u00a0original in portuguese<\/a>\u00a0by Lilian Nassi-Cal\u00f2.<\/p>\n","protected":false},"excerpt":{"rendered":"

Researchers have been discussing what would be the impact of making the review process of grant proposals more open and transparent, in order to support the preparation of better proposals and acknowledge the work of the reviewers. A recently published paper in Nature examines the impact of two articles on the open availability of the review of research proposals and the possibility of changing the assessment after publication of the results. …<\/span> Read More →<\/span><\/a><\/span><\/p>\n","protected":false},"author":22,"featured_media":1756,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_links_to":"","_links_to_target":""},"categories":[3],"tags":[18,31,48],"class_list":["post-1755","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-analysis","tag-open-access","tag-research-evaluation","tag-research-funding-agency"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/blog.scielo.org\/en\/wp-json\/wp\/v2\/posts\/1755","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.scielo.org\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.scielo.org\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.scielo.org\/en\/wp-json\/wp\/v2\/users\/22"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.scielo.org\/en\/wp-json\/wp\/v2\/comments?post=1755"}],"version-history":[{"count":2,"href":"https:\/\/blog.scielo.org\/en\/wp-json\/wp\/v2\/posts\/1755\/revisions"}],"predecessor-version":[{"id":2123,"href":"https:\/\/blog.scielo.org\/en\/wp-json\/wp\/v2\/posts\/1755\/revisions\/2123"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.scielo.org\/en\/wp-json\/wp\/v2\/media\/1756"}],"wp:attachment":[{"href":"https:\/\/blog.scielo.org\/en\/wp-json\/wp\/v2\/media?parent=1755"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.scielo.org\/en\/wp-json\/wp\/v2\/categories?post=1755"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.scielo.org\/en\/wp-json\/wp\/v2\/tags?post=1755"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}