{"id":728,"date":"2013-11-05T13:09:00","date_gmt":"2013-11-05T16:09:00","guid":{"rendered":"http:\/\/blog.scielo.org\/en\/?p=728"},"modified":"2016-01-15T16:07:08","modified_gmt":"2016-01-15T18:07:08","slug":"impact-natures-viewpoint-comments-on-special-issue-502-7471-17th-october-2013","status":"publish","type":"post","link":"https:\/\/blog.scielo.org\/en\/2013\/11\/05\/impact-natures-viewpoint-comments-on-special-issue-502-7471-17th-october-2013\/","title":{"rendered":"Impact \u2013 Nature\u2019s Viewpoint: comments on special issue 502 (7471) 17th October, 2013"},"content":{"rendered":"<div id=\"attachment_729\" style=\"width: 310px\" class=\"wp-caption alignright\"><a href=\"http:\/\/www.flickr.com\/photos\/raster\/3170571010\/\" target=\"_blank\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-729\" class=\"size-medium wp-image-729\" title=\"Photo: Pete Prodoehl\" src=\"http:\/\/blog.scielo.org\/en\/wp-content\/uploads\/sites\/2\/2013\/11\/impacto-300x199.jpg\" alt=\"Photo: Pete Prodoehl\" width=\"300\" height=\"199\" srcset=\"https:\/\/blog.scielo.org\/en\/wp-content\/uploads\/sites\/2\/2013\/11\/impacto-300x199.jpg 300w, https:\/\/blog.scielo.org\/en\/wp-content\/uploads\/sites\/2\/2013\/11\/impacto.jpg 640w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><p id=\"caption-attachment-729\" class=\"wp-caption-text\">Photo: <a href=\"http:\/\/www.flickr.com\/photos\/raster\/3170571010\/\" target=\"_blank\">Pete Prodoehl<\/a><\/p><\/div>\n<p><b>\u201c<\/b><b>Impact : the search for the science that matters<\/b><b>\u201d<\/b><b> <\/b>is the<b> <\/b>principle theme in this special issue of Nature, where six articles written by journalists and researchers examine if the current evaluation systems of science really select research that is\u00a0 truly influential and of the greatest benefit to society which, after all, finances it with the taxes it pays.<\/p>\n<p>Three principal lines of argument are highlighted in this analysis:<\/p>\n<ul>\n<li>The Impact Factor (IF), which measures the frequency with which an article is cited in a particular year or period works well enough, but falls short overall. In other words, the Journal Impact Factor continues to be a criterion considered relevant by agencies responsible for the evaluation of research performance and is used in all countries the world over where evaluations are carried out. However, there is a consensus that many aspects connected with this evaluation process slip through the net when this \u201cobjective mathematical paradigm\u201d is applied.<\/li>\n<li>Research funding agencies are beginning to incorporate other alternative ways of measuring the value of the journals which publish research output into the procedures they use to evaluate institutions and projects. Without dismissing these objective methods and the importance that they represent in the professional careers of researchers, these agencies are beginning to evaluate the \u201csocial impact\u201d of the research that they fund. This new direction is in line with the creed of \u201cpolitical correctness\u201d which is used by industrial and commercial companies under the heading of \u201csocial engagement\u201d whereby they realize that when they are part of a society and wish to gain public acceptance, they must demonstrate that what they do is really necessary, useful and applicable to the man in the street who pays the taxes.<\/li>\n<li>Even though publications in Open Access are reaching the figure of 50% of published output, access to citation databases and \u201cscientometric\u201d information is not equally available in this way, since over 95% of this information can be accessed by subscription only from two commercial providers : Thomson Reuters and Elsevier. Projects that are still being developed are put forward so that they can achieve Open Access to citations which brings with it an as yet pending task: How should works be cited which are not strictly speaking <i>papers<\/i> in the true sense of the word, but appear in other formats such as videos, microscope slides, runs of meteorological data, the genome, etc?<\/li>\n<\/ul>\n<p>The present issue, which we will call \u201cImpact \u2013 Nature\u2019s Viewpoint\u201d, begins with an editorial\u00b9 where it is accepted that those who carry out evaluations are beginning to understand that the conventional methodology which uses the following elements &#8211; citation metrics, Journal Impact Factor, and \/ or the opinions of other colleagues &#8211; are necessary but insufficient and should incorporate other aspects, among others, those elements which take into account the new field of \u201caltmetrics\u201d. This means that the question of what is important should be measured outside the confines of academia and must include the impact of research on <b>decision makers in policy and health<\/b>, and the effects on industry, the economy and social players. Conventional metrics must not continue to be used blindly and inconsistently, since they would end up by putting at risk the very reputation of those carrying out the evaluation process.<\/p>\n<p>The challenge therefore is to find a balance between the different aspects of the evaluation process. If the emphasis is put on publication in high impact journals, this will mean that academics will feel that they are under undue pressure to publish in order to advance their careers. On the other hand, if the emphasis is put on economic and social impact then they will divert their time away from academic and research activities <i>per se<\/i> towards bureaucratic and legal red-tape connected with formulating patents or ill-considered spin-out companies. It is therefore important that those that carry out evaluations have clear and transparent procedures and explicit methods for measuring impact.<\/p>\n<p>\u201cEl d\u00eda del juicio\u00b2\u201d (The Day of Judgement) (for universities) is the first article which follows the editorial. The article reports that since the end of the 1980\u2019s (the UK was the first to do this closely followed by the USA) the governments of these two countries initiated programs for the systematic evaluation of their universities. The governments of Australia, Germany and Italy, etc, then followed suit using similar methodologies and experiences. Although these evaluations have helped to improve national research evaluations by increasing the presence of their countries in citation indexes, the results of these exercises in terms of the enormous evaluation efforts required and the results, when looked at from a\u00a0 cost-benefit viewpoint, have led to critical voices being raised at the level of researchers,\u00a0 university administrators and leaders in the field of education.<\/p>\n<p>The possibility that some researchers may end up being published in the most prestigious journals, which places them in some sort of \u201cThe Golden Club\u00b3\u201d, which may well open doors for them in terms of their professional career, is currently under discussion. One of the reasons behind this criticism is that there is a growing output of works worldwide which are not published in the \u201cstar\u201d journals such as <i>Science<\/i>, <i>Cell<\/i> and <i>Nature<\/i>. Although the number of submissions sent to these journals has increased by a little more than 40% over the last 16 years \u2013 along with the rejection rate &#8211; submissions have increased worldwide by some 86%. This suggests that researchers are going to other places to get published. Although these \u201ctop\u201d journals annually publish more works that are highly cited, this growth does not keep pace with the rest of the industry, since the percentage of the total of highly cited works in those journals is declining in favor of other journals, many of which are Open Access.<\/p>\n<p>Research practices suffer because citation data\u2074 are not freely available, which brings us to the fourth article in <i>Nature<\/i>. It is a paradox that in the age of free access to information, citations in<b> <\/b>major journals which are the nucleus of academic evaluations are not freely available. Citations to Open Access journals are likewise unavailable because there are no institutions that are compiling and organizing them in a usable way.<\/p>\n<p>Citation analyses always end up using WoS and Scopus from the two commercial vendors whose contract license terms present barriers to the dissemination of the citation data of these analyses.<\/p>\n<p>The are various open sources of citations. The two principle ones are CiteSeerX, primarily in Computing Science and CitEc in Economics (years ago there was the Citebase database of citations but unfortunately it went offline). It is because of this that the author of the article that we have mentioned presents Open Citations Corpus (OCC), a project of Open Access citation sources which has been in a pilot phase over the past two years.<\/p>\n<p>If we sum up the references available in those services mentioned, and include the references available in PubMed and in OCC, they barely cover 4% of the estimated 50 million existing bibliographic references to journal articles and books. There is certainly a lot of information in recent years to other types of literature in Google Scholar and Microsoft Academic Search, but their results are subject to various criticisms among which are the lack of consistency in the results from the same search done on different days.<\/p>\n<p>OCC has established agreements with a variety of the world\u2019s most important journal publishers (commercial and Open Access) so that in the future the bibliographic reference data may be collected and which, like the abstracts, may be made available in Open Access. The references will be centrally harvested in CrossRef, so publishers will have to publish their articles with a DOI as participants in the CitedBy Linking service, with the only condition being that the publishers give their consent to include new metadata for this purpose.<\/p>\n<p>Once the citations are freely available, services can be developed to generate the desired metrics, including a service that could permit the correction of erroneous references by the interested parties.<\/p>\n<p>Bibliographic references\u00a0 have not died. Quite the contrary, they are growing in such a way that they cover the entire spectrum of research, from lines of software code to video frames. This is the topic of the next article in Nature titled \u201cReferencing: The reuse factor\u2075\u201d.<\/p>\n<p>We are entering the age of reuse. Researchers and evaluators are struggling to find ways to adequately handle citations to published research results because it is the way in which credit is formally given to research but\u2026 there is a greater amount of scientific data published in other ways, such as genetic sequences, datasets in economics, meteorology, clinical trials without significant results, etc.<\/p>\n<p>This information is being compiled, for example, in Dryad Digital Repository and Gen Bank, and also in Figshare. None of this information is found in WoS or Scopus. It is in this context that the project Research Data Alliance (RDA) was presented and created in August of 2012 by various<b> <\/b>agencies in the USA, Europe and Australia. The objective of RDA is to accelerate and facilitate the interchange of data between different disciplines that have unconventional results and are mandated to be cited by the funding agencies. Examples of unconventional data to be cited could be columns in spreadsheets and image sequences in a video, such references should not be lost within the citations of the article to be published.<\/p>\n<p>The emergence of a large number of citations with exponential growth, giving credit to a\u00a0 wide variety of results in different forms, will not be the end of the conventional bibliographic citation -of course not &#8211; but it will be the decline of the classic measure of impact which counts only citations in written works.<\/p>\n<p>Finally, the article Impact: Pack a punch<sup>6<\/sup> rounds off this reflection by indicating that reviewers and evaluators of research funds will have to increasingly focus on the social impact of academic research project proposals.<\/p>\n<p>Public and private entities around the world will have to redefine the concept of impact and the types of impact because the different stakeholders vary widely across the range of human activity, whether in health, laboratories, the economy, education or any other group in society.<\/p>\n<h3>External links<\/h3>\n<p>CiteSeerX &#8211; <a href=\"http:\/\/citeseerx.ist.psu.edu\/index\" target=\"_blank\">http:\/\/citeseerx.ist.psu.edu\/index<\/a><\/p>\n<p>CitEc &#8211; <a href=\"http:\/\/citec.repec.org\/\" target=\"_blank\">http:\/\/citec.repec.org\/<\/a><\/p>\n<p>Open Citations Corpus &#8211; <a href=\"http:\/\/opencitations.net\/\" target=\"_blank\">http:\/\/opencitations.net\/<\/a><\/p>\n<p>PubMed &#8211; <a href=\"http:\/\/www.ncbi.nlm.nih.gov\/pubmed\" target=\"_blank\">http:\/\/www.ncbi.nlm.nih.gov\/pubmed<\/a><\/p>\n<p>Google Scholar &#8211; <a href=\"http:\/\/scholar.google.com.br\/\" target=\"_blank\">http:\/\/scholar.google.com.br\/<\/a><\/p>\n<p>Microsoft Academic Search &#8211; <a href=\"http:\/\/academic.research.microsoft.com\/\" target=\"_blank\">http:\/\/academic.research.microsoft.com\/<\/a><\/p>\n<p>CrossRef &#8211; <a href=\"http:\/\/www.crossref.org\/\" target=\"_blank\">http:\/\/www.crossref.org\/<\/a><\/p>\n<p>CitedBy Linking &#8211; <a href=\"http:\/\/www.crossref.org\/citedby\/index.html\" target=\"_blank\">http:\/\/www.crossref.org\/citedby\/index.html<\/a><\/p>\n<p>Dryad Digital Repository &#8211; <a href=\"http:\/\/datadryad.org\/\" target=\"_blank\">http:\/\/datadryad.org\/<\/a><\/p>\n<p>Gen Bank &#8211;<a href=\"http:\/\/www.ncbi.nlm.nih.gov\/genbank\/\" target=\"_blank\">http:\/\/www.ncbi.nlm.nih.gov\/genbank\/<\/a><\/p>\n<p>Figshare &#8211; <a href=\"http:\/\/figshare.com\/\" target=\"_blank\">http:\/\/figshare.com\/<\/a><\/p>\n<p>Research Data Alliance &#8211; <a href=\"https:\/\/rd-alliance.org\/\" target=\"_blank\">https:\/\/rd-alliance.org\/<\/a><\/p>\n<p>Research evaluation: Impact &#8211; <a href=\"http:\/\/www.nature.com\/news\/research-evaluation-impact-1.13949\" target=\"_blank\">http:\/\/www.nature.com\/news\/research-evaluation-impact-1.13949<\/a><\/p>\n<h3>Notes<b><\/b><\/h3>\n<p>\u00b9 <a href=\"http:\/\/www.nature.com\/doifinder\/10.1038\/502271a\" target=\"_blank\">The maze of impact metrics<\/a><\/p>\n<p>\u00b2 <a href=\"http:\/\/www.nature.com\/doifinder\/10.1038\/502288a\" target=\"_blank\">Research assessments: Judgement day<\/a><\/p>\n<p>\u00b3 <a href=\"http:\/\/www.nature.com\/doifinder\/10.1038\/502291a\" target=\"_blank\">Science publishing: The golden club<\/a><\/p>\n<p>\u2074 <a href=\"http:\/\/www.nature.com\/doifinder\/10.1038\/502295a\" target=\"_blank\">Publishing: Open citations<\/a><\/p>\n<p>\u2075 <a href=\"http:\/\/www.nature.com\/doifinder\/10.1038\/502298a\" target=\"_blank\">Referencing: The reuse factor<\/a><\/p>\n<p>\u2076 <a href=\"http:\/\/www.nature.com\/doifinder\/10.1038\/nj7471-397a\" target=\"_blank\">Impact: Pack a punch<\/a><\/p>\n<p><b>References<\/b><\/p>\n<p>Nature: Impact, 2013, vol. 502, n\u00ba 7471, pp. 271-402. Available from: <a href=\"http:\/\/www.nature.com\/nature\/journal\/v502\/n7471\/\" target=\"_blank\">&lt;http:\/\/www.nature.com\/nature\/journal\/v502\/n7471\/&gt;<\/a>.<\/p>\n<p>&nbsp;<\/p>\n<h3><a href=\"http:\/\/blog.scielo.org\/en\/wp-content\/uploads\/sites\/2\/2013\/10\/spinak.jpg\" target=\"_blank\"><img loading=\"lazy\" decoding=\"async\" class=\"alignright wp-image-640\" title=\"Ernesto Spinak\" src=\"http:\/\/blog.scielo.org\/en\/wp-content\/uploads\/sites\/2\/2013\/10\/spinak-300x271.jpg\" alt=\"Ernesto Spinak\" width=\"180\" height=\"163\" srcset=\"https:\/\/blog.scielo.org\/en\/wp-content\/uploads\/sites\/2\/2013\/10\/spinak-300x271.jpg 300w, https:\/\/blog.scielo.org\/en\/wp-content\/uploads\/sites\/2\/2013\/10\/spinak-768x694.jpg 768w, https:\/\/blog.scielo.org\/en\/wp-content\/uploads\/sites\/2\/2013\/10\/spinak-1024x926.jpg 1024w, https:\/\/blog.scielo.org\/en\/wp-content\/uploads\/sites\/2\/2013\/10\/spinak-150x136.jpg 150w\" sizes=\"auto, (max-width: 180px) 100vw, 180px\" \/><\/a>About Ernesto Spinak<\/h3>\n<p>Collaborator on the SciELO program, a Systems Engineer with a Bachelor\u2019s degree in Library Science, and a Diploma of Advanced Studies from the Universitat Oberta de Catalunya (Barcelona, Spain) and a Master\u2019s in \u201cSociedad de la Informaci\u00f3n\u201d (Information Society) from the same university. Currently has a consulting company that provides services in information projects\u00a0 to 14 government institutions and universities in Uruguay.<\/p>\n<p>&nbsp;<\/p>\n<p>Translated from the original in <a href=\"http:\/\/blog.scielo.org\/es\/2013\/11\/05\/impacto-segun-nature-comentarios-al-numero-especial-502-7471-17-oct-2013\/\">Spanish<\/a>\u00a0by\u00a0<a href=\"http:\/\/www.nicholascopconsulting.com\/\" target=\"_blank\">Nicholas Cop Consulting<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The journal Impact Factor as measured by citations is a relevant yet insufficient measure in the evaluation of projects by national research funding agencies. Without denying this objective measure and the importance it has, a consensus is emerging that the social and economic impact of research funded by these agencies must also be evaluated. <span class=\"ellipsis\">&hellip;<\/span> <span class=\"more-link-wrap\"><a href=\"https:\/\/blog.scielo.org\/en\/2013\/11\/05\/impact-natures-viewpoint-comments-on-special-issue-502-7471-17th-october-2013\/\" class=\"more-link\"><span>Read More &rarr;<\/span><\/a><\/span><\/p>\n","protected":false},"author":8,"featured_media":730,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_links_to":"","_links_to_target":""},"categories":[3],"tags":[7],"class_list":["post-728","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-analysis","tag-scholarly-communication"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/blog.scielo.org\/en\/wp-json\/wp\/v2\/posts\/728","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.scielo.org\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.scielo.org\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.scielo.org\/en\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.scielo.org\/en\/wp-json\/wp\/v2\/comments?post=728"}],"version-history":[{"count":11,"href":"https:\/\/blog.scielo.org\/en\/wp-json\/wp\/v2\/posts\/728\/revisions"}],"predecessor-version":[{"id":1699,"href":"https:\/\/blog.scielo.org\/en\/wp-json\/wp\/v2\/posts\/728\/revisions\/1699"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.scielo.org\/en\/wp-json\/wp\/v2\/media\/730"}],"wp:attachment":[{"href":"https:\/\/blog.scielo.org\/en\/wp-json\/wp\/v2\/media?parent=728"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.scielo.org\/en\/wp-json\/wp\/v2\/categories?post=728"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.scielo.org\/en\/wp-json\/wp\/v2\/tags?post=728"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}