{"id":5320,"date":"2023-11-14T14:35:13","date_gmt":"2023-11-14T17:35:13","guid":{"rendered":"https:\/\/blog.scielo.org\/en\/?p=5320"},"modified":"2023-11-14T14:32:54","modified_gmt":"2023-11-14T17:32:54","slug":"chatgpt-and-other-ais-will-transform-all-scientific-research-initial-reflections-on-uses-and-consequences-part-2","status":"publish","type":"post","link":"https:\/\/blog.scielo.org\/en\/2023\/11\/14\/chatgpt-and-other-ais-will-transform-all-scientific-research-initial-reflections-on-uses-and-consequences-part-2\/","title":{"rendered":"ChatGPT and other AIs will transform all scientific research: initial reflections on uses and consequences \u2013 part 2"},"content":{"rendered":"

Rafael Cardoso Sampaio<\/strong><\/a>, Professor permanente do Programa de P\u00f3s-gradua\u00e7\u00e3o em Ci\u00eancia Pol\u00edtica e Comunica\u00e7\u00e3o da Universidade Federal do Paran\u00e1 (UFPR)<\/strong><\/p>\n

Maria Alejandra Nicol\u00e1s<\/strong><\/a>, Professora do Programa de Mestrado em Pol\u00edticas P\u00fablicas e Desenvolvimento da Universidade Federal da Integra\u00e7\u00e3o Latino-Americana (UNILA)<\/strong><\/p>\n

Tain\u00e1 Aguiar Junquilho<\/strong><\/a>, Professora do Mestrado em Direito do IDP<\/strong><\/p>\n

Luiz Rog\u00e9rio Lopes Silva<\/strong><\/a>, Professor substituto do Departamento de Ci\u00eancia e Gest\u00e3o da Informa\u00e7\u00e3o da Universidade Federal do Paran\u00e1 (UFPR)<\/strong><\/p>\n

Christiana Soares de Freitas<\/strong><\/a>, Professora dos Programas de P\u00f3s-gradua\u00e7\u00e3o em Comunica\u00e7\u00e3o e em Governan\u00e7a e Inova\u00e7\u00e3o em Pol\u00edticas P\u00fablicas Universidade de Bras\u00edlia (UNB)<\/strong><\/p>\n

Marcio Telles<\/strong><\/a>, Professor permanente do Programa de P\u00f3s-Gradua\u00e7\u00e3o em Comunica\u00e7\u00e3o e Linguagens da Universidade Tuiuti do Paran\u00e1 (UTP)<\/strong><\/p>\n

Jo\u00e3o Senna Teixeira<\/strong><\/a>, Bolsista de p\u00f3s-doutorado no Instituto Nacional de Ci\u00eancia e Tecnologia em Democracia Digital (INCT-DD) da Universidade Federal da Bahia (UFBA)<\/strong><\/p>\n

\"Image<\/a>

Image: Google DeepMind<\/a>.<\/em><\/p><\/div>\n

In the first part of this essay<\/a>,1<\/sup> we presented six main uses of Artificial Intelligence (AI) to help in various stages of the scientific endeavor; now in the second part, we discuss consequences, risks, and paradoxes in the use of AI for scientific research. Now, in this second part, we seek to present some of these risks, which arise particularly in the use of generative AI in the scientific field and in academic work. Although all the problems have not been fully mapped out, we have tried to offer initial reflections to support and encourage debate. We would like to emphasize that both essays are a summary of a larger reflection that has already been published as an article<\/a>2<\/sup> in SciELO Preprints<\/a>.<\/p>\n

1. Authorship and plagiarism<\/h3>\n

The trend is for AIs to become part of the development of academic texts, gradually making it difficult to differentiate between texts produced by humans or by machines. In particular, there is the unresolved question of whether a text is written by a human and only proofread by artificial intelligence, is that enough for it to stop being human? In another possibility, if a human creates a good part of a text, but then asks the AI to complement it, does this violate any ethical boundaries? If we ask the AI to read a text we have written and draft the summary or title, will it be a machine text? Bearing in mind that the human has proofread all the texts and put their name down as responsible. There is little consensus on these issues, except that artificial intelligences cannot be considered as authors of an academic paper, since they cannot be held responsible for the content produced and if any tool is used, it should be mentioned in the appropriate section of the academic paper.<\/p>\n

As AIs can easily generate images, graphs, tables, and even entire presentations, rather than the discussion about plagiarism or “pasting”, as discussed above, it seems to us that the most fruitful discussion will be the one related to authorship, copyright, fonts, and the limits of human-machine cooperation.<\/p>\n

It is important to note that at the time of writing this essay, no text-based AI detection tool has been evaluated as infallible or truly reliable. In fact, one study showed that it tended to have a bias against non-native English speakers. Finally, it is vital to reinforce that the large language models (LLMs) cited in this essay are NOT<\/strong> capable of detecting plagiarism or the use of AI, which is why we do not recommend detection applications.<\/p>\n

2. Diminished integrity of scientific endeavor<\/h3>\n

The lack of transparency about the criteria and algorithms used by AI can lead to a lack of understanding about how decisions are made. Why did the machine recommend reading a certain text over another? Why did it recommend doing a statistical test or a specific visual representation? Moreover, it is worth noting that LLMs can occasionally produce answers that seem plausible at first glance, but on closer examination turn out to be disconnected from the context, factually incorrect or considerably distorted by the model’s biases.<\/p>\n

As we know little or nothing about the possible unaddressed and similar biases of these models, the risk of inconsistency or inaccuracy requires a critical approach when evaluating and using its responses, underlining the importance of rigorous human supervision and meticulous review to ensure the quality and reliability of scientific writing. The use of AIs makes it difficult to trust the results and limits researchers’ ability to assess their validity and reliability, making certain aspects of the research completely unreplicable.<\/p>\n

3. Restricted range of research possibilities<\/h3>\n

The standard established by Google, also used by academic indexers, shows a series of links to pages dealing with the requested subject based on relevance criteria.<\/p>\n

In the new paradigm established by ChatGPT, the main output is initially a machine-generated response and only then links for further study. We will not need to go to the websites or read different opinions but have them summarized by artificial intelligence directly in the chat while proceeding the questions, reducing access to dissent, and crystallizing the most “relevant” positions, which could have effects on increasing inequalities in producing scientific knowledge.<\/p>\n

This situation is likely to get even worse, as these artificial intelligences, despite being trained on huge databases, primarily index North American, Anglo-Saxon, and European output, especially when we are talking about academic AIs. Therefore, they will tend to reinforce a very particular view of science, which includes certain types of methods and forms of analysis that could become even more the standard for scientific research. This could have especially damaging effects on humanities research or on all qualitative research in general.<\/p>\n

4. The paradox of knowledge production<\/h3>\n

The automation of various tasks and the outsourcing of important choices to artificial intelligences could foster a generation of future researchers who will know fewer authors and sources and will have even less knowledge of academic literature and the basis of research and programming methods.<\/p>\n

Thus, we can easily imagine future researchers who will “read” dozens of articles simultaneously; draw up automated and substantive “literature reviews”; scrape, clean, and analyze massive amounts of data; use such models and technologies to test the limits of human knowledge and, we suppose, even achieve more substantive and innovative results. Paradoxically, we will have “less trained”, “less skilled” researchers who will nevertheless be able to carry out faster and more meaningful tasks.<\/p>\n

Finally, we recognize that all this can generate a gigantic amount of worthless academic trash, with a tide of poorly written scientific texts, lacking in novelty, criteria, and rigor, seeking only to feed predatory journals and the CVs of students and professors in search of funding.<\/p>\n

5. Center-periphery relations and data colonialism<\/h3>\n

Besides breaking down language barriers in terms of reading and writing, the technology also democratizes the role of research assistants, who were only available to renowned researchers from institutions in the global north. With AI tools, researchers can perform tasks such as note-taking, citation archiving, manuscript preparation, manuscript editing, form filling, audio transcription, email writing, topic list creation, text translation, and presentation creation \u2013 tasks normally relegated to assistants. This could reduce the gap in output and publication capacity between researchers from the center and the periphery of the academic world.<\/p>\n

Even so, many of these tools are paid for, and in dollars or euros, limiting access to them for researchers from the global south. Another discussion is whether or not university resources should go towards the use of these applications and AI platforms. While this may seem strange at the moment, it is worth remembering that we have already normalized payment for servers (Microsoft, Amazon etc.), software (NVivo, Atlas.ti, Stata, SPSS, Endnote, Office etc.) and for access to certain article platforms.<\/p>\n

Considering that research resources in the global south will continue to be lower and that the centers will pay for or develop their own AIs, the danger lies not in a decrease, but in an increase in the research gap between the center and the periphery. There is a risk that researchers from the global south will increase their dependence on technologies developed by the global north, reinforcing the issue of data colonialism, which already exists on social media platforms.<\/p>\n

However, we point out a paradox in terms of research internationalization<\/em>. We may be overly dependent on the technologies created by this center, but the number of applications dedicated to language translation and correction has increased in quantity and quality, reducing the biggest obstacle to publication in the most important journals. Apps like DeepL<\/a> already deliver considerably better translations than Google Translate<\/a> and we have all the aforementioned English correction tools like Grammarly<\/a>, Quillbot<\/a>, Writefull<\/a> and the like. And all LLMs without exception are able to refine texts to be closer to standard academic text.<\/p>\n

We are not advocating an end to translation or correction by humans, just that there will be new tools for researchers to write directly in other languages and publish in the main journals.<\/p>\n

All of these reflections are very premature, but they seek to move away from the exclusive discussion about plagiarism and the falsification of tests by students, seeking to show that the changes will be much more significant than we are discussing. Discussion is therefore vital at this time.<\/p>\n

The \u201cChatGPT and other AIs will transform all scientific research: initial reflections on uses and consequences\u201d series consists of two posts<\/h3>\n