###### Abstract
They could amplify the biases we already experience, undermine our trust in information, and take away our ability to determine what is real and what is not. Equally important, they are likely to upend the creative and knowledge industries and many types of work that have been primarily conducted by well-compensated professionals, such as artists, writers, executives, and programmers [3; 24; 25]. A recent study of the potential impact on the labour market found that LLMs and LLM-powered software will affect between 47% and 56% of all work-related tasks, with higher-income jobs being more affected [4]. Of particular interest to our article is that roles that rely heavily on science and critical thinking show a negative correlation with exposure to LLMs, yet there is some exposure [4]. Thus, AI experts, journalists, policymakers, and the public are increasingly discussing a wide range of important and urgent risks of AI, such as the AI race, organizational risks, rogue Als, the reinforcement of social inequalities, reshaping labour and expertise, and the exacerbation of environmental injustices [28; 29; 30; 31; 32; 33]. Questions are being asked about safety [34; 35], capabilities [36], massive workforce redundancy [37], and legality [38; 39]. This has led to calls for a pause in AI development [40], although there are doubts that such attempts would have any impact [41]. In short, the opportunities that generative AI presents for our lives, our communities, and our society are as great as the risks it poses [42]. There are those who believe that generative AI will change our work and our lives in general for the better. There are others who believe that generative AI will disastrously encroach on areas best navigated by sentient beings. Regardless, all agree on an urgent need for safeguards [43; 44; 45; 46], and the first steps have already been taken [47]. All this is even more true for the planned next step, artificial general intelligence (hereafter AGI). The term AGI describes AI systems that will generally be more intelligent than humans and are planned to become a reality within the next 10 years. After the introduction of the first AGI, the world could be extremely different from how it is today [48]. And how does academia and the way we create and write research and scholarly articles fit in? AI chatbots have the potential to revolutionise academia and scholarly publishing [49]. In fact, it seems that academia will be among the first industries to go through this process, since academics and students represented two of the top three occupational groups among the early adopters of ChatGPT [50]. AI chatbots--most of the attention to date was directed to ChatGPT--have already been recognised as a powerful tool for scientific writing that can help organise material, proofread, draft, and generate summaries [51; 52; 53]. The scientific community is also actively testing their ability to generate entire papers with minimal human input. The consensus is that AI chatbots are able to create scientific essays and reports of scientific experiments that appear credible, but the resulting texts are a combination of true and entirely fabricated information [54; 55; 56; 57; 58; 59; 60; 61; 62; 63]. Unfortunately, there is a public perception that ChatGPT is already capable of generating academic papers that are peer-reviewed and published, e.g., [64; 65], which may add to public scepticism about science. This is not the case. For example, the ArXiv repository of preprints ([[https://arxiv.org](https://arxiv.org)]([https://arxiv.org](https://arxiv.org)); accessed on 16 August 2023), the AI community's most popular publication forum, shows no results for ChatGPT (in any variation) as a (co-)author. We are aware of a single such attempt that has attracted much public attention but has not been peer reviewed or published and has no notable scientific value [55].