## 3 Results
### Reasoning Errors, Hallucinations, Biases
The most obvious hallucinations in our case study were invented references (ChatGPT-4, Appendix C available at [[https://doi.org/10.5281/zenodo.8345088](https://doi.org/10.5281/zenodo.8345088)]([https://doi.org/10.5281/zenodo.8345088](https://doi.org/10.5281/zenodo.8345088)), accessed on 1 October 2023; henceforth Appendix C: L. 36; ChatGPT-3.5, Appendix A: L. 495-498; ScholarAI, Appendix A: L. 189-197). Similarly, attempts to inline citations by Bing (Appendix A: L. 226-257) and ChatGPT-4 w/Bing (Appendix A: L. 575-581) were largely confabulations. It would appear that the inability to provide correct inline citations is a known problem, as ChatGPT-4 w/Bing generated a warning to this effect (Appendix A: L. 154-155). Another very clear example of hallucination was the Late Antique Little Ice Age phenomenon. ChatGPT-4 dated it \"between 300 and 700 AD\" (Appendix A: L. 512). The correct dates are 536 to approximately 660 AD, which is clearly intelligible to any human by consulting the title of the reference which ChatGPT-4 correctly provides: \"... Late Antique Little Ice Age from 536 to around 660 AD\" (Appendix A: L. 532-535). The underlying issue and a key challenge for current technology is that \"AI chatbots do not know what they do not know\" and may very confidently invent facts [46] rather than formulate the sentence \"I don't know\". _Biases_ are often exhibited by AI chatbots. They are based on training data, but according to recent research, they can be amplified beyond existing perceptions in society. Biases generated by AI chatbots can therefore be informative about the underlying data, but they can also be misleading if the AI-generated content is used uncritically. The most researched biases to date are those related to gender, race, ethnicity, and disability status, e.g., [29; 98; 99; 100; 49]. In our test, we detected three different types of biases: language bias, neo-colonial bias, and citation bias. First, there was a language bias. Although there is far more relevant scholarly content written in Balkan languages than in English, 92% of the references generated by the AI chatbots in our test referred to English and none to Balkan-language publications (Figure 7). This can only be partially explained by the fact that the prompts were in English. Namely, three (8%) German references prove that English was not the only criterion for selection. When question Q2 was asked with a prompt in Slovenian, two references were again in English and the third in Slovenian was a hallucination (Appendix C: L. 36). The detected language bias is most likely due in large part to the language bias of online search engine ranking algorithms that favour English publications [101]. This bias seems to be a wasted opportunity, because all tested AI chatbots \"understand\" many languages, e.g., [13]. Figure 6: A Twitter (now X) poll asking human users to differentiate between ChatGPT and human-generated content with almost no context. Most respondents answered correctly. Second, neo-colonial bias. A total of 88% of the references are by authors from the global West. Only a minority (12%) belong to English translations of texts originally written in Slavic languages [96; 102; 103], although there are several other (more) relevant translations, e.g., [103; 104]. This reflects a scholarly hierarchy created by colonialism (until the 1910s, the Balkans were largely divided between the Austro-Hungarian and Ottoman Empires), sometimes referred to as a neo-colonial pattern in the global network of science in which the intellectual dominance of the global West is growing [105; 106]. To our knowledge, the neo-colonial bias for the study of medieval Slavs has not yet been explicitly analysed or even discovered, as it has never been revealed as clearly as through the use of AI chatbots in this case study. Third, citation bias. A total of 75% of the references are from before 2005, and the oldest was originally published in 1895 [102]. This shows a very clear bias against new and up-to-date publications.