Geopolitical biases in LLMs: what are the "good" and the "bad" countries
  according to contemporary language models
Authors: name: Mikhail Salnikov, name: Dmitrii Korzh, name: Ivan Lazichny, name: Elvir Karimov, name: Artyom Iudin, name: Ivan Oseledets, name: Oleg Y. Rogov, name: Alexander Panchenko, name: Natalia Loukachevitch, name: Elena Tutubalina

This paper evaluates geopolitical biases in LLMs with respect to various
countries though an analysis of their interpretation of historical events with
conflicting national perspectives (USA, UK, USSR, and China). We introduce a
novel dataset with neutral event descriptions and contrasting viewpoints from
different countries. Our findings show significant geopolitical biases, with
models favoring specific national narratives. Additionally, simple debiasing
prompts had a limited effect in reducing these biases. Experiments with
manipulated participant labels reveal models' sensitivity to attribution,
sometimes amplifying biases or recognizing inconsistencies, especially with
swapped labels. This work highlights national narrative biases in LLMs,
challenges the effectiveness of simple debiasing methods, and offers a
framework and dataset for future geopolitical bias research.