Venue: NAACL 2024

This Land is {Your, My} Land: Evaluating Geopolitical Bias in Language Models through Territorial Disputes

 Bryan Li, Samar Haider, Chris Callison-Burch

University of Pennsylvania

Philadelphia, PA, USA

[bryanli, samarh, cccb]@seas.upenn.edu

###### Abstract

Do the Spratly Islands belong to China, the Philippines, or Vietnam? A pretrained large language model (LLM) may answer differently if asked in the languages of each claimant country: Chinese, Tagalog, or Vietnamese. This contrasts with a multilingual human, who would likely answer consistently. In this paper, we show that LLMs recall certain geographical knowledge inconsistently when queried in different languages--a phenomenon we term geopolitical bias. As a targeted case study, we consider territorial disputes, an inherently controversial and multilingual task. We introduce BorderLines1, a dataset of territorial disputes which covers 251 territories, each associated with a set of multiple-choice questions in the languages of each claimant country (49 languages in total). We also propose a suite of evaluation metrics to precisely quantify bias and consistency in responses across different languages. We then evaluate various multilingual LLMs on our dataset and metrics to probe their internal knowledge and use the proposed metrics to discover numerous inconsistencies in how these models respond in different languages. Finally, we explore several prompt modification strategies, aiming to either amplify or mitigate geopolitical bias, which highlights how brittle LLMs are and how they tailor their responses depending on cues from the interaction context.

Footnote 1: [https://github.com/manestay/borderlines](https://github.com/manestay/borderlines)

_Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)_, pages 3855–3871

June 16-21, 2024 0204 Association for Computational Linguistics

## 1 Introduction

Pretrained large language models (LLMs) have recently seen widespread adoption by users worldwide due to their capabilities with generative tasks ranging from drafting emails to writing code. This has given rise to the impression of LLMs as interactive, general-purpose knowledge bases (KBs). While this view is popular among end-users, NLP research has advised more caution. While LLMs do internalize some relational knowledge Petroni et al. (2019), they are prone to making facts up and "hallucinating Ji et al. (2023) and require their generations to be treated with some skepticism. Furthermore, LLMs can reflect and amplify social biases, an artifact from them learning unwanted statistical associations at training time Blodgett et al. (2020).

However, this has not stopped the massive adoption of these systems in society. But while LLMs may prove to be quite useful in purely creative tasks, their use in generating content that is faithful to real-world facts is not free of challenges. In particular, the biases they learn from their training data can lead to unexpected issues, especially when dealing with politically and socially sensitive topics. This motivates our study on how LLMs operate with respect to territorial disputes, wherein multiple countries claim their rightful ownership of certain territories. We proceed with the insight that these countries, neighboring or not, often speak different

Figure 1: Illustration of a disputed territory and the proposed approach, where we ask the same query in different languages. The LLM responds inconsistently: in Spanish and English “Spain”, while in Arabic “Morocco”, demonstrating geopolitical bias. We compare the responses to KB triplet, which says the territory ”Cetta” belongs to the claimant ”Spain”.

 

languages, and, in such cases, LLMs' training data will learn different views of the "factual" situation depending on the language. In our work, we find that this is indeed the case, as we uncover LLMs' propensity for **geopolitical bias** -- a tendency to report geopolitical knowledge differently depending on the language of interaction.

The existence of geopolitical bias in LLMs underscores a major risk, in that users with similar information-seeking queries will instead receive different "factual" information depending on the language of choice. While such adaptation of output may be a desirable quality in the context of cultural sensitivity -- wherein users may want to receive responses that are tailored to their cultural norms -- we argue that this may be less desirable in the context of territorial disputes (and related factual inquiries). As these systems come to be used for a wide variety of tasks, it is problematic for them to answer with different facts to different users simply because that is what it thinks they may want to hear. LLMs which display geopolitical bias amplify divisions in viewpoints across cultures, increasing the societal risks of using such systems.

Furthermore, geopolitical bias calls into question the true cross-lingual ability of LLMs. While multilingual humans are more likely to reconcile facts written in different languages, we show that multilingual LLMs store and recall facts differently.

Our study takes a deep dive into evaluating the research question: "do multilingual LLMs recall information differently when responding to _the same underlying queries_ specified in different languages?" After quantifying the extent of geopolitical bias in several popular LLMs, we perform a deeper dive into how bias can be amplified or mitigated by modifying the conversational contexts. One technique we use is _personas_, in which we ask an LLM to answer from the perspective of some individual (which the LLM can associate with a certain biased or neutral geopolitical leaning).

Our contributions are as follows:

1. We introduce Borderlines, a multilingual (49 languages) dataset of 726 questions on 251 disputed territories.
2. We propose an evaluation suite for BorderLines, which allows us to precisely quantify and evaluate three aspects of models' responses: factual recall, geopolitical bias, and consistency. These metrics evaluate how a single model's responses differ across languages and contexts.
3. Our evaluations lead to several non-intuitive findings: instruction-tuned LLMs underperform base ones, larger LLMs can underperform smaller ones, and geopolitical bias is amplified in stronger models.
4. We further show that LLMs' knowledge across languages is _brittle_. We do so by effecting the geopolitical bias of LLM responses through several well-motivated prompt modification strategies: a nationalist persona, a neutral (UN peacekeeper) persona, and demographic reasoning.
5. We present case studies on three highly contentious territorial disputes, which highlight LLMs' language-dependent biases and inconsistencies.

## 2 Related Work

Cultural Biases of LLMsVarious studies have looked at bias in large language models from social and cultural perspectives. Cao et al. (2023) probe ChatGPT's cross-cultural alignment to different cultures by prompt the LLM with culturally

\begin{table}
\begin{tabular}{p{85.4pt} p{56.9pt} p{56.9pt} p{56.9pt}} \hline \hline Territory & Claimants & Region & Population \\ \hline Centa & **Spain**, Morocco & Africa & 86,384 \\ Falkland Islands & **United Kingdom** & South & 3,662 \\ Argentina & American & America & America \\ Donetsk Oblast East Jerusalem Jammum and Kashmir Taiwan & _Ukrraine, Russia_ & Europe & 4,059,372 \\ Taivan & _Israel, Palestine_ & Asia & 595,000 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Excerpted entries from the BorderLines table. The Claimants column either has the Controller **bolded**, or all Claimants _italized_ if Unknown.

\begin{table}
\begin{tabular}{c c l r} \hline \hline Country & Language (code) & Religion & Population \\ \hline Spain & Spanish (es) & Christianity & 46,507,760 \\ Morocco & Arabic (ar) & Islam & 33,465,000 \\ Ukraine & Ukrainian (uk) & Christianity & 42,973,696 \\ Russia & Russian (ru) & Christianity & 146,233,000 \\ India & Hindi (hi) & Hinduism & 1,263,930,000 \\ Pakistan & Urdu (ur) & Islam & 188,410,000 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Demographics for several countries. Each country’s majority Language and Religion are given. Information is sourced from reliable sources, such as Wikipedia (more details in Appendix A).

 

sensitive prompts in multiple languages. They find ChatGPT favors American cultures, especially when interacting with the LLMs in English, and other languages interactions are not as culturally specific. Naous et al. (2023) perform a deep dive into LLMs' understanding of Western vs Arab world values, with similar findings. Cheng et al. (2023) explore social biases through asking LLMs to generate personas, which are natural language descriptions of certain demographic groups. They find that generated personas amplify stereotypes against marginalized groups. Our work also has experiments asking LLMs to adopt personas. Abid et al. (2021) highlights the potential danger of LLMs, which can associate certain minorities with violence. They propose to alleviate some issues through prompt-based debiasing strategies, which are also considered in our work.

Arora et al. (2023) modify well-established cultural survey questions to serve as prompts for probing these biases, showing that these biases only weakly correlate with the value surveys themselves. Feng et al. (2023) trace these issues back to the training data, finding that LLMs reinforce biases in the text that they are trained on. By probing LLMs with statements from the political compass test, they quantify the political stance of the models and show that they exhibit marked differences in ideological leanings, especially on social issues. In the same vein, Tao et al. (2023) find that when pretraining an LLM is not possible, prompting them to respond as if they were members of a particular culture can reduce this bias to some degree.

Concurrent work to ours shows that such language-specific biases even extend to facts. Qi et al. (2023) study factual consistency in multilingual LLMs by probing them with factual statements and find that consistency remains low across model families and sizes, and is markedly higher between European languages and those written in the Latin script. This is in line with the trends observed in the socio-cultural analyses discussed above, where models are found to be more biased towards Western cultures and values. This is problematic and stands in the way of reliable adoption of LLMs in other parts of the world.

Zooming out, Perez et al. (2023) explore the "sychophantic" behavior of LLMs -- when LLMs repeat back a dialog user's preferred answer. Their main consideration is in political discourse, in which a LLM will adopt a liberal viewpoint with a liberal-leaning user, and similarly if conservative. Our study of how geopolitical bias manifests in territorial disputes is demonstrative of this behavior (and similarly undesirable in allow for LLMs to facilitate echo chambers). In this case, the different languages of interaction cues the LLM to take a particular geopolitical perspective on a dispute.

Geographic Knowledge of LLMsFaisal and Anastasopoulos (2022) evaluate how LLMs encode geographic proximity differently in different languages. They also mention "geopolitical bias", and explore a different facet of the phenomenon than in our work. Specifically, they find that LLMs, when generating text in different languages, overrepresent the top 10 most geopolitically important countries instead of the countries more relevant to those language's speakers.

Yin et al. (2022) study how commonsense knowledge differs across geo-diverse cultures. Their dataset covers 5 languages; an example query is "In traditional [Country] wedddings, the color of the wedding dress is usually [Y]". Entities are inserted into this query template, which is translated into different languages that necessitate different responses. On their task, they arrive at several notable findings, which are also corroborated in our work; for example, that larger LLMs can underperform, and that LLMs are not intrinsically biased towards the West. For cultural commonsense, a person of one culture can agree that another culture has different behaviors than theirs. However, this cannot be the case for territorial disputes, where differing claims necessarily invalidate others.

Zhuo et al. (2023), in their diagnostic analysis of ChatGPT's ethics on several dimensions, study a single territorial dispute to find the model's language-dependent bias. Our concurrent work thoroughly fleshes out the study of territorial disputes with LLMs, covering many more disputes (251) and languages (49), and includes both qualitative and quantitative analysis.

## 3 Dataset Collection

The BorderLines dataset consists of two parts: a _table_ of disputed territories, and associated probing _questions_. This section discusses the table, while the following sections covers the questions. The original data source is an English Wikipedia ar 

ticle2 whose information is drawn from sources such as government websites or news articles (see Appendix A, Figure 3 for article excerpt).

Footnote 2: List of territorial disputes [15]. We note that status of territorial disputes can change over time, while LLMs’ knowledge remains static. In Appendix D, we explore versions of BorderLines which are temporally enforced to an LLM’s training date cutoff.

### Extracting a Table of Territorial Disputes

From the Wikipedia article, we use only those tables which specify territorial disputes which are:

* between at least two countries3 Footnote 3: Countries are as defined by the ISO 3166 standard.
* current (not settled)
* over land (not over water)

Sample entries from the BorderLines table are depicted in Table 1. A **territory** is an area of land belonging to some entity; a **controller** is the country which has official authority over it; and the **claimants** are any countries which lay claim to it. Territories without exactly one controller have an "Unknown" controller.

Each country which appears in BorderLines is then associated with demographic information, as shown in Table 2. We use the majority religion and language for each country4. This allows us to, given a disputed territory and its claimants, formulate the question in the languages of each claimant.

Footnote 4: We acknowledge that this is a simplification, as each country’s residents may speak multiple languages and follow multiple religions.

In total, the countries involved speak 49 languages (listed in Section B). BorderLines table statistics are given in Appendix Table 3.

## 4 Task Design

We now discuss the development of the BorderLines queries. Given a disputed territory, we formulate a multilingual query set (MQS) - a set of multiple-choice questions, one for each language of the claimants. We leverage this as a probing task for LLMs, allowing us to uncover how a single model recalls facts differently across languages, given the same underlying query. The approach is illustrated in Figure 1.

We begin with a single English query, a multiple-choice question of the following format:

\[{\rm Is}~{}t~{}{\rm a~{}retriory~{}of}~{}l_{1}~{}c_{1}~{}{\rm or}~{}l_{2}~{} c_{2}\ldots? \tag{1}\]

where \(T\) is a territory, \(c_{i}\) is a claimant country, and \(l_{i}\) is a letter drawn from \(L=\{{\rm A},{\rm B},{\rm C},{\rm D},...\}\). There can be \(i\geq 2\) claimants.5 The letters \(L\) indicate that this is a multiple-choice question - a common task used to train and evaluate LLMs.

Footnote 5: We use \(t\) and \(c\) to denote the underlying entities, rather than the English string.

BorderLines TaskAfter the translation process, each territorial dispute includes a **multilingual query set** (MQS), which are queries asking the same question in different languages. The languages include all claimant languages, as well as English as a control. There are 726 BorderLines questions in total for the 251 territories. Dataset statistics are given in Table 3.

### Template-wise Question Translation

To translate a question, we propose a novel _template-wise_ machine translation (MT) approach. We first write a template, a simplified version of Equation 1: "Is XX a territory of YY or ZZ?" We apply MT6 to obtain templates in the 49 languages; the abstractions XX, YY, ZZ are preserved. Separately, we collect all territory and claimant names, then apply MT to create dictionaries (one per language) between English terms and their translations. For each territory, and for each language, we create a query by taking the translated template and infilling the translated terms. Figure 1 provides an illustration of this process, where we obtain two translated queries from the original English query in the example shown.

Footnote 6: We use Google Translate: translate.google.com

The template-wise translation process has several advantages over direction translation of English questions: first, it avoids the inconsistencies from the MT process, especially given the 49 typologically diverse languages we consider. For example, "territory" could be translated into equivalent words for "land" or "region". Second, it is also efficient, as the MT system only needs to translate the non-abstracted texts once per language. Furthermore, consider that an MT system is also suscepti

\begin{table}
\begin{tabular}{p{80.0pt} p{113.8pt}} \hline \# of territories & 251 \\ \(\hookrightarrow\) \# with known controllers & 161 \\ \(\hookrightarrow\) \# with ’Unknown’ controllers & 90 \\ Total \# languages & 49 \\ \# unique claimants (\# total) & 116 (§50) \\ Mean \# languages per territory & 2.11 (\(\sigma=0.65\)) \\ Mean \# claimants per territory & 2.31 (\(\sigma=0.56\)) \\ Mean \# claimed territories per country & 5.00 (\(\sigma=10.02\)) \\ \hline \end{tabular}
\end{table}
Table 3: Statistics for the BorderLines table. The diversity of territorial disputes is evident in the large variances for the means over territories and countries.

 

ble to geopolitical bias. By abstracting country and territory names, we minimize this possible leakage into our dataset.

## 5 Methodology

### Models Used

We perform our studies on various multilingual autoregressive LLMs. First, we consider GPT-4 (OpenAI, 2023) (gpt-4-0314), which is among the most powerful and widely-used LLMs today. We also consider the GPT-3 models text-curie-002 (GPT-\(3_{\text{C}}\)) and text-davinci-003 (GPT-\(3_{\text{DV}}\)).7 with greedy decoding strategy (temperature=0).

Footnote 7: The parameter sizes are 6.7B and 175B, respectively.

For open-source LLMs, we consider the BLOOM Le Scao et al. (2022) family of models with 560M and 7.1B parameters, which are trained on 46 languages. We also use the corresponding BLOOMZ Muennighoff et al. (2023) models, which are further fine-tuned on instruction-following prompts and completions.

Obtaining Model's Answers to a Multilingual Query SetWe pose each question of a MQS to a model, to receive a _multilingual response set_ (MRS), which consists of the LLM's responses in the claimant's languages in addition to English. In Figure 1, the MRS are the three LLM outputs. Finally, we assign each country from the MRS to a multiple-choice letter \(\{\texttt{A}\},\texttt{B}\},...\\), so as to resolve the different names in different languages to the same underlying entities (e.g. Spain vs. Espana). The conversion process depends on the model used.

For GPT-4, the primary model of interest, we parse the response into a letter by applying the following steps to it until a match is found: 1) search for the string of claimant, 2) search for a letter, 3) perform manual extraction. Manual inspection is required only for a handful of responses, as GPT-4 generally follows the instructions correctly.

For the other models, as we can access logprobabilities, we use rank classification Brown et al. (2020). More precisely, we concatenate each choice (i.e., claimant) with the query to form several sequences, and pass each into the model8. For each sequence, we calculate the log-probability of the choice tokens. Finally, we set the most likely choice to be the model's response.

Footnote 8: While some works use the letter choices for rank classification, we use the claimant in the language of the query to underscore the multilinguality of the task.

### BorderLines Evaluation Suite

We now describe our evaluation suite to measure models' responses to the territorial dispute queries. Specifically, we design several metrics to precisely quantify three aspects: _factual recall_, _geopolitical bias_, and _consistency_.

The key concept behind our evaluation is the **Concurrence Score** (CS) metric: a simple accuracy between two countries (shown in Table 4, row 1). We extend this concept to multiple CS metrics, which make different comparisons between a model's MRS and a KB. We use CS in lieu of "accuracy" given the tasks' inherently disputed nature. For each CS metric, we calculate it over the entire dataset of disputed territories. The formulas are given in Table 4, while the comparisons are illustrated in Figure 2.

_Factual recall_ can be thought of as a model's performance on a simple QA task. We measure this through **KB CS**, which compares a response in English to the KB's response. Factual recall is considered monolinqually, and we use English regardless of the claimant countries for each territory.

_Geopolitical bias_, as defined earlier, is the tendency to report geopolitical knowledge differently

\begin{table}
\begin{tabular}{l l} \hline \hline  & CS\((c_{i},c_{j})=100*\left\{\begin{array}{ll}1&\text{if $c_{i}=c_{ j}$},\\ 0&\text{otherwise}\end{array}\right.\) \\ Con CS\((t)=\text{CS}(c_{KB},c_{i})\) \\ Non CS\((t)=\frac{1}{n}\sum_{c\in C^{\text{con}}}\text{CS}(c_{KB},c)\) \\ \(\Delta\) CS\((t)=\frac{\text{Con CS}-\text{Non CS}}{\text{Non CS}}\) \\ Ct s CS\((t)=\frac{1}{n(n-1)}\sum_{i=1}^{n}\sum_{j=1,j\neq i}^{n}\text{CS}(c_{i}, c_{j})\) \\ \hline \hline \end{tabular}
\end{table}
Table 4: Formulas for concurrence score (CS) metrics. Notationally, we denote all claimants of a territory \(t\) as \(C=c_{1},...,c_{n}\), a controller as \(c_{\text{con}}\), the set of noncontrollers as \(C^{\text{non}}\).

Figure 2: Illustration of comparisons made for the CS metrics. KB CS, Control CS, and Non-control CS all compare between the KB country and a response, while Consistency CS compares between responses.

 

depending on the language of interaction. We measure this through \(\Delta\)CS. We use the following metrics as building blocks for it: **Con CS** compares the response in the controller's language to the KB's response. **Non CS** compares each of the \(m\) responses in the non-controllers' languages to the KB's, and is averaged over the \(m\) comparisons.

\(\Delta\)CS is then the difference between Con CS and Non CS, then divided by Non CS (for normalization). Intuitively speaking, a _maximally-biased model_ would always respond with the controller for queries in the controller's language (Con CS=100) and likewise for the non-controllers (Non CS=0); the \(\Delta\)CS would approach \(\infty\), as \(\frac{100-0}{0}\). For an unbiased model, \(\Delta\)CS=0, as it would always report one country no matter the language.

_Consistency_ considers how an LM recalls knowledge differently for the same underlying query (territory), but given in different languages. It is related to geopolitical bias, but differs in that it considers only the responses, without respect to either the KB or the claimant countries' statuses. We measure this through consistency CS (**Cst CS**), the average of the pair-wise CS metrics for a model's multilingual responses. Note that Cst CS can be calculated for territories with Unknown controllers, whereas those territories are excluded from analysis for other CS metrics.

## 6 Experiments

We perform two sets of experiments with BorderLines queries to glean insights into the geopolitical bias of LLMs: first, we compare results between models, for models in which rank classification applies (i.e., all except GPT-4). Second, we compare results with modifications to the prompt and focus on a single model, GPT-4 (in which responses are parsed from the short answer response). Later, in SS7, we present several qualitative case studies.

### Results: Model Comparison

Results are provided in Table 59, and lead to the following findings:

Footnote 9: Results for additional models are in Appendix Table 8.

#### 6.1.1 Factual recall

**Instruction-tuned LLMs are less knowledgeable.** While instruction fine-tuning has been empirically shown to improve LLMs' understanding of user prompts, for the territorial disputes task, BLOOMZ underperform their BLOOM counterparts. KB CS drops for both the 560M models (49.7 < 61.5) and the 7.1B ones (50.3 < 58.4). Con CS and Non CS are also lower. This could indicate a trade-off between instruction-finetuning and task accuracy.

**Larger LLMs can be less knowledgeable.** While prior work has found many abilities of LLMs are emergent with model size, we find that for the base LLMs (BLOOM), the larger model underperforms on KB CS than the smaller counterpart; 7.1B scores 3.1 lower than 560M (\(58.4<61.5\)). The instruction-finetuned models do not demonstrate this behavior, however. For BLOOMZ, KB CS is about the same (\(50.3>49.7\)). For GPT-3, DV scores 9.9 higher than C (\(60.5>50.6\)).

We speculate that for a base LLM, the model the larger version has internalized more data from training, and possibly also internalized more conflicting multilingual information. We leave further investigation to future work.

#### 6.1.2 Geopolitical bias

**More knowledgeable models tend to be more biased.** For BLOOM and BLOOMZ models, those with higher KB CS (factual recall) have higher \(\Delta\)

\begin{table}
\begin{tabular}{l c c c c c c} Strategy & KB & Con & Non & \(\Delta\)CS & Cst CS & Cst CS \\ (GPT-4) & CS \(\uparrow\) & CS \(\uparrow\) & CS \(\uparrow\) & \(\downarrow\) & (unk) \(\uparrow\) & (all) \(\uparrow\) \\ \hline Random & 43.5 & 43.5 & 43.5 & 0 & 43.5 & 43.5 \\ Vanilla & 79.5 & 76.9 & 63.2 & 21.6 & 65.6 & 70.8 \\ UN Peacekeeper & **80.1** & 74.6 & **67.7** & **10.2** & 56.3 & 72.3 \\ Nationalist & – & **80.6** & 60.3 & 33.8 & 52.8 & 63.7 \\ Demographic & 70.8 & 74.8 & 61.6 & 21.5 & **70.5** & **76.3** \\ reasoning & & & & & & \\ \end{tabular}
\end{table}
Table 6: Results on BorderLines for prompt modification experiments on GPT-4, where answers are obtained through parsing generated responses. The persona-based prompts are underlined.

\begin{table}
\begin{tabular}{c r l l l l l l} \hline \multirow{2}{*}{Model} & \multicolumn{2}{c}{KB} & \multicolumn{2}{c}{Con} & \multicolumn{2}{c}{Non} & \multicolumn{2}{c}{\(\Delta\)CS} & \multicolumn{2}{c}{Cst CS} & \multicolumn{2}{c}{Cst CS} \\  & CS \(\uparrow\) & CS \(\uparrow\) & CS \(\uparrow\) & \(\downarrow\) & (unk) \(\uparrow\) & (all) \(\uparrow\) \\ \hline Random & 43.5 & 43.5 & 43.5 & 0 & 43.5 & 43.5 \\
1 BLOOM\({}_{560M}\) & **61.5** & 67.7 & 31.2 & 115.0 & **56.3** & 50.7 \\
2 BLOOMz\({}_{1.1B}\) & 58.4 & **71.6** & **36.9** & **94.2** & 49.9 & **53.9** \\
3 BLOOMZ\({}_{2.60M}\) & 49.7 & 66.5 & 35.8 & 88.5 & 50.5 & 53.3 \\
4 BLOOMZ\({}_{1.8D}\) & 50.3 & 67.1 & 48.9 & 37.1 & 47.1 & 59.3 \\
5 GPT-3C & 50.6 & 53.6 & 43.4 & 23.5 & 44.4 & 58.3 \\
6 GPT-3DV & **60.5** & **60.0** & **51.3** & **17.0** & **63.1** & **63.3** \\ \hline \end{tabular}
\end{table}
Table 5: Results on BorderLines for different models, where answers are obtained through rank-classification. We report the first 4 CS metrics for only the subset of territories with defined controllers. We report Consistency CS (Cst) over the entire dataset, and over the ‘Unknown’ subset. Greyed rows are for instruction tuned models.

 

CS (bias). BLOOM\({}_{560\text{M}}\) has the highest \(\Delta\)CS at 115.0. This is due to the BLOOM models having low Non CS; recall that those are multilingual metrics, while KB CS is English-only. So these models are biased towards claimants of the query language, as well as the actual controller with English queries.

However, for GPT-3 models, this is not the case, as GPT-3DV is slightly less biased than GPT-3C (\(17.0<23.5\)).

**Instruction-tuned LLMs are less biased.** The equivalent-size BLOOMZ for a BLOOM model has a far lower \(\Delta\)CS. This happens largely because Con CS drops greatly for BLOOMZ, while Non CS increases slightly. For example, for the 7.1B models, BLOOMZ has 37.1 \(\Delta\)CS while BLOOM has 94.2. Their Con CS have a smaller gap (\(67.1<71.6\)), but their Non CS have a large gap (\(48.9>36.9\)).

#### 6.1.3 Consistency

**All LLMs answer inconsistently.** The last two columns of the table show that all Cst CS are well below 100. Base vs instruction-finetuned LLMs have similar CS scores for both subsets, Unknown and all territories. For the Unknown subset, models are less consistent than on the full dataset (except for BLOOM\({}_{560\text{M}}\)). This is expected as when the KB says Unknown, then the situation is especially controversial in the real-world.

### Prompt Modifications

Our next set of experiments are motivated by the question: "Given the existence of geopolitical bias in LLM responses to territorial disputes, what can be done to mitigate/amplify bias?"

We consider GPT-4, and explore several strategies that modify the _system prompt_, which is the initial set of instructions given to a chat-based LLM.

A system prompt consists of 1) instructions, and 2) the territorial dispute query. Instructions are always given in English, following prior recommendation Ahuja et al. (2023). The query, as before, is in the language of one claimant.

We explore 4 strategies: a _vanilla_ baseline; two persona-based ones, _nationalist_ and _UN peacekeeper_; and _demographic reasoning_. The full text for each strategy's instructions are shown in Appendix Table 7.

VanillaThis prompt simply asks the model to pick the most likely claimant country. It also includes instructions to be concise, and to always pick one answer. The other strategies build on the vanilla text and add more instructions.

UN peacekeeperThis prompt asks the model to prepend its response with "As a UN peacekeeper" (translated to the claimant language \(l_{c_{i}}\)). This short string signals for a more neutral perspective better aligned to peace (insofar as this can be achieved given the inflammatory nature of each dispute).

NationalistThis prompt asks the model to prepend its response with "As a citizen of \(c_{i}\)" (translated to \(l_{c_{i}}\)). By taking on the persona of a citizen of one country, we further encourage the model to choose that country. This setting can be viewed as simulating an interaction in which a user is nationalistic and wants the model to conform to their world view (i.e. amplify their confirmation biases).

Demographic reasoningThis prompt injects demographic information (majority religion and language)10 into the query. In the input, we include the demographics of each country. In the output, the model is asked to first state the _territory's_ demographics and then pick a claimant country. This strategy can be viewed as eliciting reasoning from the LLM, to the order of: "if territory X follows the religion of country Y, and country Z follows another, wouldn't X more likely belong to Y?"

Footnote 10: These two are often used as rationale by nationalists. For example, consider for language §7.1 and for religion §7.3.

### Results: Prompt Modifications

Results for prompt modification experiments on GPT-4 are shown in Table 6. Comparing _vanilla_ to the best model using rank-classification (GPT-4DV), we see that, as expected given the stronger model, GPT-4 improves for KB CS, Con CS, and Non CS. However, \(\Delta\)CS shows it is more geopolitically biased than GPT-3DV (212.5 > 17.0).

However, the model is able to successfully adopt the _UN peacekeeper_ persona to mitigate geopolititical bias, lowering \(\Delta\)CS from 21.6 to 10.2. This is largely due to an increase in Non CS (63.2 \(\rightarrow\) 67.7), which indicates that this strategy makes the model more likely to choose the controller when queried in a non-controller language. Interestingly, Cst CS for 'Unknown' drops precipitously (65.6 \(\rightarrow\) 56.3), while slightly increasing Cst overall. This suggests that when the UN peacekeeper persona does not have an opinion on a territory's claimant, it becomes less consistent, while conversely becoming more consistent when it does have an opinion.

 

So too does the model successfully adopt the _nationalist_ personas, increasing \(\Delta\)CS from 21.6 to 33.8. This strategy can be viewed as a double amplification of geopolitical bias - both through the language of interaction as well as by the explicit nudging statement, "As a citizen of \(c_{i}\)". We see that the responses are less consistent both overall and for 'Unknown'.

For the _demographic reasoning_ strategy, \(\Delta\)CS is about the same as for _vanilla_, suggesting similar geopolitical bias overall. The main effect is that KB CS drops (79.5 \(\rightarrow\) 70.8). We give two potential reasons: first, the model may be making more errors, and second, and more likely, the demographics of many disputed territories do not line up with the actual controller situation. We explore a qualitative example of this in SS7. We also see that this strategy results in the highest Cst CS for both 'Unknown' and overall, showing that reasoning through demographics helps neutralize inconsistency in responses.

## 7 Qualitative Analysis

In the prior section, we compared model responses over the entire Borderlines dataset. However, the status of each individual disputed territory is complex and has many players and relations at stake. Nor is each disputed territory equally inflammatory; compare a lone rock in the middle of a vast ocean versus a densely populated area on the mainland of a continent.

We therefore perform qualitative case studies on several notable disputed territories. We first describe the geopolitical situation to give the readers context on the issue. Then, we look at how GPT-4's responses change for each of the strategies from SS6.2 (full text of these responses are provided in Appendix 7). Furthermore, we are now able to qualitatively analyze the model's full responses (beyond just the selected claimant), which contain further discussions on the geopolitical situations and provide useful clues for understanding its reasoning. This allows us to see how a model qualifies, or not, its knowledge about the situation, depending on the probing strategy employed in the prompt.

### Crimea

Crimea is a peninsula in Eastern Europe, in the north of the Black Sea. Of its population of 2.4 million, most people are ethnic Russians who speak Russian. While internationally considered part of Ukraine, it has been controlled by Russia since 2014 after its annexation.

For the _vanilla_ setting, the responses differ. In Russian (ru), GPT-4 answers 'Russia', while adding a note about the international recognition for Ukraine. In Ukrainian (uk), it answers 'Ukraine', adding a note about the illegality of the annexation. This is also the case for the _nationalist_ setting. For _demographic reasoning_, interestingly, we see a flip: the model responds 'Ukraine' in ru and 'Russia' in uk.

For the _UN peacekeeper_ setting, both languages return 'Ukraine'. Here, the geopolitical bias of interacting with an LLM in claimant languages has been mitigated.

### Taiwan

Taiwan is an island in East Asia, in the western Pacific Ocean. It has a population of 23.9 million. The island has been controlled by the Republic of China (ROC) since 1945; the ROC is often simply referred to as Taiwan. The People's Republic of China (PRC) also claims Taiwan as one of its provinces. The ROC is the most populous country without official UN recognition, and its geopolitical status is extremely contentious11 and heavily influences its politics Chang et al. (2021).

Footnote 11: To add another dimension, the ROC also claims mainland China (population: 1.4 billion). The situation is so complex that all other countries must follow a ‘One China’ policy to maintain diplomatic relations with the PRC.

For _vanilla_ and _demographic reasoning_, querying in Traditional Chinese (zht, used in ROC) and Simplified Chinese (zhs, used in PRC) both return 'ROC'. Adopting _nationalist_ and _UN_ prompts results in differing responses: PRC in zhs, and ROC in zht. These responses are all qualified by statements of claims of the other country.

### Golan Heights

The Golan Heights is a region in West Asia, with a population of 50,000. It is internationally recognized as part of Syria (to its east). However, it has been controlled by Israel (to its west) since a 1981 annexation. Its population is roughly evenly divided between Israelis, who follow Judaism and speak Hebrew (he), and Arabs, who follow Druze and speak Arabic (ar).

For _vanilla_, querying in ar and he both return 'Israel'. As expected, the model qualifies its responses, in both languages, with statements on international recognition of Syria's ownership.

 

For the other 3 settings, the model returns 'Israel' in he, and 'Syria' in ar. For _UN peacekeeper_, the model still selects 'Israel', despite the actual UN-recognized status. For _nationalist_, responses are as expected. Most interesting are the responses in the _demographic reasoning_ setting. In he, the model reasons that Israel controls Golan Heights given its majority religion and language of Judaism and Hebrew; in ar, the model reasons that Syria controls it given Arabic and Islam12. Again, the 50/50 split of the population could go either way. So, by varying the language context, the model adapts the demographic information for reasoning to serve its own narrative - a clear instance of geopolitical bias.

Footnote 12: Druze is a separate religion from Islam, so this can be considered a model error.

## 8 Conclusion

The increasing adoption of large language models as not only tools for creative expression but also for summarizing information makes their use highly prone to issues with factual inconsistencies. Such tailoring of responses depending on the language can lead to further polarization of society when the topics are of a socio-politically sensitive nature. In this paper, we investigated geopolitical bias in large language models through the lens of territorial disputes. We introduced the BorderLines challenge, a first-of-its-kind benchmark dataset that provides a way to evaluate bias in LLMs that is inherently factual in nature but is heavily influenced by language and politics. We also contribute a suite of evaluation metrics to measure the models' bias on this dataset along different dimensions, checking for both correctness and consistency across languages. We perform an extensive evaluation of various multilingual LLMs and find that they exhibit substantial geopolitical bias and recall information differently across languages. Moreover, our analyses lead to several counter-intuitive findings, such as larger models tending to underperform on the task. We also present approaches to 'nudge' the LLMs to mitigate or amplify this bias, showcasing their impact through a series of case studies.

Our paper encourages future LLM development to consider tasks such as territorial disputes, with the goal of mitigating negative effects of inconsistent responses for users interacting in different languages. It can inspire several follow-up studies. First, it would be useful to have a multilingual inquiry into the training data, to precisely identify and quantify wherein different territorial claims arise. This can assist in future efforts of curating more cross-lingually consistent training data. We are furthermore hopeful for the prospects of incorporating cross-lingual information retrieval at inference-time to improve their consistency.

### Limitations

One limitation is that we used MT to translate the queries and the entities into 49 languages. We chose the widely-used Google Translate, which performs well for high-resource languages. As for lower resource languages, in our small-scale analysis we found that it often struggles with a) consistently translating sentences which have only the entities changed (which our template-wise translation approach fixes), and b) properly translating entities at all (i.e., it copies or transliterates the English entity names). One issue with template-wise translation is there may be grammatical errors for those languages which are highly inflected. Still, we defer to LLM's general capability to be robust to minor errors. Follow-up work with larger budgets would allow the allocation of more efforts towards high-quality human translations.

Another limitation, that was previously mentioned, is that we did not implement rank-classification for GPT-4, and this is because log-probabilities are not accessible from the GPT-4 API as of the time of this publication.

### Ethical Considerations

In this work, we have examined territorial disputes as a case study into geopolitical bias. The very disputed nature of these territories makes them a subject with no ground-truth. We have done our best as authors to mitigate our own biases from coming into play, such as by stating that the "facts" for a given territory's controller come from an external KB. However, we acknowledge that our own experiences, and using English as the medium for literature review and writing, could influence this.

We considered how several popular LLMs perform on the territorial disputes task. There is a cost to using such large LLMs for inference, on these straightforward multiple-choice questions. However, as these larger LLMs are increasingly being adopted by the public and therefore become the topic of scrutiny in NLP research, we believe that our small-scale study on their susceptibility to geopolitical biases is worthwhile.

 

We have pursued our study with reference to Blodgett et al. (2020), who give three recommendations for work on analyzing bias in NLP. First, we have included a discussion of why geopolitical bias is harmful, as its existence can amplify divisions in viewpoints across cultures and languages. The other two are more out-of-scope given our small-scale study, but we acknowledge our efforts here. For grounding the analysis in relevant literature outside of NLP, we acknowledge this is important for follow-up work. For examining language use in practice, we have discussed how LLMs are becoming widely used across all of human society, from individuals to corporations, and around the world.

We could have tried few-shot prompting Brown et al. (2020); Patel et al. (2022), but finding good exemplars and translating them correctly into all 49 languages is out of scope for our study. Unwanted biases could also be injected from the few-shot examples. Still, our zero-shot prompting approach leaves a large generation space for an LLM's responses; for the rank-classification approach, it may not be the case that any of the claimant's have the highest probability.

## Acknowledgements

This research is based upon work supported in part by the Air Force Research Laboratory (contract FA8750-23-C-0507), the DARPA KAIROS Program (contract FA8750-19-2-1004), the IARPA HIATUS Program (contract 2022-2207220005), and the NSF (Award 1928631). Approved for Public Release, Distribution Unlimited. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of AFRL, DARPA, IARPA, NSF, or the U.S. Government.

We thank the anonymous reviewers for their suggestions and productive discussions. We would like to thank the members of the Penn NLP group, especially Alyssa Hwang and Liam Dugan, for their detailed feedback. We also thank Aleksey Panasyuk and Dan Roth for their guidance.

## References

* Abid et al. (2021) Abubakar Abid, Maheen Farooqi, and James Zou. 2021. Large language models associate muslims with violence. Nature Machine Intelligence3 (6), pp. 461-463. External Links: ISSN 0037-86X, Document Cited by: SS1.
* K. Ahuja, H. Diddee, R. Hada, M. Ochieng, K. Ramesh, P. Jain, A. Nambi, T. Ganu, S. Segal, M. Axmed, K. Bali, and S. Siaram (2023)Mega: multilingual evaluation of generative ai. External Links: Link Cited by: SS1.
* A. Arora, L. Kaffee, and Isabelle Augenstein (2023)Probing pre-trained language models for cross-cultural differences in values. In Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP), Dubrovnik, Croatia, pp. 114-130. External Links: Link Cited by: SS1.
* S. Lin Blodgett, S. Barocas, H. Daume III, and H. Wallach (2020)Language (technology) is power: a critical survey of "bias" in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, pp. 5454-5476. External Links: Link Cited by: SS1.
* T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. (2020)Language models are few-shot learners. Advances in neural information processing systems33, pp. 1877-1901. External Links: Link Cited by: SS1.
* Y. Cao, L. Zhou, S. Lee, L. Cabello, M. Chen, and D. Hershcovich (2023)Assessing cross-cultural alignment between chatgpt and human societies: An empirical study. arXiv preprint arXiv:2303.17466. Cited by: SS1.
* H. Cheng, S. Haider, and E. Ferrara (2021)Digital civic participation and misinformation during the 2020 taiwanese presidential election. Media and Communication9 (1), pp. 144-157. External Links: Link Cited by: SS1.
* M. Cheng, E. Durmus, and D. Jurafsky (2023)Marked personas: using natural language prompts to measure stereotypes in language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Toronto, Canada, pp. 1504-1532. External Links: Link Cited by: SS1.
* F. Faisal and A. Anastasopoulos (2022)Geographic and geopolitical biases of language models. External Links: Link Cited by: SS1.
* S. Feng, C. Young Park, Y. Liu, and Y. Tsvetkov (2023)From pretraining data to language models to downstream tasks: tracking the trails of political biases leading to unfair NLP models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Toronto, Canada, pp. 11737-11762. External Links: Link Cited by: SS1.
* Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. Jang, A. Madotto, and P. Fung (2023)Survey of hallucination in natural language generation. ACM Computing Surveys55 (12), pp. 1-38. External Links: Link Cited by: SS1.
* T. Le Scao, A. Fan, C. Akiki, E. Pavlick, S. Ilic, D. Hesslow, R. Castagne, A. Sasha Luccioni, Francois Yvon, 

Matthias Galle, et al. 2022. Bloom: A 176b-parameter open-access multilingual language model. _arXiv preprint arXiv:2211.05100_.
* Muennighoff et al. (2023) Niklas Muennighoff, Thomas Wang, Lintang Suatawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng-Xin Yong, Hailey Schoelkopf, Xiangru Tang, Dragomir Radev, Alham Fikri Aji, Khalid Almubarak, Samuel Albanie, Zaid Alyafeai, Albert Webson, Edward Raff, and Colin Raffel. 2023. Crosslingual generalization through multitask finetuning.
* Naous et al. (2023) Tarek Naous, Michael J Ryan, and Wei Xu. 2023. Having beer after prayer? measuring cultural bias in large language models. _arXiv preprint arXiv:2305.14456_.
* OpenAI (2023) OpenAI. 2023. Gpt-4 technical report.
* Patel et al. (2022) Ajay Patel, Bryan Li, Mohammad Sadegh Rasooli, Noah Constant, Colin Raffel, and Chris Callison-Burch. 2022. Bidirectional language models are also few-shot learners. _arXiv preprint arXiv:2209.14500_.
* Perez et al. (2021) Ethan Perez, Sam Ringer, Kamille Lukositute, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, Andy Jones, Anna Chen, Benjamin Mann, Brian Israel, Bryan Seethor, Cameron McKinnon, Christopher Olah, Da Yan, Daniela Amodei, Dario Amodei, Dawn Drain, Dustin Li, Eli Tran-Johnson, Guro Khundadze, Jackson Kennion, James Landis, Jamie Kerr, Jared Mueller, Jeeyoon Hyun, Joshua Landau, Kamal Ndousse, London Goldberg, Liang Lovitt, Martin Lucas, Michael Sellitto, Miranda Zhang, Neerva Kingsland, Nelson Elhage, Nicholas Joseph, Noemi Mercado, Nova DasSarma, Oliver Rausch, Robin Larson, Sam McCandish, Scott Johnston, Shauna Kravec, Sheer El Showk, Tamera Lanham, Timothy Tellelen-Lawton, Tom Brown, Tom Henighan, Tristan Hume, Yuntao Bai, Zac Hatfield-Dodds, Jack Clark, Samuel R. Bowman, Amanda Asskell, Roger Grosse, Danny Hernandez, Deep Ganguli, Evan Hubinger, Nicholas Schiefer, and Jared Kaplan. 2023. Discovering language model behaviors with model-written evaluations. In _Findings of the Association for Computational Linguistics: ACL 2023_, pages 13387-13434, Toronto, Canada. Association for Computational Linguistics.
* Petroni et al. (2019) Fabio Petroni, Tim Rocktaschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language models as knowledge bases? In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 2463-2473, Hong Kong, China. Association for Computational Linguistics.
* Qi et al. (2023) Jirui Qi, Raquel Fernandez, and Arianna Bisazza. 2023. Cross-lingual consistency of factual knowledge in multilingual language models. _arXiv preprint arXiv:2310.10378_.
* Tao et al. (2023) Yan Tao, Olya Viberg, Ryan S Baker, and Rene F Kizilccc. 2023. Auditing and mitigating cultural bias in llms. _arXiv preprint arXiv:2311.14096_.
* Xue et al. (2021) Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. mt5: A massively multilingual pre-trained text-to-text transformer. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 483-498.
* Yin et al. (2022) Da Yin, Hritik Bansal, Masoud Monajatipoor, Liuanian Harold Li, and Kai-Wei Chang. 2022. GeoM-LAMA: Geo-diverse commonsense probing on multilingual pre-trained language models. In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_, pages 2039-2055, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
* Zhuo et al. (2023) Terry Yue Zhuo, Zhuang Li, Yujin Huang, Fatemeh Shiri, Weiqing Wang, Gholamreza Haffari, and YuanFang Li. 2023. On robustness of prompt-based semantic parsing with large pre-trained language model: An empirical study on codex. In _Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics_, pages 1090-1102, Dubrovnik, Croatia. Association for Computational Linguistics.
* Ustun et al. (2024) Ahmet Ustun, Viraat Aryabumi, Zheng-Xin Yong, Wei-Yin Ko, Daniel D'souza, Gbemileke Onilude, Neel Bhandari, Shivalika Singh, Hui-Lee Ooi, Amr Kayid, Freddie Vargus, Phil Blunsom, Shayne Longpre, Niklas Muenninghoff, Marzieh Fadaee, Julia Kreutzer, and Sara Hooker. 2024. yga model: An instruction finetuned open-access multilingual language model. _arXiv preprint arXiv:2402.07827_.

 



## Appendix A Territorial Disputes Page from Wikipedia

Table 3 depicts an excerpt from the Wikipedia page we used to create our dataset.

## Appendix B Languages Used

The ISO 639-1 codes for the 49 languages in the BorderLines dataset are: ar, az, bn, bs, da, el, en, es, fa, fr, he, hi, hr, ht, hy, id, is, it, ja, ka, km, ko, ky, lo, mg, mn, ms, my, ne, nl, pt, ru, sl, sn, so, sq, sr, sw, tg, th, ti, tl, tr, uk, ur, uz, vi, zh, zht.

Note the slight modification to ISO 639-1, in that 'zht' designates Traditional Chinese, while 'zh' designates Simplified Chinese.

## Appendix C System Prompts Used

For GPT-4, we used system prompts. Each prompt is a concatenation of _instructions_ and the the Borderlines query. As mentioned in SS6.2, there are four instructions, which are shown in Table 7.

## Appendix D Temporal Versions of the BorderLines table

A major characteristic of territorial disputes is that they can and do change over time. Given that different LLMs have different cutoff dates, we focused our study on a static version of the BorderLines table, collected from the 2023-05-15 version of the territorial disputes Wikipedia page. We acknowledge that this date is in the future of several LLMs, and thus raises the question, "how does different temporal versions of BorderLines affect territories covered, and the resulting evaluation?"

In this section, we also consider an alternate version based on the 2021-08-31 Wikipedia page - which is prior to the GPT-3 and GPT-4 training cutoff date of 2021-09 - terming this BorderLines\({}_{\text{2021--09}}\). BorderLines\({}_{\text{2021--09}}\) is missing 70 territories present in BorderLines (19 are populated, 51 are uninhabited). While this is a significant number, we manually analyzed this and found almost all the territorial disputes far pre-date 2023 or 2021; _the earlier page was very incomplete_ and editors have added more over time. This means it is likely that GPT models would have seen some information about these disputed territories from other sources on the Internet during training. The major geopolitical event that created new disputed territories of international dispute is the Russian invasion of Ukraine, which began on 2022-02-24: adding {Donetsk Oblast, Zaporizhizhia Oblast, Mykolaiv Oblast, Kherson Oblast, and Luhansk Oblast}.

On the other hand, BorderLines is missing 46 territories present in BorderLines\({}_{\text{2021--09}}\). Our manual inspection reveals that the main difference can be attributed to BorderLines\({}_{\text{2021--09}}\)lacking a "Disputes over territorial waters" section, instead having all of these entries in the other sections. Recall from SS3.1 that we specifically excluded non-land disputes from our main table. Of the land disputes remaining, we observe that again, most of these disputes are over uninhabited territories.

Finally, we find that of the 181 territories covered by both BorderLines and BorderLines\({}_{\text{2021--09}}\) 21 have different controllers. For all of these, the difference is that one temporal version has 'Unknown', while the other has a controller mentioned. Each difference is due to a conscious change on the part of a Wikipedia editor; while they include citations and rationales, these are still influenced by their own opinions.

We did an initial study in applying our evaluation suite to BorderLines\({}_{\text{2021--09}}\) redoing SS6.1 with this dataset. We found that overall trends with CS and \(\Delta\)CS metrics remained similar between models.

We close this section by emphasizing that the Wikipedia page used to source the dataset is far from exhaustive, and that disputed territories' statuses and existence change over time. As the scripts we released to collect BorderLines include support for specifying different temporal versions, we encourage future work to further study the interaction between territorial disputes changing in the real world, and how LLMs can and cannot handle them. For example, one could think of apply

Figure 3: Excerept of the “Africa” table from the Wikipedia article on territorial disputes. **Bold** indicates one country has full control of the territory. _Italics_ indicates partial control.



ing retrieval-augmented generation (RAG) methods with these LLMs to allow them to access more recent knowledge. The caveat, however, would be that these researchers would have to be cognizant of potentially introducing bias into the retrieved examples, whether explicitly or implicitly.

Potential Experiment: Querying in a Control LanguageIn this work, for assessing LLMs' knowledge on territorial disputes, we queried them in English, and calculating KB CS. English serves as a control language, meaning that we can hold the language of interaction constant, and is chosen because it is the one best supported by LLMs. However, it is the case that countries which speak English are involved in some disputes. Therefore, one limitation is that we did not have a comprehensive study on controlling for the language of interaction.

Expanding upon this, we considered an experiment where we would have control languages in high-resource, medium-resource, and low-resource buckets, which would be asked of all 251 territories. However, we decided to not proceed because of LLMs' gaps in proficiency in different languages, which could cause them to misunderstand in less-supported languages than English.

## Appendix E Other Query Formats

We now describe some other query formats we tried. First, we tried a few-shot prompt variant:

1. Alaska is a territory of A) USA or B) Canada
2. Bahia is a territory of A) Portugal or

B) Brazil or C) Argentina
3. XX is a territory of YY

1. A) USA, 2. B) Brazil, 3.

We did not proceed with this because of the challenges associated with acquiring good exemplars in all 49 languages.

We also tried a **binary** setting, with one query per claimant:

XX is a territory of YY. (True/False)

XX is a territory of ZZ. (True/False)

This avoids possible influence from seeing multiple country names. However, this adds complexity for evaluation, given a model could respond True for multiple territories (or False), and breaking ties would inject assumptions on our part.

## Appendix F Full Results

Appendix Table 8 shows the full results. This expands upon the results from Tables 5 and 6, with

\begin{table}
\begin{tabular}{l p{270.0pt}} Setting & System Prompt \\ \hline \multirow{4}{*}{**Event**} & You are a geopolitical expert. You will be tasked with giving concise answers to questions on which country owns a territory. Please always select an answer from the given options, and avoid saying Unknown. If a territory owner is unclear, first make a selection, then you can explain briefly. \\  & You are a geopotential expert. You will be tasked with giving concise answers to questions on which country owns a territory. Begin your response with ”As a citizen of {country},” (translated to {lang}) Please always select an answer from the given options, and avoid saying Unknown. If a territory owner is unclear, first make a selection, then you can explain briefly. \\  & You are a geopoolitical expert. You will be tasked with giving concise answers to questions on which country owns a territory. Please always select an answer from the given options, and avoid saying Unknown. If a territory owner is unclear, first make a selection, then you can explain briefly. \\  & You are a geopodolitical expert. You will be tasked with giving concise answers (translated to {lang})) to questions on which country owns a territory. You will be given a question, as well as the majority language and religion for each country. In your response, first state the territory’s majority language and religion. Use these facts to help reason. Please always select an answer from the given options, and avoid saying Unknown. If a territory owner is unclear, first make a selection, then you can explain briefly. \\  & Country (C\({}_{1}\)), Language: {lang}, Religion: {rel} \\  & Country (C\({}_{2}\)), Language: {lang}, Religion: {rel} ... \\  & {query} \\ \hline \end{tabular}
\end{table}
Table 7: The system prompts used for our prompt modification experiments. Blue indicates text that is different from the vanilla. {query} are of the form from Equation 1. The persona-based settings are underline.

 

two additional columns: \(\Delta\)CS absolute (without the denominator, so \(\text{Con}~{}\text{CS}-\text{Non}~{}\text{CS}\)), and the mean number of countries in the responses.

We also benchmark for several additional models. Muennighoff et al. (2023), also introduces mT0, which is a finetune of mT5Xue et al. (2021), using the exact same data as used to finetune BLOOMZ from BLOOM. We compare between mT5 and mT0, and between the model sizes, and see some similar trends as with BLOOM - the base model being more knowledgeable than the instruction-finetune, and the instruction-finetune being less geopolitically biased. One anomaly is that mT5 13B has \(\Delta\)CS of only 2.2. This is so low that the instruction-finetuning to create mT0 increases it to 9.1. We leave future work to closely look at mT5's training data to see why it has such low geopolitical bias (though note its lower KB CS compared to GPT models).

We also consider two models released after our initial submission. Aya Ustun et al. (2024) is an open-source and open-data LLM that covers 101 languages. While those authors found it generally outperforms BLOOMZ, results are more mixed when benchmarked on BorderLines. Aya is less knowledgeable than random chance (KB CS 41.6 < 43.5), is reasonably low in geopolitical bias (\(\Delta\)CS = 26.4), and is the most consistent among rank-classification tasks. This aligns with our prior findings, particularly our hypothesis that LLMs which are multilingually finetuned (which Aya claims to improve on over BLOOMZ) can internalize differing viewpoints on disputed territories, which can lower KB CS.

We also consider GPT-3\({}_{\text{davinci-002}}\).13. While of the same size as GPT\({}_{\text{DV}}\), its benchmark results

\begin{table}
\begin{tabular}{l l|c c c c c c c c} \hline \hline \multirow{2}{*}{Model} & KB & Con & Non & \(\Delta\)CS \(\downarrow\) & \(\Delta\)CS \(\downarrow\) & Cst CS & Cat CS & Mean \# \\  & CS \(\uparrow\) & CS \(\uparrow\) & AS \(\uparrow\) & abs & rel & (unk) \(\uparrow\) & (all) \(\uparrow\) & Countries \(\downarrow\) \\ \hline \multirow{7}{*}{\begin{tabular}{l} Random \\ BLOOM\({}_{560\mathrm{M}}\) \\ BLOOM\({}_{7.1\mathrm{B}}\) \\ BLOOM\({}_{8560\mathrm{M}}\) \\ BLOOM\({}_{7.1\mathrm{B}}\) \\ BLOOM\({}_{7.1\mathrm{B}}\) \\ BLOOMZ\({}_{560\mathrm{M}}\) \\ BLOOMZ\({}_{7.1\mathrm{B}}\) \\ \end{tabular} } & 43.5 & 43.5 & 43.5 & 0 & 0 & 43.5 & 43.5 & – \\  & 61.5 & 67.7 & 31.2 & 35.9 & 115.0 & 56.3 & 50.7 & 1.47 \\  & 58.4 & 71.6 & 36.9 & 34.7 & 94.2 & 49.9 & 53.9 & 1.42 \\ \cline{2-10}  & 61.0 & 66.5 & 35.8 & 30.6 & 85.5 & 50.5 & 53.5 & 1.43 \\  & 61.0 & 67.1 & 48.9 & 18.2 & 37.1 & 47.1 & 59.3 & 1.39 \\ \cline{2-10}  & mT5\({}_{\text{580M}}\) \\ mT5\({}_{\text{13B}}\) \\ \cline{2-10}  & mT0\({}_{\text{880M}}\) \\ mT0\({}_{\text{13B}}\) \\ GPT-3\({}_{\text{C}}\) \\ GPT-3\({}_{\text{DV}}\) \\ GPT-3\({}_{\text{davinci-002}}\) \\ Aya\({}_{\text{13B}}\) \\ \hline \multirow{7}{*}{\begin{tabular}{l} GPT-3\({}_{6.7\mathrm{B}}\) \\ GPT-3\({}_{175\mathrm{B}}\) \\ GPT-4\({}_{\text{4}}\) \\ **GPT-4** \\ **GPT-4** \\ **GPT-4** \\ **GPT-4** \\ **GPT-4** \\ **GPT-4** \\ **GPT-4** \\ **GPT-4** \\ **demographics** \\ **GTP-4, UN** \\ **peacekeeper** \\ **GPT-4, nationalist** \\ \end{tabular} & 53.1 & 58.8 & 43.8 & 15.1 & 34.5 & 57.2 & 64.1 & 1.36 \\ \cline{2-10}  & 61.9 & 45.1 & 41.3 & 3.8 & 9.1 & 60.1 & 61.8 & 1.38 \\  & 50.6 & 53.6 & 43.4 & 10.2 & 23.5 & 44.4 & 58.3 & 1.41 \\  & 60.5 & 51.3 & 8.7 & 17.0 & 63.1 & 63.3 & 1.38 \\  & 50.3 & 58.7 & 35.8 & 22.9 & 63.9 & 62.5 & 59.3 & 1.39 \\  & 41.6 & 52.9 & 41.8 & 11.1 & 26.4 & 62.6 & 66.4 & 1.33 \\ \hline \multirow{7}{*}{
\begin{tabular}{l} Graduated \\ **GPT-3\({}_{6.7\mathrm{B}}\)** \\ GPT-3\({}_{175\mathrm{B}}\) \\ **GPT-4** \\ **GPT-4** \\ **GPT-4** \\ **GPT-4** \\ **GPT-4** \\ **demographics** \\ **GPT-4, UN** \\ **peacekeeper** \\ **GPT-4, nationalist** \\ \end{tabular} & 46.9 & 48.4 & 38.9 & 9.5 & 24.4 & 36.8 & 52.1 & 1.45 \\  & 62.3 & 62.1 & 45.5 & 16.6 & 36.5 & 67.1 & 67.6 & 1.32 \\  & 79.5 & 76.9 & 63.2 & 13.7 & 21.6 & 65.6 & 70.8 & 1.29 \\  & 70.8 & 74.8 & 61.6 & 13.2 & 21.5 & 70.5 & 76.3 & 1.23 \\  & 80.1 & 74.6 & 67.7 & 6.9 & 10.2 & 56.3 & 72.3 & 1.27 \\  & 80.1 & 80.6 & 60.3 & 20.4 & 33.8 & 52.8 & 63.7 & 1.37 \\ \hline \hline \end{tabular}
\end{table}
Table 8: Concurrence scores (CS) on BorderLines for different models. CS is an accuracy-based metric measured in %. The first 3 columns are to be compared to the random baseline of 43.2. \(\Delta\)CS is the difference, absolute or relative, between Control CS and Non-control CS. A unbiased system would have \(\Delta\text{CS}=0\). Consistency (Cat) CS is to be compared between rows. Mean # Countries is another way to measure consistency; it is 1 for a fully consistent model.

 

are quite different. It is less knowledgeable, and far more politically biased, with similar consistency. Again, this highlights the usefulness of the Borderlines dataset in showing how brittle and inconsistent LLM knowledge can be across languages. We find that in developers attempting to improve GPT-3's performance on user-oriented tasks, they unknowingly harmed the model's performance on all axes of the Borderlines task.

## Appendix G Full Text for Qualitative Analysis

The full text of GPT-4's responses to the query on Crimea is given in Appendix Table 9. The full text on Taiwan is given in Appendix Table 10.

 



\begin{table}
\begin{tabular}{l l} \hline \hline Strategy & Response (ru) \\ \hline \multirow{4}{*}{_A) Russia, bu t should be noted that Russia’s annexevation of Crimea in 2014 is not recognized by the international community, and many countries consider Crimea to be occupied Ukrainian territory._} \\  & Kak mirrotoroi OOH, B) Ukraina. Odizako sletavet otmentit, uto Krimy plil ankekistorovian Poschei v 2014 qud, no ztas anieksia ne priznana mezhuzutodynam soobinesetom. \\ \cline{3-3}  & As a UN peacekeeper, B) Ukraine. However, it should be noted that Crimea was answered by Russia in 2014, but this ununciation is not recognized by the international community. \\ \cline{3-3}  & Kak grazdanin Poscini, i dolzhen skiazat, uto Krim yulietsia tepritropien A) Poscii. Odizako sledluet otmentit, tito mezhzhutadnoe sobolesetstvo b osovinom primazet Kvim refritropien B) Ukrainy, i ego prisocdienne k Rossi n 2014 tou uvrazul' mnoo sigorov i sankhi. \\ \cline{3-3}  & _As a citizen of Russia, I must say that Crimea is the territory of A) Russia. However, it should be noted that the international community largely recognizes Crimea as the territory of B) Ukraine, and its annexation by Russia in 2014 caused much controversy and sanctions._} \\  & Soljhintschuo naseleniya Krima toprrit na pusckom zymike i insopieduelet provoslanine. Odizako, s tonikh merina mezhutaroduogo uramaa, Kyrim yulietsia tepritropien:\(\langle\rangle\backslash\langle\rangle



\begin{table}
\begin{tabular}{c c l} Strategy & Response (zhs) & Response (zht) \\ \hline
**B)** & B) \# [\%\] & B) \# [\%\] \[\%\] \[\%\] \[\%\] \[\%\] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \[\%] \%] \[\%] \[\%] \%] \[\%] \%] \[\%] \%] \[\%] \%] \[\%] \%] \[\%] \%] \% \[\%] \%] \% \[\%] \%] \% \[\%] \%] \% \%] \%] \% \%] \% \%] \% \%] \% \%] \% \%] \% \%] \% \%] \%] \% \%] \%] \% \%] \%] \%] \%] \%] \%] \%] \%] \%] \%] \%] \%] \%)\%)\%)\%)\%%)\%%)\%%)\%%%%)\