# Comparative Analysis of BERT and GPT for Classifying Crisis News with Sudan Conflict as an Example
## 3 Data Sources  
A total of 423 articles (such as in Table 1) related to the Sudan Conflict, covering January 2024 to November 2024, were selected and analyzed by multiple Sudan Conflict experts as the ground truth. The annotated dataset covered 17 categories including military operations, damage or destruction of civilian critical infrastructure, willful killing of civilians, etc. (detailed in Table 2). \begin{table}
\begin{tabular}{c c} \hline \hline
**Attribute** & **Example** \\ \hline Date & 27 September 2024 \\ \multirow{2}{*}{Incident Narrative} & On September 27, continued fierce fighting between the two warring parties and their supporters for the control of El Fisher, North Durfur. Radio Dabanga reported that 19 people were killed and dozens injured by continued RSF artillery shelling on the city. The El Fisher livestock market was also reported hit by bombing. \\ \multirow{2}{*}{Ground Truth Labels} & Military operations (battle, shelling), Indiscriminate use of weapons \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The structure of the incident dataset and an example of an incident. \begin{table}
\begin{tabular}{c c} \hline \hline
**Category** & **Definition** \\ \hline Unlawful attention & Refers to the act of detaining or confining an individual without legal justification or due process. For example, if protesters are arrested and detained without any legal basis during peaceful demonstrations with no access to legal representation, this would be considered unlawful detention. \\ \multirow{4}{*}{Human trafficking} & Refers to the act of recruiting, transporting, transferring, harboring, or receiving individuals through force, fraud, coercion, or other forms of deception for the purpose of exploitation. Exploitation can take many forms, including forced labor, sexual exploitation, slavery, servitude, or the removal of organs. It is considered a severe violation of human rights and is illegal under international and domestic laws. \\ \multirow{2}{*}{Enslavement} & Refers to the act of exercising ownership or control over another person, treating them as property, and depriving them of their freedom. It often involves forcing individuals to perform labor or services under coercion, violence, or the threat of punishment. \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Category definitions for conflict-related incidents. Each article was labeled as one or more categories based on its content. To ensure consistency and accuracy, the labels were cross-verified by multiple experts. Given the overlapping nature of certain categories, challenges arose in distinguishing between similar classifications. To address this, detailed category definitions were developed by the Sudan Conflict experts who were also responsible for the labeling. These guidelines provided annotators with clear decision-making criteria, minimizing ambiguity and ensuring uniform interpretation of classification rules across all annotated articles. Table 1 presents the data structure and an example of an incident. The cell \"Incident Narrative\" was put into the various open-source LLMs, and the 'Ground Truth' field was used to verify and validate the outputs of the LLMs. Table 2 outlines each category and its corresponding definition, which was also used in the prompts for the RAG-ICL approach, where definitions guide classification via retrieved in-context examples. The dataset was partitioned into training (90%) and testing (10%) subsets, with an additional 85/15 validation split within the training portion. Table 3 summarizes the distribution of articles across incident categories for the training, validation, and test sets, highlighting the dataset's constrained size and pronounced class imbalance. \begin{table}
\begin{tabular}{c c} \hline \hline
**Category** & **Definition** \\ \hline \multirow{4}{*}{ \begin{tabular}{c} Willful killing of civilians \\ \end{tabular}  } & Refers to the intentional killing of civilians who are not directly participating in hostilities, with full knowledge of their noncombant status.