# Comparative Analysis of BERT and GPT for Classifying Crisis News with Sudan Conflict as an Example
## 4 Methodologies
### Methods  
#### 4.1.1 Bert  
To maintain a focused comparison between BERT and GPT, encoder types such as XLM-RoBERTa and DeBERTa were not included, as this study was centered on comparing the traditional BERT variants and GPT models. The chosen BERT classification framework was implemented for both MLC and Single-Label Classification from SL-MLG using two configurations: a standard BERT model with default hyperparameters and a hyperparameter-tuned model optimized via Optuna [43]. Both configurations used the theert-base-uncased and bert-large-uncased variants from the Hugging Face Transformers library, fine-tuned to predict from a fixed set of \(\mathbf{K}\) conflict-related categories. Text inputs were processed using the BertTokenizer for consistent tokenization, truncation, and padding. Furthermore, no explicit class weights or custom loss functions were applied. However, using Optuna, key hyperparameters were optimized--including learning rate, batch size, number of epochs, dropout rate, and weight decay--to improve the classification performance under a data imbalance. Each news article was denoted as an input sequence \(x_{i}\), where \(i\in\{1,\dots,N\}\) indexes the article in a dataset of size \(\mathbf{N}\). Each article was paired with a binary ground-truth label vector \(y_{i}\in\{0,1\}^{K}\), where \(y_{ik}=1\) indicates that article \(x_{i}\) belongs to category \(k\), and 0 otherwise. For the SL-MLG task, the model computes an output logit vector \(f_{\theta}(x_{i})\in R^{K}\), where \(\theta\) denotes the model parameters, and each component \(f_{\theta,k}(x_{i})\) represents the raw, unnormalized score for category \(k\). The logits are converted into class probabilities via the softmax function:  
\[\hat{y}_{i}=\text{softmax}(f_{\theta}(x_{i}))=\left[\frac{e^{f_{\theta,k}(x_{ i})}}{\sum_{j=1}^{K}e^{f_{\theta,j}(x_{i})}}\right]_{k=1}^{K} \tag{1}\]  
where \(\hat{y}_{i}\in[0,1]^{K}\) is the predicted probability distribution over all categories. \(\hat{y}_{ik}\) is the predicted probability that article \(x_{i}\) belongs to category \(k\). The sum across all \(k\) ensures that  
\[\sum_{k=1}^{K}\hat{y}_{ik}=1 \tag{2}\]  
The predicted label \(\widetilde{y}\) is selected as the index \(k\) with the highest predicted probability:  
\[\widetilde{y_{i}}=\textit{arg max}\;\hat{y}_{ik} \tag{3}\]  
This prediction is deemed correct if it appears in the original ground truth multi-label set:  
\[\widetilde{y_{i}}\in\{k\mid y_{ik}=1\} \tag{4}\]  
In the MLC setting, the model again produces logits \(f_{\theta}(x_{i})\in R^{K}\). Instead of using softmax, a sigmoid activation is applied independently to each category score:  
\[\hat{y}_{ik}=\sigma(f_{\theta,k}(x_{i}))=\frac{1}{1+e^{-f_{\theta,k}(x_{i})}} \tag{5}\]  
where \(\hat{y}_{ik}\in[0,1]\) is the predicted probability that article \(x_{i}\) belongs to category \(k\). Each score is interpreted independently, allowing multiple labels per article. The final predicted label set \(\widetilde{y_{i}}\in\{0,1\}^{K}\) is computed by applying a threshold \(\tau=0.5\) to each probability:\[\widetilde{y}_{ik}=\left\{ \begin{array}{ll}1,&\text{if }\hat{y}_{ik}\geq 0.5\\ 0,&\text{otherwise}\end{array} \right. \tag{6}\]  
This allows the model to assign multiple categories to a single article. To address the concern regarding the limited size of the test set in the initial 90/10 training-testing split, which resulted in only the same 43 articles used in the evaluation, we implemented a 5-fold cross-validation 80/20 training-testing split strategy to strengthen the validity and reliability of our results. By adopting k-fold cross-validation, we ensured that each article in the dataset was used for both training and evaluation across different folds, providing a more comprehensive and balanced assessment of the model's performance. This approach further mitigated any potential bias introduced by relying on a single, fixed test subset and helped verify that the results were not dependent on a particular partition of the data.