# Comparative Analysis of BERT and GPT for Classifying Crisis News with Sudan Conflict as an Example
## 5 Results
### Mlc  
The overall performances (Table 5) of the three GPT methodologies (Zero-Shot, RAG, and RAG ICL) and two BERT configurations (standard and hyperparameter-tuned) in the MLC task were compared, focusing on accuracy, precision, recall, F1-score, and runtime. The GPT models exhibited varying performance across different methodologies. BERT models, on the other hand, demonstrated similarly consistent performances, with hyperparameter tuning providing slight improvements in scores. The bert-large hyperparameter-tuned model achieved the highest F1-score (68.90%), outperforming the close second, the Llama3.1-70b model under RAG counterpart (68.80%). Llama3.1-70b under RAG outperformed its counterpart Llama3.1-70b under RAG ICL (67.00%). Similarly, Llama3.3-70b performed better in F1-score under RAG (67.30%) than under RAG ICL (64.42%), indicating that RAG generally outperformed RAG ICL, particularly for larger models. However, RAG ICL remained competitive, with Gemma2-27b showing comparable F1-scores. Accuracy among the GPT models remained modest, with the highest accuracy for the GPT models being recorded at 25.58% for Llama3.1-70b under RAG ICL. Recall scores, however, were consistently strong across all methodologies, highlighting GPT's ability to comprehensively capture relevant categories, even at the expense of precision. \begin{table}
\begin{tabular}{c l c c c c} \hline \hline  & **MODEL** & **TUNING** & **TRAINING** & **INFERENCE** & **ACC.** \\ \hline \hline \multirow{2}{*}{**Zero-Shot**} & & & & & \\  & Gemma2-9b & & & 20 s & 65.12\% \\  & Gemma2-27b & & & 32 s & 67.44\% \\  & Llama3.3-70b & & & 40 s & 76.74\% \\  & Llama3.2-3b & & & 9 s & 41.86\% \\  & Llama3.1-70b & & & 49 s & 72.09\% \\  & Llama3.1-7b & & & 12 s & 48.84\% \\  & Mistral-7b & & & 12 s & 51.16\% \\ \hline \hline \multirow{2}{*}{**BERT hp-tuned**} & & & & & \\  & bert-base-uncased & 137 m 57 s & 7 m 20 s & 1 s & 74.42\% \\  & bert-large-uncased & 1170 m 9 s & 73 m 19 s & 20 s & 83.72\% \\ \cline{2-6}  & bert-base-uncased & & 10 m 4 s & 1 s & 76.74\% \\ \cline{2-6}  & bert-large-uncased & & 28 m 29 s & 2 s & 74.42\% \\ \hline \hline \end{tabular}
\end{table} 
Table 4: _Cont._The BERT models demonstrated higher precision than the GPT models, with the bert-large hyperparameter-tuned configuration achieving an F1-score of 68.90% and a precision of 75.70%, outperforming the standard bert-large model (F1-score of 54.62%, precision of 64.32%). However, the BERT models exhibited lower recall scores than the GPT models, with both the hyperparameter-tuned bert-base model and standard bert-base model achieving recall scores of 57.78%, indicating that while BERT excels in precise category assignment, it struggles to comprehensively capture all relevant categories. The highest overall F1-score (68.90%) was achieved by the hyperparameter-tuned bert-large model, surpassing the best GPT model, Llama3.1-70b under RAG, with an F1-score of 68.80%. Similarly, in terms of accuracy, all the BERT models achieved higher scores than all the GPT models with the hyperparameter-tuned bert-large model achieving the highest score of 41.86%. Computational efficiency differed significantly between the BERT and GPT models. Since BERT requires both training and inference time, the standard bert-base model completed training and inference in 9 min and 43 s, and the standard bert-large model completed training and inference in 14 m 55 s. For the hyperparameter-tuned models, the runtime was divided into three parts; (1) tuning, (2) training, and (3) inference. The bert-base hyperparameter-tuned version required a total of 187 min and 45 s while the bert-large hyperparameter-tuned version took a total of 1351 m 14 s (~22.5 h) due to the ~3X larger parameter size bert-large has over bert-base. In contrast, the GPT models only required inference time, with the best model GPT model, Llama3.1-70b under RAG, running in 2 m 30 s, completing the runtime in a significantly faster margin compared to the hyperparameter-tuned bert-large model, which only beat it by 0.10%.