# Comparative Analysis of BERT and GPT for Classifying Crisis News with Sudan Conflict as an Example
## 6 Discussion
While model size contributes to the advantage in classification tasks, performance is also influenced by architectural differences between BERT and GPT, the choice of prompting strategy, and the inclusion of hyperparameter tuning. Notably, a key trend observed in this study was the low performance of Llama3.2-3b across all metrics, which demonstrated low performance in SL-MLG and MLC. This under-performance was not due to the relatively small parameter size of 3B but primarily resulted from its guardrails. For example, in the Zero-Shot and standard RAG configurations, the model frequently refused to classify violent or sensitive content and instead returned refusal messages. However, when explicit category definitions were provided through the RAG ICL approach, this behavior was largely mitigated, and the model executed the classification task as intended. This finding underscores the critical role of carefully choosing the best prompting strategy for each domain-related task in order to ensure compliant behavior when utilizing LLMs for a task such as sensitive crisis-related text classification. One of BERT's primary advantages lies in its higher precision in multi-label classification tasks, where it consistently outperforms GPT in assigning categories with fewer false positives. Its bidirectional training mechanism allows it to capture deeper contextual relationships, leading to more accurate classifications in structured tasks. In the SL-MLG tasks, BERT also demonstrated better alignment with the ground-truth labels, achieving the highest accuracy among all configurations. This suggests that BERT's fine-tuned models are better suited for tasks requiring exact category assignment. Additionally, BERT's fine-tuning process enables structured optimization for domain-specific classification, allowing for more refined learning in well-defined datasets. Furthermore, BERT models produce more stable and deterministic outputs, whereas GPT's responses may vary across different runs, making BERT more reliable for structured applications that require consistent and reproducible classifications. However, BERT had notable limitations compared to GPT. One major drawback is its lower recall in MLC tasks, meaning it often fails to capture the complete set of relevant categories for an article. This conservative classification approach can lead to under-classification, where important categories are overlooked. Additionally, BERT's reliance on supervised fine-tuning makes it computationally expensive, requiring extensive training for each new dataset. In contrast, GPT operates without the need for fine-tuning, significantly reducing the setup time. Furthermore, BERT's limited adaptability to new domains makes it less flexible in handling unseen datasets. Since BERT relies on task-specific fine-tuning, it struggles to generalize as effectively as GPT's in-context learning, which enables classification across a wide range of topics without retraining. The RAG approach without definitions provided GPT models with only category names as prompts, requiring the model to rely solely on pre-trained knowledge for classification. This approach generally outperformed Zero-Shot prompting and, in some cases, even RAG ICL, particularly for larger models. For example, Llama3.1-70b under RAG achieved an F1-score of 68.80% in MLC, demonstrating that models with extensive pre-training can classify effectively without explicit category descriptions. In the SL-MLG tasks, RAG also showed competitive results, with Llama3.3-70b achieving an accuracy of 86.05%, only slightly below RAG ICL's best-performing models. The RAG ICL approach, which incorporated expert-annotated category definitions in the prompt, provided the highest classification accuracy across the SL-MLG tasks. This method improved classification precision by providing GPT with structured, contextual definitions of each category, guiding more accurate label selection during inference. For MLC, however, the results were more mixed.