# Comparative Analysis of BERT and GPT for Classifying Crisis News with Sudan Conflict as an Example
## 2 Related Work  
### Text Classification in NLP  
Text classification is a fundamental task in NLP that involves assigning textual data to predefined labels or categories based on its content. It serves as a critical component in various applications, including sentiment analysis, spam detection, topic classification, and document classification. These applications rely on accurate classification to support efficient information retrieval and enable automated decision-making [14; 15; 16; 17]. Researchers have developed a range of approaches for text classification, evolving from rule-based methods and statistical models to machine learning (ML) and deep learning techniques [6; 18]. Early text classification models relied on traditional ML algorithms, such as Naive Bayes, Support Vector Machines (SVM), and Random Forest, which leveraged handcrafted features like term frequency-inverse document frequency (TF-IDF) and n-grams to represent text [19; 20]. While these approaches demonstrated reasonable performance in structured datasets, they struggled with capturing the complex semantic and syntactic structures of natural language. To address these limitations, deep learning (DL) models emerged as a more effective solution, significantly improving text classification accuracy by automatically learning hierarchical features from raw text. Among the early DL approaches, Convolutional Neural Networks (CNNs) were widely adopted for text classification due to their ability to capture local patterns and word dependencies through convolutional filters [21; 22]. CNN-based models performed well in tasks requiring sentiment analysis and short-text classification but were limited in handling long-range dependencies due to their fixed-size receptive fields. Meanwhile, Recurrent Neural Networks (RNNs), particularly Long Short-Term Memory (LSTM) networks, became popular for their ability to model sequential dependencies and effectively handle text inputs of varying lengths, such as short phrases or long documents [23]. LSTM demonstrated superior performance in capturing word order and contextual relationships, making it effective for tasks like document classification and named entity recognition [24]. However, both CNNs and LSTMs suffered from inefficiencies when dealing with long-range dependencies and were computationally expensive due to their sequential nature. A significant breakthrough in NLP came with the introduction of the Transformer architecture by [PERSON] et al. [25], which revolutionized text classification by addressing the shortcomings of RNN-based models. Transformer's self-attention mechanism enabled it to capture long-range dependencies more effectively by processing all tokens in parallel, rather than sequentially. This architectural innovation significantly improved performance across various NLP tasks, including machine translation, text classification, and question answering [26; 27]. Building upon the Transformer framework, BERT [2] and GPT [3] emerged as state-of-the-art models for text classification and other NLP applications. BERT introduced a bidirectional training approach, enabling a deeper contextual understanding of text, while GPT leveraged autoregressive pretraining to generate coherent and contextually relevant outputs. These models set new benchmarks for text classification by surpassing previous DL methods in accuracy and adaptability across various domains [28]. The evolution of text classification methods--from traditional statistical models to DL and Transformer-based architectures--has reshaped how textual data is analyzed and classified. While earlier approaches like CNNs and LSTMs laid the foundation for deep learning in NLP, Transformer-based models have achieved unprecedented performance by capturing complex contextual dependencies with greater efficiency.