Ustnlp16 at S em E val-2025 Task 9: Improving Model Performance through Imbalance Handling and Focal Loss (2025)

Abstract: Classification tasks often suffer from imbal- anced data distribution, which presents chal- lenges in food hazard detection due to severe class imbalances, short and unstructured text, and overlapping semantic categories. In this paper, we present our system for SemEval- 2025 Task 9: Food Hazard Detection, which ad- dresses these issues by applying data augmenta- tion techniques to improve classification perfor- mance. We utilize transformer-based models, BERT and RoBERTa, as backbone classifiers and explore various data balancing strategies, including random oversampling, Easy Data Augmentation (EDA), and focal loss. Our ex- periments show that EDA effectively mitigates class imbalance, leading to significant improve- ments in accuracy and F1 scores. Furthermore, combining focal loss with oversampling and EDA further enhances model robustness, par- ticularly for hard-to-classify examples. These findings contribute to the development of more effective NLP-based classification models for food hazard detection.

## introduction the rapid advancement of natural language processing (nlp) has facilitated the development of automated classification systems across various domains, including food hazard detection. accurately identifying and categorizing food hazards is essential for ensuring food safety and mitigating health risks associated with contaminated or unsafe food products. however, food hazard classification presents several challenges, including severe class imbalances, ambiguous and unstructured textual descriptions, and the need for high predictive accuracy. traditional approaches to hazard detection have relied on manual inspections and rulebased classification methods, which are often timeconsuming and prone to human error. in contrast, recent advancements in machine learning and nlp have enabled the automation of this process, leveraging text classification models to analyze food hazard reports and categorize them into predefined classes. transformer-based models, such as bert @xcite and roberta @xcite , have demonstrated state-of-the-art performance in text classification tasks. however, their effectiveness in imbalanced classification settings remains a challenge, as they tend to favor majority classes while underperforming in minority categories. class imbalance is a common issue where certain categories have significantly fewer instances than others, leading to biased predictions and reduced model performance on underrepresented classes. to address this issue, researchers have explored various techniques, including data augmentation, resampling methods, and modified loss functions. easy data augmentation (eda) @xcite generates additional training samples for minority classes, enhancing model generalization. similarly, focal loss @xcite modifies the traditional cross-entropy loss function to focus more on difficult-to-classify examples, improving performance on underrepresented categories. in this study, we systematically investigate the impact of data balancing techniques on transformerbased models for food hazard classification. specifically, we evaluate the effectiveness of oversampling, eda, and focal loss in mitigating class imbalance and improving classification performance. through extensive experimentation, we demonstrate that these strategies enhance model robustness, particularly in detecting minority-class hazards. our findings contribute to the development of more reliable nlp-based classification models for food safety applications, providing valuable insights into optimal approaches for handling class imbalance in text classification tasks. ## task definition and dataset semeval-2025 task 9 involves classifying short food recall reports into predefined hazard-related labels @xcite . the objective is to develop robust nlp models that accurately identify hazard types and product categories despite imbalanced, noisy, and unstructured text. the dataset comprises thousands of entries obtained from government agencies. each entry includes a title, typically a brief recall identifier, and a text description that varies in length and format, often containing domain-specific terminology. the data are marked by severe class imbalance, with many hazard categories significantly underrepresented. ## methodology recent work in food hazard detection highlights the importance of addressing data imbalance, short and unstructured text, and overlapping semantic categories (adegbenjo and ngadi, 2024). semeval-2025 task 9 intensifies these challenges by providing a real-world dataset where certain hazard classes are severely underrepresented, necessitating specialized techniques to ensure fair and robust classification. in this study, we adopt a transformerbased approach, leveraging bert @xcite and roberta @xcite , and integrate three key strategies-random oversampling, easy data augmentation (eda), and focal loss-to enhance performance on minority classes while maintaining overall accuracy. our methodology follows a structured pipeline, shown in figure 1 , which consists of data preprocessing and augmentation, transformer-based model fine-tuning, and final evaluation using standard classification metrics. the sections below describe how these components are cohesively combined to tackle the real-world complexities of food hazard reports. ## experiments our study strategically integrates three techniques-easy data augmentation (eda), oversampling and focal loss-to address class imbalance in classification tasks. the sequence of application is as follows: eda is applied before tokenization to enhance data diversity, oversampling is applied after tokenization to balance class distribution, and focal loss is utilized during training to optimize the model's focus on difficult samples. these experiments not only demonstrate the effectiveness of each individual method but also highlight the synergistic benefits of their combination. the results show that this integrated approach enhances data diversity, balances class distribution, and improves model performance by prioritizing challenging samples. ## conclusion in this paper, we strategically combined three methods -easy data augmentation (eda), oversampling, and focal loss -to tackle class imbalance in classification tasks. our model ranked 13th on st1, and 12th on st2. it surpassed the baseline in both st1, predicting product and hazard category, and st2, predicting specific hazard and product. future work will explore strategies to enhance model performance in data-constrained and domain-shift scenarios. we plan to investigate the integration of advanced augmentation techniques and refined loss functions, along with fine-tuning the existing methodology.