HU at S em E val-2025 Task 9: Leveraging LLM -Based Data Augmentation for Class Imbalance (2025)

Abstract: This paper presents a solution to the food hazard detection challenge in the SemEval-2025 Task 9, focusing on overcoming class imbalance using data augmentation techniques. We employ large language models (LLMs) like GPT-4o, Gemini Flash 1.5, and T5 to generate synthetic data, alongside other methods like synonym replacement, back-translation, and paraphrasing. These augmented datasets are used to fine-tune transformer-based models such as DistilBERT, improving their performance in detecting food hazards and categorizing products. Our approach achieves notable improvements in macro-F1 scores for both subtasks, although challenges remain in detecting implicit hazards and handling extreme class imbalance. The paper also discusses various techniques, including class weighting and ensemble modeling, as part of the training process. Despite the improvements, further work is necessary to refine hazard detection, particularly for rare and implicit categories.

## introduction ensuring food safety is a critical challenge in public health, requiring timely detection of hazards leading to product recalls. traditional methods rely on manual expert analysis, which is time-consuming and lacks scalability. recent advances in natural language processing (nlp) have enabled automated food hazard detection from recall reports, improving regulatory oversight. however, class imbalance in real-world datasets, where some classes are overrepresented, remains a challenge for accurate predictions @xcite . semeval-2025 task 9: the food hazard detection challenge focuses on classifying food hazards and products from textual reports (randl et al., 1 our code: https://github.com/msaadg/hu_ semeval_task9 2025). this task is crucial for enhancing food security and public health interventions. it consists of two subtasks: the primary challenge of this task is the extreme class imbalance, where certain classes appear far more frequently than others. figure 1 illustrates the severity of class imbalance through a probability distribution. we propose a transformer-based model augmented with synthetic data from llms like gpt-4o, gemini flash 1.5, and t5 to address this imbalance. our approach leverages distilbert, which has proven effective in handling class imbalance, and fine-tunes it on both original and augmented datasets. our system ranked 5th in st1 and 4th in st2. despite these achievements, challenges persist in identifying implicit hazards and dealing with highly imbalanced categories, which require further refinement @xcite . the results confirm that data augmentation plays a key role in overcoming class imbalance. the dataset provided by the semeval-2025 organizers contains structured recall reports sourced from food regulatory bodies. it consists of: ## related work the challenge of food hazard detection has been extensively studied, with recent advancements leveraging natural language processing (nlp) for automated risk assessment. traditional approaches have primarily relied on rule-based methods and handcrafted feature extraction, which often fail to generalize across diverse recall reports. according to @xcite , while these methods have been widely used, they struggle with the complexity and scale of modern datasets. more recently, deep learning models, particularly transformers, have demonstrated superior performance in food safety classification tasks, significantly outperforming traditional methods @xcite . ## experimental setup the experimental setup aimed to address challenges such as class imbalance and computational efficiency while ensuring robust training and evaluation. we used google colab's t4 gpu and kaggle's t4x2 gpus for efficient fine-tuning. our models were trained using the adamw optimizer with a learning rate of 5e -5 , a batch size of 8, and a maximum of 5 epochs. early stopping was applied to avoid overfitting, halting training when validation loss plateaued. to address class imbalance, class weights were computed based on inverse class frequency (see appendix c), which helped the model focus on underrepresented hazard and product categories @xcite . learning rate scheduling was applied using a linear decay schedule with a warmup phase, allowing for stable training convergence. text preprocessing involved merging the 'title' and 'text' fields into a single 'title_text' feature to maximize contextual representation. tokenization was handled using model-specific tokenizers, like 'autotokenizer' for bert and distilbert, with input sequences truncated to 512 tokens for computational feasibility. we utilized nlpaug 1.1.11 for synonym replacement, contextual augmentation, and back-translation, and gemini 1.5 flash and gpt-4o for synthetic data generation, which helped mitigate class imbalance @xcite . evaluation metrics included macro-f1, with scores calculated on both st1 and st2. ## results & analysis the results of our experiments reveal a clear trend in the impact of data augmentation and model selection on performance improvements. our final model, 'distilbert-base-uncased', demonstrated significant gains in macro-f1 scores over the baseline model, highlighting the effectiveness of augmentation techniques in addressing severe class imbalance and enhancing classification accuracy across underrepresented classes. ## conclusion this work tackled the challenges of food hazard detection in semeval-2025 task 9, focusing on extreme class imbalance and enhancing model generalization through llm-based augmentation @xcite . by employing a combination of contextualized synonym replacement, paraphrasing, back-translation, and synthetic data generation, we significantly improved classification performance, particularly for low-frequency categories @xcite . among various transformer models, 'distilbert-base-uncased' provided the best tradeoff between efficiency and accuracy, achieving a final macro-f1 of 0.7882 (st1) and 0.5099 (st2). while these improvements are noteworthy, further refinements are necessary for addressing ongoing challenges, especially implicit hazard detection.