B right C ookies at S em E val-2025 Task 9: Exploring Data Augmentation for Food Hazard Classification (2025)

Abstract: This paper presents our system developed for the SemEval-2025 Task 9: The Food Hazard Detection Challenge. The shared task’s objective is to evaluate explainable classification systems for classifying hazards and products in two levels of granularity from web-collected food recall incident reports. In this work, we propose text augmentation techniques as a way to improve poor performance in minority classes and compare their effect for each category on various transformer and machine learning models. We apply three word-level data augmentation techniques, namely synonym replacement, random word swapping, and contextual word insertion utilizing BERT. The resultsshow that transformer models tend to have a better overall performance. Meanwhile, a statistically significant improvement (P 0.05) was observed in the fine-grained categories when using BERT to compare the baseline model with the three augmented models, which achieved a 6% increase in correct predictions for minority hazard classes. This suggests that targeted augmentation of minority classes can improve the performance of transformer models.

## introduction foodborne diseases affect millions of people every year. the world health organization highlights that food contamination leads to more than 200 diseases, resulting in severe health complications and affecting the socioeconomic stability of communities and nations @xcite . there is a vast amount of publicly available information on food safety-related websites. given the importance of early detection of food hazards, there is a need to timely and accurately analyze all this publicly available information to detect food hazards. the semeval-2025 task 9: food hazard detection challenge @xcite was proposed to facilitate automated classification of food hazards in food safety-related documents. it stimulates research that combines food safety and natural language processing (nlp) for explainable multiclass classification of food recall incident reports. semeval-2025 task 9 includes two sub-tasks: classifying coarse food hazard and product categories (st1) (hazard-category, product-category), and fine-grained hazard and product categories (st2) (hazard, product). a significant challenge with the semeval-2025 task 9 dataset is its substantial class imbalance. there is a long-tailed distribution across classes, especially in the fine-grained categories. this imbalance can give poor performance of classifiers, especially for deep learning (dl) models @xcite . text augmentation techniques have been shown to mitigate the effects of imbalanced data to an extent @xcite . text augmentation can range from simple string manipulations, such as those used in easy data augmentation (eda) @xcite , to more advanced methods involving transformer-based text generation @xcite . this helps to boost the representation of minority classes, which can result in a more balanced dataset and robust models. we investigated three basic text augmentation techniques (synonym replacement, contextual word insertion, and random word swapping) to boost the representation of under-represented classes in multi-class classifications of food recall incident reports. our main research question is: can text augmentation techniques on underrepresented classes enhance a food hazard multiclass classifier's performance? we evaluated the performance of various machine learning (ml) algorithms and encoder-only transformer models, both in their baseline form and after applying each augmentation technique. to participate in the task, only one submission was allowed. we submitted our predictions in the official test set after we evaluated our models in the development set, selecting the best-performing ones for each category. in st1, our system ranked 15 th out of 27 participants, with an f 1 -macro score difference of 0.0613 from the first, and in st2, it ranked 11 th out of 26, with a 0.0944 score gap from the top (see subsection 6.1 for the exact scores). our work provides valuable insights into the efficacy of text augmentation in this field. 1 2 related work ## data the food recall incidents dataset used in semeval-2025 task 9 contains 6,644 food-recall announcements in the english language @xcite . this dataset is split into 5,082 announcements in the train set, 565 in the development set, and 997 in the test set. the data is collected from 24 different websites ( experts manually labeled each sample into four coarse classes of hazards (hazard-category) and products (product-category) and fine-grained classes (hazard and product). the classes and the number of classes per category are listed in the distribution of the four categories' classes is highly imbalanced, showing a long-tail effect (figure 4 , figure 5 ). in coarse categories, 75% of classes have 513 samples in hazard-category and 263 in product-category, while the largest classes contain 1,854 and 1,434 samples, respectively. this imbalance is even more severe in the fine-grained hazard and product classes, with 75% of classes having at most four samples per product and 24 samples per hazard, while the largest class has 185 samples per product and 665 samples per hazard. ## methods we used ml and dl and implemented multiple data augmentation strategies. the next sections describe this in more detail. ## experiments in the next subsections, we further describe the preprocessing, hyperparameter fine-tuning, and evaluation details. ## results the next subsections show quantitative results for each model in the official test set using the text field (trained in training and development sets) and an error analysis on the bert baseline model versus its augmented-trained versions. ## limitations while multiple experiments have been conducted, some limitations could be addressed in future studies. the dataset used was exclusively in english, and the augmentation techniques applied were limited to word-level adjustments. future research could explore more sophisticated augmentation methods, such as llms, to generate new samples and verify their quality. incorporating datasets in other languages could provide insight into the effectiveness of augmentation techniques. further investigation could also focus on optimizing the number of augmented samples for minority classes to enhance classification performance, especially for food hazard classification, where reliable models are required to ensure safety. lastly, to enhance even further the classifiers' performance, more complex architectures such as ensemble or hierarchical approaches could be used to compare their effectiveness on augmentation in the food hazard classification task. ## conclusion we showed that word-level text augmentation can enhance multi-class classification in minority classes. we used various machine learning and transformer models on the semeval-2025 task 9 to assess the effects of these augmentations. leveraging the text field, we discovered that transformers tend to outperform ml. augmentation techniques showed a slight increase of f 1 -macro scores, but this effect was not consistent across all augmentations. comparing bert base with each augmentation technique, a statistical significant improvement was found for fine-grained categories, which indicates that augmenting minority classes can improve the performance of transformers for these classes. teven le scao, sylvain gugger, mariama drame, quentin lhoest, and alexander rush. 2020. transformers: state-of-the-art natural language processing. in proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, pages 38-45, online. association for computational linguistics. world health organization. 2024. food safety. https: //www.who.int/news-room/fact-sheets/deta il/food-safety. accessed: february 03, 2025.