Fossils at S em E val-2025 Task 9: Tasting Loss Functions for Food Hazard Detection in Text Reports (2025)

Abstract: Food hazard detection is an emerging field where NLP solutions are being explored. Despite the recent accessibility of powerful language models, one of the key challenges that still persists is the high class imbalance within datasets, often referred to in the literature as the {textit{long tail problem}}.In this work, we present a study exploring different loss functions borrowed from the field of visual recognition, to tackle long-tailed class imbalance for food hazard detection in text reports. Our submission to SemEval-2025 Task 9 on the Food Hazard Detection Challenge shows how re-weighting mechanism in loss functions prove beneficial in class imbalance scenarios. In particular, we empirically show that class-balanced and focal loss functions outperform all other loss strategies for Subtask 1 and 2 respectively.

## introduction ensuring food safety is a critical global challenge, as contaminated food can lead to severe health risks and economic losses. contaminants such as biological hazards (e.g., salmonella, listeria, e. coli), chemical hazards (e.g., pesticide residues, heavy metals, food additives), and physical hazards (e.g., glass, plastic, metal fragments) pose significant risks to consumers. early detection of such hazards is thus essential for protecting public health. with the growing availability of text-based data sources, such as news articles and social media posts, natural language processing (nlp) techniques can represent an asset to provide scalable solutions for detecting and classifying food hazards from unstructured text. this observation has motivated our participation in the semeval-2025 task 9 @xcite , which focuses on the "food hazard detection challenge" and aims at developing classification systems for titles of food-incident reports collected despite the potential of nlp techniques for food safety monitoring, several challenges still remain. in particular, we observe that the dataset (see fig. 1 ) suffers from a long tail problem @xcite , showing a substantial aggregate frequency for classes that individually have very low frequency. in addition to this, at times, the test set may include previously unseen classes that are absent from the training data 1 . therefore, this scenario presents two distinct challenges: adapting the model to classes for which it has encountered (i) a very low number of instances, and (ii) no instances at all. moreover, when the model is trained on the training set, it inherently assumes a similar class distribution in the test set. this reflects the independent and identically distributed (iid) assumption, which presumes that the training and test datasets share the same underlying distribution. the long-tailed class imbalance represents a common problem in practical visual recognition tasks, often limiting the practicality of deep network based recognition models in real-world applications, since they can be easily biased towards dominant classes and perform poorly on tail classes. acknowledging a similar issue in the setting of the food hazard detection in text reports, in this paper we present the following contributions: (a) we participate in both subtasks of semeval-2025 task 9 on the food hazard detection challenge. (b) we direct our investigation toward assessing the effectiveness of various loss functions in mitigating the impact of long-tailed class imbalance. (c) our final submission for each subtask is an ensemble of multiple models trained using different loss functions. long tail imbalance in nlp severe class imbalance is one of the most prominent challenges that @xcite identified in the dataset, affecting overall classification performance in particular for low-frequency categories. in general, various approaches have been explored to address long-tail imbalance. among those, data augmentation techniques aim to artificially increase the number of samples in lowfrequency classes, by generating synthetic examples for underrepresented categories @xcite . other approaches include oversampling and undersampling techniques. in random under-sampling (rus), a subset of head-class samples is randomly selected while the remaining samples are discarded to match the number of tail-class instances. random over-sampling (ros) randomly reproduces tail-class samples to match the number of head-class samples. to tackle overfitting that often results from ros, synthetic minority over-sampling technique (smote) @xcite creates a new artificial sample through interpolation between each existing head-class sample. furthermore, transfer learning can enhance model performance by leveraging knowledge from head classes to improve tail-class representations @xcite , while decoupled learning @xcite divides learning into two stages: (a) applying end-to-end learning using conventional methods for representation learning, and (b) fixing the feature extractor while retraining the downstream task model for classification. another strategy that has been explored is costsensitive learning, an algorithm-level approach that assigns higher weights to underrepresented classes to mitigate bias toward frequent categories and improve model generalization. this strategy is often adopted to tackle the long-tail problem in the visual recognition field @xcite . unlike more complex solutions that often require extensive resources, loss function modifications offer a lightweight and interpretable strategy that can enhance model performance even with limited data. as a fundamental aspect of model training, they do not replace but rather complement advanced techniques, making them broadly applicable across different models. in light of this, we chose to investigate loss modification as the primary strategy for the shared task. here, p unseen is predicted probability vector of length c, where j th element of the vector corresponds to the predicted probability for x unseen associated with class j. ## results in the results are reported for different loss functions in comparison to the cross-entropy loss function (l ce ). we performed initial experiments investigating the use of only-title, only-text, and title+text. we obtained the best results with title+text and continued the rest of the experiments using title+text as input. we observe that on the validation set, for product-category and hazard-category classbalanced (l cb ) outperforms all the other loss function strategies. further, for product and hazard, focal loss (l fl ) outperforms all other loss functions. this trend is followed on the test set as well except in the case of hazard-category, where cross-entropy loss function (l ce ) outperforms class-balanced loss function (l cb ). however, when subjected to the evaluation metrics provided by the semeval-2025 task 9 @xcite , we observe a consistent trend where the class-balanced loss function (l cb ) and the focal loss function (l fl ) outperform all other loss function strategies for subtask 1 and subtask 2, respectively. ## conclusion in this paper, we presented our submission to semeval-2025 task 9: food hazard detection challenge, where we focused on addressing the challenge of heavy class imbalance in the provided dataset. our approach was designed to improve model performance despite the skewed class distribution. by exploring and implementing modifications to the loss function, we showed how techniques commonly used in visual recognition tasks to handle long-tail distributions can also be effectively applied to text classification.