Anaselka at S em E val-2025 Task 9: Leveraging SVM and MNB for Detecting Food Hazard (2025)

Abstract: Our system for the Sub-task 1 of SemEval-2025 Task 9 has been designed to tackle the complexities of identifying and categorizing food safety incidents from textual data. Through a rigorous experimental setup, we have developed a text classification solution that leveraged state-of-the-art techniques in data preprocessing, feature engineering, and model optimization.

## introduction food safety is a critical public health concern that requires efficient monitoring and early detection of potential hazards @xcite . food hazard detection challenge conducted in semeval-2025 task 9 is an important initiative aiming to develop explainable classification systems for detecting food safety issues based on textual data @xcite . this task is crucial because it can help automated crawlers identify and extract food-related incidents from sources like social media, which is crucial given the potential for significant economic impact. the challenge covers english-language food recall titles and is described in detail in the task overview paper of food hazard detection of semeval-2025 @xcite . our system utilizes a multi-pronged approach to address the text classification task in sub-task 1. first, we focus on robust text preprocessing of the dataset, such as removing special characters and numbers, converting text to lowercase, stemming, and excluding stop words. this preprocessing helps to standardize the input data and focus on the most relevant features @xcite . for extracting features, we employ bag of words (bow) @xcite and term frequency-inverse document frequency (tf-idf) @xcite representations. these techniques capture the frequency and importance of key terms within the text providing informative input to our classification models. to tackle the classification task, we leverage support vector machines (svm) @xcite as our classifier, while we use gridsearchcv for hyperparameter tuning @xcite . the svm classifier has the ability to handle high-dimensional feature spaces and identify optimal decision boundaries, which has proven to be a robust choice for this type of text classification problem. furthermore, the use of gridsearchcv allows us to systematically explore a range of hyperparameter configurations, ultimately selecting the optimal settings for our specific task and dataset. participating in this challenge has provided valuable insights into the strengths and limitations of our system. through an evaluation on the evaluation phase using test data, we are able to obtain an macro f1-score of 0.6858 in sub-task 1. although this obtained result might be categorized acceptable, we have identified areas where our system struggles, such as correctly classifying certain hazard and product categories that are less represented in the dataset. ## background semeval-2025 task 9 is the food hazard detection challenge focused on developing explainable classification systems to identify food-related safety issues from textual data. the task involves predicting the type of hazard and product category given a textual input, such as the title of a food incident report. the input for this task consists of short english-language texts describing food recalls, with an average length of 88 characters. the dataset provided by the organizers includes 6,644 such texts, which were manually labeled by food science and technology experts @xcite . task 9 consists of two sub-tasks. sub-task 1's goal is to predict the "hazard-category" and "product-category" labels, while sub-task 2's purpose is to predict the exact "hazard" and "product" values. participants in the task 9 can take part in both subtasks or they can choose to participate in one sub-task only. the task 9's organizer @xcite considered several phases, namely, a trial phase for model development, a conception phase for validation of unlabeled data, and an evaluation phase for final testing on labeled data. in this task 9, we participate in sub-task 1 only. related work in the area of explainable text classification for food safety risk detection is limited but growing. recent studies have explored both model-specific @xcite and modelagnostic @xcite approaches to provide explanations for predictions. however, the unique challenges of this domain, such as the imbalanced class distribution and the need for precise hazard and product labels, present opportunities for novel contributions. our work aims to build upon these existing methods and address the specific requirements of the sub-task 1 of semeval-2025 task 9. in particular, we strive to develop a robust and explainable system for detecting food safety risks from textual data by leveraging advanced text preprocessing techniques and powerful machine learning algorithms. ## system overview our system takes a multi-pronged approach to address the text classification task os sub-task 1 in the semeval 2025 task 9. we focus on robust text preprocessing, effective feature extraction, and the use of powerful machine learning algorithms with hyperparameter optimization. ## experimental setup we have set up our experimental workflow to run on google colab, a cloud-based jupyter notebook environment. google colab provides a free and accessible platform for running machine learning experiments, with access to gpu and tpu resources. we utilized the provided train dataset containing a total of 5,082 samples. we did have not split the provided dataset into training, development, and test sets because the organizer also provided data for evaluation. to evaluate our trained models, we used a provided development dataset. the goal was to develop explainable and high-performing classification models for the sub-task 1 of semeval-2025 task 9. to prepare the input data for the classification models, we apply a series of text preprocessing steps as follows: we utilize regular expressions to remove all non-alphabetic characters from the text, leaving only the necessary words. after the preprocessing stage, we extract features using two techniques as follows: we use bow and tf-idf to capture different aspects of the textual data and investigate which feature representation performs better for the classification task. we use the training dataset to train and optimize the models. this will allow us to perform hyperparameter tuning and select the best-performing models. we leverage gridsearchcv from scikitlearn, a technique that systematically explores a range of hyperparameter configurations and selects the optimal settings. in the evaluation stages, the semeval-2025 task 9 focuses on the macro average f1-score as an evaluation metric focusing on the hazard class @xcite . mnb svm macro average 0.1076 0.6858 ## results we submitted two models in the evaluation phase to participate in this sub-task 1 of semeval-2025 task 9. in particular, our submission consisted of multinomial naive bayes (mnb) and svm models. our obtained results in ## limitations the stark performance discrepancy of the multinomial naive bayes (mnb) classifier-training macro f1-score of 0.64 versus evaluation macro f1-score of 0.10-stems from multiple interrelated factors. first, while hyperparameter tuning was rigorously applied during training via gridsearchcv to address initial low performance, an oversight led to the inadvertent use of default hyperparameters during evaluation. this inconsistency disrupted the model's calibrated probability estimates, amplifying its inherent sensitivity to imbalanced class distributions and sparse feature representations. mnb's reliance on term independence assumptions further clashed with the evaluation set's domain-specific contextual dependencies (e.g., multi-word hazards like "heavy metal contamination"), which bow/tf-idf failed to disentangle. the absence of optimized regularization during evaluation underscores the necessity of end-to-end hyperparameter consistency, particularly for models like mnb that lack intrinsic mechanisms to mitigate distribution shifts or lexical ambiguities in short, specialized texts. ## conclusion our system for the sub-task 1 of semeval-2025 task 9 has been designed to tackle the complexities of identifying and categorizing food safety incidents from textual data. through a rigorous experimental setup, we have developed a text classification solution that leveraged state-of-the-art techniques in data preprocessing, feature engineering, and model optimization. our obtained submission results in the evaluation phase indicated that svm could perform better than mnb. in particular, our svm and mnb models achieved 0.6858 and 0.1076 of macro average f1-scores, respectively. we assume this mediocre performance due to our models had difficulties in predicting the hazard class in the evaluation phase. for this reason, focusing on predicting the hazard class should be paid attention seriously in the future to deal with this challenging task.