U ni B uc at S em E val-2025 Task 9: Similarity Approaches to Classification (2025)

Abstract: In this paper, we present a similarity-based method for explainable classification in the context of the SemEval 2025 Task 9: The Food Hazard Detection Challenge. Our proposed system is essentially unsupervised, leveraging the semantic properties of the labels. This approach brings some key advantages over typical classification systems. First, similarity metrics offer a more intuitive interpretation. Next, this technique allows for inference on novel labels. Finally, there is a non-negligible amount of ambiguous labels, so learning a direct mapping does not lead to meaningful representations.Our team ranks 13th for the second sub-task among participants that used only the title and the text as features. Our method is generic and can be applied to any classification task.

## introduction as people become more aware of the health risks associated with the global food industry, there is a growing interest to ensure the safety and quality of these products. the complexity of modern food supply chains, which involve multiple stages of production, processing, and distribution, has made it increasingly difficult to monitor and control food safety hazards. in this context, natural language processing (nlp) tools lend themselves invaluable to identifying patterns and extracting information from heterogeneous data sources. the main goal of the food hazard detection challenge is to explore the explainability of classification systems on texts associated with food safety risks. the dataset consists of food-incident reports collected from various sources written in english, representing a subset of a larger dataset created by the organizers that also includes texts in german and a few instances in four other languages @xcite . the first task requires systems to predict coarse-grained categories for hazards and products, while the second task is focused on determining the exact hazard and product @xcite . we participated only in the second sub-task to explore the viability of similarity methods. the dataset reports contain a title, the full report text and other details such as date and country. since the title does not always contain adequate information about products or hazards, we also add the full text as input to our system, ignoring the other features. next, we use a large language model (llm) to clean up this text. for classification, we employ cosine similarity between each clean text and every label @xcite without any training step. in the case of hazards, we also apply lemmatization on each word and reorder the labels by importance and specificity (detailed in section 3). beside data cleaning, our biggest challenge was to decide how to handle ambiguous labels. even though this dataset design choice appears to be intentional @xcite , we believe that the addition of almost identical classes could have been avoided, at least to some extent. we settled for a similarity-based approach because this should aid the explainability of the system and it also eliminates the challenge of heavily imbalanced data. we note that many of our wrong predictions were affected by label overlap. our team ranked 14th out of 26 teams. our system is unsupervised, employing small models and needing modest resources, while also allowing for easy interpretation. our code is publicly available foot_0 . ## system overview our system consists of two independent pipelines for hazard and product classification. each pipeline has a data cleaning step, a pre-processing step and a classification step, optionally followed by a postprocessing step, as shown in figures 1 and 2 . there is no training or fine-tuning involved. ## experimental setup data cleaning and label sorting is achieved with a llama 3.2-3b-instruct model @xcite , with 4-bit quantization (q4_k_m) using ollama, restricting output to 50 tokens. the data cleaning process is the most time-consuming, requiring at least 4 hours for the training set. ideally, this should be a one time effort. this is necessary regardless of how we decide to implement the classification, so we have to verify that the hazards and products are extracted appropriately. similarities are computed by embedding the texts and the labels with the help of sentence transformers models @xcite . we choose different embeddings for hazards 2 and products 3 , observing that larger models do not always lead to improved scores. for lemmatization we use simplemma 4 . the classification step is efficient, taking less than a minute for the entire training set of 5082 examples, which enabled us to conduct several experiments. comparatively, a single call to a llm takes a few seconds. ## discussion to better understand the types of mistakes in our system, we analyze the errors in the most optimistic situation (see figure 3b ), focusing on predictions that are wrong even if we know the label set beforehand and assume correct answers if found in the first 10 predicted labels. the idea is that we cannot attribute these inaccuracies to semantic overlap, at least in the case of hazards. ## conclusions and future work we presented our approach in the semeval 2025 task 9 @xcite , where the objective is to extract hazards and products from food-incident reports. we propose a similarity-based system that aims to tackle issues related to imbalanced classes and noisy labels in an unsupervised manner. our analysis highlights the value of quality data and the benefits of exploiting label semantics. this can prevent shortcut learning and discourage hallucinations that would result from learning to predict information absent from the input text. our approach offers explainable interpretations of the results and provides a possible solution to prioritize higher-risk hazard labels. due to time constraints and unsatisfactory results in preliminary experiments, we leave finetuning embedding models for future work. we intend to leverage training data labels to enhance the embeddings, improve the data cleaning pipeline and develop techniques to mitigate the limitations of similarity methods. while the food detection could be enhanced by leveraging specialized corpora such as the foodbase corpus @xcite , we believe the task could benefit from a redesign altogether due to significant label overlap. this endeavor is left for future studies, as we also need to research reliable methods to leverage noisy labels.