H oward U niversity- AI 4 PC at S em E val-2025 Task 9: Using Open-weight BART - MNLI for Zero Shot Classification of Food Recall Documents (2025)

Abstract: We present our system for SemEval-2025 Task 9: Food Hazard Detection, a shared task focused on the explainable classification of food-incident reports. The task involves predicting hazard and product categories (ST1) and their exact vectors (ST2) from short texts. Our approach leverages zero-shot classification using the BART-large-MNLI model, which allows classification without task-specific fine-tuning. Our model achieves competitive performance, emphasizing hazard prediction accuracy, as evaluated by the macro-F1 score.

## introduction food safety is a critical global concern, with foodborne illnesses affecting millions annually and causing significant economic losses. rapid and accurate detection of food hazards is essential to mitigate these risks, but the sheer volume of foodincident reports makes manual monitoring challenging. automated systems that identify and classify food hazards from textual data, such as recall notices or social media posts, offer a promising solution. however, these systems must be accurate since it is crucial for building trust and enabling human oversight in food safety applications. recent advancements have enabled the development of systems that can automatically detect food hazards from textual data. for example, previous work has focused on identifying foodborne illnesses from social media posts or analyzing food recall reports from official agencies. @xcite despite the progress in automated food safety systems, there is a lack of systems that provide exact predictions for food hazards. additionally, the imbalanced and ambiguous nature of food-incident data poses significant challenges for traditional supervised learning approaches.in this paper, we aim to address these challenges by developing a system for food hazard detection using zero-shot classification. our approach leverages the bart-large-mnli model to predict hazard and product categories (st1) and their exact vectors (st2) from short food recall related texts. the key contribution of our work evaluate the effectiveness of zero-shot classification using a wellstudied pre-trained open-weight model as part of semeval-2025 task 9 @xcite . ## related work utilizing advancements in deep learning and large language models to categorize and classify text is standard practice in research literature across multiple domains and languages @xcite @xcite . recently, the detection of food hazards from textual data, such as food incidents reports, has gained significant attention due to its potential to improve public health and reduce economic losses. traditional methods for identifying food hazards often rely on manual analysis of reports from official sources, which can be time-consuming and prone to delays. to address these limitations, recent research has focused on developing automated systems that can efficiently process and classify food-incident reports collected from the web. for example, studies have explored the use of machine learning (ml) techniques to analyze social media posts and web-based reports for early detection of food borne illnesses, demonstrating the potential of these systems to identify unreported events and enhance outbreak response @xcite . additionally, systems like finder have leveraged aggregated web search and location data to detect food borne illnesses in real-time, showcasing the value of integrating diverse data sources for timely hazard identification @xcite . however, many existing systems lack explainability, which is crucial for ensuring transparency and enabling human oversight in food safety applications. our work addresses this gap by introducing classification system that uses zero-shot classification to predict hazard and product categories (st1) and their exact vectors (st2) from food-incident reports which supports the development of automated crawlers that can efficiently extract and classify food hazards from web sources like social media. @xcite 3 methodology the task involves two subtasks: ## limitation a key limitation of our approach is that relying on candidate labels extracted from the training set guarantees we will miss unseen or novel labels, limiting the system's ability to generalize to new hazards or products. additionally, while our approach is preliminary, there may be better-performing models or techniques, such as fine-tuning, few-shot learning, or testing alternative architectures, that could yield improved results. furthermore, our system only utilizes text data, which could overlook valuable context from other data fields that could enhance performance. finally, the model is only english, which restricts its applicability to multilingual contexts, highlighting the need for future work to address language diversity and incorporate additional data sources for more robust and generalizable food hazard detection. our system underperforms compared to other participants, but we lack a thorough error analysis to understand why. future work should include a detailed breakdown of performance across different subcategories (e.g., biological vs. chemical hazards) and confusion matrices to identify specific areas of model weakness and common failure patterns in hazard versus product classification. ## result the intention behind the submission was to evaluate the feasibility of using a zero-shot model for food-hazard and product classification from short, unstructured texts-without relying on taskspecific fine-tuning or annotated training data, which is often scarce or unbalanced in real-world applications.our model performed better than random chance, indicating that it captures some meaningful semantic distinctions between hazards and products and score between two of them has been mentioned below: model naive random classifier bart-mnli (zero-shot) task 1 0.0043 0.2426 task 2 0.07275 0.1380 however, we agree that the performance is not sufficient for deployment and that more extensive work is needed. for all the tasks where the shared task organizers released a test data and , we used the test data for the results reported in this section (for the and the score was evaluated by calculating the macro-f1score on the participants' predicted labels using the annotated labels as ground truth.the leaderboard can be found below: ## conclusion key contributions of our work include the use of zero-shot classification to handle imbalanced data and the integration of multi-task learning and ensemble methods to improve robustness. we recognize that a more thorough error analysis would provide deeper insights, and we plan to include this in future iterations of our work.we also acknowledge the importance of comparing our zero-shot baseline to fine-tuned transformer models or fewshot learning approaches, which are likely to yield better results by leveraging task-specific supervision. while our current focus was to establish a simple yet meaningful baseline, we plan to explore and benchmark fine-tuned models and other machine learning baselines in future work to better contextualize performance.