PAT eam at S em E val-2025 Task 9: LLM -Augmented Fusion for AI -Driven Food Safety Hazard Detection (2025)

Abstract: This paper introduces the approach we adopted for the SemEval-2025 “Food Hazard Detection” task, which aims to predict coarse-grained categories (such as “product category” and “hazard category”) and fine-grained vectors (such as specific products like “ice cream” or hazards like “salmonella”) from noisy, long-tailed text data.To address the issues of dirty data, as well as the severe long-tail distribution of text labels and length in the data, we proposed a pipeline system. This system combines data cleaning, LLM-based enhancement, label resampling, and ensemble learning to tackle data sparsity and label imbalance problems.The two subtasks have strong semantic relatedness. By integrating them into a unified multiturn dialogue framework, we fine-tuned five models using a bagging approach. Ultimately, we achieved good results in both subtasks, ranking 5th (with an F1 score of 80.17% for ST1 and 52.66% for ST2).

## introduction food safety incidents pose significant risks to public health and economic stability, necessitating rapid detection and transparent decision-making systems. the semeval 2025 task on food hazard detection addresses this challenge by evaluating systems that classify food incident reports from web resources into predefined categories and specific vectors for "product" and "hazard." this task focuses on english-language reports, aiming to automate the discovery of food-related risks from social media and news platforms, where timely and interpretable predictions are critical for mitigating economic and health impacts. the task requires dual subtasks: st1 for predicting hazard and product categories (e.g., "meat, eggs, and dairy" or "pathogenic bacteria") and st2 for identifying exact hazard and product entities (e.g., "salmonella" or "ice cream"). with 1,142 unique products and 128 hazards distributed across imbalanced categories, the task demands robustness against long-tail distributions and noisy text, reflecting real-world complexities in food safety monitoring @xcite . our system integrates data augmentation, label resampling, and ensemble learning to tackle these challenges. inspired by advances in nlp for lowresource scenarios, @xcite proposed some traditional data augmentation methods: synonym replacement, random insertion, random swap, and random deletion. in addition to these, advanced strategies like metadata-aware data augmentation @xcite (e.g., substituting similar products from a food ontology) and prototypical networks @xcite show promise but remain untested in multi-task food safety contexts. against this backdrop, we employed large language models (llms) to generate synthetic summaries of raw incident reports. this approach not only enhanced the diversity of the dataset but also preserved its semantic integrity. to address the severe class imbalance in our dataset, we employed a combination of oversampling techniques for minority classes @xcite and a bagging ensemble @xcite comprising five fine-tuned models with low-rank adaptation (lora) @xcite . this approach effectively mitigated the imbalance and enhanced model performance. mtln (multidimensional type-slot label interaction network) @xcite is a neural network-based mtl framework designed to handle multiple natural language processing tasks through a unified architecture. compared with single-task learning, multitask learning (mtl) demonstrates enhanced generalization capabilities by leveraging task correlations and complementarity, which has been theoretically validated. given that st1 and st2 in our challenge are both focused on food hazard detection and are highly related, we integrated large language models (llms) to combine st1 and st2 into a multi-turn dialogue framework. this framework enables the model to effectively utilize data from multiple tasks, leading to improved generalization and adaptability, while also mitigating issues such as underfitting or overfitting @xcite . our experiments yielded competitive results: 80.17% f1-score (4th rank) for st1 and 52.66% f1-score (3rd rank) for st2. quantitative analysis demonstrated that, compared to single-model predictions, employing bagging voting with five models boosted performance by 1.09% for subtask 1 and 3.14% for subtask 2. this indicates the effectiveness of the bagging voting approach, especially in significantly enhancing the model's generalization ability when dealing with long-tailed label distributions. however, the system encountered difficulties in handling ambiguous hazard descriptions (for example, distinguishing between "listeria monocytogenes" and "listeria spp"). this reflects the limitations of fine-grained entity recognition observed in the food safety literature. qualitative errors further highlighted the need for context-aware disambiguation, particularly for overlapping hazard categories such as "sulphur dioxide and sulphites" versus "sulphates/sulphites." these findings align with the broader challenges in interpretable ai for food risk assessment @xcite , emphasizing the trade-off between model complexity and explainability. this paper demonstrates the following contributions: ## system overview as illustrated in figure 1 , our experimental workflow begins with a dataset sourced from web scrap-ing, which contains noisy data and suffers from severe label imbalance as well as a pronounced long-tail distribution of text lengths. to mitigate these issues, we first applied regular expressions to clean the data by removing elements such as hyperlinks, html formatting, and email addresses. following this, we utilized a large language model with prompt engineering to generate textual summaries of the cleaned data, aiming to augment our dataset and enhance its diversity. this was achieved by concatenating the original texts with their generated summaries to form an enriched dataset. to address the label imbalance problem within this enhanced dataset, we implemented label resampling techniques. subsequently, using a bagging approach, we sampled five subsets of data with replacement. considering the strong interrelation between the two subtasks (st1: food hazard prediction; st2: precise vector detection), we combined them into a unified framework through a multi-turn dialogue format. specifically, we performed supervised fine-tuning (sft) using the lora method across these five subsets. this process resulted in the training of five distinct models, whose outputs were aggregated through voting to determine the final predictions, thereby achieving improved performance and robustness. ## experimental setup this section describes various experiments conducted on model fine-tuning and inference, aimed at exploring the impact of different approaches on the model's f1-score. we applied the lora method to fine-tune the phi-4 models, using a rank of 4, an alpha of 8, and targeting all layers. the phi-4 model parameters were frozen, with only the low-rank adapter parameters being trained. during the lora-based domain fine-tuning, we trained large models using the llama-factory @xcite framework and the adam (kingma and ba, 2014) optimization algorithm. the training included a warm-up step of 10% and a learning rate of 5e-5. additionally, we performed distributed training using deepspeed zero-3 (rajbhandari et al., 2020) on two nvidia a100 gpus (80gb), with a batch size of 1 per gpu and gradient accumulation of 12, for a total of 5 epochs. in this study, we employed various techniques and used bagging to sample five subsets of data, training five distinct models. we then evaluated their performance on the semeval-2025 official dataset by aggregating the predictions from the five lora-fine-tuned llms. the best checkpoints were selected based on the highest f1-score on the validation set, and a voting mechanism was used to make final predictions. ## conclusion in this work, we proposed a multi-turn dialogue modeling approach combined with data cleaning, prompt-based data augmentation, label resampling, and a bagging strategy to tackle the se-meval 2025 food hazard detection challenge. our final system achieved f1 scores of 80.17% and 52.66% on subtask 1 and subtask 2, respectively, securing 4th place in st1 and 3rd place in st2. from the ablation experiments, we observed that combining multi-turn modeling with data augmentation and ensemble methods can effectively mitigate the long-tail distribution and noise issues in real-world datasets. for future work, we plan to explore more advanced model interpretability techniques, domain-specific knowledge incorporation, and automated sampling strategies to further improve both the robustness and explainability of food hazard detection systems.