ABCD at S em E val-2025 Task 9: BERT -based and Generation-based models combine with advanced weighted majority soft voting strategy (2025)

Abstract: first submission to SemEval-2025 task 9 by ABCD team

## introduction semeval-2025 task 9: the food hazard detection challenge @xcite the food hazard detection task evaluates explainable classification systems for titles of food-incident reports collected from the web. these algorithms may help automated crawlers find and extract food issues from web sources like social media in the future. due to the potentially high economic impact, transparency is crucial for this task. two sub-tasks were proposed for participants in this shared task. the first challenge is called "text classification for food hazard prediction, predicting the type of hazard and product", in the first task the participants are required to develop a system that can classify food reports into 10 hazard categories and 22 product categories. task 2, this task bears some resemblance to the first task, yet participants need to classify the text input into the exact vector of 128 hazards and 1068 products. in today's interconnected world, where information flows ceaselessly across digital platforms, ensuring food safety remains a paramount concern. the ability to quickly and accurately detect food hazards from textual data is not only advantageous but imperative. natural language processing (nlp), with its ability to parse through large amounts of text, plays a pivotal role in this endeavor. using computational linguistics and machine learning techniques, nlp equips us with the tools to sift through diverse sources of textual information from social media posts to product reviews to identify potential food hazards efficiently. as a result, in this paper, we present our solutions for both task 1 and task 2 in semeval-2025 task 9: the food hazard detection challenge @xcite . specifically, we employ three different approaches to address this task: (1) experimenting with a large battery of bert-based models, (2) utilizing generation-based models, (3) taking advantage of a custom ensemble learning method. ## related works food risk has been a major issue that poses a wide range of dangers for human health throughout history. in recent years, many researchers have taken action to tackle food danger by taking advantage of machine learning and computational force in order to predict early sight of food risk. for example, @xcite propose an integrated framework for classifying and analyzing food hazards by leveraging social media data from sina weibo. @xcite framework not only provides a robust method for classifying food hazard-related sentiments but also offers valuable insights for crisis management and policy formulation by mapping how online public opinion evolves during food safety emergencies. specifically, they have shown the bert-textcnn model demonstrates exceptional performance in distinguishing between positive and negative sentiments, effectively capturing subtle emotional nuances in the context of the food hazard incident, and the bertopic model successfully uncovers stagespecific topics and shows how public discourse evolves over time, offering insights into the thematic shifts during the incident. besides that, network analysis highlights the pivotal role of certain nodes in information dissemination, confirming that both official media and influential individual users significantly impact public sentiment. the work by @xcite explores the use of machine learning models to automate the classification of literature in systematic reviews on food hazards. the aim is to reduce the expert's workload while maintaining high accuracy in selecting relevant studies. best-performing model: an ensemble of naive bayes and the support vector machine (nb + svm) achieved the highest overall performance. the study demonstrates that machine learning, particularly ensemble models (nb + svm), can effectively support experts in systematic reviews of food hazards. the approach significantly reduces the manual screening effort without compromising quality, making it a valuable tool for food safety research. ## task description the food hazard detection task focuses on developing interpretable classification models for categorizing titles of food-incident reports sourced from the web. these models have the potential to enhance automated web crawlers in identifying and extracting food-related risks from online platforms, including social media. given the significant economic implications, ensuring model transparency is a key priority in this task. semeval-2025 includes 2 sub-tasks, which are task 1: text classification for food hazard prediction, predicting the type of hazard and product, and task 2: food hazard and product "vector" detection, predicting the exact hazard and product. ## methodology in this section, we present our approaches for task 1 and task 2 in semeval-2025 shared tasks in detail. ## experimental setup we conducted our training process using hugging-face @xcite , and all bert-based models were trained for 8 epochs. the adamw optimizer was utilized to optimize the models. we selected a learning rate of 5e-5,4e-5 for bert-based models. the batch sizes were set to 16 and 32, the random seed was set to 221, and the maximum token length was 512. cross-validation is a statistical resampling technique used to evaluate the generalization performance of models. given the high dimensionality and complex structures of textual data, effective cross-validation strategies are crucial to prevent overfitting, ensure robustness, and improve model reliability across unseen data. given the imbalanced nature of the dataset, we employed the stratified k-fold cross-validation technique @xcite with @xmath0. stratified crossvalidation ensures that the class distribution remains consistent across folds, thereby reducing bias in performance estimation caused by unequal class distributions in random splits. this approach enables a more reliable evaluation of model performance across diverse subsets of the data. due to computational resource limitations, we had to adjust system settings for fine-tuning the bart model. specifically, we reduced the batch size to 8 and employed gradient accumulation to effectively train on larger effective batch sizes. this technique allows us to accumulate gradients over multiple smaller batches before updating the optimizer, mitigating memory constraints. furthermore, we utilized mixed precision training (fp16) and gradient checkpointing to accelerate training and reduce memory usage. mixed precision training combines 16-bit and 32-bit floating-point operations, enabling efficient training of large-scale models like transformers. dynamic loss scaling was employed to maintain numerical stability. given gpu limitations, we trained bart for only 6 epochs and opted for the adafactor optimizer, known for its efficiency in training large models, instead of adamw. all models were evaluated using the metric provided by the task organizers. our team leveraged a p100 gpu, available for up to 30 free hours per week on kaggle, for computational resources. ## main results in the official final result released by the organizer, our team results in task 1 and task 2 are 0.786 and 0.458 in terms of f1-score, respectively. this result was achieved by using the class-weighted majority voting strategy, which combines bart, roberta, and deberta-v3 models. moreover, in both tasks, our team also applied back-translation and synonyms replacement to augment the specific classes with fewer records to ease the negative effect of data imbalance. however, in task 2, our team only leveraged a generative-based model classification approach, which is bart, to achieve a 0.458 f1score. task 2 has worse results since the imbalance between classes was too tremendous. our team solution achieved top 8th in task 1 and top 10th in task 2. ## limitations we think our greatest limitation is that our team can only leverage the "text" and "title" text features, but using other numerical or categorical features such as the date or country columns. this is also reduce the diversity and specificity for mod-els to generalize the data better. in addition, our generative-based approach takes a great deal of time to train since the size of generative models is mostly larger than that of bert-based models. the class weighted majority voting also needs as a much longer inference time, so we think it can not be used in a real-time application. ## conclusion and future works in this paper, we presented our approach for semeval-2025 task 9: the food hazard detection challenge. our system leveraged bert-based models, generative-based models, and an advanced class-weighted majority voting strategy to enhance classification performance. through extensive experimentation, we demonstrated that combining multiple models with a weighted ensemble technique improves predictive accuracy. our best results achieved f1-scores of 0.786 for task 1 and 0.458 for task 2, highlighting the effectiveness of our approach. for future work, we aim to explore additional features beyond textual data, such as metadata from food reports, to improve classification accuracy. we also plan to experiment with prompt-based learning using large language models (llms) and investigate efficient fine-tuning techniques to reduce computational costs.