YNU - HPCC at S em E val-2025 Task 6: Using BERT Model with R -drop for Promise Verification (2025)

Abstract: This paper presents our participation in the SemEval-2025 task 6: multinational, multilingual, multi-industry promise verification. The SemEval-2025 Task 6 aims to extract Promise Identification, Supporting Evidence, Clarity of the Promise-Evidence Pair, and Timing for Verification from the commitments made to businesses and governments. Use these data to verify whether companies and governments have fulfilled their commitments. In this task, we participated in the English task, whichincluded analysis of numbers in the text, reading comprehension of the text content and multi-label classification. Our model introduces regularization dropout based on Bert-base to focus on the stability of non-target classes, improve the robustness of the model, and ultimately improve the indicators. Our approach obtained competitive results in subtasks.

## introduction tracking and verifying promises made by businesses and governments is essential for fostering accountability and trust. however, assessing whether these promises are upheld is often hindered by the complexities of different industries, languages, and countries. semeval-2025 task 6 @xcite introduces a novel approach to multilingual, multi-industry promise verification to tackle this challenge. this task is designed to extract key information from promises made to businesses and governments, such as identifying supporting evidence for the promise, evaluating the clarity of the promise-evidence relationship, and determining the appropriate timing for verification. by leveraging this data, the goal is to assess the degree to which these promises have been honored. the subtasks of semeval-2025 task 6 can be divided into binary classification and multiclassification tasks. text classification is an important task in natural language processing (nlp) @xcite ) that aims to organize text into predefined categories automatically. according to the different types of tasks, text classification can be divided into binary, multi-class, and multi-label. in binary classification tasks, the text is divided into two mutually exclusive categories, such as spam and non-spam; multi-class classification tasks divide the text into multiple mutually exclusive categories, such as classified news; and multi-label classification tasks allow each text to belong to multiple categories at the same time, such as multi-label sentiment analysis. this process usually includes data reconstruction, feature extraction, model training, and evaluation. the text must be cleaned and segmented in the data reconstruction stage to convert it into a machine-processable format. next, the feature extraction step converts the text into a numerical vector. commonly used methods include the bagof-words model, tf-idf, word2vec, and bert @xcite . the labeled data trains the machine learning or deep learning model in the model training phase. common models include support vector machine (svm) @xcite and naive bayes (naive) @xcite . after the training is completed, the model needs to be evaluated on the test set, and the performance is evaluated using indicators such as accuracy, precision, recall, and f 1 -score. text classification @xcite has many applications, including sentiment analysis, spam filtering, topic classification, public opinion monitoring, etc. with the development of deep learning technology, pretrained language models @xcite (such as bert and gpt @xcite ) have performed well in text classification tasks, especially for large-scale data sets @xcite and complex tasks. semeval-2025 task 6 contains the following four subtasks: semeval-2025 task 6: promiseevalmultinational, multilingual, multiindustry promise verification competition features a novel multilingual dataset comprising english, french, chinese, japanese, and korean, designed to evaluate corporate esg promises and their implementation. our team has participated in this competition and focus on english datasets. to improve the robustness of our model, we introduced a regularization dropout mechanism based on bert-base. this method focuses on enhancing the stability of nontarget classes, ultimately boosting the model's performance and generalizability. our approach delivered competitive results in this task, showcasing the potential of incorporating regularization techniques into promise verification tasks. this paper discusses our methodology, results, and insights into how these advancements contribute to more effective promise verification. the rest of this paper is organized as follows. section 2 introduces the related work before our study for this task. section 3 gives an overview of our system for this task. section 4 presents the specific details of our system and discusses the experimental results. the conclusions are drawn in section 5. ## ralated work some companies use misleading information to create an overly positive environmental image, a practice known as greenwashing. to address the greenwashing phenomenon and the challenge of evaluating corporate promises, @xcite proposed ml-promise, the first multilingual dataset for deep promise verification, including chinese, english, french, japanese, and ko-rean. the dataset provides key training samples for related technologies in the field of natural language processing (nlp) to verify corporate promises in environmental, social, and governance (esg) reports. the dataset contains promise data from different countries and companies, with structured labels to facilitate the identification and evaluation of corporate promises, supporting evidence, supporting evidence quality, and verification time of the promise. the labels are divided into four main aspects to ensure a comprehensive assessment of corporate promises. @xcite proposed a recommendation system based on natural language processing (nlp) to automatically analyze the credibility and substantive content of corporate sustainability reports. the study adopted a bert-based multi-task learning framework, combined with rule matching and attention mechanism, to extract promise statements from reports and classify their credibility, and external data was used for crossvalidation. the traditional manual review process was enhanced through multimodal analysis, explainable recommendations, and cross-domain adaptation. experiments show that the system achieves excellent results in the task of sustainability report detection. the study provides a technical reference for the automated verification of corporate promises. to promote in-depth verification of promises, this paper aims to apply models in the field of natural language processing to promise verification, and to monitor corporate promises and their compliance with esg promises, as well as the promises and compliance of public figures. ## system overview our system is based on the bert-based model, and the regularization technology rdrop (regularized dropout) @xcite has been added to implement it. the overall structure of our system consists of four modules, which are described below. input layer. in this layer, we build text processing tools for performing text preprocessing and word embedding @xcite . the input is a data frame containing text data (obtained from the train data test set). the text data is converted to the bert input format (token ids) using functions provided by bert. the input is padded to ensure consistent input length within a batch. the dataset ## conclusions this paper proposes a promise verification label classification model base bert with r-drop for semeval-2025 task 6. the model successfully solves the classification problems of commitment existence status, evidence existence status, evidence quality and commitment verification timeline.in the promise verification task, r-drop simulates different sub-networks by randomly dropping neurons, making the model more robust to input perturbations and making the final output labels of each sub-task more accurate. although small models combined with few-shot learning can effectively solve binary classification tasks such as promise status and evidence status, the effect on verification timeline and evidence quality tasks is not ideal. future works will apply, text data augmentation, such as using synonym replacement, random insertion or backtranslation techniques, to expand the training set. moreover, weighted loss functions or oversampling/undersampling techniques to balance the distribution of labels and avoid the model being biased towards the majority class label.