CL a C at S em E val-2025 Task 6: A Multi-Architecture Approach for Corporate Environmental Promise Verification (2025)

Abstract: This paper presents our approach to the PromiseEval task at SemEval-2025, which focuses on verifying promises in corporate ESG (Environmental, Social, and Governance) reports. We explore three model architectures to address the four subtasks of promise identification, supporting evidence assessment, clarity evaluation, and verification timing. Our first model utilizes ESG-BERT with task-specific classifier heads, while our second model enhances this architecture with linguistic features tailored for each subtask. Our third approach implements a combined subtask model with attention-based sequence pooling, transformer representations augmented with document metadata, and multi-objective learning. Experiments on the English portion of the ML-Promise dataset demonstrate progressive improvement across our models, with our combined subtask approach achieving a private leaderboard score of 0.5268, outperforming the provided baseline of 0.5227. Our work highlights the effectiveness of linguistic feature extraction, attention pooling, and multi-objective learning in promise verification tasks, despite challenges posed by class imbalance and limited training data.

## introduction the promiseeval task at semeval-2025 @xcite addresses the critical challenge of verifying promises in esg (environmental, social, and governance) reports published by corporations across multiple languages and industries. corporate promises significantly influence stakeholder trust and organizational reputation, yet their complexity and volume make verification difficult. this task breaks down promise verification into four essential subtasks: all of the code used in the implementation of the models described in this paper is made available on github foot_0 . ## background the promiseeval task at semeval-2025 builds upon the ml-promise dataset introduced by @xcite , the first multilingual resource for promise verification in corporate esg communications. for our experiments, we focused exclusively on the english portion containing 400 training instances labelled for each of the four classification subtasks. as shown in figure 1 , the dataset exhibits class imbalance across all four subtasks, creating challenges for model development. our work explores effective model architectures, with a particular focus on linguistic feature extraction and multi-objective learning with contextual enrichment. the combined subtask model addresses these challenges through attention-based sequence pooling, transformer representations augmented with document metadata, and a training methodology incorporating focal loss and test-time augmentation to improve performance on imbalanced classes while maintaining computational efficiency. ## related work our work builds upon several interconnected areas in natural language processing (nlp) and esg text analysis. this section outlines prior work related to our approach and contextualizes our contributions. ## system overview our system explores three different model architectures, as illustrated in figure 2 . model 1: a baseline architecture with esg-bert and task-specific classifier heads processing the given text inputs as is. model 2: a feature-enhanced esg-bert model incorporating linguistic features tailored for each subtask. model 3: a combined subtask model that integrates a multi-objective architecture for subtasks 1 and 2, and uses attention pooling with a shared transformer backbone for better feature extraction. ## results and discussions since the combined model only worked on tasks 1 and 2, and kaggle required all four subtasks for evaluation, we incorporated task 1 and 2 predictions from our combined model while using our feature-enhanced model for tasks 3 and 4. the private scores show improvement across our models. starting with the base model (0.4994), we achieved better results with the feature-enhanced model (0.5094), and our combined subtask model reached 0.5268, surpassing the kaggle baseline (0.5227). while the improvements are modest, they suggest our architectural changes and feature engineering methods are effective for promise verification tasks. our base model and the feature-enhanced model show a slight improvement of approximately 0.010 on the private leaderboard (from 0.4994 to 0.5094). our minimal improvements in performance likely stem from esg-bert already implicitly capturing many of these patterns during domain-specific pre-training, creating a redundancy effect. additionally, our prepending approach may have created a structural disconnect between features and relevant text spans, while the limited training data (400 instances) constrained the model's ability to learn optimal weightings for the introduced features. the combined subtask model yields the largest gain, achieving 0.5268, a 1.74% absolute improvement over the baseline. we attribute this improvement to three factors: (1) multitask learning benefits from shared representations between promise and evidence detection, (2) attention pooling allows the model to focus on semantically relevant tokens dynamically, and (3) test-time augmentation reduces variance in prediction by ensembling multiple augmentations. however, despite achieving our highest score, the combined subtask model showed only modest gains relative to its substantially increased architectural complexity and computational requirements. the limited size of our training dataset (400 instances) may have prevented the model from fully leveraging its advanced components like attention pooling and multi-objective learning, while the potential negative transfer between promise and evidence tasks may have constrained performance gains for instances where these classifications require contradictory feature attention. ## conclusion our work explored three model architectures for promise verification in esg reports: a baseline esg-bert model, a feature-enhanced model incorporating linguistic markers, and a combined subtask model with attention pooling. the combined model achieved the best performance (0.5268 on the private leaderboard), outperforming the kaggle baseline. despite the challenge of class imbalance across all four subtasks, our linguistic feature extraction approach and multi-objective learning framework demonstrated effectiveness in promise verification with limited training data. future work could explore incorporating crosslingual promise verification through multilingual transformer models, integrating more advanced linguistic pattern recognition, and incorporating all classification tasks within a single multi-objective architecture to better capture interdependencies between promise identification, evidence assessment, clarity evaluation, and verification timing. additionally, a systematic ablation study could quantify the contribution of each backbone pre-trained language model (plm), such as bert and roberta, and each feature strategy (e.g., linguistic features, document metadata).