YNU - HPCC at S em E val-2025 Task 10: A Two-Stage Approach to Solving Multi-Label and Multi-Class Role Classification Based on D e BERT a (2025)

Abstract: A two-stage role classification model based on DeBERTa is proposed for the Entity Framework task in SemEval 2025 Task 10. The task is confronted with challenges such as multi-labeling, multi-category, and category imbalance, particularly in the semantic overlap and data sparsity of fine-grained roles. Existing methods primarily rely on rules, traditional machine learning, or deep learning, but the accurate classification of fine-grained roles is difficult to achieve. To address this, the proposed model integrates the deep semantic representation of the DeBERTa pre-trained language model through two sub-models: main role classification and sub-role classification, and utilizes Focal Loss to optimize the category imbalance issue. Experimental results indicate that the model achieves an accuracy of 75.32% in predicting the main role, while the exact matching rate for the sub-role is 8.94%. This is mainly limited by the strict matching standard and semantic overlap of fine-grained roles in the multi-label task. Compared to the baseline’s sub-role exact matching rate of 3.83%, the proposed model significantly improves this metric. The model ultimately ranked 23rd on the leaderboard. The code of this paper is available at:https://github.com/jiyuaner/YNU-HPCC-at-SemEval-2025-Task10.

## introduction in subtask 1, given a news article and all named entity mentions (nes) in that article, each mention is required to be assigned one or more role tags @xcite . two levels of characters are defined: one for the main characters @xcite , and the other for the fine-grained characters. the evaluation criterion of the task is primarily the exact match rate, which measures the consistency between the main role and the fine-grained role of the evaluation prediction and the gold standard. semantic overlap @xcite and ambiguity between fine-grained roles may occur @xcite , making it easy for models to be confused when differentiating. entity roles are often determined based on the information in the context in which they are located. in news texts, the context in which entities appear may involve various metaphors, sarcasm, or indirect expressions, requiring the system to deeply understand and capture the details of the context. in real data, some roles (especially fine-grained roles) may have small sample sizes, leading to overfitting of common categories and under-identification of rare categories during model training. in the past, rule-based methods @xcite , traditional machine learning methods, or deep learning methods such as bert @xcite and its variant deberta @xcite , have often been used to solve similar entity character annotation tasks. the two-stage classification model based on de-berta proposed in this paper utilizes the deep semantic representation of the pre-trained language model, combines the entity context, adopts focal loss @xcite to address the category imbalance problem, and employs optuna @xcite to fine-tune hyperparameters for handling entity role labeling tasks with multiple labels and categories. although the fine-grained role matching accuracy still has room for improvement, the model demonstrates effectiveness in main role recognition, with an accuracy rate of 75.32%. the rest of the paper is organized as follows: section 2 details the two-stage deberta-based role classification model, including the main role and sub-role classification sub-models, and focal loss optimization for addressing class imbalance. section 3 introduces the experimental setup, results analysis, and case study, covering dataset specifications, evaluation metrics, comparisons with base- transformer layers [cls] x1 x2 x3 x4 x5 [sep] input e0 e0 e0 e0 e0 e0 e0 e[cls] e1 e2 e3 e4 e5 e[sep] type embedding sub-word embedding h[cls] h1 h2 h3 h4 h5 h[sep] hidden states softmax argmax figure 1: the structure of the system line methods, implementation details, and a discussion of the limitations and challenges encountered in both experiments and real-world scenarios. section 4 concludes the paper by summarizing the experimental outcomes and key findings. ## deberta-based two-stage role classification model in this paper, a two-stage classification model @xcite based on deberta (decoded enhanced bert with a decoupled attention mechanism) is proposed to handle the complex task of character labeling of named entities in text. the overall architecture of the model is divided into two main sub-models: main role classification and sub-role classification. each sub-model is designed to predict character labels at different granularities. the main role refers to a more macro category (e.g., protagonist, villain, and innocent), while the subrole is a more granular category that describes the specific characteristics of an entity. additionally, focal loss is employed to address the category imbalance issue. ## experimental details datasets. the dataset is sourced from semeval 2025 task 10, subtask 1 (role recognition task) and is only studied for the english part. the dataset contains the following: training: consists of multiple english-language news articles, each stored in plain text, with the start and end of entity mentions. development: serves as the basis for training the final model, which is ultimately used for predicting on the test set. test: the final test dataset to be submitted for evaluation, the label of which will not be published until submitted. the number of entity mentions, the number of articles, and the distribution of tags for each role (e.g., main role and fine-grained sub-role) in the dataset bring challenges of multi-label, multi-category, and category imbalance to this task. ## conclusions in this paper, a deberta-based two-stage role classification model is proposed for the role recognition task in semeval 2025 task 10, subtask 1. the model successfully addresses the multi-label, multiclass role classification problem through a main role classification layer and a fine-grained sub-role classification layer, and focal loss is used to optimize the class imbalance issue. this two-stage structure allows the model to not only accurately predict the main roles in news texts but also further identify fine-grained sub-roles, improving the precision of role categorization. experimental results show that the model outperforms the baseline methods in all evaluation metrics. main role accuracy reached 75.32%, but the exact match rate for sub-roles were low, at 0.0894, mainly due to the strict matching criteria in the multi-label task and semantic overlap in fine-grained roles.