NCLT eam at S em E val-2025 Task 10: Enhancing Multilingual, multi-class, and Multi-Label Document Classification via Contrastive Learning Augmented Cascaded UN et and Embedding based Approaches (2025)

Abstract: The SemEval 2025 Task 10 Subtask2 presents a multi-task multi-label text classification challenge. The task requires systems to classify documents simultaneously across three distinct topics, the Climate Change(CC), the Ukraine Russia War(URW), and others. Several challenge were identified, including the instinct distinct of topics, the imbalance of categories, the insufficient samples, and the different distribution of develop set and test set. To address these challenges, two deep learning model have been implemented. One of the approach is the Contrastive learning augmented Cascaded UNet model(CCU), which employs a cascaded architecture to jointly process all subtasks. This model incorporates an UNet-style architecture to classify embeddings extracted by the base text encoder. A domain adaption method was implemented to facilitate joint learning across different document topics. We address the data insufficiency through contrastive learning and mitigate data imbalance using asymmetric loss function. We also implemented a shallow machine learning model. In this approach, transformer encoder models were applied to extract text embedding from various aspect, then deploy machine learning method to do the classification and compared with the base line. The UNet-style model achieves the highest f1 sample at 0.365 on the test set of 5th place compared with all approaches on leader board. Our source code developed for this paper are available at

## introduction semeval-2025 task 10 subtask 2 introduces a multilingual, hierarchical, and multilabel document classification challenge. our approaches integrate contrastive learning, hierarchical pathway modeling, and domain adaptation techniques to address these challenges. we also identified several dataspecific issues, including insufficient training samples, significant class imbalance, and distribution shifts between development and test sets. our methodology systematically addresses these challenges through strategic neural architecture design and feature engineering. in the context of processing text data across multiple languages, existing research generally follows two approaches, pre-training model on massive multilingual corpora to enable cross-lingual transfer @xcite , or distilling multilingual knowledge into monolingual language models to optimize the computational efficiency @xcite . in our work, we applied a pre-trained monolingual encoder model, fine-tuned with a multilingual dataset as our base encoder. for hierarchical multi-label classification with limited samples, prior research such as the hierarchical verbalizer model employs prompt-based learning to incorporate label hierarchy knowledge into the model @xcite . in our approach, we explicitly defined the hierarchical data structure by organizing the dataset into topic-based sub-categories, and designing a corresponding cascaded model architecture specifically tailored to this dataset taxonomy. this architectural framework enforces a hierarchical prediction flow. ensuring that classification results adhere to the inherent structure of the classification task. the proposed bst-est model utilizes the prompt templates with masked language models to leverage pre-trained knowledge for few-shot learning, dynamically capturing the label-text interaction. to mitigate the significant differences between the cc and urw topics, we implement a gradient reversal layer (grl) for domain adaptation. the grl adversarially aligns feature representations from different domains by reversing gradients during backpropagation. this technique encourages the feature extractor to produce domain-invariant features, enabling better knowledge transfer between the structurally similar but topically distinct cc and urw narratives. @xcite . during inference, attention masks were applied to ensure the model predictions adhere to the natural hierarchy of labels, forcing classifications to respect the narrative and sub-narrative relationships in the label taxonomy. in detail, the parent category probabilities effectively act as an attention mechanism that gates the flow of information to specialized pathways. this ensures that the narrative and sub-narrative predictions are only evaluated for samples belonging to the correct parent category by the masked attention mechanism. a gradient inverse layer was implemented to achieve domain adaption by learning domaininvariant features across cc and urw. contrastive learning was applied to address the insufficient data @xcite . our augmentation pipeline combines contextual word substitution and back translation to mitigate the influence of the data imbalance. during the experimentation, we observed a significant limitation in the cascaded model. when trained on both the narratives and sub-narratives classification tasks simultaneously, the model would effectively learn one task while performing poorly on the other. this phenomenon, which we identified through gradient flow visualization, appeared to be caused by gradient vanishing in one of the task pathway. by implementing asymmetric loss, we successfully addressed this problem, enabling the model to learn both tasks effectively. by combining these techniques, our approach provides a framework that achieves competitive performance for multilingual, hierarchical, multi-label text classification in low-resource scenarios. ## methodology to address the multilingual, multi-class, multi-label documentation classification task, we applied a bst-est model, and a ccu model to test the classification performance. ## experiment the training set consists of 1699 samples distributed across five languages (english, russian, hindi, portuguese, and bulgarian). the taxonomy was designed to analyze and compare propaganda narratives in two conflict domains: the russia-ukraine war and climate change. both categories employ similar broadcasting techniques that attempt to redirect responsibility, challenge opposing credibility, heighten concerns about negative outcomes, and divert attention. these approaches tend to polarize audiences and support specific political positions. we further split the dataset into subsets based on hierarchical relationships, with narratives and sub-narratives grouped by their main topic (cc or urw). this organization forced the model to learn classification patterns that respect the category hierarchies. all language materials were combined into a unified multilingual dataset. ## discussion in this semeval-2025 task 10 subtask 2, we successfully implemented our approaches and achieved 6th place. we proposed two methods to solve this multi-label multi-class multilingual text classification task. the ccu model, which integrates a pre-trained language model with a unet cascaded classifier, hierarchical dataset organization, gradient reversal for domain adaptation, asymmetric loss, and contrastive learning. the bst-est method using pre-trained transformer encoder model, to extract embeddings, also the sentiment message was added to the embeddings then classified with machine learning classifiers. furthermore, limitations still remain unsolved. for the ccu model, labels are represented using one-hot encoding rather than semantic text embeddings, which results in the loss of inherent label meaning. ## conclusion this article presents an integrated implementation of existing solutions for multi-class, multi-label text classification to address semeval-2025 task 10 subtask 2. our team developed two distinct approaches to enhance classification performance. one built upon a cascaded unet model with contrastive learning, and another leveraged multiple nlp feature extraction methods combined with machine learning classification techniques. our efforts resulted in achieving 6th place with the main metric f1 sample of 0.345. the ccu model addressed the challenges identified during experimentation. we mitigated data imbalance through augmentation process, while contrastive learning techniques helped overcome data insufficiency. the cascaded model architecture, with specialized pathways, was designed to tackle the complexities of multi-class learning across dif-ferent label granularities. these solutions contributed to performance improvements. finally, the result remains below the optimal level for a reasonable classification. furthermore, the integration of existing research offers limited novelty in the research field. additionally, the pathway learning issue we identified, where the model would learn one pathway at the expense of another, suggests opportunities for developing more balanced training techniques for hierarchical classification models.