TECHSSN at S em E val-2025 Task 10: A Comparative Analysis of Transformer Models for Dominant Narrative-Based News Summarization (2025)

Abstract: This paper presents an approach to Task 10 of SemEval 2025, which focuses on summarizing English news articles using a given dominant narrative. The dataset comprises news articles on the Russia-Ukraine war and climate change, introducing challenges related to bias, information compression, and contextual coherence. Transformer-based models, specifically BART variants, are utilized to generate concise and coherent summaries. Our team TechSSN, achieved 4th place on the official test leaderboard with a BERTScore of 0.74203, employing the DistilBART-CNN-12-6 model.

## introduction in recent years, automated narrative extraction has gained significant attention in natural language processing (nlp), particularly in understanding complex socio-political events. a central challenge in this area is ensuring that the extracted summary faithfully captures the key information from the original narrative without losing context or meaning. this study focuses on task 10 in se-meval 2025, which aims to extract, justify and summarize dominant narratives from news articles @xcite . the task is based upon shared tasks focusing on persuasion techniques and subjectivity, like the labs organized as part of se-meval 2023 @xcite , semeval 2020 @xcite , and the checkthat! @xcite labs that are part of the clef tasks. specifically, we work on subtask 3, with english data, which involves generating a concise, free-text explanation that supports the selection of a given dominant narrative. the dataset comprises articles related to the russia-ukraine war and climate change. framed as a text-to-text generation problem, this subtask requires models to produce well-structured, contextually relevant explanations. our work primarily investigates the effectiveness of transformer-based models for summarization, with a particular focus on bart. we evaluate the variants of bart, with the subtasks' official metric-bertscore @xcite . techssn achieved the 4th place on the official test leaderboard, achieving a bertscore of 0.74203 using distilbart-cnn-12-6. ## related work text summarization research has evolved through distinct methodological paradigms, primarily categorized into extractive approaches, which select and concatenate salient text segments, and abstractive methods, which generate novel paraphrases to synthesize coherent summaries, with the latter gaining prominence due to their capacity to distill complex information into fluent narratives. early efforts in abstractive summarization focused on sequence-to-sequence (seq2seq) architectures enhanced with attention mechanisms, as exemplified by @xcite , who leveraged pretrained language models like bert and gpt to address the intricate syntax and domain-specific terminology of legal texts, demonstrating the superiority of abstractive methods over extractive ones while proposing a hybrid framework that combined extractive preprocessing (to identify key sentences) with abstractive generation (to refine coherence); however, this approach lacked rigorous empirical validation against standardized benchmarks, leaving its scalability and generalizability uncertain. subsequent innovations sought to enhance semantic fidelity and structural coherence: @xcite introduced the atsdl framework, integrating lstm networks for sequential context modeling with cnns for local feature extraction, guided by a multiple order semantic parsing (mosp) method-a phrase extraction technique that hierarchically identifies multi-level semantic units (e.g., clauses, propositions) to steer abstractive generation-though its reliance on phrase-level processing limited scalability for long documents, as recursive segmentation introduced redundancy. to address length constraints, @xcite adapted the bart model with a chunking strategy, splitting inputs into segments shorter than 1,024 tokens, summarizing each incrementally and aggregating results, achieving competitive rouge-1 and rouge-2 scores (43.02% and 20.57%, respectively) on the cnn/dailymail benchmark; however, their dependency on pretraining data from news articles hindered generalization to highly abstractive domains like xsum, where brevity and conceptual synthesis are paramount. parallel efforts explored summarization's utility for downstream tasks: (tran and kruschwitz, 2022) combined extractive summarization (via distilbart) with abstractive generation (using t5-3b) for crosslingual fake news detection, translating german articles to english summaries to train classifiers, achieving a modest f1 score of 28.99% on german subsets-highlighting the challenges of languagespecific biases and information loss during crosslingual transfer. despite these advancements, critical gaps persist: hybrid frameworks remain undervalidated, long-document methods prioritize technical scalability (e.g., token limits) over conceptual nuance (e.g., preserving socio-political context), and existing models inadequately address bias propagation, particularly in contested domains like geopolitics or climate science, where dominant narratives can skew summary tone and factual balance. our work bridges these gaps by adapting bart to generate concise (80-word), domainaware summaries for topics such as the russia-ukraine war and climate change, explicitly incorporating narrative context (e.g., dominant perspec-tives, subnarratives) during generation to study bias modulation. unlike prior studies fixated on metrics like rouge, we prioritize correctness (factual alignment with source content) and unbiasedness (neutral framing of contentious claims), tackling the unique challenge of compressing complex, ideologically charged narratives into balanced summaries-a task demanding both architectural innovation and ethical rigor, as models must disentangle factual reporting from rhetorical framing without amplifying systemic biases inherent in pretraining data. ## dataset the objective of this subtask is to generate a concise, free-text explanation (up to 80 words) that elaborates on a dominant narrative within an article. subtask 3 requires models to generate explanations that align closely with established ground truth. the official evaluation metric for this subtask is the bertscore, which measures the average similarity between the predicted explanations and the gold-standard references. this ensures that the generated explanations are not only semantically accurate but also contextually aligned with human annotations. ## system overview summarization methods in general, can be grouped into abstractive summarization @xcite , extreme summarization @xcite , and dialogue summarization @xcite phrases that are not present in the original text. this allows for greater flexibility in language as well as coherence. on the contrary, extreme summarization produces highly compressed summaries, often consisting of a single sentence that captures the core idea of the input text. dialogue summarization summarizes conversational text while maintaining contextual coherence. models for dialogue summarization are usually trained on corpora like dialogsum or samsum @xcite . for this study, we employed a range of transformer-based summarization models with varying architectures, depths, and training objectives. our selection includes models based on bart (bidirectional and auto-regressive transformer), distilbart (a distilled version of bart), t5 (text-to-text transfer transformer), and fal-conai's summarization model. below, we provide an architectural overview of these models. ## experimental setup this section details the steps taken in the implementation of the summarization models, including preprocessing, model training and hyperparameters. ## results this section presents the results of the models on both the development and test sets. table a2 in the appendix illustrates the performance of various models on the development set. the bestperforming model is distilbart-cnn-12-6, achieving a bertscore of 0.74459. this is the model that we submitted to the official test leaderboard. similarly, ## conclusion this work primarily explores the use of transformer-based models for news article summarization, demonstrating their effectiveness in generating concise and coherent summaries. the findings highlight the strong performance of bart-based models compared to other transformer models. for future work, integrating large language models (llms) into the summarization process offers a promising direction, given their advanced capabilities and adaptability across diverse domains. leveraging llms can enhance the quality and coherence of generated summaries, particularly in domain-specific and real-world applications. this integration opens up new possibilities for building more effective and context-aware summarization systems.