MODABS : Multi-Objective Learning for Dynamic Aspect-Based Summarization (2024)

Abstract: The rapid proliferation of online content necessitates effective summarization methods, among which dynamic aspect-based summarization stands out. Unlike its traditional counterpart, which assumes a fixed set of known aspects, this approach adapts to the varied aspects of the input text. We introduce a novel multi-objective learning framework employing a Longformer-Encoder-Decoder for this task. The framework optimizes aspect number prediction, minimizes disparity between generated and reference summaries for each aspect, and maximizes dissimilarity across aspect-specific summaries. Extensive experiments show our method significantly outperforms baselines on three diverse datasets, largely due to the effective alignment of generated and reference aspect counts without sacrificing single-aspect summarization quality.

## introduction with the rapid increase in digital content, the need for automated text summarization systems has grown significantly, as summaries efficiently distill key information from extensive texts. among various approaches, query-focused summarization @xcite @xcite and aspect-based summarization @xcite are particularly prominent for generating contentspecific summaries. these methods, referenced in recent studies, cater to diverse information needs by emphasizing particular aspects or queries within the summaries. aspect-based summarization, illustrated in figure1, specifically focuses on extracting and summarizing information relevant to predefined aspects, offering a targeted approach to understanding large datasets. however, both methodologies encounter substantial constraints with the requirement of prespecified aspects, thereby limiting their applicability. dynamic aspect-based summarization, a recently proposed task @xcite , requires models to automatically identify and extract relevant aspects from a given text, removing the need for prior knowledge about aspect numbers or content. prior research on dabs typically adopts a two-phase methodology, initially focusing on generating aspect-based elements-such as identifying key sentences @xcite or keywords @xcite @xcite relevant to each aspect. this step is followed by employing a conventional summarization model that integrates the identified aspect information with the main article. an alternative approach utilizes the capabilities of large language models (llms), such as gpt-3.5-turbo, for aspectbased summarization without predetermined aspects @xcite . however, despite llms' potential, their effectiveness in dabs is often lower than the two-step approach, mainly due to their insufficient explicit aspect knowledge and prior information @xcite . to solve this task, we propose a novel multiobjective dynamic aspect-based summarization framework, which we call modabs. our framework dynamically identifies and summarizes the aspects without needing prior knowledge. our multiobjective model not only works towards the conventional summarization objective but also predicts the number of aspects and works towards maximizing the divergence (i.e., minimizing the overlap) between the summaries of different aspects. empirical evaluations on three datasets for this task show that our framework outperforms the existing models in predicting the number of aspects and generating high-quality aspect-based summaries. furthermore, our ablation analysis reveals the crucial role of multi-objective learning in enhancing performance. the contributions foot_0 of this work are as follows: ## datasets in our study, we focus on three datasets designed for dynamic aspect-based summarization: disordered-dabs (guo and vosoughi, 2024) and oasum @xcite . disordered-dabs is crafted for disordered dynamic aspect-based summarization, encompassing two datasets: d-cnndm, adapted from cnn/daily mail @xcite , and d-wikihow from wik-ihow @xcite . these datasets utilize existing summarization datasets to generate aspect-based summaries by merging multiple samples (d-cnndm) or segmenting topics into detailed steps (d-wikihow), introducing shuffled input sentences to mimic the disorder often found in online discussions. oasum is a dynamic aspect-based summariza-tion dataset containing over three million samples. unlike disordered-dabs, aspects in oasum may exhibit hierarchical structures, where one aspect may be nested within another, e.g., "history" and "history: japanese invasion". to ensure a fair comparison with other baselines, we preprocess oasum to retain only aspects with the narrowest scope. similar to disordered-dabs @xcite , sentence shuffling is introduced to create disorderly scenarios. both ddabs and oasum datasets include samples with very long inputs, single-aspect summaries, or multiple aspects. following @xcite and @xcite , we preprocess these datasets by truncating source articles and summaries and limiting the number of aspects based on their average length and variability, as detailed in appendix c.3. given that the original oasum dataset includes about half of its samples with only one aspect, we exclude these single-aspect samples from our experiments. ## methodology our fine-tuning process, illustrated in figure 2 , diverges from standard summarization fine-tuning in two key areas: first, we modify the decoder embeddings with the intent to account for multiple aspects, a feature not commonly considered in traditional summarization tasks. by transforming the embeddings, our model generates multiple aspectspecific summaries concurrently, offering a more comprehensive summary of the input text. second, we incorporate a multi-objective learning scheme into our framework. these objectives include not only the standard text summarization task but also the maximization of the distance between different aspect-based summarization representations. furthermore, our model is designed to predict the number of aspects based on the decoding representation. in effect, the multi-objective learning approach encourages our model to create more distinct and specific summaries per aspect, thus enhancing the granularity and precision of the summarization. ## ablation analysis in this section, we present examples of summaries generated using different methods. we also conduct ablation analyses to investigate (1) the influence of the multi-objective settings and (2) the influence of discrepancies in the aspect numbers between the reference and generated summaries. ## conclusion and future directions this study introduces a robust multi-objective framework tailored for dynamic aspect-based summarization, addressing the complexities of disordered information prevalent in today's digital landscape. by innovatively merging aspect discovery with aspect-based summarization, our approach surpasses traditional methods in accurately predicting aspect counts and significantly enhances summary quality. the empirical evidence drawn from extensive experiments across various datasets underscores the efficacy of our framework. looking ahead, the integration of advanced attention mechanisms and adaptive layers promises to further refine our model, paving the way for even more precise and efficient summarization techniques.