OAS um: Large-Scale Open Domain Aspect-based Summarization (2023)

Abstract: Aspect or query-based summarization has recently caught more attention, as it can generate differentiated summaries based on users’ interests. However, the current dataset for aspect or query-based summarization either focuses on specific domains, on a relatively small scale, or contains only a few aspect types. Such limitations hinder further explorations in this direction. In this work, we take advantage of crowd-sourcing knowledge on Wikipedia and automatically create a high-quality, large-scale open-domain aspect-based summarization dataset named OASum, which contains more than 3.7 million instances with around 1 million different aspects on 2 million Wikipedia pages. We provide benchmark results on OASum and demonstrate its ability for diverse aspect-based summarization generation. To overcome the data scarcity problem on specific domains, we also perform zero-shot, few-shot, and fine-tuning on seven downstream datasets. Specifically, zero/few-shot and fine-tuning results show that the model pre-trained on our corpus demonstrates a strong aspect or query-focused generation ability compared with the backbone model. Our dataset and pre-trained checkpoints are publicly available.

## introduction text summarization aims to provide accurate, concise, and useful information about the original inputs for users to fast browse. existing generic summarization or aspect agnostic summarization methods @xcite @xcite @xcite ) typically generate only one summary for all different requests which is not optimal for diverse demands. it could fail to preserve the required information that the user needs or miss important details (woodsend and lapata, figure 1 : the left section titles are naturally adopted from the wikipedia page to serve as different aspects, while the middle abstract is the head section serving as an overall summary of the article. the right part is the corresponding aspect-based summary. however, existing datasets for aspect-based summarization are either on a small scale @xcite @xcite , only focusing on a specific domain @xcite , or with limited aspects @xcite . to the best of our knowledge, there is no existing dataset with millions of aspects and instances for large-scale open-domain aspect-based summarization. models trained in a small-scale dataset with limited instances or aspects may fail to adapt to other aspects or domains in realistic open-domain scenarios. to tackle the limitations of the existing aspectbased summarization datasets, we propose a largescale open-domain aspect-based summarization dataset named oasum. to create the data, as illustrated in fig. 1 , we take advantage of crowd-sourcing knowledge in a stands for "automatic", m stands for "manual". #input tk. and #output tk. represent the number of input and output token lengths, respectively. #asp. type is the number of all aspect types. #instances stands for the total number of (article, summary) pairs in the corresponding dataset. english wikipedia pages and parse them to collect the information on each page including the title of each section and its contents. on the one hand, the head section is a natural abstract of each wikipedia page. on the other hand, the remaining sections describe different aspects of that page. therefore, we use the section titles as the aspect inputs and apply a rule base process to automatically select sentences in the abstract section as the matched summary for different aspects. specifically, we use the wikipedia dump on 2022/06/21. it contains around 6.3 million pages after parsing. after preprocessing, we keep approximately 2 million pages that contain around 3.7 million instances in total. our dataset includes 1,045,895 different aspects on 32,956 different domains (categorized with the original wikipedia pages), providing plenty of useful information for open-domain aspect-based summarization. it also provides abstractive summaries that are not directly extracted from the original inputs. to ensure the quality, we perform a manual evaluation with randomly selected 66 pages, and the overall satisfaction score is 3.13 out of 5. based on our curated million-level aspect-based summarization corpus, we pretrain longformer-encoder-decoder (led) @xcite model on oasum in an endto-end way. compared with the backbone model, our pretrained model achieves better performance on six out of seven downstream tasks for the finetuning and zero-shot settings and all six downstream tasks for the few-shot setting. the contributions of our work are in two folds: wikipedia as data. wikipedia has been widely used as a rich source for many nlp tasks, including language modeling @xcite , question answering @xcite , information extraction @xcite , dialogue @xcite , and summa-rization @xcite @xcite . wikiasp @xcite directly uses external documents to generate the corresponding section contents with limited aspect types. comparatively, oasum employs a matching method to obtain the aspect-based summaries from the head section of a wikipedia page according to their similarities to the remaining page, resulting in more than one million aspect types. ## downstreams to verify the knowledge inside oasum provides transfer ability, we further use the model pretrained on oasum for seven abstractive downstream datasets (see appendix b.1) including three query-based summarization datasets and four aspect-based summarization datasets, across different domains. we test our model with zeroshot, few-shot, and fine-tuning abilities on these 7 datasets to see whether oasum can benefit the downstream tasks. in general, the model pretrained on the oasum outperforms the backbone model on 6 out of 7 tasks in the fine-tuning and zero-shot setting, 6 out of 6 tasks (w/o wikiasp) in the few-shot setting. 10 ## conclusions in summary, we contribute the first large-scale open-domain aspect-based summarization corpus collected using wikipedia section titles as aspects by rules with good quality. detailed statistics reveal many different aspects of the corpus, confirming its broader coverage. we also outline the methods we use for pre-training the generative language models and present abstractive and extractive results as a baseline for future work. furthermore, we prove that our pre-trained model can consistently improve seven widely-used downstream tasks, especially in few-shot and zero-shot settings. we hope our data and pre-trained models can further foster relevant research in this area.