D an S um T 5: Automatic Abstractive Summarization for D anish (2023)

Abstract: Automatic abstractive text summarization is a challenging task in the field of natural language processing. This paper presents a model for domain-specific sum marization for Danish news articles, Dan SumT5; an mT5 model fine-tuned on a cleaned subset of the DaNewsroom dataset consisting of abstractive summary-article pairs. The resulting state-of-the-art model is evaluated both quantitatively and qualitatively, using ROUGE and BERTScore metrics and human rankings of the summaries. We find that although model refinements increase quantitative and qualitative performance, the model is still prone to factual errors. We discuss the limitations of current evaluation methods for automatic abstractive summarization and underline the need for improved metrics and transparency within the field. We suggest that future work should employ methods for detecting and reducing errors in model output and methods for referenceless evaluation of summaries.

## introduction 1.automatic text summarization automatic text summarization is the automatic generation of short text which condenses the most salient points of a longer text. much of the research in this field to date has focused on automatic extractive summarization @xcite , which directly extracts and concatenates sentences from the original text. various methods have been developed for selecting sentences for extractive summaries. some, such as textrank @xcite , rely on measures of sentence importance; others rely on simple heuristics, such as sentence location. for instance, the simple lead-3 heuristic selects the first three sentences of a given article @xcite . extractive summarization thereby ensures grammaticality but tends to suffer from a lack of coherence. additionally, extractive summaries can be afflicted with dangling anaphoras, in which an extracted sentence refers to a preceding sentence not included in the summary @xcite . in recent years, the move towards deep learning in natural language processing (nlp) has brought to the forefront automatic abstractive summarization. abstractive summaries paraphrase and condense the main points from the original text. this approach views summarization as a text-to-text problem on which models can be trained and finetuned. in many cases, abstractive methods enable more informative summarization, since rephrasing and compressing the text allows for more information to be conveyed by fewer sentences. however, factual credibility is not ensured, and the gener-ated summaries may contain statements that are inconsistent with the original text @xcite . the majority of existing work on both types of automatic summarization has been in english @xcite . in this paper, we develop an abstractive text summarization model for danish news data. we achieve state-of-the-art results by fine-tuning mt5 on a cleaned subset of the danewsroom dataset @xcite ) consisting of news articles and their corresponding abstractive summaries. ## negative results during the training of these models, we attempted a few additional ideas, most of which were shown to be unpromising. this section briefly describes these attempts: ## conclusion this paper presents dansumt5, a set of models achieving state-of-the-art results in automatic abstractive summarization for danish news articles. these results were achieved by fine-tuning mt5 models and implementing more thorough cleaning of the danewsroom dataset. we present state-of-the-art rouge and bertscore performance for danish abstractive summarization with our dansumt5 large . human inspection of the relative quality of the generated summaries revealed that they were generally grammatical and coherent. we discuss several limitations of the quantitative metrics, emphasizing that rouge penalizes lexical diversity inherent to abstractive summarization, while high quantitative performance could obscure low qualitative performance. this emphasizes the need for more transparency in the field, and we argue that research should include more nuanced metrics, as well as manual evaluation of the density and overall quality of the model output. limitations of our work include data quality and the prevalence of factual errors in the generated summaries. all code related to this project is open-sourced via github foot_3 , and the model is made freely available via huggingface for public use foot_4 .