E ur L ex S ummarization – A New Text Summarization Dataset on EU Legislation in 24 Languages with GPT Evaluation (2024)

Abstract: Legal documents are notorious for their length and complexity, making it challenging to extract crucial information efficiently. In this paper, we introduce a new dataset for legal text summarization, covering 24 languages. We not only present and analyze the dataset but also conduct experiments using various extractive techniques. We provide a comparison between these techniques and summaries generated by the state-of-the-art GPT models. The abstractive GPT approach outperforms the extractive TextRank approach in 8 languages, but produces slightly lower results in the remaining 16 languages. This research aims to advance the field of legal document summarization by addressing the need for accessible and comprehensive information retrieval from lengthy legal texts.

## introduction the task of automatically summarizing legislative documents poses a formidable challenge, primarily due to the extensive nature of these texts, making them intricate to comprehend and process. this complexity is further compounded when dealing with 24 languages, as there is a relative scarcity of pre-trained models for summarization in comparison to more widely-used languages such as english. while existing text summarization models have shown promise, they are generally trained on shorter texts, such as social media posts or news articles, which significantly differ in complexity and length from legislative documents. in the forthcoming section, there is an overview of text summarization approaches tailored for longer documents. this paper presents a new dataset comprising eu legislative documents available in 24 languages. this dataset has undergone cleaning and preprocessing to ensure its utility and accessibility for research and development in the field of legal document summarization. the contributions of this paper can be summarized as follows: ## related work text summarization is a fundamental task in natural language processing, with a wide range of applications, from news article summarization to document summarization. in this section, we discuss related work in two key areas: text summarization datasets and the summarization of long and legislative documents. ## dataset collection and preprocessing the dataset contains legislative documents and their summaries in 24 languages from the european union foot_0 . in addition to producing a new dataset to serve other researchers, these were used for various experiments. the data is downloaded from the official website of the european union 2 . it contains legal texts on various topics, including laws, acts and others. each paper has a summary generated by an expert in the field. all documents and summaries, apart from english, have been translated into up to 23 other languages. some summaries summarize more than 1 document. at the time of data collection, there are 1,816 summaries and their corresponding full documents in english. a comparison between the data in different languages is made in the following sections. many characteristics of the dataset make it valuable to the research field. it contains subjectspecific text that is challenging for overtrained generic text models. the texts vary in length, with some documents being extremely long. having them in different languages provides an opportunity to compare the results and validate whether the experiments work only in a particular case. the small number of documents further complicates the task. ## conclusion the paper introduces a new multilingual dataset of european legislative laws, encompassing 24 languages. this dataset, characterized by significant variations in document length and a limited number of documents per language, presents a valuable resource for the field of automatic summarization. we conducted extensive experiments, employing three extractive summarization approaches, and introduced a novel two-step methodology that harnesses the capabilities of gpt models for summarization. notably, our two-step approach outperforms existing methods in some languages, demonstrating its potential as an effective summarization technique. however, in certain cases, textrank surpasses it in performance. in addition to our summarization findings, we offer valuable insights into the practical considerations of using gpt, including detailed information on associated costs. this information is essential for researchers and practitioners looking to leverage state-of-the-art models in real-world applica-proceedings of clib 2024 tions. by addressing the complex task of summarizing european legislative laws in diverse languages, our work contributes to the advancement of the field, offering a valuable resource and novel techniques for future research and applications in automatic summarization and multilingual natural language processing.