Darbarer @ A uto M in2023: Transcription simplification for concise minute generation from multi-party conversations (2023)

Abstract: This document reports the approach of our team Darbarer for the main task (Task A) of the AutoMin 2023 challenge. Our system is composed of four main modules. The first module relies on a text simplification model aiming at standardizing the utterances of the conversation and compressing the input in order to focus on informative content. The second module handles summarization by employing a straightforward segmentation strategy and a fine-tuned BART-based generative model. Then a titling module has been trained in order to propose a short description of each summarized block. Lastly, we apply a post-processing step aimed at enhancing readability through specific formatting rules. Our contributions lie in the first, third and last steps. Our system generates precise and concise minutes. We provide a detailed description of our modules, discuss the difficulty of evaluating their impact and propose an analysis of observed errors in our generated minutes.

## introduction the covid-19 pandemic has led to substantial changes in our way of communicating, interacting and collaborating. as the virus has required social distancing measures and the implementation of remote working across many industries, communication methods have shifted from traditional face-to-face interactions to virtual platforms. consequently, the reliance on digital tools and technologies has grown exponentially, altering not only the nature of our conversations but also the means by which they are documented and managed. in this paper, we introduce a novel approach to automatic minuting tools tailored to address the unique challenges of online communication. we submitted this system for task a of the automin2023 challenge @xcite . the primary objective of this task is to develop an automated system capable of generating minutes from multiparty meeting transcripts. the performance of the resulting summaries are to be assessed using a combination of automatic and manual evaluation metrics. for this system, we only used the task training data as well as the additional data that was recommended. we did not use large language models nor any additional training data, which positions our submission in the constraint category. instead, we used "classical" language models derived from bart. while there is no strict parameter count that officially defines if a language model is "large", at the time this paper was written, the consensus seems to be that any model exceeding 1 billion parameters with the capacity to be prompted qualifies as such in the work of @xcite . however, bart does not meet these criteria. we first describe in section 2 the data provided for the au-tomin shared task, being the elitr and the eu-roparlmin corpus. then we provide related work in 3 before describing in section 4 the different modules of our system. finally, we provide in section 5 insights on the results by analyzing the effect of each module on the metrics and detailing the different errors we've encountered in the generated minutes. ## overview of the submitted system this section presents the darbarer system foot_0 submitted for automin2023 task a, in the constraint category. the overall system is illustrated in figure 1. our system is composed of four main modules. the first module relies on a text simplification model aiming at standardizing the utterances of the conversation and compressing the input in order to focus on informative content. the second module handles summarization by employing a straightforward segmentation strategy and a finetuned bart-based generative model. then a titling module has been trained in order to propose a short description of each summarized block. lastly, we apply a post-processing step aimed at enhancing readability through specific formatting rules. ## results the automin 2023 challenge provided three test sets: elmics and elmien for elitr meeting in czech and english respectively and europarl in english. full results and details about the evaluation process are provided in @xcite . we obtain a rouge-1 score of 0.31 on elmics, 0.39 on elmien and 0.27 on europarl. manual evaluation has been produced with the align-meet tool @xcite , focusing on adequacy, grammaticality, fluency, relevance and at two different levels of granularity : at the documentlevel and the hunk-level (a hunk is defined as a set of dialog acts belonging to a summary point). ## conclusion and discussion in this paper, we described our system for the au-tomin 2023 challenge task a and detailed its four different modules: simplification, summarization, titling and formatting. our submitted system produces meeting minutes that are concise, intelligible and that may already be usable without further modifications, in a multitude of use cases. however, it is not error-proof and still subject to improvement, regarding the way we could cleverly split the conversation into coherent segments, or how to ensure correctness in regards to grammar, semantics and person tracking. moreover, this work highlights the need for better metrics for evaluating the results of abstractive summarization systems in order to make better informed decisions for the design of the whole pipeline. we believe this shared task to be very relevant, especially in times where automatic content summarization is becoming more and more common.