Team NTR @ A uto M in 2023: Dolly LLM Improves Minuting Performance, Semantic Segmentation Doesn’t (2023)

Abstract: This paper documents the approach of Team NTR for the Second Shared Task on Automatic Minuting (AutoMin) at INLG 2023. The goal of this work is to develop a module for automatic generation of meeting minutes based on a meeting transcript text produced by an Automated Speech Recognition (ASR) system (Task A). We consider minuting as a supervised machine learning task on pairs of texts: the transcript of the meeting and its minutes. We use a two-staged minuting pipeline that consists of segmentation and summarization. We experiment with semantic segmentation and multi-language approaches and Large Language Model Dolly, and achieve Rouge1-F of 0.2455 and BERT-Score of 0.8063 on the English part of ELITR test set and Rouge1-F of 0.2430 and BERT-Score of 0.8332 on the EuroParl dev set with the submitted Naive Segmentation + Dolly7b pipeline.

## introduction discussions and meetings are an integral part of any human activity that involves a group of people. on important meetings, an audio recording is often made, and specially appointed people create a brief summary of the most important things that happened at the meeting. this process is quite laborious. the ability to produce high-quality documentation of business meetings decisions without allocating additional human resources can improve the productivity of the organizations. this way important points and decisions made will not be lost due to an information overflow. thus, automated minuting of business meetings is becoming an increasingly desirable solution. an automated minuting system can be useful not only for businesses but also for government agencies and educational institutions. hundreds of meetings are held daily, and the ability to automatically generate a summary of the most important decisions made can significantly reduce the time and resources spent on documenting. thanks to an automatic minuting system, meeting participants can focus on important points without spending time on note-taking. the goal of this work is to develop a module for automatic generation of meeting minutes based on a meeting transcript text produced by an automated speech recognition (asr) system (automin 2023 task a, @xcite ). ## related work meeting summarization as a scientific problem came to light in the early 2000s @xcite . isci meeting project @xcite resulted, among other outcomes, in creating icsi meeting corpus of audio recorded from informal, natural, and even sometimes impromptu meetings @xcite . about simultaneously, klaus zechner's work on summarization of meeting speech and dialogues @xcite helped to shape the investigations in this topic further. augmented multi-party interaction (ami) project followed soon, producing the ami meeting corpus @xcite . it has been recognized early on that for a wide spectrum of applications identifying and including action items into minutes delivers the key business value @xcite . detecting decisions in multi-party dialogues happened to be as important for the minuting @xcite . the calo meeting assistant system @xcite that appeared soon after that was an important step, but the overall level of nlp at the time limited (with a few exceptions, for example, @xcite ) proliferation of minuting research and applications until recently. scientific interest to minuting reemerged about 2018, sparked both by an important review by @xcite and overall technology readiness. the interest came into a full swing in 2021 when many works have appeared @xcite @xcite @xcite @xcite @xcite and the first automin competition was held at interspeech @xcite . the works that are the most close to ours are likely those by automin 2021 winners @xcite and @xcite . the authors of the former use a bart model and train it on the samsum dialogue summarization dataset. their pipeline first splits the given transcript into blocks of smaller conversations, eliminates redundancies with a specially-crafted rule-based algorithm, summarizes the conversation blocks, retrieves the block-wise summaries, cleans, structures, and finally integrates the summaries to produce the meeting minutes. the authors of the latter introduce a russian minuting dataset and use an approach similar to @xcite . they also introduce semantic segmentation that improves rouge and bertscore metrics of minutes on the above dataset by 1%-10% compared to naive segmentation. ## datasets two main datasets are considered in the automin 2023 task a: the datasets are compared in ## methods all transformer @xcite language models have a limit on the size of the input context window and do not work well with long texts, such as transcripts of long meetings. thus, to make it possible to apply transformer-based models to the transcript text summarization, we, similarly to the winners of the automin 2021 competition @xcite decompose the task of minuting into two subtasks: in addition to the naive segmentation just fitting the chunk to the model's window size, we explore semantic segmentation in a hope to obtain higher quality reporting. the pipeline for the semantic segmentation is as follows: ## conclusion and future work in this work, we describe our system run for the second automin shared task a on automatic minuting. our proposed system leverages a pretrained large language model dolly to generate readable minutes from multi-party meeting proceedings. in the future, we plan to implement similar pipelines for different languages, including low-resource ones.