Align then Summarize: Automatic Alignment Methods for Summarization Corpus Creation (2020)

Abstract: Summarizing texts is not a straightforward task. Before even considering text summarization, one should determine what kind of summary is expected. How much should the information be compressed? Is it relevant to reformulate or should the summary stick to the original phrasing? State-of-the-art on automatic text summarization mostly revolves around news articles. We suggest that considering a wider variety of tasks would lead to an improvement in the field, in terms of generalization and robustness. We explore meeting summarization: generating reports from automatic transcriptions. Our work consists in segmenting and aligning transcriptions with respect to reports, to get a suitable dataset for neural summarization. Using a bootstrapping approach, we provide pre-alignments that are corrected by human annotators, making a validation set against which we evaluate automatic models. This consistently reduces annotators’ efforts by providing iteratively better pre-alignment and maximizes the corpus size by using annotations from our automatic alignment models. Evaluation is conducted on publicmeetings, a novel corpus of aligned public meetings. We report automatic alignment and summarization performances on this corpus and show that automatic alignment is relevant for data annotation since it leads to large improvement of almost +4 on all ROUGE scores on the summarization task.

## rst iteration: diagonal alignment the first iteration started without any reference file, therefore, we had no way of quantitatively evaluating the auto-alignment process. still, in order to provide a pre-alignment to human annotator, we used a naive approach that aligns segments diagonally: we do not compute similarity ( s i , @xmath0, ∀(i , j )) and move into the alignment matrix to stay on a diagonal i.e. we replace the position history matrix h of eq. 5 to be: and r i , @xmath1: exploring scoring functions during the second iteration we mainly explored different sentence representations and scoring functions. using plain text, we measure rouge scores @xcite , more precisely r1-f, r2-f, and rl-f. we use vector representations of text based on (i) t f • i d f and latent semantic analysis; and measuring similarities between sliding windows instead of sentences directly was meant to reduce impact of isolated sentences of low similarities. in fact, because our data don't perfectly match, there may be sentences with a very low similarity inside segments that actually discuss the same point. parameters related to the sliding windows are the window size, and the overlap. we experimented with all combinations of @xmath2, 1, 2, 3, 4, 5, 10} , @xmath3, 2, 3, 5}. related to its scores we consider aggregation and reduction function as parameters and experiment with ag @xmath4, mean, max} and r @xmath5; pr od uc t . ## rd iteration: fine tuning embedding based models during the alignment phase, we found that the dynamic programming algorithm may keep the same direction for a long time. for example one report sentence may get high similarities with a lot of transcription sentences, resulting in a too monotonical alignment. to limit this behavior, we introduce horizontal and vertical decay factors (respectively hd and vd ), typically in [0; 1], that lower scores in the same direction. we then consider a decayed alignment matrix a such as: the decay is reset to d i , @xmath6.