Matching The Statements: A Simple and Accurate Model for Key Point Analysis (2021)

Abstract: Key Point Analysis (KPA) is one of the most essential tasks in building an Opinion Summarization system, which is capable of generating key points for a collection of arguments toward a particular topic. Furthermore, KPA allows quantifying the coverage of each summary by counting its matched arguments. With the aim of creating high-quality summaries, it is necessary to have an in-depth understanding of each individual argument as well as its universal semantic in a specified context. In this paper, we introduce a promising model, named Matching the Statements (MTS) that incorporates the discussed topic information into arguments/key points comprehension to fully understand their meanings, thus accurately performing ranking and retrieving best-match key points for an input argument. Our approach has achieved the 4th place in Track 1 of the Quantitative Summarization – Key Point Analysis Shared Task by IBM, yielding a competitive performance of 0.8956 (3rd) and 0.9632 (7th) strict and relaxed mean Average Precision, respectively.

## introduction prior work in opinion summarization often followed the extractive strategy, which identifies the most representative pieces of information from the source text and copies them verbatim to serve as summaries @xcite . abstractive summarization is a less popular strategy compared to the previous one yet offers more coherent output texts. approaches governed by this vein could generate new phrases, sentences or even paragraphs that may not appear in the input documents @xcite . both extractive and abstractive methods are the straightforward applications of multi-document summarization @xcite , which has been an emerging research domain of natural language processing in recent years. as is well known, in traditional multi-document summarization methods, the role of an individual or a subset of key points among the summaries is often neglected. to be more specific, @xcite posed a question regarding the summarized ability of a small group of key points, and to some extent answered that question on their own by developing baseline models that could produce a concise bullet-like summary for the crowd-contributed arguments. with a pre-defined list of summaries, this task is known as key point matching (kpm). figure 1 provides a simple illustration of the kpm problem, where the most relevant key points are retrieved for each given argument within a certain topic (i.e. context). inspired by the previous work that studied the problem of learning sentence representation @xcite and semantic similarity @xcite , we propose matching the statements (mts), which further takes the topic information into account and effectively utilizes such proper features to learn a high performance unified model. our approach has benefited from the recent developments of pre-trained language models such as bert @xcite , albert @xcite or roberta @xcite . our contributions in this paper could be depicted as follows: key points: -we should let everyone retire when they are ready -a mandatory retirement age decreases institutional knowledge -a mandatory retirement age harms the economy -a mandatory retirement is not fair/discriminatory a mandatory retirement age decreases institutional knowledge a mandatory retirement age harms the economy a mandatory retirement is not fair/discriminatory we should let everyone retire when they are ready • secondly, our method adopts the pseudolabels mechanism @xcite , where we label arguments that belong to the same key point (and the key point itself) by the same index. the goal is to learn an embedding space in which the embedded vectors of mutual supportive statement pairs (i.e. having the same label) are pulled closer whereas unrelated ones are pushed apart. the rest of this paper is organized in the following way: section 2 briefly reviews the related work, while section 3 formulates the kpm problem. next, we describe our methodology in section 4, followed by the experimental results in section 5. finally, section 6 will conclude our work and discuss future directions for further improvements. ## related work a standard approach for key points and arguments analysis is properly extracting their meaningful semantics. our model stems from recent literatures that are based on siamese neural networks @xcite to measure the semantic similarity between documents. even though, mts has its own unique characteristics to incorporate context information. ## problem definition in this shared task, we are provided with a dataset consisting of 28 different topics. each topic con-tains a set of associated arguments and key points in the form of matching (with label 1) or nonmatching (with label 0) pairs. the stances of these statements (whether a claim agrees or disagrees with its topic) are also exposed, we further evaluate the impact of this information in section 5.6.2. in short, the key point matching problem is formulated as follows: given a controversial topic t with a list of m arguments and n key points a 1 , a 2 , . . . , a m ; k 1 , k 2 , . . . , k n , along with their corresponding stances s 1 , s 2 , . . . , s m+n (s @xmath0, 1}), which imply the attack or support relationships against the topic, our task is to rank key points that have the same stance with an input argument by the matching score. this priority is dependent on both the topic and the semantic of statements. ## methodology the proposed mts architecture is graphically shown in figure 2. it takes four separate inputs: (i) discussed topic, (ii) first statement, (iii) second statement, and (iv) their stance toward the topic. the final output is the similarity score of the fed in statements with respect to the main context. in the remainder of this section, we would like to describe three main components of mts: encoding, context integration and statement encoding layers. ## experiment to verify the effectiveness of the matching the statements model, we conduct extensive experiments on the argkp-2021 @xcite dataset and compare the performance of mts against baselines. @xcite , the data set used in the quantitative summarization -key point analysis shared task, is split into training and development sets with the ratio of 24 : 4. the training set is composed of 5583 arguments and 207 key points while those figures in the development set are 932 and 36. each argument could be matched to one or more key points, yet the latter ones account for a small proportion of the data, as stated in section 4.1. the texts presented in argkp-2021 are relatively short, with approximately 18.22 ± 7.76 by words or 108.20 ± 43.51 by characters. ## conclusion in this paper, we present an efficient key point matching method based on supervised contrastive learning. we suppose that clustering the statements will be beneficial for the model training, and empirically verify this conclusion in the experiments. in addition, we found a simple and effective technique to encode these statements, and thus yields superior performance. in terms of model architecture, the components are carefully designed to ensure productivity. results on track 1 of quantitative summarization -key point analysis show our method is a conceptually simple approach yet achieves promising performance.