Venue: NAACL 2024

# Exploring Key Point Analysis with

Pairwise Generation and Graph Partitioning

Xiao Li\({}^{\diamond}\), Yong Jiang\({}^{\dagger}\)1 , Shen Huang\({}^{\dagger}\), Pengjun Xie\({}^{\dagger}\)

**Gong Cheng\({}^{\diamond}\); Fei Huang\({}^{\dagger}\)**

Footnote 1: Yong Jiang and Gong Cheng are the corresponding authors.

###### Abstract

Key Point Analysis (KPA), the summarization of multiple arguments into a concise collection of key points, continues to be a significant and unresolved issue within the field of argument mining. Existing models adapt a two-stage pipeline of clustering arguments or generating key points for argument clusters. This approach rely on semantic similarity instead of measuring the existence of shared key points among arguments. Additionally, it only models the intra-cluster relationship among arguments, disregarding the inter-cluster relationship between arguments that do not share key points. To address these limitations, we propose a novel approach for KPA with pairwise generation and graph partitioning. Our objective is to train a generative model that can simultaneously provide a score indicating the presence of shared key point between a pair of arguments and generate the shared key point. Subsequently, to map generated redundant keys points to a concise set of key points, we proceed to construct an arguments graph by considering the arguments as vertices, the generated key points as edges, and the scores as edge weights. We then propose a graph partitioning algorithm to partition all arguments sharing the same key points to the same subgraph. Notably, our experimental findings demonstrate that our proposed model surpasses previous models when evaluated on both the ArgKP and QAM datasets.

+
Footnote †: _Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)_, pages 5657–5667 June 16-21, 2024 02024 Association for Computational Linguistics

## 1 Introduction

Analyzing a large number of arguments and making decisions is a long standing issue present in real life scenarios such as product review analysis Barham et al. (2021); Cattan et al. (2023), opinion surveys Bar-Haim et al. (2020), etc. However, manually reading through a vast amount of arguments is not a feasible solution. In this context, key point analysis (KPA, Bar-Haim et al., 2020) has been proposed. KPA involves distilling a multitude of arguments into a concise set of key points, followed by a quantitative evaluation, typically by calculating the frequency of each key point related to arguments. An example is shown in Figure 1.

Existing Methods and Limitations.The generation of key points from arguments represents the most pivotal and challenging aspect of the KPA task. Initially, a straightforward approach Cattan et al. (2023) involves selecting a few short arguments as key points and then determining whether other arguments match these key points. Subsequently, advanced model Li et al. (2023) divides the KPA task into two stages. Firstly, a semantic feature based clustering model is utilized to cluster arguments into multiple clusters. Then, a generation model is employed to generate correspond

Figure 1: An example in the ArgKP dataset. Only a subset of arguments and key points are shown due to space limitation. Our goal is to generate key points (right side) within a specific topic based on the provided arguments (left side). A single argument may correspond to multiple key points.



ing key points for arguments in each cluster by concatenating the arguments. However, these existing methods possess certain limitations. Cattan et al., 2023 posits that the mere presence of appropriate key points within arguments is insufficient. Li et al., 2023's approach is based on clustering arguments using their semantic features and then positing that arguments within a cluster share the same key point. However, this approach does not fully align with the objectives of the KPA task. It is possible for arguments that are not semantically similar to still share a key point, as an argument can correspond to more than one key point. Moreover, Li et al., 2023's approach only models the relationships between arguments that share a key point (i.e., intra-cluster arguments) and neglects the relationships between arguments that do not share a key point (i.e., inter-cluster arguments).

Our Approach.In order to address the deficiencies mentioned above, we present an approach which trains a generative model to simultaneously utilize the relationships between intra-cluster and inter-cluster arguments. This model generates a key point for a pair of inter-cluster arguments and provided a score indicating whether there exists shared key points between the pair of arguments. During the prediction phase, we utilize the score provided by the generative model, which indicates whether a pair of arguments is inter-cluster arguments, rather than the semantic similarity of the arguments, to partition the arguments into multiple clusters. This score is provided by a fine-tuned generative model, which is aligned with the data distribution within the domain, rather than employing a clustering model that is unrelated to the domain. We then construct an argument graph and formulate the relationships among arguments as a graph-based problem. Specifically, we represent arguments as vertices in the graph, the shared key point between two arguments as edges, and the scores as edge weights, resulting in a weighted argument graph. We introduce an iterative graph partitioning algorithm, which generates several subgraphs, each representing a collection of arguments that share the same or similar key point. From these subgraphs, we select a representative key point for further analysis and utilization.

To summarize, our contributions include

* proposing a pairwise generation approach that simultaneously utilizes the information between intra-cluster arguments and inter-cluster arguments and providing a score to indicate the presence of the shared key point between a pair of arguments, and
* introducing a weighted graph partition algorithm designed specifically for the KPA task. Our algorithm evaluates the connections between arguments by the scores provided by the generative model, as semantic similarity may not fully align with the objectives of the KPA task.

Outline.We elaborate our approach in Section 2, present experiments in Section 3, discuss related work in Section 4, and conclude in Section 5.

Code.Our code and data are available on GitHub: [https://github.com/Alibaba-NLP/KeyPoint-Analysis](https://github.com/Alibaba-NLP/KeyPoint-Analysis).

## 2 Approach

On the KPA task, the input consists of a set of arguments under a given topic \(T\), where \({\sf arg}_{i}\) is the argument text and \({\sf stance}_{i}\) is the stance for \({\sf arg}_{i}\). The stance ("pro" or "con") indicate the stance of argument towards the topic. The ideal outcome is a concise set of key points \(\mathbb{K}=\left\{{\sf kp}_{1},{\sf kp}_{2},\cdots\right\}\), where each key point in \(\mathbb{K}\) corresponds to one or more arguments in \(\mathbb{A}\). That is, our objective is to learn a surjective function \(f:\mathbb{A},T\rightarrow\mathbb{K}\). Afterwards, the importance of each key point can be expressed by calculating the percentage of arguments corresponding to each key point.

Figure 2 provides an overview of our proposed method, which comprises two principal components. The initial component is dedicated to the generation of key points, as depicted at the bottom of Figure 2. During this phase, we process the given topic \(T\) and pairs of arguments \(\mathbb{A}\times\mathbb{A}\) to ascertain if each pair shares a key point and, if so, generate the corresponding key point. The subsequent component entails graph partitioning, illustrated at the top of Figure 2. In this stage, we construct a graph of arguments, key points, and their associated scores. We then apply our specially developed graph partitioning algorithm to divide the graph into multiple subgraphs, allowing for the possibility of the same argument being present in more than one subgraph, as a single argument may relate to multiple key points. Finally, we select a representative key point from each subgraph to



form the final concise set of key points \(\mathbb{K}\). Notably, unlike previous pipeline approaches, we transmit the scores generated by the first stage to the second stage, avoiding issues such as feature propagation interruption and error accumulation that may arise in multi-stage pipelines.

### Key Point Generation

Inspired by contrastive learning Khosla et al. (2020); Gao et al. (2021), our approach takes a fine-grained perspective by pairing arguments and determining whether a pair of arguments shares the same key point. We posit that it is equally important to model the relationships between inter-cluster arguments that possess different key points and intra-cluster arguments that share a key point. The previous models either mapped each argument to a single key point Alshomary et al. (2021); Kapadnis et al. (2021) or clustered arguments together and concatenated multiple arguments within a cluster to generate a key point Li et al. (2023).

Construction of Training Data.We pack a pair of arguments into an input text

\[\mathtt{input}_{ij}=\mathtt{topic}\mid\mathtt{stance}_{i}\:\arg_{i}\mid \mathtt{stance}_{j}\:\arg_{j},\]

and prefix the output text with a special word to indicate whether the input pair of arguments are intra-cluster arguments or inter-cluster arguments, which is

\[\mathtt{output}_{ij}=\begin{cases}\texttt{Yes.}&\{\kp_{ij}\},\\ \text{No.},&\text{for intra-cluster arguments},\\ \end{cases}\]

We construct training data by pairing arguments, disregarding the order between a pair of arguments. That is, we consider \(\mathtt{input}_{ij}\) while discarding \(\mathtt{input}_{ji}\) (where \(i<j\)). We determine whether a pair of arguments are intra-cluster arguments or inter-cluster arguments by assessing if the corresponding key points of the arguments are identical. To avoid overfitting, we limit the occurrence of each argument as an intra-cluster argument in the training data to no more than five times.

To facilitate a better understanding of the construction of the training data, we present two examples in Table 1.

Generation Model.Subsequently, we feed \(\mathtt{input}_{ij}\) to the generative model 1,

Footnote 1: In our paper, we exemplify our approach by employing a model based on the encoder-decoder architecture. However, our approach can also be applied to decoder-only models.

\[\mathbf{h}_{1},\mathbf{h}_{2},\cdots,\mathbf{h}_{n}=\mathtt{Decoder}( \mathtt{Encoder}(\mathtt{input}_{ij})),\]

where \(\mathbf{h}_{k}\) denotes the representation obtained from the Decoder during the \(k\)-th decoding step and \(n\) is the decoder output length. During the decoding phase, a linear transformation LM_Head is employed to map the representations outputted by the

Figure 2: Overview of our approach. Each argument possesses a stance representing its polarity (“pro” or “con”). For the sake of brevity, it has been omitted in the figure.

 

Decoder to the probability distribution for the tokens in the vocabulary at step \(k\), denoted as

\[S_{k}=\texttt{LM\_Head}(\texttt{h}_{k}).\]

The \(k\)-th token is decoded by

\[\texttt{token}_{k}=\texttt{vocab}[\texttt{argmax}(S_{k})],\]

where vocab is the vocabulary.

We extract scores for token "Yes" and "No" from \(S_{1}\) (i.e., \(S_{1}[\text{Yes}]\) and \(S_{1}[\text{No}]\), respespectively) and obtain the score indicating whether arg\({}_{i}\) and arg\({}_{j}\) sharing the same key point by

\[score_{ij}=\frac{\texttt{exp}(S_{1}[\text{Yes}])}{\texttt{exp}(S_{1}[\text{ Yes}])+\texttt{exp}(S_{1}[\text{No}])}. \tag{1}\]

The loss during the training process is

\[\mathcal{L}=-\frac{1}{n}\sum_{1\leq k\leq n}\log\frac{\texttt{ exp}(S_{k}[\texttt{output}_{ij}[k]])}{\sum_{t\in\texttt{vocab}}\texttt{exp}(S_{k}[t])},\]

where output\({}_{ij}[k]\) is the \(k\)-th token in output\({}_{ij}\).

By employing this approach, we have integrated the binary classification task of determining the presence of shared key points and the task of generating key points into a single generative model, thereby mitigating potential information loss that may arise from coupling two different models.

### Graph Partitioning

After the generation of key points, it is necessary to aggregate all intra-cluster arguments and determine the shared key point. We transform this issue into a graph partitioning problem, such that each partition yields a subgraph representing a cluster of arguments that share key points. Previous approaches Li et al. (2023) relied on semantic similarity-based clustering methods. However, it is important to recognize that two arguments sharing the same key point may not always exhibit high semantic similarity, since an argument can encompass multiple key points. An alternative direct approach involves clustering the key points directly. However, the semantic gap between the generative models and the clustering models may introduce performance drop in this approach. We have mitigated this issue by transferring \(score_{ij}\) between the two stages. Specifically, in Equation 1, we calculate the score \(score_{ij}\) to quantify the presence of a shared key point between argument \(\texttt{kp}_{i}\) and argument \(\texttt{kp}_{j}\). The score \(score_{ij}\) is then used to guide the partitioning process.

Argument Graph Construction.We employ arguments A as vertices and each edge possesses two attributes, an edge weight

\[score\in\mathbb{S}=\{score_{i,j}|1\leq i,j\leq|\mathbb{A}|,i\neq j\},\]

which indicates the probability of the presence of a shared key point, and the generated shared key point

\[\texttt{kp}\in\hat{\mathbb{K}}_{\text{Gen}}=\{\texttt{kp}_{ij}|1\leq i,j\leq |\mathbb{A}|,i<j\}.\]

We have removed the edges for which the model outputs "No", corresponding to the edges between argument vertices that do not share a key point.

In doing so, we construct a weighted argument

\[G=\langle\mathbb{A},(\mathbb{S},\hat{\mathbb{K}}_{\text{Gen}})\rangle.\]

Partitioning by Local Search.We employ local search algorithm to partition argument graph. We use the BAAI-bge-large Xiao et al. (2023) model to obtain representations of arguments, and then apply k-means clustering Lloyd (1982) to cluster the arguments, which serves as the initial partitioning for our graph. We then obtain the initial subgraphs \(\mathbb{G}_{\text{subgraph}}=\{g_{1},g_{2},\cdots,g_{s}\}\), where \(s\) is a hyperparameter. Subsequently, we iteratively move vertices across the current subgraphs by optimizing a cost function, until no improvement of cost is possible or the limit of iteration steps denoted by \(l\) is reached. We finally obtain subgraphs \(\mathbb{G}^{\prime}_{\text{subgraph}}\) and form the concise set of key points \(\mathbb{K}_{\text{Gen}}\) from each subgraph by choosing the key points with the highest edge weights.

Cost Function.We define the weight wt\((g)\) of a subgraph \(g\) as the average weight of all edges in \(g\). For a randomly selected subgraph and within it, a randomly selected vertex arg, we calculate the change in the graph weight resulting from the movement of vertex arg from the subgraph \(g_{\text{out}}\) to another subgraph \(g\) by our cost function

\[\texttt{cost}( g,g_{\text{out}},\texttt{arg})=\] \[\texttt{wt}(g_{\text{out}}\setminus\{\texttt{arg}\})-\texttt{wt }(g_{\text{out}})+\] \[\texttt{wt}(g\cup\{\texttt{arg}\})-\texttt{wt}(g).\]

We then search for a target subgraph \(g_{\text{in}}\) by

\[g_{\text{in}}=\underset{g\in\mathbb{G}_{\text{subgraph}}\backslash\{g_{\text{ out}}\}}{\texttt{arg}\max}\texttt{cost}(g,g_{\text{out}},\texttt{arg}),\]

which means that moving the selected vertex arg from \(g_{\text{out}}\) to \(g_{\text{in}}\) would result in the maximum change in graph weight and the selected vertex will only be moved when the weight change is positive.

 

Soft Partition.Furthermore, since an argument may encompass multiple key points, it is permissible to allow an argument to belong to multiple subgraphs simultaneously, which we refer to as soft partition. We set a threshold \(h\), such that if the removal of the selected argument from its original subgraph would cause the weight of the subgraph to decrease beyond the threshold \(h\), we will retain the argument within the original subgraph. This means that the selected argument can contribute a weight gain greater than \(h\) in the original subgraph.

It is worth noting that we have not imposed any restrictions on the connectivity of subgraphs in our algorithm. This is because, given a specified number of subgraphs, we cannot guarantee that all the resulting subgraphs will be connected (as the number of connected components in \(G\) may be less than the number of subgraphs). However, it is important to note that our optimization objective (i.e., graph weight) tends to favor connected subgraphs or subgraphs with higher connectivity.

## 3 Experiments

### Datasets

We conducted experiments on ArgKP Friedman et al. (2021) and QAM Guo et al. (2023). ArgKP is a KPA dataset annotated by crowd sourcing, with training/validation/test sets consisting of 24/4/3 topics, 5583/932/724 arguments, and corresponding 207/36/33 key points. Each argument and key point possess a stance attribute, either "pro" (i.e., positive) or "con" (i.e., negative), indicating whether they support the current topic. The QAM dataset is originally designed for extracting quadruples \(\langle\text{key point},\text{stance},\text{argument},\text{evidence type}\rangle\) from documents with a given topic. QAM is annotated through crowdsourcing, and we have transformed the QAM dataset into the format required for the KPA task. The training/validation/test sets consists 96/52/53 topics, 6635/808/948 arguments, and corresponding 2366/314/337 key points.

### Evaluation Metrics

We employed Rouge Lin (2004) and soft-Precision/Recall/F1 as metrics. Following Li et al. (2023), we obtained Rouge scores by comparing the concatenation of generated key points and the concatenation of reference key points. We reported the scores of Rouge-1 and Rouge-2. For soft-Precision/Recall/F1 (denoted by sP, sR and sF1, respectively), we employed BLEURT Sellam et al. (2020) to measure the similarity between individual generated key point and the reference key point. The sP, sR and sF1 to compare the similarity between the sets of key points were given by

\[\textbf{sP}=\frac{1}{|\mathbb{K}_{\text{Gen}}|}\sum_{\mathsf{kp} _{\text{Gen}}\in\mathbb{K}_{\text{Go}}}\max_{\mathsf{kp}\in\mathbb{K}}\texttt{BLEURT }(\mathsf{kp},\mathsf{kp}_{\text{Com}}),\] \[\textbf{sR}=\frac{1}{|\mathbb{K}|}\sum_{\mathsf{kp}\in\mathbb{K} }\max_{\mathsf{kp}_{\text{Gen}}\in\mathbb{K}_{\text{Co}}}\texttt{BLEURT}( \mathsf{kp},\mathsf{kp}_{\text{Gen}})~{}\text{and}\] \[\textbf{sF1}=\frac{2\times\textbf{sp}\times\textbf{sR}}{\textbf{ sP}+\textbf{sR}}.\]

### Implement Details

We conducted experiments on NVIDIA A100 (80GB) GPUs. Our implementations were based on PyTorch 2.0.1 and HuggingFace Transformers 4.30.2. We set \(\textit{learning rate}=1e-5\) selected from \(\{1e-5,3e-5,6e-5\}\), \(\textit{batch size}=64\) selected from \(\{32,64,128,256\}\), \(\textit{random seed}=42\), \(\textit{maximum sequence length}=512\), \(\textit{epoch}=5\). We set \(h=0.008\) for ArgKP, \(h=0.006\) for QAM and \(l=200\) for ArgKP, \(l=80\) for QAM. In order to ensure a fair comparison with previous work Li et al. (2023), we set the number of selected key points, which is the number of subgraphs \(s\), to be equal to the number of reference key point set.

### Previous State-of-the-Art Approaches

We compared with three state-of-the-art models: graph based summarization GBS, Alshomary et al. (2021), Enigma Kapadnis et al. (2021) and supervised key point modelling SKPM, Li et al. (2023). The GBS model was a representative of the extractive key point generation models. It employed arguments to establish a graph, where each argument vertex was examined to determine their suitability as a key point. The PageRank Page et al. (1999) algorithm was then utilized to choose the key points. Enigma generated a key point for each argument based on PEGASUS Zhang et al. (2020) and selected the highest-scoring key point using the precision of Rouge-1 compared with reference key point set. SKPM was a two-stage model that first clusterd arguments and then generated key points for arguments within each cluster.

### Comparison with Previous State-of-the-Art Approaches

In comparison with the existing methods as shown in Table 2, our approach demonstrated advantages over the previous SOTA approaches. Specifically, on the ArgKP dataset, we observed improvements 

of 19.51-31.31 points in Rouge-1 and 1.34-9.34 points in sF1. On the QAM dataset, our approach yielded an improvement of 1.60 points in sF1. The results demonstrated the effectiveness of our method, which were analyzed in detail in the subsequent sections.

### Effectiveness of Key Point Generation

We conducted several experiments to evaluate the effectiveness of our generative model in identifying and generating the shared key point between a pair of arguments. These experiments were structured around two research sub-questions. **SQ1** (Shared Key Point Detection) focused on the ability of our model to determine the presence of a shared key point, which was essentially a binary classification task. **SQ2** (Key Point Generation) assessed the model's capability to generate high-quality key points that effectively covered the reference key point set. In tackling these research sub-questions, we prioritized the recall metric because our subsequent step involves selecting a concise subset from an extensive set of key points, as explained in Section 3.7).

#### 3.6.1 SQ1: Shared Key Point Detection

We applied a representative model RoBERTa-large Liu et al. (2019) to classify whether two arguments shared a key point and compared its performance with that of our generative model. In our generative model, if a key point was generated for a pair of arguments, the output was classified as positive (i.e., the output began with "Yes"). Conversely, if no key point was generated, the output was classified as negative (i.e., the output began with "No"). For the RoBERTa-large model, we utilized the CLS vector, followed by a linear layer that produced two scores, one for positive and one for negative, to serve as the classifier.

We evaluated the recall and accuracy of two models in a binary classification task aimed at identifying whether a pair of arguments shared a key point on ArgKP. As indicated in Figure 3, our approach outperformed by 10.56 points in recall, the metric of greater significance in our scenario. Generating key points may also help in determining whether there are shared key points among arguments. With pairwise generation, despite the Flan-T5-large being an encoder-decoder model primarily intended for generative tasks, it still delivered satisfactory performance in the classification task.

#### 3.6.2 SQ2: Key Point Generation

In this section, we designed an experiment to verify that the relationship between a pair of intra-cluster arguments with shared key points and inter-cluster arguments without shared key points is beneficial for key point generation. We removed the pairs of arguments that did not share key points from our training data as ablation experiment, which is the result of the "Ours w/o inter-cluster argument pairs" row in Figure 4. To verify the effectiveness of pair

\begin{table}
\begin{tabular}{l c c c c c c c c c c} \hline \hline  & \multicolumn{8}{c}{ArgKP} & \multicolumn{8}{c}{QAM} \\ \cline{2-11}

- \\ EnigmaPSIGASUS & 20.00 & 4.80 & 58.00 & 57.00 & 57.00 & **32.61** & 7.17 & 46.96 & 44.43 & 45.51 \\ SKPMFlan-TS-base & 30.30 & 8.90 & 59.00 & 58.00 & 59.00 &
- \\ SKPMFlan-TS-large & 31.40 & 9.10 & 57.00 & **62.00** & 60.00 &
- \\ \hline \multicolumn{11}{l}{**Ours**} \\ w/ Flan-T5-base & 37.38 & 11.39 & 61.29 & 58.08 & 59.59 & 30.31 & 7.44 & 48.62 & 43.81 & 45.96 \\ w/ Flan-T5-large & **50.91** & **21.23** & **62.57** & 60.26 & **61.34** & 30.44 & **8.02** & **49.38** & **45.27** & **47.11** \\ \hline \hline \end{tabular}
\end{table}
Table 2: Comparison with previous state-of-the-art approaches. The best results are in bold and the second best results are underlined. The dash signifies that we did not obtain the results due to issues such as code availability. The results of previous state-of-the-art approaches on ArgKP are taken from Li et al. (2023).

Figure 3: Shared key point detection on ArgKP.

 

wise generation, we trained a Flan-T5-large model with the concatenation of the topic, stance and a single argument (i.e., \(\texttt{input}=\texttt{topic}~{}|~{}\texttt{stance}~{}|~{}\texttt{arg}\)) as the input, and an output of concatenated key points implied by the single argument. We denoted this approach as "One2KP". Here, we also focused on the recall metric (i.e., sR), which reflected the coverage of the reference key point set during the key point generation stage.

From Figure 4, it can be seen that our method had an advantage over the One2KP method, leading by 9.21 points on the sR metric. After separating inter-cluster argument pairs from the training set, a decrease of 5.84 point on the sR metric was observed. Experiments in Figure 4 validated that generating key points from pairwise arguments was effective. It was not only necessary to consider the relationships among intra-cluster arguments, but also inter-cluster arguments.

Additionally, addressing two related tasks simultaneously within the same model did not introduce a drop on the model performance. In our task, as seen in Figure 3 and Figure 4, the carefully designed tasks and the use of pairwise generations method brought about a gain in model performance. This allowed our model to not only exhibit superior performance in key point generation but also achieve satisfactory discriminative ability in intra-cluster arguments and inter-cluster arguments classification.

### Effectiveness of Graph Partitioning

In this section, we aimed to validate the effectiveness of our graph partitioning method. We compared our graph partitioning method with existing clustering methods and graph partitioning methods in Section 3.7.1. Additionally, we investigated the influence of the number of iterations of our graph partitioning method on the performance in Section 3.7.2.

#### 3.7.1 Comparison with Clustering and Graph Partitioning Methods

We substituted our graph partitioning algorithm with either clustering methods or graph partitioning methods. Our key point generation model, exemplified by the Flan-T5-large model, produced key points for these methods and provided scores to determine if two arguments shared a key point for graph partitioning methods. The clustering algorithms we compared were of two types, one clustered arguments to obtain argument subgraphs and then selected key points with highest edge weights, and the other directly clustered key points and then selected the key point at the center of the cluster. We utilized well-established clustering algorithms, namely k-means Lloyd (1982) , for our analysis. Two models were utilized to derive representations for arguments: 1) the renowned RoBERTa-large Liu et al. (2019) and 2) BAAI-bge-large Xiao et al. (2023), which is the state-of-the-art model in sentence representation. Furthermore, we employed three well-known graph partitioning methods, including

* Greedy Modularity Maximization Clauset et al. (2004), which is a graph partitioning algorithm that utilizes a greedy strategy to maximize modularity and identify community structures,
* Louvain Community Detection Blondel et al. (2008), which is a heuristic method based on modularity optimization by iteratively merging similar vertices to uncover community structures, and
* Asynchronous Label Propagation Raghavan et al. (2007), which relies on label propagation iterations to divide the graph into communities by updating vertex labels.

From Table 3, our model exhibited superior performance compared to the three sets of baseline models on Rouge-2, sR and sF1 metrics. Specifically, on the ArgKP dataset, our model exhibited a lead of 0.60-6.27 points in sF1. On the QAM dataset, our model demonstrated a lead of 0.58-11.88 points in sF1.

Our approach amalgamated the strengths of both by identifying shared key points through our generative model, taking into account semantic information, while simultaneously utilizing graph structure information via the graph partitioning method, resulting in improved outcomes.

Figure 4: Comparison of different models on key point generation on ArgKP. All the models were based on Flan-T5-large.

 



#### 3.7.2 Analysis of Iterations Steps (\(l\))

We conducted an analysis on the impact of the number of iterations steps in the graph partition algorithm. When the number of steps equaled 0, it represented a ablation experiment, indicating the exclusion of any execution of the graph partition algorithm. As shown in Table 4, it could be seen that, excluding the subtle fluctuations in the data, the overall trend of sF1 was first an increase and then a stabilization. Additionally, it could be seen that as the algorithm iterations progress, sR did not change much overall, and our graph partitioning algorithm improved the overall performance by improving sP.

## 4 Related Work

### Key Point Analysis

Unlike multi-document summarization Mani and Bloedorn (1997); Xiao et al. (2022), KPA Bar-Haim et al. (2020a) requires the production of a concise collection of key points rather than a cohesive paragraph. Furthermore, KPA entails a quantitative analysis of these key points, involving quantifying their prevalence within the arguments.

Existing methods for key points generation in KPA rely on extractive summarization and abstractive summarization techniques. Extractive summarization methods Bar-Haim et al. (2020b, 2021); Alshomary et al. (2021) involve selecting short and representative arguments as key points. However, extractive summarization approaches are limited by the quality of the arguments and do not ensure the presence of an optimal key point within the arguments. On the other hand, abstractive summarization methods utilize generative models to produce key points for the arguments. Kapadnis et al. (2021) generates key points for each argument individually and then select a collection of key points based on the Rouge Lin (2004) scores. Li et al. (2023) first employs clustering methods to group semantically similar arguments into the same cluster, and then concatenates the arguments within the cluster to generate key points.

In contrast, our approach simultaneously takes into account the relationships between arguments with shared key points and those without shared key points. Additionally, by replacing semantic similarity scores with scores for the presence of shared key points between arguments, we address the shortcomings of existing models.

### Graph Partitioning

Unlike clustering methods which depends on the similarity or distance of clustering objects, graph partitioning aims to optimize a certain metric of the resulting subgraphs Buluc et al. (2013), which is the probability that the arguments within a common subgraph share a common key point in the KPA scenario. The algorithms Kernighan and Lin (1970); Krauthgamer et al. (2009); Margo and Seltzer (2015) based on optimizing the minimum vertex/edge cuts has been extensively utilized. Additionally, algorithms Clauset et al. (2004); Blondel et al. (2008); Dugue and Perez (2015); Traag et al. (2018) have

\begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**\#steps** & Rouge-1 & Rouge-2 & **sP** & **sR** & **sF1** \\ \hline
0 & 49.01 & 18.64 & 60.94 & 60.08 & 60.47 \\
50 & 51.46 & 20.55 & 61.38 & 60.63 & 60.96 \\
100 & 47.53 & 19.77 & 60.92 & 59.25 & 60.03 \\
150 & 52.13 & 22.39 & 61.22 & 60.52 & 60.85 \\
200 & 50.91 & 21.23 & 62.57 & 60.26 & 61.34 \\
250 & 49.23 & 19.45 & 62.32 & 59.93 & 61.09 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Analysis of iterations steps (\(l\)).

\begin{table}
\begin{tabular}{l l r r r r r r r r r} \hline \hline  & \multicolumn{6}{c}{ArgKP} & \multicolumn{6}{c}{QAM} \\ \cline{2-11} Model & Rouge-1 & Rouge-2 & **sP** & **sR** & **sFI** & Rouge-1 & Rouge-2 & **sP** & **sR** & **sF1** \\ \hline \multicolumn{11}{l}{**Clustering Methods (on arguments)**} \\ k-means\$_toBERT-large & 50.70 & 21.01 & 59.38 & 59.49 & 59.34 & 29.30 & 7.26 & 48.30 & 44.67 & 46.28 \\ k-means\$_to-class\$_al\$_top-large & 49.01 & 18.64 & 60.94 & 60.08 & 60.47 & 30.13 & 7.85 & 47.75 & 44.99 & 46.21 \\ \hline \multicolumn{11}{l}{**Clustering Methods (on key points)**} \\ k-means\$_to-bert-large & **52.71** & 20.16 & 61.30 & 60.21 & 60.74 & 28.28 & 6.77 & 48.35 & 43.48 & 45.67 \\ k-means\$_to-target & 51.75 & 19.57 & 61.42 & 58.80 & 59.99 & 29.40 & 7.50 & 48.79 & 44.70 & 46.53 \\ \hline \multicolumn{11}{l}{**Graph Partitioning Methods**} \\ Greedy Modularity Maximization & 40.85 & 14.75 & 56.15 & 54.04 & 55.07 & 18.59 & 4.79 & 39.34 & 33.85 & 36.19 \\ Louvain Community Detection & 43.01 & 18.03 & 56.81 & 56.65 & 56.69 & 17.96 & 4.68 & 38.28 & 32.97 & 35.23 \\ Asynchronous Label Propagation & 17.32 & 5.54 & **63.76** & 53.31 & 58.01 & 14.71 & 4.41 & **54.40** & 40.62 & 46.04 \\ \hline
**Ours** & 50.91 & **21.23** & 62.57 & **60.26** & **61.34** & **30.44** & **8.02** & 49.38 & **45.27** & **47.11** \\ \hline \hline \end{tabular}
\end{table}
Table 3: Comparison with clustering and graph partitioning methods with Flan-T5-large. The best results are in bold and the second best results are underlined.



approached graph partitioning by optimizing the modularity of subgraphs. Another avenue [14, 26] involves achieving graph partitioning through vertex labels propagation.

In the KPA task, our objective is to maximize the probability that arguments within a subgraph share the same key point while allowing arguments that contain more than one key point to exist in multiple subgraphs simultaneously. Given this requirement, the aforementioned graph partitioning algorithms are not suitable for the KPA task.

## 5 Conclusion

This paper presents a method that merges pairwise generation with graph partitioning to efficiently produce key points within arguments. Our integrated approach models both the relationships between arguments sharing key points and those that do not, aiding in key point generation. We employ an iterative algorithm to partition the argument graph and extract representative key points from subgraphs, yielding a concise set of key points. In the future, we will continue to explore better ways to utilize generative models to model the relationships among arguments. For instance, we will investigate improved methods of outputting scores indicating whether arguments share key points and develop graph partition algorithms that better align with generative models. Our approach may provide inspiration for tasks including clustering tasks based on generative models [30, 7], set generation tasks [8], and so forth.

## Limitations

The input consists of pairwise arguments, which increases the amount of training data and the computational cost of training. Currently, training time of Flan-T5-large (780M) on ArgKP is approximately 12 hours, and on QAM it is about 6 hours. Our model does not fully utilize the graph structure during the graph partitioning process, and we plan to incorporate more graph structural information into the model in the future. The datasets currently used for experiments are related to survey review topics, and there is a lack of datasets from other fields (such as product reviews in the e-commerce scenario) in the research community. Future research could explore the effectiveness on datasets from different domains.

## References

* Alshomary et al. (2021) Milad Alshomary, Timon Gurcke, Shahbaz Syed, Philipp Heinisch, Maximilian Spliethover, Philipp Cimiano, Martin Potthast, and Henning Wachsmuth. 2021. Key point analysis via contrastive learning and extractive argument summarization. In _Proceedings of the 8th Workshop on Argument Mining_, pages 184-189, Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Bar-Haim et al. (2020a) Roy Bar-Haim, Lilach Eden, Roni Friedman, Yoav Kantor, Dan Lahav, and Noam Slonim. 2020a. From arguments to key points: Towards automatic argument summarization. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 4029-4039, Online. Association for Computational Linguistics.
* Bar-Haim et al. (2021) Roy Bar-Haim, Lilach Eden, Yoav Kantor, Roni Friedman, and Noam Slonim. 2021. Every bite is an experience: Key Point Analysis of business reviews. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 3376-3386, Online. Association for Computational Linguistics.
* Bar-Haim et al. (2020b) Roy Bar-Haim, Yoav Kantor, Lilach Eden, Roni Friedman, Dan Lahav, and Noam Slonim. 2020b. Quantitative argument summarization and beyond: Cross-domain key point analysis. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 39-49, Online. Association for Computational Linguistics.
* Blondel et al. (2008) Vincent D. Blondel, Jean-Loup Guillaume, Renaud Lambiotte, and Etienne Lefebvre. 2008. Fast unfolding of communities in large networks. _Journal of Statistical Mechanics: Theory and Experiment_, 2008:P10008.
* Bulucybernek et al. (2013) Aydin Bulucy, Henning Meyerhenke, Ilya Safro, Peter Sanders, and Christian Schulz. 2013. Recent advances in graph partitioning. In _Algorithm Engineering_.
* Cattan et al. (2023) Arie Catttan, Lilach Eden, Yov Kantor, and Roy Barium. 2023. From key points to key point hierarchy: Structured and expressive opinion summarization. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 912-928, Toronto, Canada. Association for Computational Linguistics.
* Clauset et al. (2004) Aaron Clauset, Mark E. J. Newman, and Cristopher Moore. 2004. Finding community structure in very large networks. _Physical review. E, Statistical, nonlinear, and soft matter physics_, 70 6 Pt 2:066111.
* Cordasco and Gargano (2010) Gennaro Cordasco and Luisa Gargano. 2010. Community detection via semi-synchronous label propagation algorithms. In _2010 IEEE International Workshop on: Business Applications of Social Network Analysis (BASNA)_, pages 1-8 

* Dugue and Perez (2015) Nicolas Dugue and Anthony Perez. 2015. Directed louvain : maximizing modularity in directed networks.
* Friedman et al. (2021) Roni Friedman, Lena Dankin, Yufang Hou, Ranit Aharonov, Yoav Katz, and Noam Slonim. 2021. Overview of the 2021 key point analysis shared task. In _Proceedings of the 8th Workshop on Argument Mining_, pages 154-164, Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Gao et al. (2021) Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. SimCSE: Simple contrastive learning of sentence embeddings. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_, pages 6894-6910, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Guo et al. (2023) Jia Guo, Liying Cheng, Wenxuan Zhang, Stanley Kok, Xin Li, and Lidong Bing. 2023. AQE: Argument quadruplet extraction via a quad-tagging augmented generative approach. In _Findings of the Association for Computational Linguistics: ACL 2023_, pages 932-946, Toronto, Canada. Association for Computational Linguistics.
* Kapadnis et al. (2021) Manav Kapadnis, Sohan Patnaik, Sibu Panigrahi, Varun Madhavan, and Abhilash Nandy. 2021. Team enigma at ArgMining-EMNLP 2021: Leveraging pre-trained language models for key point matching. In _Proceedings of the 8th Workshop on Argument Mining_, pages 200-205, Punta Cana, Dominican Republic. Association for Computational Linguistics.
* Kernighan and Lin (1970) B. W. Kernighan and S. Lin. 1970. An efficient heuristic procedure for partitioning graphs. _The Bell System Technical Journal_, 49(2):291-307.
* Khosla et al. (2020) Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020. Supervised contrastive learning. In _Advances in Neural Information Processing Systems_, volume 33, pages 18661-18673. Curran Associates, Inc.
* Krauthgamer et al. (2009) Robert Krauthgamer, Joseph (Seffi) Naor, and Roy Schwartz. 2009. Partitioning graphs into balanced components. In _Proceedings of the Twentieth Annual ACM-SIAM Symposium on Discrete Algorithms_, SODA '09, page 942-949, USA. Society for Industrial and Applied Mathematics.
* Li et al. (2023) Hao Li, Viktor Schlegel, Riza Batista-Navarro, and Goran Nenadic. 2023. Do you hear the people sing? key point analysis via iterative clustering and abstractive summarisation. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 14064-14080, Toronto, Canada. Association for Computational Linguistics.
* Lin (2004) Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In _Text Summarization Branches Out_, pages 74-81, Barcelona, Spain. Association for Computational Linguistics.
* Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Daniq Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Robertra: A robustly optimized bert pretraining approach. _ArXiv_, abs/1907.11692.
* Lloyd (1982) Stuart P. Lloyd. 1982. Least squares quantization in pcm. _IEEE Trans. Inf. Theory_, 28:129-136.
* Madaan et al. (2022) Aman Madaan, Dheeraj Rajagopal, Niket Tandon, Yiming Yang, and Antoine Bosselut. 2022. Conditional set generation using seq2seq models. In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_, pages 4874-4896, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
* Mani and Bloedorn (1997) Indrejeuet Mani and Eric Bloedorn. 1997. Multi-document summarization by graph search and matching. In _AAAI/IAAI_.
* Margo and Seltzer (2015) Daniel Margo and Margo Seltzer. 2015. A scalable distributed graph partitioner. _Proc. VLDB Endow._, 8(12):1478-1489.
* Page et al. (1999) Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. 1999. The pagerank citation ranking : Bringing order to the web. In _The Web Conference_.
* Raghavan et al. (2007) Usha Nandini Raghavan, Reka Albert, and Soundar Kumara. 2007. Near linear time algorithm to detect community structures in large-scale networks. _Phys. Rev. E_, 76:036106.
* Sellam et al. (2020) Thibault Sellam, Dipanjan Das, and Ankur Parikh. 2020. Bleurt: Learning robust metrics for text generation. In _Proceedings of ACL_.
* Traag et al. (2018) Vincent Antonio Traag, Ludo Waltman, and Nees Jan van Eck. 2018. From louvan to leiden: guaranteeing well-connected communities. _Scientific Reports_, 9.
* Viswanathan et al. (2023) Vijay Viswanathan, Kiril Gashevski, Carolin (Haas) Lawrence, Tongshuang Sherry Wu, and Graham Neubig. 2023. Large language models enable few-shot clustering. _ArXiv_, abs/2307.00524.
* Xiao et al. (2023) Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. 2023. C-pack: Packaged resources to advance general chinese embedding. _ArXiv_, abs/2309.07597.
* Xiao et al. (2022) Wen Xiao, Iz Beltagy, Giuseppe Carenini, and Arman Cohan. 2022. PRIMERA: Pyramid-based masked sentence pre-training for multi-document summarization. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 5245-5263, Dublin, Ireland. Association for Computational Linguistics.
* Zhang et al. (2020) Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter J. Liu. 2020. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization. In _Proceedings of the 37th International Conference on Machine Learning_, ICML'20. JMLR.org 

Yuwei Zhang, Zihan Wang, and Jingbo Shang. 2023. Clusterllm: Large language models as a guide for text clustering. _ArXiv_, abs/2305.14871.