Discussion Structure Prediction Based on a Two-step Method (2021)

Abstract: Conversations are often held in laboratories and companies. A summary is vital to grasp the content of a discussion for people who did not attend the discussion. If the summary is illustrated as an argument structure, it is helpful to grasp the discussion’s essentials immediately. Our purpose in this paper is to predict a link structure between nodes that consist of utterances in a conversation: classification of each node pair into “linked” or “not-linked.” One approach to predict the structure is to utilize machine learning models. However, the result tends to over-generate links of nodes. To solve this problem, we introduce a two-step method to the structure prediction task. We utilize a machine learning-based approach as the first step: a link prediction task. Then, we apply a score-based approach as the second step: a link selection task. Our two-step methods dramatically improved the accuracy as compared with one-step methods based on SVM and BERT.

## introduction meetings are often held in laboratories and companies to come up with new research ideas and management strategies. a summary is vital to grasp the content of a discussion for people who did not attend the discussion. summaries are suitable for understanding the main points in discussions. assume that a summary is illustrated as a discussion structure. the summary is more powerful and helpful to understand the main points in the discussion because users can immediately capture the flow of the discussion by using links between utterances. for this purpose, we need to predict the discussion structure of each discussion. argument mining is one of the tasks to construct a structure of sentences @xcite . it automatically derives the structure of argumentation from unstructured documents such as essays. it consists of four subtasks as follows: component identification, component classification, relation identification, and relation classification. component identification is a task that extracts argument components from a given document. argument components denote sentences and paragraphs related to the discussion structure. component classification is a task that assigns a label, e.g., claim, to each argument component. relation identification is a task that predicts whether each pair of argument components is related or not. relation classification is a task that assigns a label, such as "attack" and "support," to the related pairs of argument components. in this paper, we deal with relation identification for constructing a discussion structure in a multi-party conversation. in other words, we construct a link prediction model for nodes consisting of some utterances. methods in previous work often predicted the discussion structures by using machine learning approaches, such as neural networks. @xcite have reported that such machine learning models tended to overgenerate links between nodes. here we focus on some rules in discussion structures; e.g., a child node has one parent note. we incorporate scorebased selection rules with the machine learning model as post-processing to improve the accuracy. we introduce top-down and bottom-up approaches for selecting edges. the result shows that the proposed methods are more accurate than the method without the selection. ## related work in recent years, argument mining is attracting attention in natural language processing. argument mining is a task to construct the structure of a document. it is applied to many natural language processing tasks such as document summarization @xcite , the automatic scoring of essays @xcite , the paper writing support @xcite , the information retrieval @xcite and so on. @xcite have tackled the relation identification for essays written by students. they created some features capturing the characteristics of the essay and then predicted links between argument components. the essay is usually formalized, such as the form of a claim followed by premises. however, multi-party conversations are not always formalized because many people freely speak to assert their opinions. discussion structures can be regarded as a kind of graph structure. in fact, some studies handled the selection of edges of argument structures as the shortest path problem of graphs @xcite . the shortest path problem is an optimization problem to find the path with the minimum weight among the paths connecting two given nodes in a weighted graph. in the shortest path problem, the method usually optimizes the entire path from parent nodes to multiple terminal nodes. we also apply this idea to our task, namely relation identification. however, it is not always suitable to adapt our task directly because there are some conditions in discussions: e.g., only one parent node for a child node. in this paper, we propose a link selection method for local utterance pairs to handle the characteristics of the discussions. in addition, many studies have focused on the visualization of discussions @xcite . in this paper, we also visualize predicted links to understand the result easily. ## proposed method in this section, we explain our method with two steps: link prediction and link selection. figure 2 shows an overview of our method. the first step (link prediction) is based on machine learning techniques. we compare two models; one is support vector machines @xcite with word embeddings and selected features, and the other is bert @xcite . however, the models tend to over-generate links between nodes in this task. hence, we introduce link selection approaches as the second step: top-down and bottomup approaches. ## conclusions in this paper, we proposed two types of discussion structure prediction methods. they were based on a two-step architecture: link prediction and link selection. for the link prediction, we evaluated two machine learning models, namely svm and bert. these models tended to over-generate links be-tween nodes. to solve this problem, we introduced top-down and bottom-up approaches to the link selection task. our methods outperformed svm and bert without the link selection approaches (0.53 vs. 0.66 on f1 for svm and 0.24 vs. 0.41 on f1 for bert). in the experiment, the bottom-up approach was better than the top-down approach. we visualized the discussion structures from the outputs. from the visualized data, we can see qualitatively that our methods dramatically improved the discussion structure prediction. to obtain higher accuracy, the recall rate of the link prediction model is the most important factor. it was indicated by the results of svm; the higher recall rate led to the best performance, as compared with the bert-based model. therefore the important future work is to improve the link prediction models by machine learning, especially the recall rate.