Syntopical Graphs for Computational Argumentation Tasks (2021)

Abstract: Approaches to computational argumentation tasks such as stance detection and aspect detection have largely focused on the text of independent claims, losing out on potentially valuable context provided by the rest of the collection. We introduce a general approach to these tasks motivated by syntopical reading, a reading process that emphasizes comparing and contrasting viewpoints in order to improve topic understanding. To capture collection-level context, we introduce the syntopical graph, a data structure for linking claims within a collection. A syntopical graph is a typed multi-graph where nodes represent claims and edges represent different possible pairwise relationships, such as entailment, paraphrase, or support. Experiments applying syntopical graphs to the problems of detecting stance and aspects demonstrate state-of-the-art performance in each domain, significantly outperforming approaches that do not utilize collection-level information.

## introduction collections of text about the same topic such as news articles and research reports often present a variety of viewpoints. @xcite proposed a formalized manual process for understanding a topic based on multiple viewpoints in their book, how to read a book, applying dialectics to collection browsing. this process consists of four levels of reading, the highest of which is syntopical reading. syntopical reading is focused on understanding a core concept by reading a collection of works. it requires finding passages on the * work done while interning at adobe research. core concept that agree or disagree with each other, defining the issues, and analyzing the discussion to gain a better understanding of the core concept. the goal of the paper at hand is to operationalize the syntopical reading process computationally in order to help individuals make sense of a collection of documents for a given topic. viewed through the lens of computational argumentation, these documents state claims or conclusions that can be grouped by the aspects of the topic they discuss as well as by the stance they convey towards the topic @xcite ). an individual aiming to form a thorough understanding of the topic needs to get an overview of these viewpoints and their interactions. this may be hard even if adequate tool support for browsing the collection is available @xcite @xcite . we seek to enable systems that are capable of reconstructing viewpoints within a collection, where a viewpoint is expressed as a triple @xmath0, aspect, stance). we consider the argumentative unit of a claim to be the minimal expression of a viewpoint in natural language, such that a single viewpoint can have many claims expressing it. as an example, consider the following two claims: "nuclear energy emits zero co 2 ." "nuclear can provide a clean baseload, eliminating the need for fracking and coal mining." within a collection these claims express: @xmath1, env. impact, @xcite the goal of the systems we envision is thus to identify, group, and summarize the latent view-viewpoints @xcite ## related work first attempts at stance detection used contentoriented features @xcite . later approaches, such as those by @xcite and hasan and ng (2013), exploited common patterns in dialogic structure to improve stance detection. more tailored to argumentation, bar-haim et al. (2017a) first identified the aspects of a discussed topic in two related claims and the sentiment towards these aspects. from this information, they derived stance based on the contrastiveness of the aspects. later, bar-haim et al. (2017b) mod-eled the context of a claim to account for cases without sentiment. our work follows up on and generalizes this idea, systematically incorporating implicit and explicit structure induced by the topics, aspects, claims, and participants in a debate. in a similar vein, @xcite embedded debate posts and authors jointly based on their interactions, in order to classify a post's stance towards the debate topic. @xcite encoded related pairs of claims using bert to predict the stance and specificity of any claim in a complex structure of online debates. however, neither of these exploited the full graph structure resulting from all the relations and interactions in a debate, which is the gap we fill in this paper. @xcite model collective information about debate posts, authors, and their agreement and disagreement using probabilistic soft logic. whereas they are restricted to the structure available in a forum, our approach can in principle be applied to arbitrary collections of text. we also tackle aspect detection, which may at first seem more content-oriented in nature. accordingly, previous research such as the works of @xcite and @xcite employed word-based features or contextualized word embeddings for topic-specific aspect clustering. @xcite , whose argument frames dataset we use, instead clustered aspects with latent semantic analysis (lsa) and topic modeling. but, in general, aspects might not be mentioned in a text explicitly. therefore, we follow these other approaches, treating the task as a clustering problem. unlike them, however, we do not model only the content and linguistic structure of texts, but we combine them with the debate structure. different types of argumentation graphs have been proposed, covering expert-stance information @xcite , basic argument and debate structure @xcite , specific effect relations @xcite , social media graphs @xcite , and knowledge graphs @xcite . our main focus is not learning to construct ground-truth graphs, but how to use an approximated graph to derive properties such as stance and aspect. our work resembles approaches that derive the relevance of arguments @xcite or their centrality and divisiveness in a discussion @xcite ) from respective graphs. @xcite used a neural graph attention network to classify speech stance based on a graph with texts, speakers, and topics as nodes. while we also use a relational graph convolutional network for learning, the graph we propose captures implicit claim relations as well as explicit structure. in addition, text-based graph neural models have been proposed to facilitate classification, such as textgcn @xcite as well as the followup work bertgcn @xcite . these approaches build a graph over terms (using normalized mutual information for edge weights) as well as sentences and documents (using tf-idf for edge weights) to improve sentence-or documentlevel classification. our work generalizes this approach, focusing on incorporating many edge types with different meanings, such as relative stance or relative specificity. we compare our approach with a bertgcn baseline, and we ablate all considered edge types, in order to show the importance of capturing these different textual relationships. ultimately, we seek to facilitate understanding of the main viewpoints in a text collection. @xcite used clustering-based viewpoint discovery to study the impact of the interaction of topics and users in forum discussions. @xcite used multi-document summarization techniques to mine and organize the main points in a debate, and vilares and he (2017) mined the main topics and their aspects using a bayesian model. @xcite introduced the idea of keypoint analysis, grouping arguments found in a collection by the viewpoint they reflect and summarizing each group to a salient keypoint. while our graph-based analysis is likely to be suitable for finding keypoints, we instead focus on reconstructing latent viewpoints by grouping claims, leaving open the option to identify the key claims in future work as it would require manual evaluation. ## syntopical graphs we now introduce the concept of a syntopical graph. the goal of our syntopical graph is to systematically model the salient interactions of all claims in a collection of documents. then, properties of claims (say, their stance towards a topic or the aspects they cover) can be assessed based not only on the content of the claim alone, but on the entirety of information available in their context. to capture this context, we build a graph where documents and claims are nodes. edges between the nodes are documents and claims, and there are 0+ weighted and typed edges between any pair of nodes. in downstream applications, we add the representation of the topic to the claim nodes. ## viewpoint reconstruction a viewpoint can be understood as a judgment of some aspect of a topic that conveys a stance towards the topic. the goal of viewpoint reconstruction is to identify the set of viewpoints in a collection given a topic, starting with the claims. an example of this process is shown on the right in figure 1 . to denote viewpoints, we borrow notation in line with the idea of aspect-based argument mining (trautmann, 2020), which in turn was inspired by aspectbased sentiment analysis. in particular, we express a viewpoint as a triple v : @xmath2, aspect, stance) a claim is an expression of a viewpoint in natural language, and a single viewpoint can be expressed in several ways throughout a collection in many claims. aspects are facets of the broader argument around the topic. while some actual claims may encode multiple viewpoints simultaneously, henceforth we consider each claim to encode one viewpoint for simplicity. to tackle viewpoint reconstruction computationally, we decompose it into two sub-tasks, stance detection and aspect detection, along with a final grouping of claims with same aspect and stance. stance detection stance detection requires assigning a valence label to a claim with respect to a particular topic. though content-only baselines can work in many cases, there are also cases where the stance of a claim might only make sense in relation to a broader argument. for example, the claim "nuclear power plants take 5 years to construct" is difficult to assign a stance a priori. however, in the context of other claims such as "solar farms often take less than 2 years to commission", it might be viewed as having a negative stance. to exploit this additional contextual information, we use syntopical graphs as input to a graph neural network, in particular a relational graph convolutional network (r-gcn) @xcite . we treat stance detection as a supervised node classification task. the goal is to output a prediction in the set {pro, con} for each claim node relative to a topic. r-gcns were developed to perform node classification and edge prediction for knowledge bases, which are also typed multigraphs. as such, the abstractions of the syntopical graph slot neatly into the abstractions of r-gcns. the input to an r-gcn is a weighted, typed multigraph with some initial node representation. the network is made up of stacked relational graph convolutional layers; each layer computes a new set of node representations based on each node's neighborhood. in effect, each layer combines the edge-type-specific representation of all of a node's neighbors with its own representation. the representations are influenced by the node, and all of its neighbors, attenuated through the edge weight. an r-gcn thus consumes a set of initial claim representations, transforms them through stacks of relational graph convolutional layers, and outputs a final set of node vectors, which are fed into a classifier to predict the claim stance. aspect detection following the work of @xcite , we treat aspect detection as an unsupervised task. as aspects are an open class, we use a community detection approach, modularity-based community detection @xcite . the key intuition of modularity-based community detection is that communities are graph partitions that have more edges within communities than across communities. modularity is a value assigned to a graph partition, which is higher when there are fewer edges across communities than within them; a modularity of 0 represents a random partition, while higher modularities indicate tighter communities. the goal of modularity-based community detection is to maximize modularity by finding dense partitions. this intuition works well for aspects in a syntopical graph -claims that discuss a similar aspect are likely to have salient interactions. as aspects themselves are independent of stance, the direction of the interactions (e.g., support or refute) does not matter, but their salience does. to capture only the intensity of the interaction between two claims, we apply a transformation to signed collapse the multi-edges of a syntopical graph (denoted sg) to a positive-weighted graph (g): ## experiments to evaluate the effectiveness of our approach at reconstructing viewpoints, we consider three datasets across the two subtasks of stance and aspect detection. we hypothesize that syntopical graph approaches will outperform content-only baselinesincluding the ones used to initialize the claim representations -because they are able to make use of not only the claim content, but also the claim context. we further hypothesize that syntopical graph approaches will outperform graph-based baselines that use only textual similarity edges, because the latter's claim context is not as rich. for our experiments, we construct a syntopical graph as described in section 3. we further evaluate our model by conducting several additional experiments, including removing the use of document nodes or initial claim representations, analyzing the performance of each edge type in isolation and when left out, and an analysis of the differences in predictions between the syntopical graph and the content-only baselines. stance detection for the stance detection experiments, we use two datasets: first, the heterogeneous cross-topic argumentation mining dataset (argmin) from @xcite , and second, the claim-stance dataset (ibmcs) from @xcite . the argmin dataset contains about 25k sentences from 400 documents across eight controversial topics, ranging from abortion to school uniforms. following @xcite , we filter only the claims, resulting in 11.1k claims. the ibmcs dataset contains 2.4k claims across 55 topics. we use the splits from @xcite , which ensure that the topics in the training and test sets are mutually exclusive. claims are given a stance label drawn from {pro, con}. we evaluate using macro-averaged f 1 and accuracy. we use a syntopical graph for each dataset as the input to a relational graph convolutional network (r-gcn), implemented in dgl @xcite and pytorch @xcite . for document node representations, we use a pretrained sentence transformer and concatenate all of the sentences as input @xcite . for the claim node representations, we use a roberta model pretrained on an nli task @xcite to encode both the claim and topic; the resulting vectors are fixed throughout training. aspect detection for clustering-based aspect detection, we use the argument frames dataset from @xcite . the dataset contains roughly 11k sentences drawn from 465 different topics. each sentence has a specific aspect (or frame, in the original paper), drawn from a set of over a thousand possible aspects. following the authors, we evaluate with a clustering metric, bcubed f 1 @xcite . we transform the graph as described in section 4 to use as an input to modularity-based community detection, using τ of 0.6 tuned on held-out topics. ## results and analysis the remainder of this section investigates the robustness of the syntopical graph approach to stance and aspect detection: first, we analyze the contribution of each edge type, running experiments without and with only each edge type. we also examine the accuracy of the edges in our graph when applied out of domain as well as analysis to understand the types of claims for which this model improves performance. ## conclusion in this paper, we have introduced a data structure, the syntopical graph, which provides context for claims in collections. we have provided empirical evidence that syntopical graphs can be used as input representations for graph-structured approaches