Similarity-weighted Construction of Contextualized Commonsense Knowledge Graphs for Knowledge-intense Argumentation Tasks (2023)

Abstract: Arguments often do not make explicit how a conclusion follows from its premises. To compensate for this lack, we enrich arguments with structured background knowledge to support knowledge-intense argumentation tasks. We present a new unsupervised method for constructing Contextualized Commonsense Knowledge Graphs (CCKGs) that selects contextually relevant knowledge from large knowledge graphs (KGs) efficiently and at high quality. Our work goes beyond context-insensitive knowledge extraction heuristics by computing semantic similarity between KG triplets and textual arguments. Using these triplet similarities as weights, we extract contextualized knowledge paths that connect a conclusion to its premise, while maximizing similarity to the argument. We combine multiple paths into a CCKG that we optionally prune to reduce noise and raise precision. Intrinsic evaluation of the quality of our graphs shows that our method is effective for (re)constructing human explanation graphs. Manual evaluations in a large-scale knowledge selection setup verify high recall and precision of implicit CSK in the CCKGs. Finally, we demonstrate the effectiveness of CCKGs in a knowledge-insensitive argument quality rating task, outperforming strong baselines and rivaling a GPT-3 based system.

## introduction computational argumentation is a growing field with relevant applications, such as argument retrieval @xcite , argument analysis @xcite @xcite @xcite or generation @xcite @xcite . argumentation requires deep understanding of argumentative statements figure 1 : cckg connecting the premise "a person is unhappy if she is dissatisfied with her body." to the conclusion "plastic surgery raises patients' self esteem and allows them to lead normal happy lives." concepts associated with premise and conclusion are colored in violet and orange, respectively. the graph makes explicit that plastic surgery causes looking better, which in turn causes happiness, which is distinct from dissatisfaction. and how they relate to each other. often, commonsense knowledge (csk) is needed to understand how a premise connects to its conclusion, as these connections are often left implicit, as shown in figure 1 . while humans can easily infer implied knowledge, for machines extra mechanisms are needed to inject missing knowledge to better solve argumentative tasks @xcite @xcite . methods to inject such knowledge either rely on parametric memory, where csk is stored in the parameters of large language models (llms), or non-parametric memory, where csk is stored in external knowledge bases. in the llm approach, latent csk can be directly exploited in downstream tasks @xcite or the llm is fine-tuned to generate the csk in explicit form @xcite . however, approaches based on parametric memory have drawbacks: they often are difficult to adapt to new domains @xcite or suffer from risk of hallucinations and unsafe generations @xcite since they are not traceably grounded. explicit and structured csk is available in commonsense knowledge graphs (kgs) @xcite @xcite . but kgs are large and not contextualized, which makes selecting relevant knowledge difficult. we can extract knowledge in the form of individual triplets @xcite , but this does not allow for multi-hop reasoning over (potentially disconnected) triplets. extracting paths consisting of multiple triplets allows multi-hop reasoning @xcite , but systems cannot exploit potential interactions between multiple paths. our approach extends the idea of multi-hop path extraction by combining multiple such paths into a graph -our contextualized commonsense knowledge graph. the cckgs are small and tailored to a specific argument, as shown in figure 1 , which makes them applicable in joint reasoning models @xcite . similar to retrieval models @xcite ) that extract relevant passages from text for knowledge extension, our approach extracts relevant subgraphs from structured kgs. we can find connecting paths in large kgs by extracting shortest paths that link pairs of concepts. but the paths are not guaranteed to provide relevant knowledge for a given context, as intermediate triplets might be off-topic. to mitigate this problem, we compute edge weights to rate the semantic similarity of individual kg triplets to the argument at hand, and extract weighted shortest paths that are maximally similar to the argument. combining the paths into a cckg encapsulates relevant csk. we compute the edge weights using sbert without extra fine-tuning, and rely on graph algorithms for cckg construction. hence, our method is unsupervised and applicable in zero-shot settings. our main contributions are: i.) we present an unsupervised contextualized commonsense knowledge graph (cckg) construction method that enriches arguments with relevant csk, by combining similarity-based contextualization with graph algorithms for subgraph extraction. ## background and related work when humans debate a topic, they typically leverage a vast body of background knowledge, some already known to them and other knowledge subject to addition, e.g., by looking up a wikipedia entry. therefore, with the availability of large-scale kgs @xcite , and with the advent of llms that have been shown to learn knowledge during self-supervised training @xcite , we observe growing interest in incorporating knowledge into computational argumentation systems @xcite @xcite . of particular interest is the (re-)construction of implicit commonsense knowledge (csk) @xcite @xcite within or between arguments. usually, the goal is to improve downstream-task performance of systems, e.g., improving argumentative relation classification by connecting concepts with paths found in @xcite , or improving argument quality prediction by extracting kg distance features @xcite . but the aim can also extend to argumentative explanations, propelled by an emergent need for more transparency of model predictions @xcite , which is crucial for argumentative decision making @xcite . @xcite manually created small csk explanation graphs and developed fine-tuned language models to generate such graphs automatically. prior approaches for retrieving csk suffer from several issues, e.g., @xcite enrich single tokens but can't provide longer reasoning paths. by contrast, works that construct reasoning paths either do not exploit their interactions, are intransparent on which paths are used for prediction @xcite , employ thresholds that are hard to tailor to different tasks @xcite , or depend on knowledge generated from llms @xcite @xcite , which may decrease trust in the provided knowledge due to hallucinations @xcite @xcite . in our work, we aim to unify the strengths of such approaches while mitigating their weaknesses: our cckg construction method is i) context-sensitive without requiring threshold selection, and extracts csk graphs that provide ii) accurate multi-hop reasoning structures and iii) are transparently grounded in a large kg, and hence, iv) yield strong task performance in a transfer setting. ## cckg construction given an argument, we aim to enrich it with csk that connects the argument's premise and conclusion via a contextualized commonsense knowledge graph (cckg). figure 2 shows an overview of our method. in a pre-processing step we compute a semantic embedding for each triplet of the kg. using these embeddings we compute the semantic similarity of each triplet to the premise, to the conclusion and to the argument as a whole. by selecting the triplets with maximal similarity scores, we obtain relevant concepts associated with the premise and conclusion. next we aim to connect these concepts with relevant reasoning paths, i.e., short paths containing triplets that match the argument. we thus convert the argument similarity scores to edge weights, and connect the extracted concepts with weighted shortest paths that maximize the semantic similarity between the path and the argument. optionally, we further prune the cckg. 2 below we describe each step in detail. pre-processing we compute a sentence embedding for each triplet in the kg by first verbalizing the triplets using a designated template for each relation (see §b.1.2). we then use a s(entence)bert (reimers and gurevych, 2019) encoder to compute an embedding for each verbalized triplet. verbalization and triplet encoding is independent from the argument, so this step is executed only once. 2 the pruning is not shown in figure 2 . ## experiments we evaluate our cckg construction method intrinsically ( §4.1) in an argumentative commonsense graph generation task and extrinsically ( §4.2) in a knowledge-intense conclusion classification task. ## conclusion in this work we proposed an unsupervised method to construct contextualized commonsense knowledge graphs (cckgs). our extensive evaluations show that cckgs are of high quality, outperform context-insensitive baselines and rival strong supervised graph construction methods on diverse argumentation tasks, while offering increased robustness. being grounded in a kg, the information captured in our cckgs is traceable and hence interpretable. future work could explore incorporation of more specific kgs to address particular domains. using our compact, high-quality cckgs in stronger interaction with llms is another step to address in future work.