External Knowledge-Driven Argument Mining: Leveraging Attention-Enhanced Multi-Network Models (2024)

Abstract: Argument mining (AM) involves the identification of argument relations (AR) between Argumentative Discourse Units (ADUs). The essence of ARs among ADUs is context-dependent and lies in maintaining a coherent flow of ideas, often centered around the relations between discussed entities, topics, themes or concepts. However, these relations are not always explicitly stated; rather, inferred from implicit chains of reasoning connecting the concepts addressed in the ADUs. While humans can infer such background knowledge, machines face challenges when the contextual cues are not explicitly provided. This paper leverages external resources, including WordNet, ConceptNet, and Wikipedia to identify semantic paths (knowledge paths) connecting the concepts discussed in the ADUs to obtain the implicit chains of reasoning. To effectively leverage these paths for AR prediction, we propose attention-based Multi-Network architectures. Various architecture are evaluated on the external resources, and the Wikipedia based configuration attains F-scores of 0.85, 0.84, 0.70, and 0.87, respectively, on four diverse datasets, showing strong performance over the baselines.

## introduction argument mining involves identifying the argumentative structure within a text. it includes segmenting arguments into argumentative discourse units (adus) @xcite , distinguishing argumentative units from nonargumentative ones, classifying adus, labeling argument relation (ar) between adus, and identifying argument schemes @xcite @xcite . this study focuses on classifying the ar between adus into three categories: inference (ra) (when one adu supports the other), conflict (ca) (when one adu attacks the other), and none (when there is no ar). the nature of ar is inherently contextdependent @xcite @xcite , relying on maintaining a coherent flow of interconnected ideas. this cohesion is often centered around the connections between the discussed entities, topics, themes or concepts, commonly referred to as local coherence @xcite . local coherence facilitates smooth idea transitions between adus by recognising inherent regularities in entity distribution. similarly, other entity-based theories of discourse @xcite and centering theory @xcite propose that these regularities contribute to the coherence of discourse by guiding the organisation of ideas around salient entities. following a similar framework, aspect-based argument mining techniques use the relationships between the concepts discussed in adus, to identify argument structures @xcite @xcite . yet, the contexts required to link these concepts are not always explicit and are often inferred from background knowledge. pre-trained large language models (llms) have transformed nlp, moving from traditional feature engineering to data-driven approaches. studies indicate that these models implicitly capture various types of knowledge, including relational, commonsense, and structural linguistic knowledge, within their parameters @xcite @xcite . while excelling in various nlp tasks, their ability to encode the necessary background knowledge for identifying ars remains uncertain @xcite . for example, @xcite revealed their limitations in chaining multiple steps of complex logical reasoning, while @xcite demonstrated they fail to comprehend the semantics behind commonsense reasoning tasks. this limitation is critical in ar iden-tification, as linking adus relies on the implicit chain of reasoning, often inferred from the chain of relations between the concepts discussed in the adus. this highlights the need for supplementary contextual information from external sources to establish these connections. consider the adus from the 2016 presidential election debate corpus @xcite in leveraging knowledge from external resources has been shown to improve performance in am @xcite @xcite and related tasks, such as semantic plausibility @xcite , identifying inferences @xcite , and determining entailment @xcite . however, existing studies on ar prediction exclusively utilise structured knowledge bases and overlook semi-structured resources like wikipedia, which contains over 6,805,837 articles (as of april 1, 2024), offering richer connections through hyperlinks embedded within articles. moreover, these methods rely on entities, events, and factual information sourced from structured databases, limiting their applicability to specific domains. in contrast, using generic semantic relation types that encode ar ensures adaptability across domains (refer to in this paper, we propose traversing wikipedia, wordnet, and conceptnet to find semantic paths linking concepts mentioned in adu pairs. ars between adus are identified by leveraging these paths using attention-based multi-network architectures. to establish a benchmark, we evaluate llms across various configurations, comparing the knowledge obtained from external resources with that inherent in llms. the evaluation demonstrated that integrating external resources consistently enhances performance, showing strong performance over the baselines and comparison approaches. additionally, we assess the effectiveness of the attention-based multi-network architecture in leveraging external knowledge, demonstrating its superiority over the standard linear classification baseline. the contribution of this paper is fourfold: (a) the utilisation of both structured and semistructured external resources for ar prediction, (b) architecture for effectively leveraging external knowledge, (c) features adaptable across domains, and (d) the state-of-the-art performance. ## related works in the literature, am has been approached using various configurations, including dependency parsing @xcite , discourse parsing @xcite , sequence tagging @xcite , and sequence classification configurations @xcite @xcite . various works tackle specific am tasks. some focus exclusively on argument segmentation @xcite , while others start with segmented data and focus solely on ar identification @xcite @xcite . @xcite train an encoder-decoder @xcite with attention mechanism @xcite to identify ar. @xcite decompose adus into fine-grained components and use classifiers to predict ar based on the relations between these components. @xcite identify argument components and ars within both inter-turn and intra-turn interactions in dialogues. they classify ars as a binary prediction, determining only the presence of a relation without specifying its type. their findings indicate that using distant-labeled data and integrating discourse relations from rhetorical structure theory @xcite improve performance. end-to-end am approaches address multiple am tasks, simultaneously. @xcite and @xcite adopt a pipeline architecture and train separate models for each subtask to then utilise an integer linear programming (ilp) model to encode global constraints. @xcite propose a neural end-to-end approach, framing the task in various configurations including dependency parsing and token-based sequence tagging. they also employ a multi-task setup to leverage the dependencies between am tasks, including component identification and ar prediction. their best-performing configuration achieves an f1-score of 0.51 for ar identification on the aaec dataset. @xcite aim to map rst trees to argumentation structures @xcite using sub-graph matching and an evidence graph model. they evaluate various features of their system on the amt dataset and achieve an overall f-measure of 0.76 in identifying ars. similarly, @xcite introduce an end-to-end cross-corpus training strategy that facilitate information transfer between datasets. @xcite address argument component and relation identification on a dataset comprising various disease treatments. the approach involves combining static and dynamic embeddings using various configurations of rnn and crfs. they demonstrate the efficacy of specialised llms like scibert @xcite , highlighting their relevance in medical domain adaptations. however, most of these works rely on the information explicitly provided in the argument alone. recent am works fine-tuned llms in sequence classification fashion @xcite . studies show that such llms implicitly capture relational, commonsense, and structural linguistic knowledge @xcite @xcite . despite their significant performance, the ability of llms to encode the requisite background knowledge for identifying ars remains uncertain, raising concerns about relying solely on llms for this task @xcite . for instance, @xcite exposed their limitations in complex logical reasoning, while @xcite showed they struggle in comprehending the semantics of commonsense reasoning tasks. the works most related to ours are those of @xcite and @xcite , as they also leverage external knowledge bases to identify ar. however, their methodologies differ significantly from ours. firstly, they rely on structured knowledge bases with predefined relation types, while we also use semi-structured resources like wikipedia that cover diverse relations. furthermore, they struggle to effectively integrate external information into model architectures, relying instead on conventional feature engineering techniques that exploit structural features obtained from sub-graph extracted from external knowledge bases. for instance, @xcite use features like the frequency of relations existing between adus. similarly, plenz et al. (2023) leverage the similarity between external knowledge and adus to identify relevant sub-knowledge graphs and exploit the sub-graph to extract categorical features, such as the number of shared concepts between adu pairs and the path lengths between the concepts. additionally, the formalisation of the "concepts" used for alignment with external resources is vague, relying on arbitrary entity mentioned in the adus. moreover, their approach for ar identification has not been evaluated. ## methodology our approach comprises two main stages: first, we align a pair of adus (premise-conclusion) with external knowledge resources and extract the relevant knowledge paths connecting them; second, we incorporate these knowledge paths into our model architecture using attention-based multi-networks to predict the ar between the adus. the following subsections provide details on the data used and the processes involved in each step. ## conclusion our exploration of various model configurations underscored the importance of external resources and multi-network architecture with attention mechanisms in ar prediction. models augmented with external resources consistently outperform those relying solely on llms. this emphasises the necessity of leveraging supplementary knowledge sources to enrich llms for ar prediction. furthermore, multi-network architectures with attention mechanisms, notably the attention-based triplet network architecture, demonstrates superiority across all configurations. further work is required to delve deeper into attention analysis, to shed-light on its role in encouraging the model to focus in aligning the premise with the conclusion, as well as in linking the premise-conclusion pair with external knowledge. while configurations leveraging wikipedia outperformed those using other resources, more work is required to evaluate the quality of keywords representing semantic relations between concepts identified from wikipedia against the standard semantic relation types in ontologies. furthermore, alternative methods for extracting these keywords should be explored.