Enhancing Argument Structure Extraction with Efficient Leverage of Contextual Information (2023)

Abstract: Argument structure extraction (ASE) aims to identify the discourse structure of arguments within documents. Previous research has demonstrated that contextual information is crucial for developing an effective ASE model. However, we observe that merely concatenating sentences in a contextual window does not fully utilize contextual information and can sometimes lead to excessive attention on less informative sentences. To tackle this challenge, we propose an Efficient Context-aware ASE model (ECASE) that fully exploits contextual information by enhancing modeling capacity and augmenting training data. Specifically, we introduce a sequence-attention module and distance-weighted similarity loss to aggregate contextual information and argumentative information. Additionally, we augment the training data by randomly masking discourse markers and sentences, which reduces the model’s reliance on specific words or less informative sentences. Our experiments on five datasets from various domains demonstrate that our model achieves state-of-the-art performance. Furthermore, ablation studies confirm the effectiveness of each module in our model.

## introduction argument structure extraction (ase) aims to identify the argumentative discourse structure in documents @xcite @xcite . as a basis of evidence-based reasoning applications, ase has attracted increasing attention from researchers in a wide spectrum of domains, such as legal documents @xcite @xcite , online posts @xcite @xcite , and scientific articles @xcite . 1 figure 1 shows an exam-figure 1 : an example of argument structure extraction in peer reviews. the argumentative discourse markers are marked in red. it has been verified that the contextual information of documents plays a crucial role in detecting argumentative relations @xcite @xcite . according to our preliminary experiments based on the context-aware ase model (case) @xcite , several findings motivate us to further explore the efficient use of contextual information. firstly, based on our experimental results shown in figure 2 (a), we find that the performance of the model is initially improved with the increase of the context window, but then decreases as the context window continues to grow. these results indicate that contextual information can enhance the performance of the model, but the excessively long context may cause the model to overly focus on some irrelevant sentence information and behave relatively weak performance. secondly, the performance of case decreases as the distances of argumentative pairs increase, as shown in figure 2 (b), which suggests that detecting argumentative relations over long distances is still challenging for case. it also implies that there is still room for improvement by making full use of contextual information rather than simply concatenating the sentences in a context window. thirdly, figure 2(c) shows that the model mostly achieves better performance in argumentative pairs where there are discourse markers, such as 'so', 'thus', and et. these specific words are regarded as significant signals for identifying argumentative relations @xcite ), but they may hinder the model from fully exploiting the contextual information between sentences. based on the above observations, we propose an efficient context-aware ase model (ecase), which enhances the leverage of contextual information from two perspectives, i.e., modeling capacity and data augmentation. regarding the modeling capacity, we adopt a sequence attention module on roberta @xcite to aggregate the contextual information and optimize the attention distribution between sentences. we also use a distanceweighted similarity loss to reinforce the representation similarity in view of argumentative relations. in terms of data augmentation, we randomly mask the discourse markers and sentences in training sets to mitigate the model's dependence on specific words and less informative sentences. hence, the model is encouraged to thoroughly comprehend the contextual information. experiments on five datasets from different domains show that our model achieves state-of-theart performance in both the head-given setting and the end-to-end setting of the ase task. ablation studies also demonstrate the effectiveness of each module in our model. the codes are released in https://github.com/luoxiaoheics/ecase . ## experiments setup we carry out experiments on five ase datasets from different domains, namely ampere, essays, abstrct, echr, and cdcp, whose statistics are shown in ## conclusion in this study, we proposed an efficient contextaware ase model by enhancing modeling capacity and augmenting training data. our experiments demonstrated the effectiveness of our model, and the ablation study proved the significance of each module. our study highlighted the essential role of efficient use of contextual information.