Constituency Tree Representation for Argument Unit Recognition (2023)

Abstract: The conventional method of extracting arguments from sentences solely relies on word proximity, disregarding the syntactic structure of the sentence. This approach often leads to inaccuracies, especially when identifying argumentative span boundaries. In this research, we investigate the benefits of utilizing a constituency tree representation of sentences to predict Argument Discourse Units (ADUs) at the token level. We first evaluate the effectiveness of utilizing the constituency tree representation for capturing the structural attributes of arguments within sentences. We demonstrate empirically that the constituency structure surpasses simple linear dependencies among neighboring words in terms of effectiveness. Our approach involves leveraging graph neural networks in conjunction with the constituency tree, adapting it specifically for argument unit recognition. Through extensive evaluation, our model outperforms existing approaches in recognizing argument units at the token level. Furthermore, we employ explainability methods to assess the suitability of our model architecture, providing insights into its performance.

## introduction argument identification within documents serves as the initial step in studying rhetorical speech processes, student essays, or political debates. the objective is to accurately identify argument discourse units (adus), defined as minimal analysis units, within sentences, and predict their stance and relation to each other. previous works on token-level argument analysis @xcite have employed language models such as bidirectional encoder representations from transformers (bert) @xcite , in conjunction with probabilistic models like conditional random field (crf) @xcite . this combination enhances overall prediction coherence with constrained fine-tuning. the study of arguments and discourse has been approached from a grammatical perspective, including frameworks such as rhetorical structure theory @xcite , one of the conclusion from the annotation guideline @xcite is the use of syntax to better identify the elementary discourse units (edu). building grammatical parsers is a complex task that has received extensive research attention. the results achieved thus far are promising and can serve as a foundation for various applications. in this research, we investigate the benefits of incorporating grammatical structure into a bert-crf model for argument unit recognition, with a specific focus on the constituency tree representation of sentences (as illustrated in figure 1 ). this representation consists of a tree where interior nodes represents the grammatical structure of the sentence, along with leaf nodes (nodes without children) corresponding to the words within the sentence. the primary objectives of this paper are: argumentation theory the precise definition of the concept of argument is an important step when creating dataset annotation rules. the identification of argument is strongly related to the discourse structure of the text and the identification of elementary discourse units. as introduced in the rhetorical structure theory (rst) by @xcite , the elementary discourse units (edu) refers to a minimal unit of meaning within a larger discourse or conversation. it represents a self-contained piece of information that contributes to the overall structure and coherence of a discourse. a non-elementary discourse unit (du) is called a complex discourse unit (cdu). the structure of a document is the set of linked dus. as presented by @xcite , argument discourse unit (adu) are units of meaning that contribute to the development and presentation of an argumentative structure. adus typically contain propositions, claims, evidence, or reasoning that support or challenge a particular standpoint or claim. in practical applications, while certain studies rely on the annotator's judgment to determine the boundary of an adu, many studies prefer to utilize a set of syntactic rules as a foundation @xcite . this approach is favored because employing syntactic structure for annotating a sizable corpus at the token level is comparatively easier @xcite . tree structure representation in natural language processing (nlp) substantial evidence @xcite supports the hypothesis that semantic interpretation of sentences by humans involves a tree-structured, hierarchical computation, where smaller constituents recursively combine into larger constituents, until we reach the full sentence. in nlp, pioneer work @xcite ) presenting the benefits of using constituency tree representation has failed to scale into production. according to the authors, this is caused by the lack of a reliable model to generate constituency tree representation of the sentences. however, recent promising results @xcite were made in consistency and dependency parsing. other papers have recently studied the use of tree structure to incorporate syntactic information to their models. @xcite uses graph convolutional networks (gcns) based on the dependency tree structure of the sentence for semantic role labeling. @xcite uses gnns for generation tasks from abstract meaning representation. recently, @xcite demonstrate that for some tasks, transformers models become more "tree-like" over the course of training and in some cases unsupervisedly recovering the same trees as supervised parsers. showing the importance of constituency tree in the learning process of the transformers models. segmentation of argumentative units in texts has been explored in ajjour et al.. the research indicates that both structural and semantic features are pivotal for segmenting argument units across various domains. however, within specific domains, semantic features stand out as the most effective for identifying the boundaries of these units. ## evaluation of the node similarity in the subsequent section, we evaluate the effectiveness of utilizing the constituency tree representation for capturing the structural attributes of arguments within sentences. in subsection 3.2, we introduce three metrics to assess the proximity of nodes in the tree concerning their argumentative label. additionally, in subsection 3.3, we propose modifications to the tree to enhance its suitability for argument recognition. ## presentation of our model in this section, we present a detailed overview of the architecture and components of our proposed model for argument unit recognition. ## experimental setup in the next sections, we present a comprehensive evaluation of our proposed model for argument unit recognition using constituency tree representations and gnns with a crf layer. we compare the performance of our model information from the constituency tree. first, we describe the experimental setup and datasets used for evaluation 5.1 data source arg2020 @xcite is an argument mining corpus annotated with argumentative structure composed of "claims" and "premises". it is composed of 145 english argumentative essays selected through the writing mentor educational app. it is based on middle school students writing. the claims is defined as a potentially arguable statement that indicates a person is arguing for or arguing against something. the premises is de-fined as the reasons given by either for supporting or attacking the claims. argument unit recognition and classification (aurc) @xcite is a corpus for argument mining that includes annotations for argumentative structure information, capturing the polarity of arguments on a given topic. the corpus consists of 8000 sentences, evenly distributed across 8 topics. the authors distinguished between pro (supporting), con (opposing) arguments, and non (non-argumentative) words for each topic, in order to construct sentence-level labels. their labeling rule is as follows: if only non words occur, the sentence is labeled as non. if both non and figure 3 : illustration of the model architecture on an example of a sentence. we present the three distinct modules composing the model with their associated input/output. the colored node after the graph neural network refers to the nodes where the label predicted in "pro", this phrase is taken from the aurc dataset. the e x refers to the embeddings from the bert model, the interior nodes are initialized with the 0 vector. the cornell erulemaking corpus (cdcp) @xcite is a corpus for argument mining that includes annotations for argumentative structure information, specifically capturing the evaluability of arguments. the corpus comprises 731 user comments on the consumer debt collection practices rule issued by the consumer financial protection bureau. the resulting dataset contains a total of 4931 elementary unit annotations and 1221 support relation annotations. argument annotated essays corpus (ukp) consists of a collection of persuasive essays gathered by @xcite . this essay corpus is equipped with annotations of argument components at the clause level, as well as argumentative relations. specifically, it includes annotations for major claims, claims, and premises, which are interconnected through argumentative support and attack relations. the corpus was annotated by three raters, achieving an inter-annotator agreement of @xmath0.72 for argument components and @xmath1.81 for argumentative relations. in total, the corpus consists of 90 essays containing 1673 sentences. the models are trained individually on each of the four datasets, conforming to the respective label schemes they offer. for comparison with the baseline, we adhere to the train-test splits presented in the original datasets' experiments when available. in cases where these are not provided, we employ a random sampler to allocate 20% of the sentences for testing and 80% for training. subsequently, the sentences are segmented into chunks of 64 tokens each. ## conclusion in conclusion, this research study introduces a novel method for identifying the boundary of adus using the sentence constituent tree representation. our model effectively spreads information across the graph and achieves promising results on a small dataset. previously identified errors in these datasets include the incorrect recognition of argumentative segment spans and inaccurate classification of stances. in this study, we focus on improving the span detection problem and successfully enhance the method for identifying adu boundaries. however, it is worth noting that some argument mining datasets does not strictly adhere to grammatical correctness, as noted in @xcite . this limitation arises from sentences where subjects are absent, which hampers the performance of models relying solely on grammatical structure. this issue could be resolved by devising annotation rules that more strictly align with the syntactic structure of sentences. furthermore, the second type of error, which pertains to position identification, is primarily attributed to the limitations of the bert model. the dataset only provides sentences with a maximum length of 64, thereby restricting the available context for adus and impeding our model's capability. many arguments require deeper domain knowledge to fully comprehend the underlying issues.