Global and Local Hierarchy-aware Contrastive Framework for Implicit Discourse Relation Recognition (2023)

Abstract: Due to the absence of explicit connectives, implicit discourse relation recognition (IDRR) remains a challenging task in discourse analysis. The critical step for IDRR is to learn high-quality discourse relation representations between two arguments. Recent methods tend to integrate the whole hierarchical information of senses into discourse relation representations for multi-level sense recognition. Nevertheless, they insufficiently incorporate the static hierarchical structure containing all senses (defined as global hierarchy), and ignore the hierarchical sense label sequence corresponding to each instance (defined as local hierarchy). For the purpose of sufficiently exploiting global and local hierarchies of senses to learn better discourse relation representations, we propose a novel GlObal and Local Hierarchy-aware Contrastive Framework (GOLF), to model two kinds of hierarchies with the aid of multi-task learning and contrastive learning. Experimental results on PDTB 2.0 and PDTB 3.0 datasets demonstrate that our method remarkably outperforms current state-of-the-art models at all hierarchical levels.

## introduction implicit discourse relation recognition (idrr) aims to identify logical relations (named senses) between a pair of text segments (named arguments) without an explicit connective (e.g., however, because) in the raw text. as a fundamental task in discourse analysis, idrr has benefitted a wide range of natural language processing (nlp) applications such as question answering @xcite , summarization @xcite , information extraction @xcite , etc. the critical step for idrr is to learn high-quality discourse relation representations between two arguments. early methods are dedicated to manually figure 1 : an idrr instance in the pdtb 2.0 corpus @xcite . argument 1 is in italics, and argument 2 is in bold. the implicit connective is not present in the original discourse context but is assigned by annotators. all senses defined in pdtb are organized in a three-layer hierarchical structure (defined as global hierarchy in our paper), and the implicit connectives can be regarded as the most fine-grained senses. however, existing hierarchy-aware methods still have two limitations. firstly, though they exploit the fact that there are complex dependencies among senses and such information should be encoded into discourse relation representations, their manners of encoding the holistic hierarchical graph of senses may not be sufficient, since they fail to strengthen the correlation between the discourse relation representation and its associated sense labels, which is highly useful for classification @xcite . secondly, they only consider the graph of the entire label hierarchy and ignore the benefit of the label sequence corresponding to each instance. as shown in figure 2 , the label sequences of instances (1) and (2) differ at both the top and second levels, while the label sequences of instances (1) and ( 3 ) only differ at the most fine-grained level. the similarity between label sequences provides valuable information for regularizing discourse relation representations, e.g., by ensuring that the distance between representations of instance (1) and ( 2 ) is farther than the distance between representations of instance (1) and ( 3 ). under such an observation, we categorize the sense hierarchy into global and local hierarchies to fully utilize the hierarchical information in idrr. we define global hierarchy as the entire hierarchical structure containing all senses, while local hierarchy is defined as a hierarchical sense label sequence corresponding to each input instance. therefore, global hierarchy is static and irrelevant to input instances, while local hierarchy is dynamic and pertinent to input instances. built on these motivations, we raise our research question: how to sufficiently incorporate global and local hierarchies to learn better discourse relation representations? to this end, we propose a novel global and local hierarchy-aware contrastive framework (golf), to inject additional information into the learned relation representation through additional tasks that are aware of the global and local hierarchies, respectively. this is achieved via the joint use of multi-task learning and con-trastive learning. the key idea of contrastive learning is to narrow the distance between two semantically similar representations, meanwhile, pushing away representations of dissimilar pairs @xcite . it has achieved extraordinary successes in representation learning @xcite . finally, our multi-task learning framework consists of classification tasks and two additional contrastive learning tasks. the global hierarchy-aware contrastive learning task explicitly matches textual semantics and label semantics in a text-label joint embedding space, which refines the discourse relation representations to be semantically similar to the target label representations while semantically far away from the incorrect label representations. in the local hierarchy-aware contrastive learning task, we propose a novel scoring function to measure the similarity among sense label sequences. then the similarity is utilized to guide the distance between discourse relation representations. the main contributions of this paper are threefold: ## problem definition given m hierarchical levels of defined senses @xmath0, ..., s m , ..., s m ), where s m is the set of senses at the m-th hierarchical level, and a sample input consisting of two text spans, or x @xmath1, arg 2 ), our model aims to output a sequence of sense y @xmath2, ..., y m i , ..., y m i ), where y m @xmath3. ## methodology figure 3 illustrates the overall architecture of our multi-task learning framework. beginning at the left part of figure 3 , we utilize a discourse relation encoder to capture the interaction between two input arguments and map them into a discourse relation representation h. after that, the discourse relation representation h is fed into a staircase classifier to perform classification at three hierarchical levels dependently. while training, we will use two additional tasks, the global hierarchy-aware contrastive loss l global (in the upper right part of figure 3 ) and the local hierarchy-aware contrastive loss l local (in the lower right part of figure 3 ) as additional regularization to refine the discourse relation representation h. during inference, we only use the discourse relation encoder and the staircase classifier for classification and discard the global and local hierarchy-aware contrastive learning modules. detailed descriptions of our framework are given below. ## ablation study firstly, we investigate the efficacy of individual modules in our framework. for this purpose, we remove the multi-head interactive attention (mhia), the "staircase" in classifier, the global hierarchy-aware contrastive loss l g , and the local hierarchy-aware contrastive loss l l from golf one by one. note that removing the "staircase" in classifier means that we keep the crossentropy loss but remove the dependence between logits from different hierarchical levels.