Context Tracking Network: Graph-based Context Modeling for Implicit Discourse Relation Recognition (2021)

Abstract: Implicit discourse relation recognition (IDRR) aims to identify logical relations between two adjacent sentences in the discourse. Existing models fail to fully utilize the contextual information which plays an important role in interpreting each local sentence. In this paper, we thus propose a novel graph-based Context Tracking Network (CT-Net) to model the discourse context for IDRR. The CT-Net firstly converts the discourse into the paragraph association graph (PAG), where each sentence tracks their closely related context from the intricate discourse through different types of edges. Then, the CT-Net extracts contextual representation from the PAG through a specially designed cross-grained updating mechanism, which can effectively integrate both sentence-level and token-level contextual semantics. Experiments on PDTB 2.0 show that the CT-Net gains better performance than models that roughly model the context.

## introduction implicit discourse relation recognition (idrr) aims to identify logical relations between two adjacent sentences in discourse without the guidance of connectives (e.g., because, but), which is one of the major challenges in discourse parsing. with the rise of deep learning, lots of sentence-modeling based methods @xcite @xcite @xcite have emerged in the field of idrr. these methods typically focus on modeling the local semantics of these two sentences, without considering wider discourse context. contextual information plays an important role in understanding sentences. take the paragraph @xmath3, s 2 , s 3 , s 4 } in figure 1 as an example, the ground-truth relation between s 3 and s 4 is "comparison". combining the contextual information carried by s 1 and s 2 , we can more easily identify the "comparison" relation reflected by the manufacturer went public at @xmath029 per-share price. strong earnings growth helped achieve that price far ahead of schedule. the stock has since softened, trading around @xmath115.75 a share" to "@xmath229 pershare" to "$25 a share"). @xcite move one step on utilizing wider discourse context, where they use a hierarchical bilstm (h-lstm) to model the whole paragraph rather than only the two sentences, to obtain context-aware sentence representation. however, there are still two limitations in their model. first, they roughly merge all the information in the paragraph, which dilutes the role of key context that closely related to the current sentence. second, the h-lstm suffers from the long-distance forgetting problem, which may fail to model the long-distance and non-continuous dependency across multiple sentences (like green lines in figure 1 ). to overcome these limitations, we propose a novel context tracking network (ct-net), which can track essential context for each sentence from the intricate discourse, without being affected by the spatial distance. the ct-net computes contextual representation through two main steps. firstly, it converts the paragraph into the paragraph association graph (pag) (figure 1 ), which contains three types of edges between sentences, namely (1) adjacency edge (black lines): connecting adjacent sentences, (2) co-reference edge (purple lines): connecting sentences with co-reference associations, and (3) lexical chain edge (green lines): connecting sentences containing related words. each sentence can track closely related context along these edges, including long-distance sentences involving the same object or topic. secondly, the ct-net extracts contextual representation over the pag. to effectively incorporate fine-grained information carried by tokens, we propose the cross-grained updating mechanism, which will be executed multiple recurrent rounds. at each round, it performs semantic exchange via three processes: after multiple rounds, the ct-net obtains the contextual representation that fully combines sentencelevel and token-level contextual semantics. our main contributions are two folds. 1 first, we propose a novel ct-net for idrr, which builds the pag to track closely related context for each sentence in the intricate discourse, and incorporates multi-grained contextual semantics via the crossgrained updating mechanism. second, experiments on pdtb 2.0 demonstrate that the ct-net gains better performance than a variety of approaches that roughly model the discourse context. 1 code is available at: https://github.com/ yxuezhang/ctnet ## model the input of the ct-net is a paragraph @xmath4, s 2 , ..., s n-1 , s n ). here, s n-1 and s n are the adjacent sentences to be classified, while s 1 , ..., s n-2 are context with background information. our goal is to identify the relation between s n-1 and s n . we firstly build a paragraph association graph (pag) for p (section 2.1), then employ the cross-grained updating mechanism on the pag to extract the contextual representation of s n-1 and s n (section 2.2). the contextual representation is then used for the final classification (section 2.3). ## conclusion we propose a novel graph-based context tracking network (ct-net) to model the context for implicit discourse relation classification. the ct-net first converts the paragraph into the paragraph association graph (pag), where each sentence tracks their appropriate context through different edges, then employs the cross-grained updating mechanism to combine sentence-level and token-level contextual information. experiments on pdtb 2.0 demonstrate that the ct-net captures more effective contextual information than carefully designed baselines with different context encoders.