Extending Implicit Discourse Relation Recognition to the PDTB -3 (2020)

Abstract: The PDTB-3 contains many more Implicit discourse relations than the previous PDTB-2. This is in part because implicit relations have now been annotated within sentences as well as between them. In addition, some now co-occur with explicit discourse relations, instead of standing on their own. Here we show that while this can complicate the problem of identifying the location of implicit discourse relations, it can in turn simplify the problem of identifying their senses. We present data to support this claim, as well as methods that can serve as a non-trivial baseline for future state-of-the-art recognizers for implicit discourse relations.

## introduction most readers will be familiar with the pdtb-2 @xcite . at the time of its creation, it was the largest public repository of annotated discourse relations (over 43k), including over 18.4k signalled by explicit discourse connectives (coordinating or subordinating conjunctions, or discourse adverbials). in the corpus, discourse relations comprise two arguments labelled arg1 and arg2, with each relation anchored by either an explicit discourse connective or adjacency. in the latter case, annotators inserted one or more implicit connectives to signal the sense(s) they inferred to hold between the arguments. the size and availability of the pdtb-2 spawned work on shallow discourse parsing, as in the 2015 and 2016 conll shared tasks @xcite . with the release of the pdtb-3 1 , there are now ∼12.5k additional intra-sentential relations annotated (i.e., relations that lie wholly within the projection of a top-level s-node) and ∼1k additional inter-sentential relations @xcite . but there is another reason as well: work on the pdtb-2 has assumed (correctly) that non-explicit discourse relations (i.e., implicit relations, altlex relations @xcite and entity relations) only hold between adjacent sentences as they did in the pdtb-2, so that a sentence boundary is the only position that needs to be checked for the presence of a non-explicit relation. the difficult problem lay in assigning sense-labels to implicit relations. in section 2, we show that, with the pdtb-3, this is no longer the case because non-explicit relations can hold within sentences as well as between them. this in turn motivates a new approach to handle implicit discourse relations in shallow discourse parsing, involving both finding them as well as identifying their senses (section 3). after showing that the sense-distribution of implicit relations within sentences differs from that between them (cf. section 4), we argue that one should be able to take advantage of this fact in sense-labelling these relations. 2 section 5 describes two different ways of doing so, along with a way of dealing with another difference in sense distribution -that of implicit relations that co-occur with explicit relations and implicit relations that do not. while the particular methods used here for sense-labelling may not advance the state-of-the-art, it is the way we use them that should deliver a new baseline for recognizing a fuller range of implicit relations and contribute to the next generation of shallow discourse parsers. 3 ## discourse annotation in pdtb-3 discourse annotation in the pdtb-3 differs from that in the pdtb-2 in two major ways: (1) many more discourse relations are annotated within sentences, and (2) there are changes in the sense hierarchy used in annotating them. while only the first requires changes to shallow discourse parsing, presenting changes to the senses used in annotating relations will allow us to show differences in the distribution of senses associated with different types of implicit discourse relations. ## basic model architecture the sense classifiers for implicit relations used in this paper are based on a basic model whose properties reflect consideration of data size and the interaction between lexical information and structural information. (a full description of the basic model is given in appendix a.) the architecture of basic model is shown in figure 1 . it consists of two lstms @xcite and max-pooling layers, a hidden layer, a dense layer, and a softmax layer. inputs to the model consist of pairs of discourse arguments, each represented as a sequence of word vectors. the output is a probability distribution of the senses between the discourse argument spans. the two sequences of word vectors are encoded by lstms in order to capture positional information within the sequential structure. max-pooling on the output of the lstms is used to compose meaning and reduce parameters for the model, as it has been proven effective in @xcite . modeling the interaction between discourse arguments follows @xcite , who argue that discourse relations can only be determined by jointly analyzing the arguments. in addition, @xcite observed the influence of different configurations on the performance of the model for the implicit sense classification task, suggesting an interaction between the lexical information in word vectors and the structural information encoded in the model itself. we follow them in adopting a 300-dimension word2vec @xcite word embedding and hidden size of 100 for the basic model. as for the 1753 implicits that co-occur with explicit relations, ## inter-and intra-sentential implicits differences in the distribution of implicit relations within sentences and across sentences suggest that we exploit this difference in sense-labelling implicit relations. in this section, we first assume that we know where implicit relations are located within a sentence, so that we can simply consider their arguments. we then present work we have done towards relaxing this assumption. ## conclusion and future work we have shown that recognizing implicit discourse relations as annotated in the pdtb-3 now requires finding them, as well as figuring out what sense relation(s) holds between the arguments. however, we have also shown that the latter task is simplified by differences in the sense distribution of different implicit relations. we still have to develop a way of recognizing precisely where implicit relations hold in those sentences that can be identified as containing them, and a more accurate approach to sense labelling implicit relations that co-occur with explicit ones. we are also interested in whether these different sense distributions hold in other news corpora and other genres. while it is likely not the case that all languages show the same difference in the sense distribution of discourse relations, we would not be surprised if the discourse relations realized within sentences differed from those realized across sentences. in conclusion, we hope that the current effort will contribute to future work on shallow discourse parsing as annotated in the pdtb-3. hyperparameter settings: (rutherford et al., 2017) observed the influence of different configurations on the performance of the model for the implicit sense classification task, suggesting an interaction between the lexical information in word vectors and the structural information encoded in the model itself. to determine the configuration for our model, we trained our model with different combinations of the dimension of word embedding (50, 300) and hidden size (50, 100), and evaluate it on level 2 labels on the wsj section 23.