Modal Dependency Parsing via Language Model Priming (2022)

Abstract: The task of modal dependency parsing aims to parse a text into its modal dependency structure, which is a representation for the factuality of events in the text. We design a modal dependency parser that is based on priming pre-trained language models, and evaluate the parser on two data sets. Compared to baselines, we show an improvement of 2.6% in F-score for English and 4.6% for Chinese. To the best of our knowledge, this is also the first work on Chinese modal dependency parsing.

## introduction modal dependency parsing (mdp) is the task of parsing a text into a modal dependency structure (mds) @xcite in which each event in the text is linked to a conceiver, the information source of the event. an mds is a graph in which the nodes are events and conceivers, and the edges represent the level of certainty that a conceiver holds with respect to the event. an example mds is presented in figure 1 , where jeroen weimar is the conceiver of the event travelled, and is certain that the traveling event has happened, as indicated by the edge label pos. @xcite define 6 categories of modal strength, or levels of certainty, and they are full positive (pos), partial positive (prt), positive neutral (neut), negative neutral (neutneg), partial negative (prtneg) and full negative (neg). the root node of an mds is always the author (author) of a document, the ultimate source of all information sources mentioned in the text. modal dependency parsing is thus the task of taking a text as input and parsing it into a modal dependency structure. mdp departs from previous approaches to event factuality prediction that cast it as an event classification (e.g. @xcite ) or regression (e.g. @xcite ) problem aimed at just predicting the level of certainty of an event. the level of certainty alone is insufficient in judging the factuality of an event, and knowing the information source (conceiver) is also crucially important. for example, in figure 1 , our judgment of whether the event travelled has happened also crucially depends on the credibility of the information source, jeroen weimar, in addition to the level of certainty the information source holds towards the event. the victorian government's covid-19 response commander jeroen weimar confirmed 'this individual has travelled to melbourne.' " @xcite develop the first modal dependency parser by first separately extracting events and conceivers, then building up the mds bottomup with a ranking model. one shortcoming of this approach is that it fails to capture the fact that the status of an entity as a conceiver is conditioned on its being the information source of an event. for instance, in figure 1 , a person is an entity but is not a conceiver as it is not the source of any event. as a result, @xcite report relatively low conceiver extraction f-score compared to event extraction (70.4% for conceiver extraction vs. 90.8% for event extraction). errors in conceiver extraction will propagate to the structure building stage, leading to lower overall mds parsing accuracy. in this paper, we describe an approach to mdp based on language model priming in which we construct a prompt with an event and use it to predict its chain of conceivers as well as the level of certainty the conceivers hold foot_0 . this approach avoids the error propagation problem in @xcite and also takes better advantage of powerful pretrained language models. our experiments show that this approach outperforms previous models for both english and chinese. ## approach we approach mds parsing by first performing event extraction, then use the extracted events to construct the prompt for the purpose of identifying their conceivers. given a document, a language model such as bert @xcite is used to obtain the contextualized representation for each token. a standard bio tagging model is then applied to identify events, where b, i, o refer to the beginning, inside, and outside of an event respectively. the next step in mds parsing is to identify conceivers for each extracted event. more formally, given an event e i as a child node, the task is to extract (e i , c i , cc i ), where c i is the conceiver of e i , and cc i is the conceiver of c i . in theory, a child event can have a chain of conceivers longer than two, but in over 96% of the cases, an event has a chain of two conceivers or less. we thus made the simplifying assumption that an event can have a chain of two conceivers at most. our model receives an event-specific text sequence as input, then predicts a tag from a target set {b-c, i-c, b-coc, i-coc, o} for each token in the sequence. b-c and i-c labels are for tokens in c i , and b-coc, i-coc are for tokens in cc i . we construct the event-specific sequence, seq i , by concatenating a prompt and a context sequence in the form of [cls] a prompt [sep] a context sequence @xcite . let s i denote the sentence containing e i , we add token @xmath0, </@xmath1. for a child event e i , its parent conceiver can usually be found within a window surrounding s i . thus, the context sequence for e i is constructed by taking the surrounding sentences of s i in a window, followed by two special @xmath2. figure 2 shows an example of the input sequence seq i with gold tags. the input sequence seq i is then encoded with a pre-trained language model. let @xmath3, ..., h m ) denote a sequence of contextualized representations for the input tokens in seq i , the score for the tag of the j-th token is: to learn the edge label between a child node and its parent node, we use a separate feed-forward neural network to map h j to the edge label set. the edge label set includes the modal relations in the data set plus the n/a label, which is chosen when there is no relation between the child node e i and token j, i.e. when token j is neither part of the conceiver of e i nor part of the conceiver of the conceiver of e i . the score for the edge label of the j-th token is: in the training phase, we minimize the following cross-entropy loss: inference in an mds, each child node only has one parent node. to enforce a well-formed mds, we apply two rules in the inference stage: (i) if more than one conceiver is predicted for e i , the first prediction is taken; (ii) if a conceiver doesn't have a conceiver, by default it is attached to the author with the majority label in the data set. ## related work early works cast event factuality prediction (efp) as a classification or regression problem and have employed rule-based @xcite or machine learning approaches @xcite @xcite . more recently, different types of neural models have been applied to this problem, such as lstm-based rnns @xcite , generative adversarial networks @xcite , or graph neural networks @xcite . @xcite and @xcite extended the sentence level task to document-level efp. our work is most closely related to that of @xcite , which casts efp as modal dependency parsing. however, they first extract events and conceivers and then build the mds by ranking the candidate parents for each event. in contrast, we perform modal dependency parsing by constructing a prompt with the event to predict its conceiver parent, simplifying the pipeline. our prompt-based approach also bears resemblance to works applying prompt-based learning to other nlp tasks, such as event extraction @xcite , relation extraction @xcite , named entity recognition @xcite and coreference resolution @xcite . ## conclusion in this paper, we propose a model for modal dependency parsing based on priming pre-trained language models. we evaluate the model on an english modal dependency data set, and for the first time, evaluate the model on a chinese modal dependency data set. experimental results show that our model consistently outperforms baselines on both data sets. the models for 3 runs with different seeds, and report the average f-scores across runs. each epoch takes about 45 minutes for english, 19 minutes for chinese.