Modal Dependency Parsing via Biaffine Attention with Self-Loop (2025)

Abstract: A modal dependency structure represents a web of connections between events and sources of information in a document that allows for tracing of who-said-what with what levels of certainty, thereby establishing factuality in an event-centric approach. Obtaining such graphs defines the task of modal dependency parsing, which involves event and source identification along with the modal relations between them. In this paper, we propose a simple yet effective solution based on biaffine attention that specifically optimizes against the domain-specific challenges of modal dependency parsing by integrating self-loop. We show that our approach, when coupled with data augmentation by leveraging the Large Language Models to translate annotations from one language to another, outperforms the previous state-of-the-art on English and Chinese datasets by 2% and 4% respectively.

## introduction at a time when we find ourselves inundated with endless streams of new information and knowledge, being able to identify a source of information and the confidence level with which it is conveyed is often helpful-if not sometimes critical-for better understanding the context behind a text or discourse. modal dependency structure (mds) @xcite is designed with such representation in mind, where the events and the sources (also known as concievers 1 ) take the center stage as the vertices of the graph, while the edges denote (1) source of factualiy via its direction and (2) level of certainty via its label, which is a combination of 3 modal strengths (full, partial, and neutral) and 2 polarities (affirmative and negative) based on the annotation scheme from factbank @xcite . 1 in what follows, 'conceivers' are preferred over 'sources.' according to her, they had not paid for their ring." figure 1 shows an example of a modal dependency graph for a sample document of two sentences: an abstract root node at the top ensures that the structure is single-rooted. immediately below is an abstract author node, whose presence is implicitly presumed for every document as its creator. in general, an mds typically shows heavy traffic through the author node as a principal conceiver of various events in the document. in the first sentence, the author states that may's accusation did take place without any hesitation. this is represented in the mds with a ':full-affirmative' edge between the author and the 'accused' event node. if it were later revealed that may in fact never accused the couple of shoplifting, we may conclude that the author is responsible for providing the false information. the author then relays a comment made by may in the second sentence, indicating that the cited statement does not necessarily reflect the author's point of view. this form of reporting, wherein the author (or some conceiver) simply reiterates information from another source, is an important phenomenon in the mds that is represented with a conceiver-to-conceiver triple ('author' :full-affirmative 'may'). since it is may who claims that shoplifting by the couple did happen (because it is her allegation) and their paying for the ring did not, the event nodes 'shoplifting' and 'pay' are subsequently linked to 'may' conceiver node with ':full-affirmative' and ':full-negative' edge labels respectively. figure 2 illustrates this point further by comparing a simple statement (figure 2a ) against a quotation (figure 2b ). in figure 2a , the author does not rely on any external sources; hence, the author is the conceiver of the events 'declined' and 'request'. in addition, the use of the term 'likely' indicates that the author is only partially certain about the 'declined' event, justifying the assignment of the modal edge label ':partial-affirmative.' in contrast, figure 2b shows the same sentence as a quotation made by dr. kim. it follows that the events embedded within the quotation 'declined' and 'request' should be connected to 'dr. kim' who is the author of the quoted statement and hence the conceiver of the said events. it is important to note that, while april serves as the protagonist in both instances, she is not regarded as a conceiver because she is not cited in any capacity. in order to obtain such a modal dependency tree foot_0 from text, modal dependency parsing (mdp) needs to perform a few different tasks. first, spans of events and conceivers must be identified and labeled accordingly. second, the modal relations must be established for the identified spans by predicting the correct modal arc and edge label. to tackle this problem, we present a simple yet effective solution in the form of a biaffine attention with added support for self-loop. the merits of our approach are as follows: we further experiment with data augmentation by leveraging the large language models (llms) to translate annotations in english or chinese into the other language while preserving the annotated spans of modal nodes and edges, resulting in a significantly increased number of training samples. our experiments show that the proposed approach significantly outperforms the previous state of the art by 2% for english and 4% for chinese. the code is available at https://github. com/umr4nlp/mdp_biaffine . ## related work traditionally, event factuality prediction (efp) was seen as a classification or regression problem that involved rule-based @xcite or statistical approaches @xcite @xcite . with widespread adoption of deep learning came a surge of neural models tackling this problem, for instance based on lstms @xcite , gans @xcite or gnns @xcite . @xcite is the first work that casted efp as modal dependency parsing and reported baseline results on english, while releasing the crowdsourced dataset which is publicly available foot_1 . this was followed up by a prompt-based model @xcite with the first reported results on chinese mdp trained on annotations from @xcite , along with an incremental improvement for english. with the recent integration of mds into uniform meaning representation (umr) (van gysel et al., 2021), the prompt-based model has also been used as part of the umr parsing pipeline @xcite . this implies that improved modal dependency parsing performance can have beneficial downstream impact for umr parsing. given the status of the prompt-based model as the current state-of-the-art in mdp, we briefly summarize its core setup. here an event and its sentence-known as prompt-is paired with some local context as defined by the number of sentences before, including, and after the prompt sentence. the parser is then trained to predict the event's parent and grand-parent conceivers from the context sentences, based on the simplifying assumption that an event has a chain of one or two conceivers 96% of the time @xcite 4 . while this is a prudent approach that alleviates the multi-tasking complexity of the parsing process by first focusing on events whose definition is more widely accepted and less context-dependent than conceivers, it is susceptible to error propagation during decoding due to its pipelined setup. furthermore, since each event requires an individual forward pass over the local context, both training and decoding can be slow, particularly when processing lengthy documents with numerous event candidates. finally, the simplifying assumption that enables the approach also restricts the model from generating certain forms of modal structures, reducing its generalization capabilities. in contrast, our proposed biaffine parser has a simpler end-toend setup that requires just a single forward pass over the entire document and does not rely on any simplifying assumptions. this line of approach based on the deep biaffine scoring mainly traces its roots to dependency parsing @xcite @xcite but has also been explored in other areas such as ner tagging @xcite and constituency parsing @xcite . to the best of our knowledge, however, no existing research has examined the utility of modeling the self-loop within the biaffine-based parsing framework. as a result, previous approaches have primarily relied on a pipeline framework @xcite . although these efforts successfully established a strong baseline performance, they fall short of fully capturing the complexity of modal dependency parsing due to the simplifying assumptions auth null "april likely declined the request," dr. kim said. full-affirmative full-affirmative partial-affirmative full-affirmative span conceiver none none conceiver event none event conceiver conceiver event that constrain their structural expressiveness, such as constraining any subtree to a depth of 2. this study is an attempt to develop a streamlined yet comprehensive framework capable of fully representing the modal dependency relations without relying on any simplifying assumptions. although a biaffine parser is seen as an effective solution for dependency parsing in various contexts, its ability to fully address the complexities of modal dependency parsing is only realized when support for self-loop is incorporated. this is because while the biaffine parser has been conventionally used to generating arcs between different tokens, the addition of a self-loop enables it to additionally behave as a tagger, which enables more effective handling of conceiver node identification while eliminating the need for a pipelined setup. as illustrated in figure 2 , conceiver identification is a highly contextdependent sub-task that significantly benefits from access to the overall structural information. our approach of training the biaffine module to be both a parser and a tagger provides the flexibility to leverage partially constructed parse graphs in the decoding process that is not afforded when using a separate tagger. in what follows, we provide details on model setup and architecture. setup building on the fundamental assumption that a modal dependency structure is inherently a tree and therefore single-headed @xcite , our parser employs the biaffine attention to locate the most suitable head for each token and to label the newly formed edges. this is a natural way of modeling the modal arcs and relations between different nodes-sub-tasks (3) and (4)-which is similar to that of a traditional dependency parsing configuration. these arcs and relation labels are drawn on top of the tokens in figure 3 . however, not all phrases act as events or conceivers within the modal dependency structure. this is why mdp can be understood as a form of sparse dependency parsing, and the non-modal tokens must be pruned first. to this end, we introduce self-loop for every token. since self-loops are structurally self-evident, it is only the predicted edge label that is of interest, chosen from three possible options: 'event', 'conceiver' or 'none'. when a token is labeled 'none', it is neither an event or a conceiver and is hence subject to be be pruned. otherwise, the selfloop label is used to classify between an event and a conceiver-sub-task (2)-as seen with the edges below each token in figure 3 . meanwhile, dependency structure is arguably not the most intuitive method of representing multiword spans, because some arbitrary token must be raised as a head to ensure structural integrity. this is also an issue for mdp as an event or a concept may extend across multiple words but must be treated as a single node in the mds. we address this by assuming that the leftmost token is the representative head in our modeling where there are multiple words in a node foot_3 . any token to the right within a multi-word span should hence be headed by the leftmost token with a special label of ':span,' which triggers a special interpretation during decoding for a flat structure. the key strength of this solution lies in its consistency with the single-headedness of mds, while facilitating the use of the existing biaffine arc generator for span identification-sub-task (1)-without any modifications. for instance, 'kim' points to 'dr.' in figure 3 with ':span' edge label. therefore, any relation to and from 'dr.' should be structurally interpreted as involving the span of 'dr. kim' as a whole. model architecture our proposed model network consists of two modules: (1) a document encoder based on a pre-trained language model (plm), followed by (2) a biaffine module consisting of two biaffine layers that predict arc presence and edge labels respectively. figure 4 visualizes the network architecture of our approach. one of the major bottlenecks faced by the document encoder is the input length limitation imposed by the choice of the plms. this is especially relevant as the typical mdp text input is generally a multi-sentence document. we cope with this challenge by splitting a long input sequence into smaller segments that are embedded independently, before being merged together @xcite . as such, we find that it is advantageous to choose a plm with a long context window which can thereby reduce the frequency of such sentence fragmentation during encoding. the contextualized embeddings from the document encoder are first projected into arc and relation hidden states by the biaffine parser, shown as 'h_arc' and 'h_rel' in figure 4 respectively. then arc and relation scores 's_arc' and 's_rel' are produced by the biaffine classifier that considers the arc and relation hidden states of any two positional tokens. while typical biaffine mechanism only considers two different positions, our model also considers self-loop. since self-loops are self-evident edges, only the relation label needs to be predicted. these self-loop label scores are denoted as 'sl_rel' in the figure. ## data augmentation with llm due to the challenging and costly nature of the mds annotation process via crowd-sourcing @xcite , the number of annotated documents has remained the same since the initial release. in an effort to artificially increase the number of training samples, we obtain silver data by leveraging the llms to translate annotated examples across languages. throughout this process, we strive to maintain the spans of annotated modal events and conceivers, because the edges are automatically retained as long as the source and target vertices exist in the translation. this is fundamentally a two-step process. for instance, consider a sample sentence from figure 2b : "april likely declined the request." first, the llm translates the raw english sentence to chinese ("april可能拒绝了请求。"). then, in a separate conversation context, the llm is given an augmented english sentence as shown in figure 5 as 'src', where the two events 'declined' and 'request' are surrounded by the special markers 'e1' and 'e2' respectively foot_4 . the llm is then instructed to insert the same set of special markers 'e1' and 'e2' in the chinese translation. figure 5 shows a successful completion of our data augmentation pipeline, where 'tgt' contains not only the correct chinese translation but also the special markers that denote the same events in english. hence, any edges in the english annotation that use these events as endpoints will be preserved. for instance, the modal edge ('author' :full-affirmative 'request') from figure 2b can now be mapped to ('author' :full-affirmative '请 求'), since both 'request' and '请 求' are tagged by the same special marker 'e2' in figure 5 . figure 6 visualizes the final annotation in chinese. in practice, the llm is instructed to translate a document sentence-by-sentence in a series of backand-forths that keeps track of its previous translations. we hypothesize that access to the entire document text, along with its prior translations, can enhance the translation quality by ensuring consistent mapping of named entities across sentences. moreover, sentence-level translation allows the model to focus on only a limited number of tags at a time during the tagging phase, potentially increasing the likelihood of preserving special markers. see figure 8 in appendix d for an illustration. ## analysis for english mdp, the conceiver identification still remains a major bottleneck as a highly contextdependent task that may span across multiple sentences. nevertheless, the improvement in conceiver identification by our proposed approach is evident with at least 2% gain across the board. since both the prompt-based model and ours share the generally same approach of first prioritizing event detection followed by conceiver identification, we attribute the increase to the biaffine classifier as well as the customized decoding process. to test this hypothesis, we perform an experiment with a simple decoding that does not use the bottom-up decoding described in section e. rather, we accept the self-loop label predictions as-is and generate the nodes in a single step, which is more akin to the prompt-based setup. we observe a slight dip in conceiver identification by 0.7 and 0.3 in dev and test sets respectively, which suggests that our customized decoding is indeed beneficial to the overall parsing performance. the experimental results also show higher improvement in chinese compared to english. since the prompt-based model consistently shows lower conceiver identification scores for chinese as well, the earlier discussion of the efficacy of customized decoding process is again relevant. in fact, it is important to emphasize that our approach is fundamentally language-agnostic aside from the choice of pre-trained language models. this is in contrast to the prompt-based model, where a language-specific parameter for determining the size of the context window needs to be manually adjusted @xcite based on the statistics shown in figure 7 , showing a bigger portion of modal heads in prior sentences for chinese. compared to english which is more clustered around the current sentence, it follows that the chinese modal dependency parser should be able to generate comparatively long-distance edges. indeed, on the development set, the average number of sentences crossed by the biaffine classifier is 0.2 for english and 0.78 for chinese. although the promptbased model comes similar for english at 0.18, it also favors generating edges that are significantly shorter for chinese, at 0.62. in terms of computational efficiency foot_5 , the prompt-based model takes about 30 minutes per epoch during training and 5 minutes for inference on the development and test set. the biaffine parser in comparison is considerably faster, requiring only 10 minutes per training epoch and 2 minutes for inference. it is also worth noting that the promptbased model further requires a separate training phase for the event tagger. ## conclusion this work presents a biaffine modal dependency parser that is simple yet effective. by incorporating self-loop, our proposed approach is able to fully and efficiently address the multi-tasking nature of modal dependency parsing. we also show that training on silver data generated by using the llms to translate the annotated samples from one language to another leads to improved performance. the model is evaluated on the english and chinese datasets and in both instances outperform the previous state-of-the-art.