Semantic Role Labeling as Dependency Parsing: Exploring Latent Tree Structures inside Arguments (2022)

Abstract: Semantic role labeling (SRL) is a fundamental yet challenging task in the NLP community. Recent works of SRL mainly fall into two lines: 1) BIO-based; 2) span-based. Despite ubiquity, they share some intrinsic drawbacks of not considering internal argument structures, potentially hindering the model’s expressiveness. The key challenge is arguments are flat structures, and there are no determined subtree realizations for words inside arguments. To remedy this, in this paper, we propose to regard flat argument spans as latent subtrees, accordingly reducing SRL to a tree parsing task. In particular, we equip our formulation with a novel span-constrained TreeCRF to make tree structures span-aware and further extend it to the second-order case. We conduct extensive experiments on CoNLL05 and CoNLL12 benchmarks. Results reveal that our methods perform favorably better than all previous syntax-agnostic works, achieving new state-of-the-art under both end-to-end and w/ gold predicates settings.

## introduction semantic role labeling (srl) is a fundamental yet challenging task in the nlp community, involving predicate and argument identification, as well as semantic role classification. as srl can provide informative linguistic representations, it has been widely adopted in downstream tasks like question answering @xcite , information extraction @xcite , and machine translation @xcite , etc. recent works of srl mainly fall into two lines: 1) bio-based; 2) span-based. the former views srl as a sequence labeling task @xcite @xcite . for each predicate, each token is tagged with a label starting with bio prefixes indicating if it is at the beginning, inside, or outside of an argument. the latter @xcite @xcite , in contrast, opts to jointly predict all predicate and argument span pairs using a spangraph formulation. despite ubiquity, there are some drawbacks that limit the expressiveness of the two methods. first, framing predicate-argument structures as a biotagging scheme is less effective as it lacks explicit modeling of span-level representations, so that long adjacencies of argument phrases can be ignored @xcite @xcite . second, span-based method seeks to pick very few (typically ă10%) positive examples from opn 3 q candidate predicateargument pairs, thus suffering from severe class imbalance problem @xcite . to alleviate this issue, span-based method relies on heavy pruning @xcite to reduce the searching space, potentially impairing the performance. meanwhile, both formulations share some common flaws in terms of lacking explicit modeling of internal argument structures, which appear to be beneficial to srl. taking fig. 1 as an example, internal dependencies of words ("in other european markets") inside the span provide strong clues for recognizing it as a locative modifier ("am-loc") of the predicate "closed". besides, the predicateargument relation can be naturally reflected by the dependency from the predicate to the span headword ("closed am-loc ý ýýýý ñ in"), and we can properly recognize the argument span boundaries by retrieving all descendants of the subtree. such observa-tions have motivated many attempts on utilizing relations inside arguments @xcite @xcite . however, stuck on the fact that span-style srl has no determined internal structure realizations, existing works have to resort to making use of external human-annotated syntax knowledge to bridge the gap @xcite . our main goal in this work is to explicitly take internal argument structures into account meanwhile keeping our framework end-to-end. to this end, we propose to model flat arguments as latent subtrees, thus paving the way for reducing srl to dependency parsing seamlessly: we view predicateargument structures as partially-observed trees where exact subtrees for each argument are not realized yet. in this way, we reframe span-style srl as parsing word-to-word relations by encoding all predicate-argument relations into a unified dependency graph. unlike span-based methods @xcite , a dependency graph contains no more than opn 2 q possible dependencies, so that the class imbalance issue can be side-stepped effortlessly. specifically, we make use of treecrf @xcite , which provides a viable way for probabilistic modeling of tree structures, to learn the partially-observed trees and marginalize the latent structures out during training. unlike canonical treecrf, which enumerates all possible trees, in our setting, we have to impose many span constraints to reflect the argument boundaries on subtrees correctly. to accommodate this, we further design a novel span-constrained treecrf to adapt it to our learning procedure, which explicitly prohibits invalid edges across different arguments as well as multi-head subtrees @xcite . there are further advantages to our reduction. conversion to tree structures enables us to easily conduct global optimization @xcite in polynomial time, which has already been shown to often lead to improved results and more meaningful predictions @xcite @xcite compared to local unconstrained methods. on the other hand, by drawing on the experience in the parsing literature, we can further extend our method to some well-studied high-order methods @xcite without any obstacle. we experiment with sibling factors in this work and find significant gains, in line with many parsing works @xcite . our contributions can be summarized as follows: foot_0 • aware of the benefits of internal argument structures, we propose to model flat argument spans as latent subtrees, thereby reducing srl to dependency parsing seamlessly. ## overview in span-style srl, an argument of a predicate corresponds to one word or multiple continuous words. in the latter case, each word in the argument span is treated as equal, and the internal structure of a multi-word argument, i.e., the relationship between words inside the argument, is usually overlooked due to the lack of corresponding annotations. in this work, we propose to explicitly model internal structures of multi-word arguments and treat arguments as latent subtrees. our approach deals with each predicate separately, and assumes each corresponds to a single-root tree. consequently, each argument subtree is attached to the predicate. during the training process, all possible structures are enumerated and accumulated to compose the argument representation. while decoding, we seek to find a 1-best tree and recover arguments from the subtrees belonging to the resulting structure. we highlight four key points. they1 want2 to3 do4 more5 .6 a0 a1 (a) original structure: arguments of the predicate are located in the upper half-plane of the sentence and do not overlap with each other. they1 want2 to3 do4 more5 .6 prd a0 a1 ∅ (b) training: convert the predicate-argument structure to a dependency tree with (dotted) latent annotations; non-argument spans are assigned "∅" for distinction. they1 want2 to3 do4 more5 .6 prd a0 a1 ∅ (c) decoding: realize a tree rooted at the predicate with the arc labeled as "prd"; (dashed) arcs labeled as "∅" are discarded. they1 want2 to3 do4 more5 .6 prd a0 a1 (d) recovery: collapse all (dashed) subtrees governed by the predicate into flat argument spans. figure 2 : illustration of our srlñtree conversion (fig. 2a and fig. 2b ), and its inverse treeñsrl process (fig. 2c and fig. 2d ). we emphasize the predicate "want" in the figures for clarity. the two arguments with roles "a0" and "a1" are framed by red and blue rectangles, respectively. ## methodology now we elaborate the architecture of our proposed model for training the parser. following @xcite ; @xcite , our model consists of a contextualized encoder and a (secondorder) scoring module. we further propose a spanaware treecrf to compute the probabilities of the converted partially-observed trees. ## experiments we measure our proposed first-order crf and second-order crf2o models on two srl benchmarks: conll05 and conll12. full implementation details are given in § a. data ## discussions and future works the basic idea of this work is to mimic srl structures with a combination of multiple latent trees. this new perspective sheds light on some natural extensions of our work to other tightly related semantic parsing tasks, e.g., amr @xcite and ucca @xcite . 10 tasks fall into this type exhibit very flexible graph representation schemes (e.g., reentrancy and disconti-nuity) @xcite , which are intractable by principled decoding algorithms like dynamic programming. we believe that employing structured inference in spirit of our approaches can provide considerable help in getting rid of greedy span/dependency selections and finding globally optimal structures. we prefer to reduce srl to dependency-based tree parsing rather than another paradigm, i.e., constituency parsing, partly because dependencies provide a more transparent bilexical governordependent encoding of predicate-argument relations @xcite . we also do not pursue the way of jointly modeling dependencies and phrasal structures with lexicalized trees @xcite @xcite as our approach enjoys a lower time complexity of opn 3 q. nonetheless, we admit potential advantages of this kind of modeling @xcite and leave this as our future work. there are other interesting perspectives deserve further explorations: given that span-style srl substantially benefits from our formulation of recovering srl structures from trees, can the induced dependency trees learn plausible syntactic structures? or in other words, can they agree with linguisticmotivated annotations @xcite ? we conduct thorough analyses in spirit of @xcite ; @xcite and give affirmative answers. due to space limitations, we refer readers to § d and § e for details. ## conclusions in this paper, we propose to reduce span-style srl to dependency parsing by viewing flat phrasal arguments as latent subtrees, and design a novel spanconstrained treecrf to accommodate the span structures. taking inspirations from the parsing literature, we also build a second-order extension and find further gains. our models are syntax-agnostic and do not rely on any external linguistic syntax knowledge. experimental results show that, our proposed methods outperform all previous comparable works, achieving new state-of-the-art on both conll05 and conll12 benchmarks. extensive analyses confirm that our approach enjoys some merits of global structural constraints, meanwhile maintaining acceptable time complexity. furthermore, we find our modeling of latent subtrees provides effective assistance in terms of long-range dependencies and global consistency.