Learning Semantic Role Labeling from Compatible Label Sequences (2023)

Abstract: Semantic role labeling (SRL) has multiple disjoint label sets, e.g., VerbNet and PropBank. Creating these datasets is challenging, therefore a natural question is how to use each one to help the other. Prior work has shown that cross-task interaction helps, but only explored multitask learning so far. A common issue with multi-task setup is that argument sequences are still separately decoded, running the risk of generating structurally inconsistent label sequences (as per lexicons like Semlink). In this paper, we eliminate such issue with a framework that jointly models VerbNet and PropBank labels as one sequence. In this setup, we show that enforcing Semlink constraints during decoding constantly improves the overall F1. With special input constructions, our joint model infers VerbNet arguments from given PropBank arguments with over 99 F1. For learning, we propose a constrained marginal model that learns with knowledge defined in Semlink to further benefit from the large amounts of PropBank-only data. On the joint benchmark based on CoNLL05, our models achieve state-of-the-art F1’s, outperforming the prior best in-domain model by 3.5 (VerbNet) and 0.8 (PropBank). For out-of-domain generalization, our models surpass the prior best by 3.4 (VerbNet) and 0.2 (PropBank).

## introduction semantic role labeling @xcite aims to understand the role of words or phrases in a sentence. it has facilitated other natural language processing tasks including question answering @xcite , sentiment analysis @xcite , information extraction @xcite , and machine translation @xcite . semantic role labeling can take various forms, each associated with different datasets. predicates ˚work done at the university of utah. as a modeling problem, srl requires associating argument types and phrases with respect to an identified predicate. the two labeling tasks (i.e., verbnet srl and propbank srl) are closely related; but they differ in their treatment of predicates and have disjoint label sets. learning jointly can improve data efficiency across the different labeling srl tasks. a common formulation used to instantiate this idea in prior work is multitask learning (e.g. @xcite : each label set is treated as a separate labeling task, and sometimes also modeled with inter-task feature interaction or consistency losses. while multitask learning often works well in such cases, the loss formulation represents a conservative view over label compatibilities of different tasks. at prediction time, subtask modules still run independently and are not constrained by each other. consequently, decoded labels may violate structural constraints with respect to each other. in such settings, constrained inference (e.g., @xcite has been found helpful. however, this raises the question of how to involve such inference during learning for better data efficiency. furthermore, given the wider availability of propbankonly data (e.g., @xcite , how to efficiently benefit from such data also remains a question. in this paper, we argue that the two disjoint but compatible labeling tasks can be more effectively modeled as one task using their compatibility structures that are already explicitly defined in the form of semlink @xcite . semlink offers mappings between various semantic ontologies including propbank and verbnet. @xcite devised a deterministic conversion from propbank label sequences to verbnet ones using only the unambiguous mappings in semlink. this conversion gives a test bed that has half of the predicates in conll05 srl dataset @xcite with both verbnet and propbank jointly labeled. given this setting, we propose a simple and effective joint crf model for the verbnet srl and propbank srl tasks. in addition to the joint crf, we propose an inference constraint that uses compatible label structures defined in semlink, and show that our constrained inference achieves higher overall srl f1-the average of verbnet and probbank f1 scores-than the current stateof-the-art. indeed, when propbank labels are observed, it achieves over 99 f1 on the verbnet srl, suggesting the possibility of an automated annotation helper. we show that our formulation naturally extends to a constrained marginal model that learns from the more abundant propbank-only data in a semi-supervised setting. when learning and predicting with constraints, it achieves even better srl f1 in out-of-domain generalization. foot_0 2 joint task of semantic role labeling we consider modeling verbnet (vn) and prop-bank (pb) srl as a joint labeling task. given a sentence x, we want to identify a set of predicates (e.g., verbs), and for each predicate, generate two sequences of labels, one for verbnet arguments y v , the other for propbank arguments y p . with respect to vn parsing, a predicate is associated with a verbnet class that represents a group of verbs with shared semantic and syntactic behavior, thereby scoping a set of thematic roles for the class. similarly, the predicate is associated with a propbank sense tag that defines a set of pb core arguments along with their modifiers. a example is shown in tab. 1. we treat predicate classification and argument labeling as separate tasks and focus on the latter. 2 assuming predicates u and their associated vn classes η and pb senses σ are given along with x, we can write the prediction problem as: ## a joint crf model to eliminate the errors discussed in sec. 2.2, we propose to model the disjoint srl tasks using a joint set of labels. this involves converting multitask modeling into a single sequence labeling task whose labels are pairs of pb and vn labels. doing so not only eliminates the bio inconsistency, but also exposes an interface for injecting semlink constraints. our model uses roberta @xcite as the backbone to handle textual encoding, similarly to the srl model of @xcite . at a high level, we use a stack of linear layers with gelu activations @xcite to encode tokens to be classified for a predicate. for the problem of predicting arguments of a predicate u, we have an encoding vector ϕ u,i for the i-th word in the input text x. here, map sums up word-piece embeddings to form a sequence of word-level embeddings, the functions f u and f a are both linear layers, and f ua denotes a two-layer network with gelu activations in the hidden layer. we use a dedicated module of the form in eq. 4 for the vn and pb subtasks. this gives us a sequence of vectors ϕ v u for vn and a sequence of vectors ϕ p u for pb. next, we project the vn and pb feature sequences into a |y v | ˆ|y p | label space: here, g is another two-layer gelu network followed by a linear projection that outputs |y v | |y p | scores, corresponding to vn-pb label pairs. the final result z u denotes a sequence of vn-pb label scores for a specific predicate u. in addition, we use a crf as a standard first-order sequence model over z u (treating it as emission scores), and use viterbi decoding for inference. the training objective is to maximize: reduced joint label space. we use the crossproduct the two label sets, prefixed with a bio prefix foot_2 . a brute-force cross product leads to a in practice, it is important to keep the joint label space at a small scale for efficient computation, especially for the crf module. therefore, we condense it by first disallowing pairs of the form (b-*, i-*) and predicateto-argument pairs. the former enforces that the verbnet arguments do not start within probbank arguments, while the latter ensures that the predicate is not part of any argument. next, we observe the co-occurence pattern of vn and pb arguments, disabling semantically invalid pairs such as (theme, argm-loc) 4 . this reduces the label space by an order of magnitude (from 144 ˆ105 " 15, 120 to 685). ## semi-supervised learning with propbank-only data compared to data with both verbnet and prop-bank fully annotated, there is more data with only propbank labeled. the semlink corpus helps in unambiguously mapping " 56% of the conll05 data @xcite . therefore, a natural question is: can we use propbank-only data to improve the joint task? here, we explore model variants based on the joint crf architecture described in sec. 3. we will focus on modeling for propbank-only sentences. ## inference with semlink here we discuss the implementation of the y seml u . remember that each pair of verbnet class η and propbank sense σ uniquely determines a set of joint argument labels for the predicate u. for brevity, let us denote this set as semlpuq (e.g., tab. 1). eventually, we want the viterbi-decoded label sequence to comply with semlpuq. that is, this constraint can be easily implemented by a masking operation on the emission scores of the joint crf, thus can be used at both training and inference time. during inference, it effectively ignores those label sequences with semlink violations during viterbi decoding: in sec. 6, we will show that using eq. 10 always improves the overall srl f1 scores. ## experiments in this section, we aim to verify whether the compatibility structure between verbnet and propbank (in the form of semlink) has a positive impact on their sequence labeling performances. ## analysis we report statistical metrics in sec. 7.1. in sec. 7.2, we analyze the use of constrained learning on the jointly labeled data. ## limitations towards fully end-to-end parser. our model architecture is on the track of end-to-end srl parser, but it still assumes gold predicate positions and predicate attributes are given. a fully end-to-end parser can take sentences in raw text and output disjoint label sequences. while doing so can make computation less efficient (e.g., requiring substantially larger memory for training), it can bring users convenience. involving document context. @xcite showed that using neighboring sentence prediction with transformer positively impacts parsing f1. in contrast, we assumed sentences in the corpus are independent. why does propbank seem more difficult? we hypothesized the reason to be less granularity in argument labels and more ambiguous label assignments. as mentioned in sec. 3, prior work benefited from using dedicated label text/definition as an auxiliary input. we only used such features at the predicate level, implicitly trading off potential gains on pb f1 for more efficient computation. marginal model's capacity at handling constraints. in this paper, we focused on the sem-link constraint for compatible label sequences. there is a broad spectrum of srl constraints in prior work @xcite , some of them do not easily fit in the marginalization formulation, such as the unique core role constraint.