Comparing Span Extraction Methods for Semantic Role Labeling (2021)

Abstract: In this work, we empirically compare span extraction methods for the task of semantic role labeling (SRL). While recent progress incorporating pre-trained contextualized representations into neural encoders has greatly improved SRL F1 performance on popular benchmarks, the potential costs and benefits of structured decoding in these models have become less clear. With extensive experiments on PropBank SRL datasets, we find that more structured decoding methods outperform BIO-tagging when using static (word type) embeddings across all experimental settings. However, when used in conjunction with pre-trained contextualized word representations, the benefits are diminished. We also experiment in cross-genre and cross-lingual settings and find similar trends. We further perform speed comparisons and provide analysis on the accuracy-efficiency trade-offs among different decoding methods.

## introduction semantic role labeling (srl) is a core natural language processing (nlp) task that aims to identify predicate-argument structures in text @xcite . following the neural encoder-decoder paradigm, we can view an srl model as combining an encoder, which builds hidden representations for the input words, with a decoder, which extracts the argument spans based on the encoded representations. while recent srl models achieve high performance on popular benchmarks @xcite @xcite @xcite , most improvements come from better neural encoders, such as the transformer @xcite and pre-trained contextualized word representations, such as bert @xcite . however, influence on end-task performance due to the choice of decoder has become less clear. ## model for a given predicate, foot_1 srl aims to extract all argument spans and assign them role labels. to model this task, we follow the neural encoder-decoder paradigm: the encoder produces hidden representations for the input words, upon which the decoder decides the structured outputs. all our models adopt the same encoding architecture: a deep transformer encoder @xcite , which has been shown effective for srl @xcite . for a given input sequence of words {w 1 , . . . , w n }, we obtain their contextualized representations {h 1 , . . . , h n } from the encoder. upon these, we stack different decoders to extract the argument spans corresponding to different extraction strategies, which will be described in the following. ## related work argument extraction before the incorporation of end-to-end neural models, traditional srl systems usually depend on input constituency trees to obtain argument candidates @xcite . although straightforward, this may suffer from error propagation from syntax parsers. recent neural systems utilize endto-end models to solve the task. casting srl as bio-based sequence labeling problem is the most common decoding scheme and can obtain impressive results @xcite @xcite @xcite . on the other hand, span-based methods @xcite directly select and label among argument span candidates. this is actually similar to the traditional approaches, though the argument candidates are obtained by the model rather than from input syntax trees. in addition to span-based srl, the focus of this work, there is another category of dependency-style srl, which only requires the extraction of head words of argument spans @xcite . inspired by this, for span-based srl, we can extract argument head words as the first step and then expand to the full spans in a second step. this idea has also been applied in information extraction, such as coreference resolution @xcite , entity detection @xcite and event argument extraction @xcite . another interesting direction is considering the structured constraints of the arguments, including works on integer linear programming @xcite , dynamic programming @xcite and structure-aware tuning @xcite . syntax and srl there has been discussion of the relation between syntax and srl @xcite , considering the close connections between these two tasks. though syntax trees are usually the inputs to traditional srl systems, some recent works find that syntax-agnostic neural models also work well @xcite . nevertheless, with recent neural models, syntax information has still been found helpful for srl in various ways, including multi-task learning @xcite , argument pruning @xcite , and tree-based modeling @xcite @xcite . in this work, our "headsyntax" decoder incorporates syntax in a partial way, utilizing dependency trees to decide the head words in training. this method indeed performs the best overall if only adopting static word embeddings. however, the incorporation of bert features diminishes the advantages. this indicates that bert may already cover much of the syntactic (surface) features of the input sentences, as suggested by recent works on bert interpretation @xcite @xcite . cross-lingual srl there has also been increasing interest in cross-lingual transfer for srl, where data transfer and model transfer are the main approaches. data transfer usually depends on translation and annotation projection to obtain training resources for target languages @xcite @xcite @xcite . on the other hand, model transfer techniques directly reuse an srl model trained on source languages to transfer to target languages @xcite , based on common representations. in particular, the recent development of multilingual neural representations, such as multilingual bert, has been shown to be effective for cross-lingual transfer @xcite . in this work, we explore a simple zero-shot unlabeled setting for cross-lingual srl. we leave more explorations on this to future work. ## conclusion in this work, we empirically compare several span extraction methods for srl. extensive results show that in fully supervised settings, simple biotagging is a robustly good option when utilizing bert features. similar trends are also found in cross-genre and cross-lingual settings. we also analyze the accuracy-efficiency trade-offs for different decoders; although methodologically more complex, two-step approaches are still efficient in decoding. future work could explore other nlp tasks that require extracting textual spans.