Fast and Accurate End-to-End Span-based Semantic Role Labeling as Word-based Graph Parsing (2022)

Abstract: This paper proposes to cast end-to-end span-based SRL as a word-based graph parsing task. The major challenge is how to represent spans at the word level. Borrowing ideas from research on Chinese word segmentation and named entity recognition, we propose and compare four different schemata of graph representation, i.e., BES, BE, BIES, and BII, among which we find that the BES schema performs the best. We further gain interesting insights through detailed analysis. Moreover, we propose a simple constrained Viterbi procedure to ensure the legality of the output graph according to the constraints of the SRL structure. We conduct experiments on two widely used benchmark datasets, i.e., CoNLL05 and CoNLL12. Results show that our word-based graph parsing approach achieves consistently better performance than previous results, under all settings of end-to-end and predicate-given, without and with pre-trained language models (PLMs). More importantly, our model can parse 669/252 sentences per second, without and with PLMs respectively.

## introduction as a fundamental natural language processing (nlp) task, semantic role labeling (srl) uses predicate-argument structure to represent the shallow semantic meaning of sentences. srl structure is shown to be helpful for many downstream nlp tasks, such as machine translation @xcite and question answering @xcite . there exist two forms of concrete srl formalism in the community, i.e., word-based (also known as dependency-based srl) and span-based, depending on whether an argument consists of a single word or a word span. compared with word-based srl, span-based srl is more complex due to difficulties in determining argument they want to do more . they want to do more . a0 a1 a0 a1 figure 1 : an example of span-based srl, where "want" and "do" are two predicates. in recent years, span-based srl has achieved substantial performance boost due to the tremendous progress made by deep neural network models, especially by pre-trained language models (plms). currently, there are mainly two mainstream approaches, i.e., bio-based @xcite sequence labeling and span-based graph parsing @xcite . the bio-based sequence labeling approach first identifies the predicates and then finds arguments for each predicate independently by labeling every word with bio tags, like "b-a0" or "i-a0". its major weakness is that a sentence has to be encoded and decoded for multiple times, each time for one predicate @xcite , thus proportionally reducing the training and inference efficiency. 1 @xcite concatenate an indicator embedding to each input token, where the focused predicate corresponds to 1, and others to 0. @xcite append the focused predicate word to the end of the sentence before getting into bert @xcite . the span-based graph parsing approach directly considers all word spans as candidate argument nodes and links them to predicate nodes @xcite . however, this approach also suffers from a severe inefficiency problem, since there are o(n) candidate predicates and o(n 2 ) candidate arguments, leading to a big search space of o(n 3 ). previous works usually employ heuristic pruning techniques to improve efficiency. inspired by recent works on semantic dependency graph parsing (sdgp) @xcite @xcite , this work for the first time proposes a word-based graph parsing approach for end-to-end span-based srl. end-to-end means that all predicates and arguments in a sentence are inferred simultaneously and by a single model. the key challenge is how to represent span-based arguments in word-based graphs in which nodes correspond to single words. once this is solved, we can build our parser on the shoulder of existing word-based graph parsing models. this work employs the second-order model of @xcite . in summary, our work makes the following contributions: we release our code, configuration files, and models at https://github.com/ zslin177/srl-as-gp . the bio-based approach usually predicts predicates first and then recognizes arguments for each predicate via sequence labeling. for each predicate, @xcite indicates the position of the predicate via indicator embedding, and then encode the sentence using multi-layer bilstms, and finally apply a crf layer to find the best label sequence. @xcite append the focused predicate word to the original sentence, and then feed the sentence into bert, and then apply bil-stm for further encoding. the span-based graph parsing approach is proposed by @xcite . the idea is directly predicting relations between candidate predicates (single words) and arguments (word spans) in a graph. @xcite apply the approach to the word-based srl task. besides the two mainstream approaches, researchers have explored other interesting directions. @xcite propose a two-step span recognition approach, i.e., first identifying a head word and then extending the word into a span. @xcite cast the srl task under the predicategiven setting as a sequence-to-sequence task like machine translation. given a predicate, its srl structure is converted into a token sequence. their approach achieves competitive performance by using bart @xcite . concurrently, @xcite cast spanbased srl as a tree parsing approach. given a predicate, the word span corresponding to an argument is represented as latent trees. the sentence is encoded once without using predicate indicators, but each predicate require an independent decoding process. syntax-enhanced srl. due to the close connection between syntax parsing and srl, there has been a lot of works on syntax-enhanced srl. strubell et al. (2018) and zhou et al. (2020) ## proposed graph schemata this work proposes to cast end-to-end span-based srl as a word-based graph parsing task. the key challenge is to design a suitable graphical schema so that all predicates and their span-based arguments can be represented simultaneously in one graph without ambiguity. and the graph can be transformed to its corresponding srl structure without performance loss. ## model based on our designed graphical schema, we can address span-based srl as a word-based graph parsing task. following @xcite and @xcite , the framework of our model consists of two stages: 1) predicting all edges and 2) assigning labels for edges. ## conflict resolution during inference, we first use the edge prediction module to build the graph skeleton, and then use the label prediction module to assign labels to predicted edges. after that, we use a simple procedure to check whether the generated graph is legal. concretely, for each predicate, we scan the edges of the predicate from left to right. for example, in the bes schema, a "b- * " edge must be followed by a "e- * " edge; "s- * " edge and "e- * " can be followed by a "b- * " edge or "s- * " edge. if the generated graph is legal, we can directly recover the corresponding srl structure through graph-to-srl procedure described in 3.2. however, since the label prediction module handles each edge independently, the resulting graph may contain conflicts, as shown in the upper part of figure 5 (a) foot_2 . first, if two consecutive edges are both labeled as "e- * ", such as the two "e-a0" edge, then it is impossible to recover the corresponding arguments. another conflicting scene is when there exists a single outlier edge labeled as "b- * " or "e- * ", such as "e-a1" edge in the figure. ## conclusions this paper proposes four new graph representation schemata for transforming raw span-based srl structures to word-based graphs. based on the schema, we cast the span-based srl as a wordbased graph parsing task and present a fast and accurate parser. moreover, we propose a simple postprocessing method based on constrained viterbi to handle conflicts in the output graphs. experiments show that our parser 1) is much more efficient than previous parsers, and can parse over 600 sentences per second; 2) reaches consistently better performance than previous results on conll05, conll12 datasets. the in-depth comparison between four schemata shows that the boundary information counts a lot when recognizing arguments. in addition, distinguishing single-word arguments from multi-words arguments can also improve the final performance. these clear findings may help researchers think about srl from a new perspective in the future.