High-order Refining for End-to-end C hinese Semantic Role Labeling (2020)

Abstract: Current end-to-end semantic role labeling is mostly accomplished via graph-based neural models. However, these all are first-order models, where each decision for detecting any predicate-argument pair is made in isolation with local features. In this paper, we present a high-order refining mechanism to perform interaction between all predicate-argument pairs. Based on the baseline graph model, our high-order refining module learns higher-order features between all candidate pairs via attention calculation, which are later used to update the original token representations. After several iterations of refinement, the underlying token representations can be enriched with globally interacted features. Our high-order model achieves state-of-the-art results on Chinese SRL data, including CoNLL09 and Universal Proposition Bank, meanwhile relieving the long-range dependency issues.

## introduction semantic role labeling (srl), as the shallow semantic parsing aiming to detect the semantic predicates and their argument roles in texts, plays a core role in natural language processing (nlp) community @xcite @xcite . srl is traditionally handled by two pipeline steps: predicate identification @xcite and argument role labeling @xcite . more recently, growing interests are paid for developing end-to-end srl, achieving both two subtasks, i.e., recognizing all possible predicates together with their arguments jointly, via one single model @xcite . the end-to-end joint architecture can greatly alleviate the error propagation problem, thus helping to achieve better task performance. currently, the end-to-end srl methods largely are graph-based * corresponding author. neural models, enumerating all possible predicates and their arguments exhaustively @xcite @xcite . however, these first-order models that only consider one predicateargument pair at a time can be limited to short-term features and local decisions, thus being subjective to long-range dependency issues existing at large surface distances between arguments @xcite . this makes it imperative to capture the global interactions between multiple predicates and arguments. in this paper, based on the graph-based model architecture, we propose to further learn the higherorder interaction between all predicate-argument pairs by performing iterative refining for the underlying token representations. figure 1 illustrates the overall framework of our method. the bil-stm encoder @xcite first encodes the inputs into the initial token representations for producing predicate and argument representations, respectively. the biaffine attention then exhaustively calculates the score representations for all the candidate predicate-argument pairs. based on all these score representations, our highorder refining module generates high-order feature for each corresponding token via an attention mechanism, which is then used for upgrading the raw token representation. after total n iterations of the above refining procedure, the information between the predicates and the associated arguments can be fully interacted, and thus results in global consistency for srl. on the other hand, most of the existing srl studies focus on the english language, while there is little work in chinese, mainly due to the limited amount of annotated data. in this study, we focus on the chinese srl. we show that our proposed high-order refining mechanism can be especially beneficial for such lower-resource language. meanwhile, our proposed refining process is fully parallel and differentiable. we conduct experiments on the dependencybased chinese srl datasets, including conll09 @xcite , and universal proposition bank @xcite . results show that the graph-based end-to-end model with our proposed high-order refining consistently brings task improvements, compared with baselines, achieving state-of-the-art results for chinese end-to-end srl. prior studies traditionally separate srl into two individual subtasks, i.e., predicate disambiguation and argument role labeling, mostly conducting only the argument role labeling based on the pre-identified predicate @xcite @xcite @xcite . more recently, several researches consider the end-to-end solution that handles both two subtasks by one single model. all of them employs graph-based neural model, exhaustively enumerating all the possible predicate and argument mentions, as well as their relations @xcite @xcite . most of these end-to-end models, however, are first-order, considering merely one predicate-argument pair at a time. in this work, we propose a high-order refining mechanism to reinforce the graph-based end-to-end method. note that most of the existing srl work focuses on the english language, with less for chinese, mainly due to the limited amount of annotated data @xcite . in this paper, we aim to improve the chinese srl and make compensation of the data scarcity by our proposed high-order model. given an input sentence @xmath0, • • • , w n }, the system is expected to output a set of triplets @xmath1, where @xmath2, • • • , p m } are all possible predicate tokens, @xmath3, • • • , a l } are all associated argument tokens, and r are the corresponding role labels for each a i , including a null label indicating no relation between a pair of predicate argument. ## conclusion we proposed a high-order end-to-end model for chinese srl. based on the baseline graph-based model, our high-order refining module performed interactive learning between all predicate-argument pairs via attention calculation. the generated higher-order featured token representations then were used to update the original ones. after total n iterations of refinement, we enriched the underlying token representations with global interactions, and made the learnt features more informative. our