Zero Pronouns Identification based on Span prediction (2021)

Abstract: The presence of zero-pronoun (ZP) greatly affects the downstream tasks of NLP in pro-drop languages such as Japanese and Chinese. To tackle the problem, the previous works identified ZPs as sequence labeling on the word sequence or the linearlized tree nodes of the input. We propose a novel approach to ZP identification by casting it as a query-based argument span prediction task. Given a predicate as a query, our model predicts the omission with ZP. In the experiments, our model surpassed the sequence labeling baseline.

## introduction pro-drop languages, such as japanese, chinese, or arabic, allow omissions of essential phrases or arguments, e.g., nouns, which could be easily inferred by humans given contexts in a sentence. the omitted argument is called zeropronoun (zp), or (small) "pro", which is an instance of empty categories in linguistics. en this cake is delicious. i like (it). in the japanese example above, the object argument (obj) is omitted from the second sentence because japanese speakers can predict from the context that the obj is "it", and the omission is natural for the japanese speakers. downstream tasks involving pro-drop languages could easily suffer from the existence of zps. in the machine translation task, it has been reported that supplementing the zp information when translating from pro-drop languages to non-pro-drop languages improves the performance @xcite . when identifying a zp from the sentence where the argument is omitted, the predicate information is the key. the zp identification is solved in many previous works as a labeling task for input sentence tokens @xcite or nodes in a parse tree @xcite . in this study, we treat zp identification as an instance of span prediction tasks inspired by the qa method proposed in @xcite . there are two steps to solve the zp identification in our approach. 1) given a predicate as a query, our model extracts each argument, such as subject or object, as the answer from the input sentence. 2) if our model cannot extract any corresponding argument from the input sentence, the model predicts whether or not it is a zp. in the above example, given a predicate 気に入った "like", our model should predict that the subject argument is 私は "i" in the sentence and the object argument is a zp. by explicitly providing predicates as queries in this way, our approach allows the model to capture information about the zp cue from the input sentence, thereby improving the zp identification performance. our contributions are as follows: 1)we proposed a novel approach for zp identification. ## related work most of the researchers considered the zp detection or zp identification as a labeling task. @xcite and @xcite used parse trees as input and detected empty categories, including zps, by labeling a node representing the maximal projection of a predicate, namely ip or vp. @xcite proposed jointly learning zp resolution and zp identification by treating it as sequence labeling on every word boundary. @xcite considered word positions before or after each vp node as zp location candidates and predicted whether the candidate has zp or not as a binary classification task. to the best of our knowledge, our approach is the first work that formalizes zp identification as a qa task. in recent years, approaches for solving various tasks as qa-based span prediction problems have been proposed. @xcite made questions corresponding to ner entity tags. then, their model predicted the entity span giving the question and a sentence as qa tasks to tackle the nested ner problem. in the coreference resolution task, @xcite generated queries based on each mention and extracted the text spans of coreferences as answers to given queries. @xcite improved the performance of word alignment task by giving a word in the source language sentence as a question and predicting its corresponding word span in the target language sentence. ## span-based zp identification treebanks have phrase structure tree information, and in some treebanks, empty categories are also annotated as null terminal nodes @xcite . however, we focused only on zp identification, not dealing with other empty categories, such as trace and pro, in this paper. we formally define the zp identification as span-based prediction as follows: given a tokenized sentence @xmath0, ..., x |x| , we denote a span of the sentence as x qs:qe (1 ≤ @xmath1, i.e., verb or adjective. the task is to identify the span of the sentence x corresponding to the argument required by the predicate x qs:qe . when no span is detected, there are three possible cases: (i) the argument is dropped as a kind of zp; (ii) the argument is not dropped as a zp, but as another empty category such as trace or pro; (iii) it is not required by the predicate at all. we grouped the latter two cases into one class, the non-zp class. therefore, our model predicts one of the zp classes or the non-zp class for the required but omitted argument. the prediction is applied for each grammatical function of the argument, such as sbj, obj, etc. our argument span prediction is inspired by bert fine-tuning for the qa task @xcite . inputs follow a bert style formulated as "[cls] query [sep] sentence [sep]", where [cls] is a special token to output the classification result and [sep] denotes the boundary of "query" and "sentence." the query in the input is defined as follows: in the example sentence (1) , there are five words in the tokenized input sentence excluding a null token φ foot_1 . given "着き" as a predicate with @xmath2, the query is represented as follows: ## conclusion we proposed a zp identification method based on span prediction and evaluate it on japanese and chinese datasets. our model is the first approach to consider zp detection as a qa task. in experiments, the f1 values of our method were higher than the baseline method using sequence labeling for both japanese and chinese. future works include to analyze arguments that appeared overtly in tasks such as semantic role labeling. as a setting closer to the real problem, we will use a tagger to create queries instead of using gold data. the other future work is comparison with a baseline which predicts all arguments at once by sharing the model parameters of bert as our proposal model. we also consider extending our proposed method to coreference resolution tasks in pro-drop languages.