Data Augmentation Methods for Anaphoric Zero Pronouns (2021)

Abstract: In pro-drop language like Arabic, Chinese, Italian, Japanese, Spanish, and many others, unrealized (null) arguments in certain syntactic positions can refer to a previously introduced entity, and are thus called anaphoric zero pronouns. The existing resources for studying anaphoric zero pronoun interpretation are however still limited. In this paper, we use five data augmentation methods to generate and detect anaphoric zero pronouns automatically. We use the augmented data as additional training materials for two anaphoric zero pronoun systems for Arabic. Our experimental results show that data augmentation improves the performance of the two systems, surpassing the state-of-the-art results.

## introduction in pronoun-dropping (pro-drop) languages such as arabic @xcite , chinese @xcite , italian (di @xcite and other romance languages (e.g., portuguese, spanish), japanese @xcite , and others @xcite , certain arguments can be omitted in which a pronoun is used in english: such arguments are called zero-pronouns (zp) in nlp. anaphoric zero pronouns (azp) are a special case of zero pronouns. azps refer to one or more noun entities previously mentioned in a text, as in the following example from the arabic portion of ontonotes (in arabic ontonotes 5.0, zps are denoted as * in arabic text). ironically, bush did not show any enthusiasm for the international conference, because since the beginning, (he) wanted to attend another conference ... in the example, the azp, indicated with '*', refers to a singular masculine person that has been mentioned previously, bush ( ‫ب‬ ‫و‬ ‫ش‬ ). english is not a pro-drop language @xcite so the null argument in arabic has to be expressed using an overt pronoun (he). azp resolution involves two tasks: identification and resolution. azp identification is the task of identifying the locations where the pronoun was omitted (the * in the example). azp resolution is the task of finding the antecedent that an azp refers to (bush in the example). contextual information is required for these two tasks, such as that provided by the verb which has the azp as an argument (in the example the verb is "wanted/ ‫ي‬ ‫ر‬ ‫ي‬ ‫د‬ )". the azp problem has inspired much research because azp interpretation benefits many natural language processing (nlp) tasks such as machine translation @xcite . although azps are common @xcite , they are not always annotated in nlp corpora. there are two reasons for this. first, azps have no surface realization and the focus in coreference is usually on arguments realized on the surface @xcite . second, pro-drop languages have several types of zps, not all of which anaphoric, which can make it challenging to identify azps. therefore, the number of datasets with annotated azps is small @xcite . we investigate therefore automatic methods for augmenting these existing data, applying them to arabic. our contributions are as follows: the rest of the paper is organized as follows. we discuss azp and data augmentation related literature in section 2. we explain our dataaugmentation methods in section 3. we discuss the evaluation settings and results in section 4 and section 5 respectively. we conclude in section 6. ## methodology we applied the following five methods to generate azps: ## results we use the systems from @xcite and @xcite as baselines for azp identification and resolution respectively. to see the effect of the collected samples, we use their systems following the same settings. we evaluate the augmented data of each method with different settings, and see their performance on the precision, recall, and f1 scores. azp identification in azp resolution we evaluate each augmenteddata method and the results are shown in combinations we also try combining two methods together to see if they can outperform when we use every method separately. we show the azp identification results in ## conclusion we tested five methods to detect and generate azps. we suggested a simple and effective way to resolve azps in the automatically generated samples. we evaluated the augmented data on the azp identification and resolution tasks for arabic. training the models with the augmented data achieved new state-of-the-art results for both azp tasks.