Z -coref: T hai Coreference and Zero Pronoun Resolution (2024)

Abstract: Coreference Resolution (CR) and Zero Pronoun Resolution (ZPR) are vital for extracting meaningful information from text. However, limited research and datasets pose significant challenges in Thai language. To address this, we developed an annotated joint CR and ZPR dataset. Additionally, we introduced the Z-coref model, capable of simultaneously handling CR and ZPR tasks by adjusting the span definition of a prior CR architecture to include token gaps. The proposed model trained on our dataset outperformed the state-of-the-art in resolving both coreference resolution and zero-pronoun resolution, while taking less time to train.

## introduction coreference resolution (cr) is the task of identifying and linking words or phrases referring to the same entity in a text. it is a crucial step in natural language processing (nlp) taken to determine the meaning of a text by resolving ambiguity. one of the tasks in cr is known as zero pronoun resolution (zpr). the main goal of zpr is to determine the reference of a missing pronoun, or so-called a zero pronoun (zp) -a linguistic phenomenon in which a pronoun in a sentence can be omitted because its referent is clear from the context. this omission is often easily recognizable by humans but presents a challenge for machines. zero pronoun resolution still remains a difficult task in pro-drop languages like thai, chinese, and japanese. figure 1 illustrating a news headline written in thai and its english translation, exemplifies the challenge of zps. nouns and zero pronouns (∅) marked with blue squares refer to the wife of a taxi driver, while those in red squares refer to the taxi driver. it can be noticed that there are several occurrences of zps although the headline and language style are succinct. these brief sentences present a challenge for a machine to interpret. while there exist various baseline models and large annotated datasets for cr in english, there is a paucity of research in this area for the thai language. only one dataset and one baseline model by han-coref are publicly available (phatthiyaphaibun and limkonchotiwat, 2023); however, neither covers the case of zero pronouns. therefore, this study makes the following contributions: (1) we have taken the initiative to create the first dataset that combines both cr and zpr for thai language; ## methodology in this paper, we established a cr dataset that contains details of zps and modified the cr model's architecture to handle zps. ## dataset the dataset has been annotated by eight annotators. due to time constraints and the challenging nature of the task, each annotator was only able to annotate a subset of the dataset. however, by combining the annotations from all annotators, the entire dataset of 1,338 documents was covered with at least two annotations per document. details of dataset are further described in appendix a ## experiment and results this experiment aims to evaluate our proposed model against the e2e-joint-coref model developed by @xcite using our annotated dataset. the models were trained for 150 epochs to compare their performance and training time requirements. both models were trained using an nvidia geforce rtx 3090 gpu with no other processes running concurrently during the training sessions for the fairness of time comparison. detailed experiment setting are discussed in appendix b as shown in as illustrated in in addition, z-coref with phayathaibert encoder exhibits superior performance compared to others. nevertheless, when employing mbert encoder, z-coref is unable to surpass the performance of e2e-joint-coref. in the case of xlm-roberta and wangchanberta, further elaboration on these results is presented in appendix b as the performance observed in ## conclusion due to the lack of a dataset and baseline model for cr in the thai language, as well as the nature of pro-drop languages that can cause original cr to overlook zps that frequently occur in informal language such as news articles, we have created the first thai joint dataset of cr and zpr. we also introduce z-coref, a lightweight joint cr and zpr model. z-coref with phayathaibert encoder achieves higher performance than previous work from @xcite and significantly reduces training time. our method effectively resolves the majority of zps. however, it may face limitations when multiple zps occur within the same gap. for example, in the sentence "they would hit (it) (so) (it) flees", the words in parentheses can be omitted in thai. consequently, the gap between "hit" and "flees" contains two zps: the object of the first subsentence and the subject of the second subsentence. this scenario highlights a potential challenge for our approach.