A Multi-Pass Sieve Coreference Resolution for I ndonesian (2021)

Abstract: Coreference resolution is an NLP task to find out whether the set of referring expressions belong to the same concept in discourse. A multi-pass sieve is a deterministic coreference model that implements several layers of sieves, where each sieve takes a pair of correlated mentions from a collection of non-coherent mentions. The multi-pass sieve is based on the principle of high precision, followed by increased recall in each sieve. In this work, we examine the portability of the multi-pass sieve coreference resolution model to the Indonesian language. We conduct the experiment on 201 Wikipedia documents and the multi-pass sieve system yields 72.74% of MUC F-measure and 52.18% of BCUBED F-measure.

## background many natural language processing (nlp) tasks need to incorporate linguistic comprehension beyond semantics understanding. coreference resolution is an important discourse-level nlp pipeline that can be used to support a number of nlp applications, such as question answering, summarization, and dialogue system. coreference resolution task aims to evaluate whether a set of expressions in the text refer to each other, in other words whether they describe the same entity in real-world situation @xcite there are not many coreference resolution studies in indonesian. @xcite worked on the indonesian coreference resolution task by applying the association rules. @xcite developed a coreference resolution system using supervised classifier. they utilized lexical and syntactic features to connects pronouns to named entities, between named entities, and between pronouns. stanford nlp lab introduced multi-pass sieve approach for coreference resolution task @xcite . then, this method has been widely adapted for the same task in various languages other than english, such as korean @xcite , @xcite , arabic, and chinese @xcite . the multi-pass sieve concept is based on the principle of high precision, followed by increased recall in each pass. in this approach, the breakdown of the correlation relationship is divided into several layers of sieves. in spite of emphasis on hand-crafted linguistics features, this is a robust technique to tackle the coreference resolution task. in addition, the results of this approach have a high level of interpretability, so that the analysis of the results has the potential to be used in long-term research. in this research, we utilize this multi-pass sieve technique for indonesian coreference resolution task. in addition, we build a data set for testing the system. we publish our code and data for research purpose foot_0 . ## noun phrases in indonesian the set of referential phenomena in indonesian includes several types, i.e., pronouns, demonstratives, noun phrases, and named-entities. in this section, we briefly describe the first three types of nominal, while there is no difference of names between indonesian and english. pronouns pronouns are a class of words that are used to refer to another noun @xcite . a personal pronoun is a pronoun which refers to one or more persons. personal pronouns in indonesian are differentiated into singular and plural pronouns. unlike english, pronouns in indonesian are not differentiated by gender, nor by its function in a sentence, i.e., same set of pronouns are used as a subject, an object, or to indicate possession. for example, the word "saya" is a personal pronoun in indonesian which can be used as a subject pronoun, e.g., "saya makan" (in english "i am eating."), an object pronoun, e.g., "pak bob memanggil saya." (in english "mr.bob is calling me.", and a possesive pronoun, e.g., "ini buku saya." (in english "this is my book."). indonesian pronouns can be in form of the clitics @xcite . a clitic is a morpheme that is attached to another word or phrase. clitic pronouns in indonesian include "-ku" as first-person pronouns, "-mu" as second-person pronouns, and "-nya" as a third-person pronoun. demonstratives demonstrative, also classified as a demonstrative pronoun by @xcite , are words that refer to a noun or noun phrase. in indonesian, there are two common words that are classified as demonstratives, namely "ini" (in english:"this" or "these") and "itu" (in english: "that" or "those"). nouns nouns can be seen semantically as words that represent humans, animals, objects, meanings, and concepts that exist in the world as described in @xcite . syntactically, a noun can be negated with "bukan" (in english: "not to be"), but cannot be negated with "tidak" (in english: "do not"). nouns can be followed by one or more adjectives, either directly or connected by the word "yang" (in english: "which"). several indonesian nouns are multiword expression @xcite , e.g., "kamar tidur" (in english: "bedroom") and "rumah sakit" (in english: "hospital") noun phrases a noun phrase may consist of one or more noun(s), pronoun(s), numeral(s), verb(s), adjective(s), and demonstrative(s). a noun phrase in indonesian is constructed by expanding the noun to the left, with a determiner, or to the right, with modifiers. the initial noun before the expansion is named as the core noun (the head word). while english put modifiers before the head word, modifiers come after the head word in indonesian, e.g., "buku matematika" vs. "mathematics book". here are some rules for expanding a head word into a noun phrase in indonesian @xcite . we design multi-pass sieve to resolve coreference problem in indonesian text. the multi-pass sieve approach works by receiving input in the form of mention pairs (m1, m2) and classifying the correlation relationship between the two mentions according to the definition of each sieve layer sequentially and stops when the pair (m1, m2) has been declared to have a correlation relationship or when the pair have visited the last sieve. the model is implemented in six tiers, i.e., exact string match, precise constructs, string head match, proper head word match, relaxed head match, and pronoun sieve. several other sieves in original stanford model @xcite are not adapted in our model due to the differences of linguistic characteristics between english and indonesian languages. in contrast to english, demonym relation in indonesian are not expressed by changing words. for example: "indonesia" and "orang indonesia" (in english: "indonesia" and "indonesian"). a sieve-pass using morphological properties is not included in ours. indonesian word does not possess gender, number (singular vs. plural), and animacy attribute. on the other hand, lexical chain sieve is not implemented due to the lack of language resources. existing indonesian wordnet @xcite does have neither adequate synsets nor lexical semantic relation. in addition, we do not find proper dictionary to construct alias sieve. we also exclude relaxed string match as adverb clauses are out of scope of mention detected in our coreference resolution system. discourse processing sieve is skipped since the data used in our experiment does not contain any direct sentence. ## data annotation since there is no publicly available data of coreference resolution for indonesian, we construct new data set in this research. we collect the data from wikipedia in indonesian language. we filter the wikipedia pages that fulfill three criteria 1. the pages contain many noun phrases. we hypothesizes they are the wikipedia pages discussing one of following topics. (i) fictional plots, e.g., subtitles for films, tv show episodes, and novel stories; (ii) biographies (incl. fictional characters); and (iii) historical events or important events. ## conclusion we conclude that the multi-pass sieve approach provides a strong baseline performance for coreference resolution task in indonesian language. the multi-pass sieve approach achieves muc fmeasure up to 72.74% and bcubed f-measure up to 52.18%. the use of the exact string match feature provides high precision for the multi-pass sieve model, while the increase in recall is also influenced by the pronoun sieve. the highest fmeasure results for the muc metric were obtained by multi-pass sieve for the combined implementation of pass 1-6 and pass 1-3 models for the bcubed metric. on the other hand, the proper head match and relaxed head match features do not appear to have a major impact on system performance, contributing to a performance increase of around 1%. there is still a lot of room to improve the coreference resolution system for indonesian. in the future, we plan to implement an end-to-end model coreference resolution system, in which the resolved phrases or words can be detected automatically using a more perfect chunking and namedentity recognition system.