End to End Multilingual Coreference Resolution for I ndian Languages (2024)

Abstract: This paper describes an approach on an end to end model for Multilingual Coreference Resolution (CR) for low resource languages such as Tamil, Malayalam and Hindi. We have done fine tune the XLM-Roberta large model on multilingual training dataset using specific languages with linguistic features and without linguistic features. XLM-R with linguistic features achieves better results than the baseline system. This shows that giving the linguistic knowledge enriches the system performance. The performance of the system is comparable with the state of the art systems.

## introduction the task of coreferenceresolution (cr) has attracted considerable attention in natural language processing due to its importance in deep language understanding and its potential as a subtask in a variety of complex natural language processing problems. it is a key module in many nlp systems and is used in many higher-level nlp tasks, such as information extraction, causeeffect identification, cross lingual information retrieval, document summarization and question answering. cr is the task of finding all referring expressions for a referent (an anaphor) that refers to the same real-world entity (the antecedent) with in a document or text. a referring expression or a mention is a noun phrase (np) , a named entity (ne), or a clause, which refer to an entity in the real world known as the referent @xcite . the referents (anaphors) can be a pronoun, a reflexive, a reciprocals, a noun, in short anything that requires a reference in front. a grouping of referring expressions with the same referent or antecedent is called a coreference chain or cluster. the goal of a coreference resolution system is to identify such chains or clusters in a given document and give each chain as output. the span of coreferential expressions can be from a single sentence or can be separated by one or more sentences. in many cases, it is seen that the entire document needs to be considered for identifying the referent accurately. the coreference resolution task can be divided into two subtasks, 1) identify entity mentions and group them together according to the real-world entity they refer to and 2) to resolve the task of anaphora resolution which is closely related to coreference resolution @xcite . multilingual coreference resolution is the process of determining if expressions in multiple languages refer to the same entity. the rest of the paper is organized as follows. section 2 discusses the related work. section 3 discusses the data and section 4 outlines our approach. whereas the section 5 discuss the evaluation and section 6 the conclusion. ## related work in general, coreference resolution is solved by first predicting the coreference mentions and subsequently performing coreference linking (clustering) of the predicted mentions. in recent years this method got changed and an end-to-end approach proved to be better performing @xcite @xcite . recently, different approaches have been presented for coreference resolution in english. latent antecedents @xcite or neural models end to end multilingual coreference resolution for indian languages sobha lalitha devi, vijay sundar ram and pattabhi rk rao au-kbc research centre, mit campus of anna university, chennai, india sobha@au-kbc.org @xcite have gained popularity and achieved state-of-the art results. hybridisation of techniques has been also proposed, for example in @xcite , which presents a hybrid architecture of their stanford system. this approach incorporates both rule-based and statistical machine learning methods. interest in coreference resolution in languages other than english has been increasing in the last few years @xcite , and because of this many papers about adaptations of coreference resolution systems to a language other than the one for which they were created have been published. the beautiful anaphora resolution toolkit (bart) @xcite has been adapted to many languages: originally created for english, its flexible modular architecture facilitates its portability to other languages. there has therefore been a lot of work on extending the bart coreference toolkit to languages other than english including italian @xcite , german @xcite , polish @xcite , arabic and chinese @xcite , and indian languages @xcite . there are other systems using crfs such as @xcite . the first end-to-end neural coreference resolution system was introduced by @xcite , and many subsequent neural coreference resolution systems are based on their model. the end-to-end approach has been improved by ( kirstain et al. 2021) not to explicitly construct the span representations, and by (dobrovolskii 2021) to consider only the word level, ignoring the span level altogether during coreference linking. simultaneously, (wu et al. 2020) formulated coreference resolution in a question answering setting. ## data in this work we have used aligned multilingual corpus for annotating for coreference. cr is a discourse phenomenon and it requires both syntactic and sematic information for resolution. in this study we have developed a corpus of aligned hindi, malayalam, tamil texts to identify coreferring expressions. our task focused on the same kind of coreference as considered in the past muc competitions, namely the identity coreference. identity coreference links nouns, pronouns and noun phrases (including proper names) to their corresponding antecedents. we created our multilingual collection of data by translating the english data collected for the discourse translation project to tamil, hindi and malayalam. the translation was done manually by bilingual experts. the data is from three domains, agriculture, health and governance. the data is parallel aligned and annotated for anaphors, its antecedents and coreference chains. the details of the data is given in ## our approach this our model builds on the transformer-based end-to-end coreference resolution model that was originally proposed by @xcite . this predicts the antecedents for all possible mention spans without detecting the mentions in the previous stage. we based our model on xlm roberta large which is trained on 100 languages which includes hindi, tamil and malayalam. our coreference annotated dataset consists of hindi, tamil, malayalam. xlm-roberta, is an advanced multilingual model based on the transformer architecture and has shown promising results in various nlp tasks, including coreference resolution. xlm-roberta is an extension of the roberta model, designed for multilingual tasks. it is pre-trained on a large corpus of multilingual text using a masked language modelling objective. its architecture allows it to capture semantic relationships and context effectively across multiple languages, making it suitable for tasks that require understanding deeper references within texts. training involves fine-tune the pre-trained xlm-roberta model on the coreference resolution dataset for hindi, tamil and malayalam. the dataset is parallel across languages as discussed in section 3. we fine-tune the model for each specific language. data preparation: the data is formatted such that it can be given as input to the training system. a sequence of tokens representing the entire text and an attention mask to distinguish between real tokens and padding are given as the input. model training (fine-tuning): incorporating linguistic information can significantly enhance the model. we include the following linguistic features: i) entity type -such as person, organization ii) number and gender -plural/singular, male/female to encode linguistic information, we add to each token representation the four feature information. in more detail, we convert the linguistic information into numerical information. for example for the entity type, we use one-hot encode. and for binary features such as gender we use binary representation 0/1 (eg: 1 for female, 0 for male). this we name it as ling_emb. this representation is then concatenated with roberta embedding of each token. this results in richer representation of each token. along with the above step of embedding the following 3 steps are also involved in the process of fine tuning. ## evaluation we evaluate the trained models for all the 3 language. we consider the baseline system as the one which does not have linguistic features as explained in the previous section. the results are tested on our own dataset described in section 3. as it can be observed from the results in