Coreference Resolution Using A dapter F usion-based Multi-Task learning (2023)

Abstract: End-to-end coreference resolution is the task of identifying the mentions in a text that refer to the same real world entity and grouping them into clusters. It is crucially required for natural language understanding tasks and other high-level NLP tasks. In this paper, we present an end-to-end architecture for neural coreference resolution using AdapterFusion, a new two stage learning algorithm that leverages knowledge from multiple tasks. First task is in identifying the mentions in the text and the second to determine the coreference clusters. In the first task we learn task specific parameters called adapters that encapsulate the taskspecific information and then combine the adapters in a separate knowledge composition step to identify the mentions and their clusters. We evaluated it using FIRE corpus for Malayalam and Tamil and we achieved state of art performance.

## introduction coreference resolution is the task of grouping mentions in a text that refer to the same real-world entity into clusters @xcite . end-toend coreference resolution is the task of identifying and grouping mentions in a text such that all mentions in a cluster refer to the same entity. many traditional coreference systems, either rule based @xcite or learning-based @xcite @xcite , solve the problem in two separate stages: initially a mention identifier to identify entity mentions from the text, then a coreference resolver to cluster identified mentions. at both stages, they depend upon finegrained, conjoined features using heuristics. this two staged pipeline approach can cause cascading errors, and since both stages has dependency over syntactic parser and handcraft features, it is difficult to scale up the system for new data sets and languages. the first state of the art end-to-end neural coreference resolution system was proposed by @xcite . here all text spans are considered as potential mentions and therefore eliminate the need of carefully hand-engineered mention detection systems. in addition, the representation of pre-trained word embeddings and deep neural networks have made the model use a minimal set of hand-engineered features (speaker id, document genre, span distance, and span width). the core of the end-to-end neural coreference resolver is the scoring function to compute the mention scores for all possible spans and the antecedent scores for a pair of spans. one of the major challenges of coreference resolution is that most mentions in the document are singleton or non-anaphoric, i.e., not coreferent with any previous mention @xcite . since the data set only have annotations for mention clusters, the end-to-end coreference resolution system needs to detect mentions, detect anaphoricity, and perform coreference linking. therefore, good designs of the scoring architecture and the learning strategy for both mention detection and antecedent identification needs to be arrived at given only the gold cluster labels. to this end, we propose to use adapterfusion-based multi task learning methodology proposed by @xcite for an end-to-end coreference resolution system. ## our approach for developing an end-to-end coreference resolution system, we used adapterfusion-based multi-task learning methodology @xcite . this transfer learning method has two the two predominant approaches in transfer learning for sharing knowledge across multi-task are  sequential fine-tuning: this involves sequentially updating all the weights of the model on each task. this approach works well for two sequential tasks and beyond that leads to catastrophic forgetting. ## experiment end-to-end coreference resolution requires the following task, named entity recognizer, pronominal resolution, noun-noun anaphora resolution and coreference entity clustering. we train each task as a single task adapter. the adapters are combined in the adapterfusion task and fusion parameters ψ is learned. ## dataset the availability of coreference annotated data is the major bottleneck in indian languages. ontonotes, have created standard coreference annotated data and has made available for arabic, chinese and english. in indian languages, following the convention adopted in ontonotes, data was created and shared in icon 2011 shared task. it has anaphora annotated sentences in bengali, hindi and tamil. socanares (20221 and 2022) has anaphora annotated microblog data from twitter, in hindi, @xcite . in the present study we have used socanares2022 corpus (fire 2021 and 2022 corpus) which has conversation tweets collected by recursively retrieving the parent tweets to construct the full conversational tree structure. to construct the full tree, 5 iterations were performed and considered each chain of tweets as a document. out of the three languages we have taken malayalam and tamil for our study. it has anaphors and antecedent annotated. for the present task, we have annotated named entities, noun-noun anaphors which includes definite description using palinka @xcite , which is an open source tool from university of wolverhampton. the corpus statistics is given in the description malayalam tamil no. of files 370 482 no. of tweets 1880 2855 table1: corpus statistics s.no type number of occurrence malayalam tamil 1 noun-noun anaphora 1854 2424 2 total no. of anaphors 1170 991 3 anaphors having antecedent 455 665 the results in s.no exp muc b 3 ceafe ml ta ml ta ml ta pr re f1 pr re f1 pr re f1 pr re f1 pr re f1 pr re f1 1 af word -level 12 10 11 19 13 15 33 24 28 42 31 36 18 25 21 24 33 28 2 af morp hlevel 15 13 1 21 17 19 36 28 32 47 34 39 22 29 25 29 38 33