Coreference Resolution for the Biomedical Domain: A Survey (2021)

Abstract: Issues with coreference resolution are one of the most frequently mentioned challenges for information extraction from the biomedical literature. Thus, the biomedical genre has long been the second most researched genre for coreference resolution after the news domain, and the subject of a great deal of research for NLP in general. In recent years this interest has grown enormously leading to the development of a number of substantial datasets, of domain-specific contextual language models, and of several architectures. In this paper we review the state of-the-art of coreference in the biomedical domain with a particular attention on these most recent developments.

## introduction coreference resolution is the process of identifying entities in a text and finding all mentions that refer to the same entities. it is a fundamental and challenging nlp task, supporting downstream tasks such as information extraction and question answering. in the biomedical domain, issues with coreference resolution are one of the most frequently mentioned challenges for information extraction from the biomedical literature @xcite . biomedical coreference resolution has become an essential task to support the discovery of complex information by identifying coreference links in biomedical texts. in recent years in particular, biomedical coreference resolution has attracted a great deal of attention due both to its great potential for application, and to its theoretical interest e.g., as an application of knowledge embeddings and entity linking. several biomedical coreference corpora have been made available, especially for protein coreference which is a supporting task for bionlp 2011 shared task @xcite . biomedical coreference is quite different from the general domain coreference, such as different markable types. therefore, domain-specific knowledge is important for bridging the gap. figure 1 shows two examples of biomedical coreference in the craft-cr dataset @xcite . in example a), the mention this protein refers to the antecedent the hspb2 gene product. to understand the coreference relation, we need the background knowledge that proteins are fundamental encoded by genes. in example b), to correctly resolve the coreference relation between these different biomedical entities, the biomedical-domain knowledge that cholinergic starburst amacrine cell is a kind of interneuron and belongs to retinal neuron are required. a large number of coreference models for the biomedical domain have already been proposed, from rule-based models @xcite @xcite @xcite @xcite , machine learning-based models @xcite @xcite @xcite to recent deep learning-based models @xcite @xcite . these models usually integrate biomedical specific information, typically specific rules, pre-trained embeddings and features. this paper reviews and analyses coreference datasets and models for the biomedical domain, as well as recent biomedical language representation models which can enhance coreference models with domain-specific knowledge. in addition, we conduct experiments to evaluate the ability of these language represetation models for biomedical coreference task. the structure of this paper is as follows. in section 2 we briefly provide some background on coreference resolution in the general domain. section 3 reviews the main datasets used to study biomedical coreference. overviews of biomedical language representation models and biomedical coreference models are provided in sections 4 and 5, respectively. section 6 introduces the methodology of comparing the biomedical language representation models for coreference. section 7 presents the evaluation results including the performance of previous models and our experiments, and section 8 concludes. ## background coreference resolution in the general domain has a long history of being studied from early heuristicbased and rule-based approaches to recent learningbased approaches. @xcite proposed the first end-to-end neural coreference resolution model which uses lstm encoder. based on the end-to-end model, many extensions to the model have been proposed. bert and spanbert were proposed to replace the lstm encoder and achieved better performance on ontonotes dataset @xcite . @xcite adapted questionanswering framework on coreference resolution, and achieved the state-of-the-art result with 83.1% f1 score on ontonotes dataset. @xcite proposed a novel language representation model corefbert, which can capture the coreferential relations in context. however, these general coreference systems do not work well in the biomedical domain due to the lack of domain knowledge. for example, the end-to-end model @xcite only achieved 33.85% and 61.25% f1 scores on craft-cr and bionlp datasets respectively @xcite , but achieved 68.8% f1 score on ontonotes dataset @xcite , which covers multiple genres, such as newswire, broadcast news and web data. ## biomedical coreference datasets several biomedical datasets with coreference annotations exist, but different document selection criteria, annotation schemes, domains and coreference types were used. the best known include: medstract @xcite is a corpus consisting of medline abstracts with coreference annotation. it is mainly concerned with two forms of anaphora: pronominal and sortal (definite noun phrase) anaphora. this corpus adapted the muc-7 annotation scheme @xcite ; in addition, semantic types from umls @xcite were also annotated. flyslip @xcite ) contains anaphoric links among noun phrases, including coreferent and associative relations. different from medstract, full-text biomedical articles were annotated in this corpus. flyslip was annotated according to a domain-specific annotation scheme. genia-medco @xcite is a coreferentially annotated version of the genia corpus @xcite , which in turn consists of 1999 med-line abstracts. this corpus follows the muc-7 annotation scheme, but adds more linguistic based relations. drugnerar @xcite was created to study anaphoric expressions in the task of extracting drug-drug interactions in pharmacological literature. this corpus consists of 49 fulltext from the drugbank database, which contains 4900 drug entries. bionlp-st'11 coref @xcite ) was created in support of one of the tasks of the bionlp 2011 shared task, focusing on finding anaphoric protein references, and based on the observation that one of major difficulties in event extraction is coreference resolution. this corpus was derived from three resources: medco coreference annotation @xcite , genia event annotation @xcite , and genia treebank @xcite . hanapin (batista-navarro and ananiadou, 2011) is comprised of 20 full-text articles from biochemistry literature. in addition to nominal and pronominal anaphora, this corpus also annotated abbreviation/acronyms and numerical anaphora. craft-cr @xcite consists of 97 full-text biomedical journal articles. similar to the general domain, this corpus was annotated with coreferent chains in full-text articles, while most other biomedical coreference datasets focuse on annotating the pairwise coreference relation between an anaphor and its antecedent. in addition, all coreference expressions were annotated regardless of semantic type. these datasets are summarized in ## biomedical language representation models the news domain and the biomedical domain are different in a number of respects, such as markable types. some authors have argued that biomedical domain knowledge is the key to bridging the gap @xcite , and that therefore, incorporating biomedical specific representation is beneficial for resolving coreferring expressions in the biomedical domain. in this section, we will give a brief introduction to biomedical language representation models. ## comparing the biomedical language representation models for coreference in section 4, we introduced a series of biomedical language representation models. to investigate the ability of these models for biomedical coreference task, we conduct experiments to evaluate these models on craft-cr dataset. ## results in this section, we first present the performance achieved by previous biomedical coreference models decribed in section 5. then we describe our experiment and report the results. ## conclusion in this paper, we review and analyse the progress of biomedical coreference datasets, biomedical language representation models and coreference models for the biomedical domain. biomedical coreference is an essential but challenging task. some efforts have been made in this field, but there is still a much room for improvement. the experiments which we conducted indicate biomedical domain knowledge from either pre-training on biomedical texts or integrating biomedical knowledge bases can enhance coreference models for the biomedical domain.