Venue: NAACL 2021

# Stay Together: A System for Single and Split-antecedent Anaphora Resolution

Juntao Yu1, Nafise Sadat Moosavi2, Silivu Paun1, Massimo Poesio1

###### Abstract

The state-of-the-art on basic, single-antecedent anaphora has greatly improved in recent years. Researchers have therefore started to pay more attention to more complex cases of anaphora such as split-antecedent anaphora, as in _Time-Warner is considering a legal challenge to Telecommunications Inc's plan to buy half of Showtime Networks Inc-a move that could lead to all-out war between the two powerful companies_. Split-antecedent anaphora is rarer and more complex to resolve than single-antecedent anaphora; as a result, it is not annotated in many datasets designed to test coreference, and previous work on resolving this type of anaphora was carried out in unrealistic conditions that assume gold mentions and/or gold split-antecedent anaphors are available. These systems also focus on split-antecedent anaphors only. In this work, we introduce a system that resolves both single and split-antecedent anaphors, and evaluate it in a more realistic setting that uses predicted mentions. We also start addressing the question of how to evaluate single and split-antecedent anaphors together using standard coreference evaluation metrics.1

Footnote 1: The code is available at [https://github.com/juntaoy/dali-full-anaphora](https://github.com/juntaoy/dali-full-anaphora)

\({}^{1}\)Queen Mary University of London

\({}^{2}\)UKP Lab, Technische Universitat Darmstadt

\({}^{1}\){juntao.yu, s.paun, m.poesio}@qmul.ac.uk

\({}^{2}\)moosavi@ukp.informatik.tu-darmstadt.de

## 1 Introduction

Thanks in part to the latest developments in deep neural network architectures and contextual word embeddings (e.g., ELMo Peters et al. (2018) and bert Devlin et al. (2019)), the performance of models for single-antecedent anaphora resolution has greatly improved Wiseman et al. (2016); Clark Manning (2016); Lee et al. (2017, 2018); Kantor and Globerson (2019); Joshi et al. (2020). So recently, the attention has turned to more complex cases of anaphora, such as anaphora requiring some sort of commonsense knowledge as in the Winograd Schema Challenge Rahman and Ng (2012); Peng et al. (2015); Liu et al. (2017); Sakaguchi et al. (2020); pronominal anaphors that cannot be resolved purely using gender Webster et al. (2018), bridging reference Hou (2020); Yu and Poesio (2020), discourse deixis Kolhatkar and Hirst (2014); Marasovic et al. (2017); Kolhatkar et al. (2018) and, finally, split-antecedent anaphora Zhou and Choi (2018); Yu et al. (2020) - plural anaphoric reference in which the two antecedents are not part of a single noun phrase.

However, a number of hurdles have to be tackled when trying to study these cases of anaphora, ranging from the lack of annotated resources to the rarity of some of these phenomena in the existing ones. Thus, most previous work on resolving these anaphoric relations focused on developing dedicated systems for the specific task. The systems are usually enhanced by transfer-learning to utilise extra resources, as those anaphoric relations are sparsely annotated. The most frequently used extra resource is single-antecedent anaphors. Due to the complexity of these tasks, previous work is usually based on assuming that either gold anaphors Hou (2020); Yu et al. (2020) or gold mentions Zhou and Choi (2018); Yu and Poesio (2020) are provided. By contrast, in this work we introduce a system that resolves both single and split-antecedent anaphors, and is evaluated in a more realistic setting that does not rely on gold anaphors/mentions. We evaluate our system on the arrau corpus Poesio and Artstein (2008); Uryupina et al. (2020), in which both single and split-antecedent anaphors are annotated, although the latter are much rarer than the former. We use the state-of-the-art coreference resolution system on arraauYu et al. (2020) as our base system for single-antecedent anaphors. This cluster-ranking system interprets single-antecedent anaphors, singletons and non-referring expressions jointly. In this work, we extend the system to resolve split-antecedent anaphors. The extended part of the system shares mention representations

and candidate clusters with the base system, and outputs binary decisions between a mention and individual candidate clusters. We configure our system to learn the split-antecedent part and the base system in both joint and pre-trained fashion. The results show both versions work much better than naive baselines based on heuristics and random selection. The pre-trained version works equally well as the joint version on split-antecedent anaphors, but it is better for the other aspects of anaphoric interpretation.

In the paper we also begin to address the question of how a system carrying out both single and split-antecedent anaphora resolution should be evaluated. Specifically, we introduce an extended version of lea(Moosavi and Strube, 2016), a standard coreference metric which can be used to give partial credit for resolution, to evaluate single and split-antecedent anaphors together. Using this metric, we find that our best model achieves a better lea score than the baselines.

We further evaluate our best system in the gold setting to compare with the Yu et al. (2020) system. The model achieved better performance when compared to their system that is designed solely for split-antecedent task.

## 2 Related Work

### Neural Approaches to Single-anteced Anaphoran Resolution

Single-antecedent anaphora resolution is an active research topic. The first neural model was introduced by Wiseman et al. (2015) and later extended in (Wiseman et al., 2016). Clark and Manning (2016) introduced a hybrid cluster/mention-ranking approach, whereas Clark and Manning (2016) adapted reinforcement learning to a mention-ranking model. Lee et al. (2017) introduced the first end-to-end system, performing mention detection and coreference resolution jointly. The Lee et al. (2017) system was also simpler than previous systems, using only a small number of hand-coded features. As a result, the Lee et al. (2017) system has become the blueprint for most subsequent systems. Lee et al. (2018) and Kantor and Globerson (2019) showed that employing contextual ELMo Peters et al. (2018) and bert(Devlin et al., 2019) embeddings in the system by Lee et al. (2017) can significantly improve performance. Joshi et al. (2019, 2020) fine-tuned bert and SpanBERT to further improve performance. Recently, Wu et al. (2020) framed coreference resolution task as question answering and showed that the additional pre-training on a large question answering dataset can further improve performance. However, those systems are only focused on single-antecedent anaphors and do not consider the other anaphoric relations.

### Other Aspects of Anaphoric Interpretation

Interpreting nominal expressions with respect to a discourse model is not simply a matter of identifying identity links; it also involves recognizing that certain potential anaphors are in fact non-referring, or singletons; other expressions refer to entities which have to be introduced in the discourse model via accomodation processes involving for instance the construction of a plural object out of other entities, as in the case of split-antecedent anaphors; other expressions again are related to existing entities by associative relations, as in _one_-anaphora or bridging reference. These other anaphoric interpretation processes are much less studied, primarily because the relevant information is not annotated in the dominant corpus for coreference, OntoNotes Pradhan et al. (2012). Systems such as the Stanford Deterministic Coreference ResolverLee et al. (2013) do use linguistically-based heuristic rules to recognize and filter singletons and non-referring expressions, but these aspects of the system are not evaluated. Carrying out such an evaluation requires a corpus with richer anaphoric annotations, such as arrau(Uryupina et al., 2020).

Yu et al. (2020) is the only neural system that targets singletons and non-referring expressions. The system uses the mention representation from Lee et al. (2018); Kantor and Globerson (2019) and applies a cluster-ranking algorithm to incrementally attach mentions directly to their clusters. Yu et al. (2020) showed that performance on single-antecedent anaphors improves by up to \(1.4~{}\text{p.p.}\) when jointly training the model with non-referring expressions and singletons. We use Yu et al. (2020) as our base system, and extend it to resolve split-antecedent anaphors.

A few systems resolving split-antecedent anaphors have been proposed in recent years. Vala et al. (2016) introduced a system to resolve plural pronouns _they_ and _them_ in a fiction corpus they themselves annotated. Zhou and Choi (2018) introduced an entity-linking corpus based on the transcripts of the _Friends_ sitcom. The mentions (in 

cluding plural mentions) are annotated if they are linked to the main characters. Coreference clusters are then created for mentions linked to the same entities. One issue with this corpus is that it is mainly created for entity-linking, so it is problematic as a coreference dataset, as many mentions are linked to general entities that are not annotated in the text. Zhou and Choi (2018) trained a cnn classifier to determine the relation between mention pairs, jointly performing single and split-antecedent resolution.

Another issue with this work is evaluation. Zhou and Choi (2018) evaluate their system using the standard conll scorer; in order to do this, they encode split-antecedent anaphora by adding the plural mention to each cluster. So, for instance, in _John met Mary. They went to the movies_, they would have two gold clusters: {John, They} and {Mary, They}. This is clearly problematic, as _They_ is not a mention of the individual entity John, but of the set consisting of John and Mary. In this work, we propose an alternative, an extended version of leaMoosavi and Strube (2016) that does joint evaluation of single/split-antecedent anaphors by explicitly representing plural entities.

Yu et al. (2020) introduced the first system to resolve all split-antecedent anaphors annotated in the arrau corpus. Their work focuses on the data sparsity problem; split-antecedent anaphora resolution is helped using four auxiliary corpora created from a crowdsourced corpus and other anaphoric annotations in the arrau corpus. However, their approach focuses on split-antecedent anaphora only, and assumes gold split-antecedent anaphors and gold mentions are provided during the evaluation, which is not realistic. In this work, we resolve both single and split-antecedent anaphora and evaluate our system on predicted mentions.

## 3 The Resolution Method

### The Base System

In this work, we use the system of Yu et al. (2020) as starting point, and extend it to handle split-antecedent anaphora. Yu et al. (2020) is a cluster-ranking system that jointly processes single-antecedent anaphors, singletons and non-referring expressions. The system uses the same mention representations as in Lee et al. (2018); Kantor and Globerson (2019). The input to the system is a concatenation of contextual bertDevlin et al. (2019) embeddings, context-independent glove embeddingsPennington et al. (2014) and learned character-level embeddings based on convolutional neural network (cnns). The system then uses a multi-layer bilstm to encode the document at the sentence level to create the word representations (\(T_{i}\)). The candidate mention representations (\(M_{i}\)) are created by the concatenation of the word representations at the start/end positions of the mention as well as a weighted sum of all the tokens within the mention boundary. After that, the candidate mentions are pruned according to their mention scores (\(s_{m}(i)\)) computed by applying a feed-forward neural network (fmn) to the \(M_{i}\). The top-ranked candidate mentions are then used by the cluster-ranking model to form the entity clusters and to identify the non-referring expressions.

The cluster-ranking model incrementally links the candidate mentions to the clusters according to the scoring function (\(s(i,j)\)) between candidate mention \(M_{i}\) and partial clusters created so far (\(C^{j}_{i-1}\)). More precisely, \(s(i,j)\) is defined as:

\[s(i,j)=\left\{\begin{array}{l l}s_{no}(i)&j=\text{no}\\ s_{nr}(i)+s_{m}(i)&j=\text{nr}\\ s_{dn}(i)+s_{m}(i)&j=\text{dn}\\ s_{m}(i)+s_{c}(j)+s_{mc}(i,j)&j\in C_{i-1}\end{array}\right.\]

where \(s_{no}(i)\), \(s_{nr}(i)\) and \(s_{dn}(i)\) are the likelihood for a candidate mention to be a non-mention (no), a non-referring expression (nr) or a discourse new mention (dn) respectively. \(s_{m}(i)\), \(s_{c}(j)\) and \(s_{mc}(i,j)\) are the mention scores (computed for mention pruning), cluster scores (a weighted sum of \(s_{m}\) for the mentions in the cluster) and cluster-mention pairwise scores. The system employs additional methods to enhance performance-e.g., keeping cluster histories and training the system on the oracle clusters. We refer the reader to Yu et al. (2020) for more details. We use the default settings of the system in our experiments.

### Resolving Split-antecedent Anaphors

To resolve split-antecedent anaphors, we follow Yu et al. (2020) who framed the task as a binary classification task. The system uses a scoring function to assign each cluster-mention pair a score \(s_{p}(i,j)\) specifying the likelihood that that cluster is one of the split-antecedents of the mention. During training, we add a dummy score (\(s_{\epsilon}(i)=0\)) for the cases in which a mention is not a split-antecedent anaphor. Formally, \(s_{p}(i,j)\) is calculated as follows:

\[s_{p}(i,j)=\left\{\begin{array}{l l}0&j=\epsilon\\ s_{m}(i)+s_{c}(j)+s_{pmc}(i,j)&j\in C_{i-1}\end{array}\right.\]

The extension for split-antecedents uses the same mention/cluster representations as well as the can 

didate mentions/clusters of the single-antecedent component. This benefits the split-antecedent anaphors part of the system, that can share the representations learned from more numerous single-antecedent anaphors. As a result, the extension shares the same \(s_{m}(i)\) and \(s_{c}(j)\) scores as the base system. \(s_{pmc}\) is calculated by applying a ffnn to the cluster-mention pairwise representations. At test time, we convert \(s_{p}(i,j)\) into probabilities (\(p_{p}(i,j)\)), and assign split-antecedents to plural mentions when the \(p_{p}(i,j)\) between the plural mentions and the candidate clusters are above the threshold (e.g., \(0.5\)). \(p_{p}(i,j)\) is calculated by applying a sigmoid function to \(s_{p}(i,j)\):

\[p_{p}(i,j)=\frac{1}{1+e^{-s_{p}(i,j)}}\]

To make sure the final system outputs (single-antecedent anaphors, singletons, non-referring expressions and split-antecedent anaphors) do not contradict each other, we only allow discourse-new mentions to become split-antecedent anaphors. We also constrain split-antecedent anaphors to have at least two and at most five antecedents.

Since we are working with predicted clusters, to evaluate using lenient and strict scores as in Yu et al. (2020a), we need to find a way to align the predicted clusters with the gold clusters. Here we use the standard coreference alignment function CEAF\({}_{\phi 4}\) to align predicted and gold clusters. The alignment between predicted and gold clusters is at the centre of the CEAF\({}_{\phi 4}\) scores, which gives exactly what we need for our evaluation.

### Training Strategies

To train, we add to the original loss (\(loss_{s}\)) a second dedicated loss (\(loss_{p}\)) for split-antecedent anaphors. We use marginal log-likelihood loss, and optimize on all oracle clusters that belong to the gold split-antecedent cluster list gold\({}_{p}(i)\) of split-antecedent anaphors \(M_{i}\). Formally,

\[loss_{p}=log\prod_{j=1}^{N}\sum_{\hat{c}\in\textsc{gold}_{p}(j)}s_{p}(\hat{ c},j)\]

Since the vast majority of mentions (99%) are negative examples (non-split-antecedent anaphors), training is highly imbalanced. So during training we also use the mentions from the same cluster as the split-antecedent anaphors as additional positive examples. In this way we managed to nearly double the number of positive training examples. We multiply the losses of the negative examples an adjustment parameter \(\alpha\) to balance the training.

We train our system both in joint and pre-trained mode. For joint learning, we train our system on the sum of two losses and weigh them by a \(\beta\) factor that determines the relative importance of the losses. Formally, we compute the joint loss as follows:

\[loss_{j}=(1-\beta)loss_{s}+\beta loss_{p}\]

To use a joint loss the split-antecedent part of the system can have an impact on the mention representations hence might lead to better performance.

Our pre-trained approach is based on the hypothesis that mention/cluster representations trained on the single-antecedent anaphors are sufficient as pre-trained embeddings for downstream tasks like split-antecedent anaphors. The pre-trained approach minimises the changes to the base system, and one can even reuse the models trained solely for the base system. The training for the split-antecedent part is inexpensive. We use the pre-trained models for our base system to supply mention/cluster representations and other necessary information and optimise the split-antecedent part of the system solely on \(loss_{p}\).

## 4 Evaluating Coreference Chains with Split Antecedents

If the interpretation of a split-antecedent anaphor were only given credit when all antecedents are correctly detected and grouped together, without giving any reward to systems that find at least some of the antecedents, systems that get closer to the gold would be unfairly penalized, particularly for the cases with 3 or more split antecedents (25% in our data). Consider example 4.1, in which "their\({}_{i,j}\)" refers to the set {"Mary", "John"}, and "they\({}_{i,j,p}\)" to the set {"May", "John", "Jane"}. And take two systems A and B that resolve "their\({}_{i,j}\)" to {"Alex", "Jane"} and {"Mary", "Jane"}, respectively and "they\({}_{i,j,p}\)" to {"Alex"} and {"Myar\({}_{i}\)", "John\({}_{j}\)"}, respectively. Neither system is perfect, but intuitively, system B is more accurate in resolving split-antecedent anaphors (it correctly identifies 1 antecedent of "their\({}_{i,j}\)" and 2 of "they\({}_{i,j,p}\)", versus 0 for A)-yet both systems will receive the same 0 score if only a perfect match is credited.

**Example 4.1**.: Mary\({}_{i}\) and John\({}_{j}\) were on their way to visit Alex\({}_{k}\) when Mary\({}_{i}\) saw Jane\({}_{p}\) on their\({}_{i,j}\) way and realized they\({}_{i,j,p}\) all wore the same shirt.

This example indicates that in order to score a system carrying out both single and split 

antecedent resolution three issues have to be addressed. First of all, it is necessary to have some way to represent plural entities. Second, we need some way of ensuring that systems that propose different but equivalent resolutions for split-antecedent plurals score the same. Third, we need a metric allowing some form of partial credit.2 We discuss how we addressed each issue in turn.

Footnote 2: This third issue is the reason why Vala et al. (2016); Yu et al. (2020a) used lenient metrics for scoring split-antecedent resolution, although ones that did not score single antecedent resolution as well.

Plural mentionsFirst of all, we propose to have two types of mentions in our coreference chains: in addition to the standard **individual** mentions ("Mary"), we also allow **plural** mentions ({"Mary", "Jane"}).

Normalizing coreference chainsAs Example 4.1 shows, a text may contain multiple individual mentions of the same entity that participate in a plural mention (e.g. 'Mary'). Plural mentions whose antecedents are mentions of the same entity should be equivalent. To do this, we use the first mention of each gold coreference chains as the representative of the entity. We **normalize** every plural mention in a system- produced coreference chain by (i) aligning the system-produced coreference chains for the individual mentions in the plural mention to the gold coreference chains using ceaf, and (ii) replacing each individual mention in the plural mention with the first mention in the aligned gold coreference chains.

Partial creditA natural way to obtain a scorer for coreference resolution giving partial credit is to extend the lea evaluation metric Moosavi and Strube (2016) to handle split-antecedents. For each entity \(e\), lea evaluates (a) how important is \(e\), and (b) how well it is resolved. Thus, for computing recall, lea evaluates a set of system-detected entities \(E\) as follows:3

Footnote 3: We can compute precision by switching the role of system and key entities in lea computations.

\[\frac{\sum_{e\in E}\text{importance}(e)*\text{resolution-score}(e)}{\sum_{e\in E} \text{importance}(e)} \tag{1}\]

where _resolution-score_ is the ratio of correctly resolved coreference links in the entity, and the _importance_ measures how important is entity \(e\) in the given text. In the default implementation, _importance_ is set to the size of the entity. However, it can be adjusted based on the use case.

Let \(e\) be an entity in the system output \(E\) consisting of \(n\) mentions, and \(K\) be the set of gold entities. The _resolution-score_ (\(RS\)) of \(e\) is computed as:

\[RS(e)=\frac{1}{|\mathbb{L}(e)|}\sum_{l\in\mathbb{L}(e)}\mathbb{B}(l,K) \tag{2}\]

where \(\mathbb{L}(e)\) is the set of all coreference links in \(e^{4}\), and \(\mathbb{B}(l,K)\) is defined as

\[\mathbb{B}(l,K)=\begin{cases}1&\{\exists_{k\in K}|l\in\mathbb{L}(k)\}\\ 0&\text{otherwise}\end{cases} \tag{3}\]

(3) states that for each coreference link \(l\) in system entities, the system receives a reward of one if \(l\) also exists in gold entities, and zero otherwise. If any of the mentions that are connected by \(l\) is a partially resolved plural mention, the system receives a zero score.

To extend lea to handle split-antecedents, we change \(\mathbb{B}\) to also reward a system if any of the corresponding mentions of \(l\), i.e., mentions that are connected by \(l\), is a plural mention and is partially resolved. Let \(\hat{\mathbb{P}}(m)\) be an ordered list of all subsets of \(m\), including \(m\), by descending order of their size. If \(m\) is a singular mention, \(\hat{\mathbb{P}}\) will only contain \(\{m\}\). If \(m\) is a plural mention, \(\hat{\mathbb{P}}\) will contain \(m\) as well as all the subsets of \(m\)'s containing mentions. For instance, \(\hat{\mathbb{P}}\)({"Mary", "John"})=[ {"Mary", "John"}, {"John"}, {"May"}]. Assuming the corresponding mentions of \(l\) are \(m_{i}\) and \(m_{j}\), we update \(\mathbb{B}(l,K)\) as follows:

\[\small\begin{cases}\frac{|s_{i}|*|s_{j}|}{|m_{i}|*|m_{j}|}&\{\exists_{k\in K, s_{i}\in\hat{\mathbb{P}}(m_{i}),s_{j}\in\hat{\mathbb{P}}(m_{j})}|l_{s_{i},s_{ j}}\in\mathbb{L}(k)\}\\ \frac{|s_{i}|*|m_{j}|}{|m_{i}|*|m_{p}|}&\{\exists_{k\in K,m_{i}\in\hat{ \mathbb{P}}(m_{k}),m_{j}\in\hat{\mathbb{P}}(m_{p})}|lm_{m_{k},m_{p}}\in\mathbb{ L}(k)\}\\ 0&\text{otherwise}\end{cases}\]

where \(l_{s_{i},s_{j}}\) is the link connecting \(s_{i}\) and \(s_{j}\) that are the largest subset of \(\hat{\mathbb{P}}(m_{i})\) and \(\hat{\mathbb{P}}(m_{j})\), respectively, that exist in gold entities and are coreferent. \(m_{k}\) and \(m_{p}\) are gold coreferring mentions that \(m_{i}\) and \(m_{j}\) are a subset of, respectively.

For instance, consider the system chain {\(m_{1}\)={"Mary", "Jane"}, \(m_{2}\)="their\({}_{i,j}\)"} for Example 4.1. The coreference link between \(m_{1}\) and \(m_{2}\) does not exist in the gold entities. However, \(m_{1}\) is a subset of a gold mention, i.e., \(m_{k}\)={"Mary", "John", "Jane"}, and \(m_{1}\subset\hat{\mathbb{P}}(m_{k})\). Therefore, system B receives a reward of \(\frac{2*1}{3*1}\) for resolving the coreference link between \(m_{1}\) and \(m_{2}\) based on \(RS\) 

ImportanceAs discussed, the number of entities that contain split-antecedents in our annotated data is negligible compared to entities with singular mentions. Therefore, we will not see a big difference in the overall score when the system resolves both singular and plural mentions. In order to put more emphasize on harder coreference links, i.e., resolving split-antecedents, we adapt the _importance_ measure to assign a higher weight to entities containing split-antecedent as follows:

\[{\rm importance}(e)=\frac{{\rm importance\mbox{-factor}}(e)*|e|}{\sum_{e_{i}}{ \rm importance\mbox{-factor}}(e_{i})*|e_{i}|}\]

The _importance-factor_ assigns Imp\({}_{split}\) times higher importance on plural entities compared to entities of singular mentions:

\[{\rm importance\mbox{-factor}}(e)=\begin{cases}{\rm Imp}_{split}&\quad\text{If $e$ is a plural entity}\\ 1&\quad\text{If $e$ is singular}\end{cases}\]

## 5 Experiments

### Datasets

We evaluated our system on the rst portion of the arraau corpus Uryupina et al. (2020). arrau provides a wide range of anaphoric information (referring expressions including singletons and non-referring expressions; split-antecedent plurals; generic references; discourse deixis; and bridging references) and was used in the crac shared task Poesio et al. (2018); rst was the main evaluation subset in that task; the rst portion of the arraau corpus consists of 1/3 of the Penn Treebank (news texts). Table 1 summarizes the key statistics about the corpus.

### Separate and Joint Evaluation Methods

In **separate evaluation**, we follow standard practice to report conll average F1 score (macro average of MUC, B\({}^{3}\) and CEAF\({}_{\phi 4}\)) for single-antecedent anaphors, and F1 scores for non-referring expressions. For split-antecedent anaphors, we report three F1 scores: the strict F1 score that only gives credit when both anaphors and all their split-antecedents are resolved correctly6; the lenient F1 score that gives credit to anaphors that resolved partially correct Vala et al. (2016); and the anaphora recognition F1 score.

Footnote 6: Here we report F1 instead of accuracy used in Yu et al. (2020) as our evaluation is based on predicted mentions.

For **joint evaluation** of single/split-antecedent anaphors, we report the lea score using the upgraded script described in Section 4.

### Hyperparameters

We use the default parameter settings of Yu et al. (2020) and use their hybrid approach for handling the non-referring expressions. The split-antecedent part of the system uses an ffnn with two hidden layers and a hidden size of 150. The negative example loss adjustment parameter \(\alpha\) and the loss weight parameter \(\beta\) (used for joint learning) are set to 0.01 and 0.1 respectively after tuning on the development set. Table 2 provides details on our parameter settings.

## 6 Results and Discussions

### Separate Evaluation on Single/Split-antecedent Anaphors

We first evaluate our two proposed systems in the separate evaluation setting, in which we report separate scores for single-antecedent anaphors, non-referring expressions and split-antecedent anaphors. Showing individual scores for different aspects provide a clear picture of the different models.

Training settingsIn the joint setting, the system is trained end-to-end with a weighted loss func

\begin{table}
\begin{tabular}{l l l} \hline \hline  & **Train/Dev** & **Test** \\ \hline Documents & 353 & 60 \\ Sentences & 7524 & 1211 \\ Tokens & 195676 & 33225 \\ Mentions & 61671 & 10341 \\ Singletons & 26368 & 4158 \\ Non-referring expressions & 8159 & 1391 \\ Single-antecedent anaphors & 20127 & 3568 \\ Split-antecedent anaphors & 356 & 60 \\ Split-antecedents & 878 & 137 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Statistics about the corpus used for evaluation.

 

tion. In the pre-trained setting, we use the pretrained model provided by Yu et al. (2020), and train only the split-antecedent part of the system.

BaselinesLike Vala et al. (2016); Yu et al. (2020), we include baselines based on heuristic rules or random selection. For all baselines, we use the same model as used by our pre-trained approach to supply the candidate split-antecedent anaphors/singular clusters. The anaphora recognition baseline classifies as split-antecedent anaphors the discourse-new mentions belonging to a small list of plural pronoun (e.g., _they_, _their_, _them_, _we_).6 The **recent-x baseline** chooses the \(x\) closest singular clusters as antecedents for these candidates. The **random baseline** assigns two to five antecedents randomly to each chose split-antecedent anaphors.

Footnote 6: We also tried a random selection based approach, but such an approach only gets less than 5% split-antecedent anaphors correctly.

ResultsTable 3 shows the comparison between our two systems and the baselines. Since plural pronouns are the most frequent split-antecedanaphors, the simple heuristic gives a reasonably good F1 score of up to 36.2% for anaphora recognition. In term of the scores on full resolution, the baselines only achieved a maximum F1 of 17% and 5.7% when evaluated in the lenient and strict settings respectively. The low F1 scores indicate that split-antecedent anaphors are hard to resolve.

When compared with the baselines, both of our approaches achieved much better scores for all three evaluations. Our models achieved substantial improvements over the baselines of up to 19%, 19.9% and 14.7% for anaphora recognition, full resolution (lenient and strict) respectively. The model trained in a joint setting achieves a better recall for both lenient evaluation and anaphora recognition, while the pre-trained setting has much better precision. We expect this is because the joint system could have an impact on candidate mentions/clusters, hence potentially recover more antecedent-anaphora pairs. By contrast, the candidate mentions/clusters are fixed in the pre-trained setting. Overall, the joint model achieves a slightly better lenient F1 score but a lower strict F1 score, whereas the pre-trained setting has a better overall performance when compared with the joint model. The joint system also has a lower conll average F1 score and non-referring F1 score when compared with the system trained in a pre-trained fashion. This indicates that jointly training is not helpful for the single-antecedent anaphors and non-referring expressions. Hence we use the pre-trained approach for further experiments.

### Evaluating single and split antecedent anaphors jointly

We then evaluate our models with the newly extended lea scores to show how split-antecedanphors could impact the results when evaluated together with single-antecedent anaphors. Table 4 shows the lea score comparison between our best

\begin{table}
\begin{tabular}{l c c c c c c c} \hline  & \multicolumn{3}{c}{**Imp\({}_{split}\) = 1**} & \multicolumn{3}{c}{**Imp\({}_{split}\) = 10**} \\  & **R** & **P** & **F1** & **R** & **P** & **F1** \\ \hline Recent-2 & 70.5 & 66.9 & 68.7 & 61.5 & 61.3 & 61.4 \\ Recent-3 & 70.5 & 66.9 & 68.7 & 61.6 & 61.1 & 61.4 \\ Recent-4 & 70.6 & 66.9 & 68.7 & 61.8 & 61.1 & 61.5 \\ Recent-5 & 70.5 & 66.9 & 68.7 & 61.5 & 61.2 & 61.3 \\ Random & 70.4 & 66.7 & 68.5 & 60.9 & 60.0 & 60.4 \\ \hline Our model & **70.8** & **67.2** & **69.0** & **63.8** & **64.4** & **64.1** \\ \hline \end{tabular}
\end{table}
Table 4: lea evaluation on both single- and split-antecedent anaphors. Imp\({}_{split}\) indicates the split-antecedent importance.

\begin{table}
\begin{tabular}{l c c c c c c c c c c c c} \hline  & \multirow{2}{*}{CoNLL} & \multicolumn{3}{c}{**Non-referring**} & \multicolumn{3}{c}{**Anaphora Rec\({}_{split}\)**} & \multicolumn{3}{c}{**Lenient\({}_{split}\)**} & \multicolumn{3}{c}{**Strict\({}_{split}\)**} \\  & **F1** & **R** & **P** & **F1** & **R** & **P** & **F1** & **R** & **P** & **F1** & **R** & **P** & **F1** \\ \hline Recent-2 & - & - & - & - & 31.7 & 42.2 & 36.2 & 10.3 & 15.6 & 12.4 & 5.0 & 6.7 & 5.7 \\ Recent-3 & - & - & - & - & 31.7 & 42.2 & 36.2 & 16.9 & 17.0 & 17.0 & 0.0 & 0.0 & 0.0 \\ Recent-4 & - & - & - & - & 30.0 & 40.9 & 34.6 & 18.4 & 14.2 & 16.0 & 0.0 & 0.0 & 0.0 \\ Recent-5 & - & - & - & - & 28.3 & 39.5 & 33.0 & 16.9 & 10.7 & 13.1 & 0.0 & 0.0 & 0.0 \\ Random & - & - & - & - & 31.7 & 42.2 & 36.2 & 5.9 & 3.7 & 4.5 & 0.0 & 0.0 & 0.0 \\ \hline joint & 77.1 & **72.6** & 77.2 & 74.8 & **50.0** & 51.7 & 50.9 & **39.0** & 35.3 & **37.1** & 15.0 & 15.5 & 15.3 \\ pre-trained & **77.9** & 72.4 & **78.0** & **75.1** & 45.0 & **71.1** & **55.1** & 30.2 & **46.1** & 36.4 & **16.7** & **26.3** & **20.4** \\ \hline \end{tabular}
\end{table}
Table 3: Separate evaluation of our systems on the test set. X\({}_{split}\) are the scores for the split-antecedent anaphors 

model (pre-trained) and the baselines. As only half of the test documents contain split-antecedent anaphors, we report the results on those test documents to give a clear picture on the evaluation.

We carried out two evaluations. The first setting is the traditional evaluation setting for coreference, in which split-antecedent anaphors are weighed equally as single antecedent anaphors (i.e., they are treated in lea as a single mention, Imp\({}_{split}\) = 1). We do not believe, however, that treating all anaphors equally is the most informative approach to evaluating coreference, for it is well-known that some anaphors are much easier to resolve than others [32, 4]. lea makes it possible to give more weight to anaphors that are harder to resolve. So in our second evaluation we give more importance to split-antecedent anaphors (Imp\({}_{split}\) = 10) since they are much harder to resolve and also infrequent when compare to the single-antecedent anaphors. To have slightly higher importance for split-antecedents will give us a better view of their impact. The results in Table 4 show that our best model achieved moderate improvements of 0.3% - 0.5% on the first lea score setting when compared with the baselines. This is mainly because the split-antecedent anaphors are less than 1% of the mentions. But ss expected, the improvements become more clear in the second evaluation setting, in which our model is 2.6% - 3.7% better than the baselines.

### State-of-the-art Comparison

To compare with the state-of-the-art system on arrau, [89], we train our best setting (pre-trained) as yu2020learning did, i.e., assuming both gold mention and gold split-antecedent anaphors are provided. We first train the base model using gold mentions, then train the split-antecedent part of the system using gold mentions and gold split-antecedent anaphors. Since yu2020learning's system is evaluated on the full arrau corpus and with a customised train/test split priorities the split-antecedent anaphors, we retrain their system using the same standard rst split as used in our evaluation. We train their system with both baseline and the best settings using a single auxiliary corpus (single-coref).7 As shown in Table 5, our best model achieved both better lenient and better strict accuracy than the yu2020learning system, even though theirs is a dedicated system concerned only with split-antecedent anaphora. The results suggest the pre-trained mention/cluster representations are suitable for low-resource tasks that reply heavily on such representations.

Footnote 7: The best setting, that uses multi-auxiliary corpora, is more complex to train and only moderately improves the results.

### Analysis

In this section, we carry a qualitative analysis on the system outputs to find out the main courses of the performance gaps between the gold and predicted settings. We also report a more detailed comparison between our system and the yu2020learning system to see if there is a systematic difference between the two systems on the gold settings.

**The Challenge of Using Predicted Setting** The split-antecedent anaphora resolution task is more complex than its single-antecedent counterpart. The semantic relation between each individual antecedent and the anaphora is not identity, but element-of; and the number of antecedents can also vary. The results on evaluations with gold mentions and gold split-antecedent anaphors provided are promising. However, when evaluated using predicted mentions we have two main challenges: anaphora recognition and noisy candidate mentions/clusters. For anaphora recognition, our best model (pre-trained) only recalls 45% of the anaphors. The performance of our anaphora recognition is affected by the predicted mentions, and further capped by the fact that we only attempt to classify as split-antecedent the mentions classed as discourse-new by the base model. To assess the impact of these two factors, we computed the recall of split-antecedent anaphors by predicted mentions and discourse-new mentions. Virtually all split-antecedent anaphors are recalled among the predicted mentions-98.33%-but only 65% are recalled among the discourse-new mentions. This has a big impact on our results for split anaphora recognition, since 35% of the anaphors are not accessible to our system. To understand the impact of this gap on the result, we supply to our system the 98.33% of split-antecedent anaphors recognized as

\begin{table}
\begin{tabular}{l c c c c} \hline \hline  & \multicolumn{3}{c}{**Lenient**} & \multicolumn{2}{c}{**Strict**} \\  & **R** & **P** & **F1** & **Accuracy** \\ \hline Yu et al. Baseline & 61.0 & 52.5 & 56.5 & 21.7 \\ Yü et al. Best model & 69.1 & 63.9 & 66.4 & 35.0 \\ \hline Our model & **71.3** & **65.1** & **68.1** & **45.0** \\ \hline \hline \end{tabular}
\end{table}
Table 5: State-of-the-art comparison on the test set.

 

predicted. We keep everything else- predicted mentions, and clusters-unchanged. When run this way, our system achieves a lenient F1 score of 47.7%, which is 11.3 p.p. better than the score (36.4%) achieved using predicted anaphors, although still 20.4% lower than the model trained and evaluated with gold mentions and gold split-antecedent anaphors (68.1%). We suggest this additional difference is mainly a result of noise in the predicted mentions and clusters. Overall, then, the noise in the predicted mentions and clusters contributed 2/3 of the score difference, while problems with anaphora recognition are responsible for the rest.

**In Depth Comparison with Yu et al. (2020a).** Next, we compared our model's outputs in the gold setting with those of the best model of Yu et al. (2020a) in more detail. We split the test set in two different ways and compute the system performances on different categories. First, we follow Yu et al. (2020a) and split the split antecedent anaphors in the test set into two classes according to the number of gold split-antecedents: one class includes the anaphors with two split-antecedents, whereas the second class includes the anaphors with three or more split-antecedents (about 23% of the total). Table 6 compares these two classes. As we can see from the Table, with lenient evaluation the two systems work equally well for the anaphors with two split-antecedents, but our model is 8.5% better for mentions with three or more split-antecedents. In terms of strict evaluation, our model outperforms the Yu et al. (2020a) model by 8.7% and 14.3% for two classes respectively. Overall, the model presented here achieved substantial performance gains on anaphors with three or more split-antecedents.

We then split the data into two classes according to a different criterion: the part-of-speech of the anaphor. The first class consists of pronoun anaphors, such as "they" or "their". The second class consists of all other split antecedent anaphors, such as "those companies" or "both". As shown in Table 7, the Yu et al. (2020a) model achieves better scores for pronoun anaphors (mainly "they" and "their"). However, our new model outperforms the old system with non-pronominal anaphors by 5.4% according to lenient F1, and doubled their strict accuracy.

## 7 Conclusions

In this paper, we introduced a neural system performing both single and split-antecedent anaphora resolution, and evaluated the system in a more realistic setting than previous work. We extended the state-of-the-art coreference system on arrau to also resolve split-antecedent anaphors. The proposed system achieves much better results on split-antecedent anaphors when compared with the baselines using heuristic and random selection when using the predicted mentions/clusters. Our system also achieves better results than the previous state-of-the-art system on arrauYu et al. (2020a), which only attempted single-antecedent anaphora resolution from gold mentions, when evaluated on the same task.

In addition, we also proposed an extension of the lea coreference evaluation metric to evaluate both single and split-antecedent anaphors in a single metric.

## Acknowledgements

This research was supported in part by the DALI project, ERC Grant 695662.

## References

* Barbu and Mitkov [2001] Catalina Barbu and Ruslan Mitkov. 2001. Evaluation tool for rule-based anaphora resolution methods. In _Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics_, pages 34-41, Toulouse, France. Association for Computational Linguistics.
* Clark and Manning [2016a] Kevin Clark and Christopher D. Manning. 2016a. Deep reinforcement learning for mention-ranking coreference models. In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_, pages 2256-2262, Austin, Texas. Association for Computational Linguistics.
* Clark and Manning [2016b] Kevin Clark and Christopher D. Manning. 2016b. Improving coreference resolution by learning entity

\begin{table}
\begin{tabular}{l c c c c c} \hline \hline  & \multicolumn{3}{c}{**Yu et al.**} & \multicolumn{3}{c}{**Our model**} \\  & **Count** & **Lenient** & **Strict** & **Lenient** & **Strict** \\ \hline
2 & 46 & 71.9 & 45.7 & 70.9 & 54.4 \\
3+ & 14 & 52.5 & 0.0 & 61.0 & 14.3 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Scores for anaphors with different number of antecedents.

\begin{table}
\begin{tabular}{l c c c c c} \hline \hline  & \multicolumn{3}{c}{**Yu et al.**} & \multicolumn{3}{c}{**Our model**} \\  & **Count** & **Lenient** & **Strict** & **Lenient** & **Strict** \\ \hline PRP & 24 & 82.4 & 58.3 & 76.4 & 54.2 \\ Other & 36 & 57.5 & 19.4 & 62.9 & 38.9 \\ \hline \hline \end{tabular}
\end{table}
Table 7: Scores for pronoun and other anaphors.

 

level distributed representations. In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 643-653, Berlin, Germany. Association for Computational Linguistics.
* Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_, pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
* Hou (2020) Yufang Hou. 2020. Bridging anaphora resolution as question answering. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 1428-1438, Online. Association for Computational Linguistics.
* Joshi et al. (2020) Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2020. SpanBERT: Improving pre-training by representing and predicting spans. _Transactions of the Association for Computational Linguistics_, 8:64-77.
* Joshi et al. (2019) Mandar Joshi, Omer Levy, Luke Zettlemoyer, and Daniel Weld. 2019. BERT for coreference resolution: Baselines and analysis. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_, pages 5803-5808, Hong Kong, China. Association for Computational Linguistics.
* Kantor and Globerson (2019) Ben Kantor and Amir Globerson. 2019. Coreference resolution with entity equalization. In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_, pages 673-677, Florence, Italy. Association for Computational Linguistics.
* Kolhatkar and Hirst (2014) Varada Kolhatkar and Graeme Hirst. 2014. Resolving shell nouns. In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 499-510, Doha, Qatar. Association for Computational Linguistics.
* Kolhatkar et al. (2018) Varada Kolhatkar, Adam Roussel, Stefanie Dipper, and Heike Zinsmeister. 2018. Anaphora with nonnominal antecedents in computational linguistics: a Survey. _Computational Linguistics_, 44(3):547-612.
* Lee et al. (2013) Heeyoung. Lee, Angel. Chang, Yves. Peirsman, Nathaneal. Chambers, Mihai. Surdeanu, and Dan. Jurafsky. 2013. Deterministic coreference resolution based on entity-centric, precision-ranked rules. _Computational Linguistics_, 39(4):885-916.
* Lee et al. (2017) Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_, pages 188-197, Copenhagen, Denmark. Association for Computational Linguistics.
* Lee et al. (2018) Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018. Higher-order coreference resolution with coarse-to-fine inference. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_, pages 687-692, New Orleans, Louisiana. Association for Computational Linguistics.
* Liu et al. (2017) Quan Liu, Hui Jiang, Andrew Evdokimov, Zhen-Hua Ling, Xiaodan Zhu, Si Wei, and Yu Hu. 2017. Cause-effect knowledge acquisition and neural association model for solving a set of winograd schema problems. In _Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17_, pages 2344-2350.
* Marasovic et al. (2017) Ana Marasovic, Leo Born, Juri Opitz, and Anette Frank. 2017. A mention-ranking model for abstract anaphora resolution. In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_, pages 221-232, Copenhagen, Denmark. Association for Computational Linguistics.
* Moosavi and Strube (2016) Nafise Sadat Moosavi and Michael Strube. 2016. Which coreference evaluation metric do you trust? a proposal for a link-based entity aware metric. In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 632-642, Berlin, Germany. Association for Computational Linguistics.
* Peng et al. (2015) Haoruo Peng, Daniel Khashabi, and Dan Roth. 2015. Solving hard coreference problems. In _Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 809-819, Denver, Colorado. Association for Computational Linguistics.
* Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 1532-1543, Doha, Qatar. Association for Computational Linguistics.
* Peters et al. (2018) Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_, pages 2227-2237, New Orleans, Louisiana. Association for Computational Linguistics.
* Poesio and Artstein (2008) Massimo Poesio and Ron Artstein. 2008. Anaphoric annotation in the ARRAU corpus. In _Proceedings of 

the Sixth International Conference on Language Resources and Evaluation (LREC'08)_, Marrakech, Morocco. European Language Resources Association (ELRA).
* Poesio et al. (2018) Massimo Poesio, Yulia Grishina, Varada Kolhatkar, Nafise Moosavi, Ina Roesiger, Adam Roussel, Fabian Simonjetz, Alexandra Uma, Olga Urvupina, Juntao Yu, and Heike Zinsmensiter. 2018. Anaphora resolution with the ARRAU corpus. In _Proceedings of the First Workshop on Computational Models of Reference, Anaphora and Coreference_, pages 11-22, New Orleans, Louisiana. Association for Computational Linguistics.
* Pradhan et al. (2012) Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Urvupina, and Yuchen Zhang. 2012. CoNLL-2012 shared task: Modeling multilingual unrestricted coreference in OntoCNotes. In _Proceedings of the Sixteenth Conference on Computational Natural Language Learning (CoNLL 2012)_, Jeju, Korea.
* Rahman and Ng (2012) Altaf Rahman and Vincent Ng. 2012. Resolving complex cases of definite pronouns: The Winograd schema challenge. In _Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning_, pages 777-789, Jeju Island, Korea. Association for Computational Linguistics.
* Sakaguchi et al. (2020) Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2020. Winogrande: An adversarial winograd schema challenge at scale. _Proceedings of the AAAI Conference on Artificial Intelligence_, 34:8732-8740.
* Uryupina et al. (2020) Olga Uryupina, Ron Artstein, Antonella Bristot, Federica Cavicchio, Francesca Delogu, Kepa J. Rodriguez, and Massimo Poseio. 2020. Annotating a broad range of anaphoric phenomena, in a variety of genres: the ARRAU corpus. _Journal of Natural Language Engineering_.
* Vala et al. (2016) Hardik Vala, Andrew Piper, and Derek Ruths. 2016. The more antecedents, the merrier: Resolving multi-antecedent anaphors. In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 2287-2296, Berlin, Germany. Association for Computational Linguistics.
* Webster et al. (2018) Kellie Webster, Marta Recasens, Vera Axelrod, and Jason Baldridge. 2018. Mind the GAP: A balanced corpus of gendered ambiguous pronouns. _Transactions of the Association for Computational Linguistics_, 6:605-617.
* Wiseman et al. (2015) Sam Wiseman, Alexander M. Rush, Stuart Shieber, and Jason Weston. 2015. Learning anaphoricity and antecedent ranking features for coreference resolution. In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_, pages 1416-1426, Beijing, China. Association for Computational Linguistics.
* Wiseman et al. (2016) Sam Wiseman, Alexander M. Rush, and Stuart M. Shieber. 2016. Learning global features for coreference resolution. In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_, pages 994-1004, San Diego, California. Association for Computational Linguistics.
* Wu et al. (2020) Wei Wu, Fei Wang, Arianna Yuan, Fei Wu, and Jiwei Li. 2020. CorefQA: Coreference resolution as query-based span prediction. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 6953-6963, Online. Association for Computational Linguistics.
* Yu et al. (2020a) Juntao Yu, Nafise Sadat Moosavi, Siliviu Paun, and Massimo Poesio. 2020a. Free the plural: Unrestricted split-antecedent anaphora resolution. In _Proceedings of the 28th International Conference on Computational Linguistics_, pages 6113-6125, Barcelona, Spain (Online). International Committee on Computational Linguistics.
* Yu and Poesio (2020b) Juntao Yu and Massimo Poesio. 2020. Multitask learning based neural bridging reference resolution. In _Proceedings of the 28th International Conference on Computational Linguistics_, pages 3534-3546, Barcelona, Spain (Online). International Committee on Computational Linguistics.
* Yu et al. (2020c) Juntao Yu, Alexandra Uma, and Massimo Poesio. 2020c. A cluster ranking model for full anaphora resolution. In _Proceedings of the 12th Language Resources and Evaluation Conference_, pages 11-20, Marseille, France. European Language Resources Association.
* Zhou and Choi (2018) Ethan Zhou and Jinho D. Choi. 2018. They exist! introducing plural mentions to coreference resolution and entity linking. In _Proceedings of the 27th International Conference on Computational Linguistics_, pages 24-34, Santa Fe, New Mexico, USA. Association for Computational Linguistics.