Venue: NAACL 2016

# Learning Global Features for Coreference Resolution

Sam Wiseman and Alexander M. Rush and Stuart M. Shieber

School of Engineering and Applied Sciences

Harvard University

Cambridge, MA, USA {swiseman,srush,shieber}@seas.harvard.edu

###### Abstract

There is compelling evidence that coreference prediction would benefit from modeling global information about entity-clusters. Yet, state-of-the-art performance can be achieved with systems treating each mention prediction independently, which we attribute to the inherent difficulty of crafting informative cluster-level features. We instead propose to use recurrent neural networks (RNNs) to learn latent, global representations of entity clusters directly from their mentions. We show that such representations are especially useful for the prediction of pronominal mentions, and can be incorporated into an end-to-end coreference system that outperforms the state of the art without requiring any additional search.

## 1 Introduction

While structured, non-local coreference models would seem to hold promise for avoiding many common coreference errors (as discussed further in Section 3), the results of employing such models in practice are decidedly mixed, and state-of-the-art results can be obtained using a completely local, mention-ranking system.

In this work, we posit that global context is indeed necessary for further improvements in coreference resolution, but argue that informative _cluster_, rather than mention, level features are very difficult to devise, limiting their effectiveness. Accordingly, we instead propose to learn representations of mention clusters by embedding them sequentially using a recurrent neural network (shown in Section 4). Our model has no manually defined cluster features, but instead learns a global representation from the individual mentions present in each cluster. We incorporate these representations into a mention-ranking style coreference system.

The entire model, including the recurrent neural network and the mention-ranking sub-system, is trained end-to-end on the coreference task. We train the model as a local classifier with fixed context (that is, as a history-based model). As such, unlike several recent approaches, which may require complicated inference during training, we are able to train our model in much the same way as a vanilla mention-ranking model.

Experiments compare the use of learned global features to several strong baseline systems for coreference resolution. We demonstrate that the learned global representations capture important underlying information that can help resolve difficult pronominal mentions, which remain a persistent source of errors for modern coreference systems Durrett and Klein (2013); Kummerfeld and Klein (2013); Wiseman et al. (2015); Martschat and Strube (2015). Our final system improves over 0.8 points in CoNLL score over the current state of the art, and the improvement is statistically significant on all three CoNLL metrics.

## 2 Background and Notation

Coreference resolution is fundamentally a clustering task. Given a sequence \((x_{n})^{N}_{n=1}\) of (intra-document) mentions - that is, syntactic units that can refer or be referred to - coreference resolution involves partitioning \((x_{n})\) into a sequence of clusters \((X^{(m)})^{M}_{m=1}\) such that all the mentions in any particular cluster

\(X^{(m)}\) refer to the same underlying entity. Since the mentions within a particular cluster may be ordered linearly by their appearance in the document,1 we will use the notation \(X_{j}^{(m)}\) to refer to the \(j\)'th mention in the \(m\)'th cluster.

Footnote 1: We assume nested mentions are ordered by their syntactic heads.

A valid clustering places each mention in exactly one cluster, and so we may represent a clustering with a vector \(\boldsymbol{z}\!\in\!\{1,\ldots,M\}^{N}\), where \(z_{n}\!=\!m\) iff \(x_{n}\) is a member of \(X^{(m)}\). Coreference systems attempt to find the best clustering \(\boldsymbol{z}^{*}\in\mathcal{Z}\) under some scoring function, with \(\mathcal{Z}\) the set of valid clusterings.

One strategy to avoid the computational intractability associated with predicting an entire clustering \(\boldsymbol{z}\) is to instead predict a single _antecedent_ for each mention \(x_{n}\); because \(x_{n}\) may not be anaphoric (and therefore have no antecedents), a "dummy" antecedent \(\epsilon\) may also be predicted. The aforementioned strategy is adopted by "mention-ranking" systems Denis and Baldridge (2008); Rahman and Ng (2009); Durrett and Klein (2013), which, formally, predict an antecedent \(\hat{y}\!\in\!\mathcal{Y}(x_{n})\) for each mention \(x_{n}\), where \(\mathcal{Y}(x_{n})\!=\!\{1,\ldots,n-1,\epsilon\}\). Through transitivity, these decisions induce a clustering over the document.

Mention-ranking systems make their antecedent predictions with a _local_ scoring function \(f(x_{n},y)\) defined for any mention \(x_{n}\) and any antecedent \(y\!\in\!\mathcal{Y}(x_{n})\). While such a scoring function clearly ignores much structural information, the mention-ranking approach has been attractive for at least two reasons. First, inference is relatively simple and efficient, requiring only a left-to-right pass through a document's mentions during which a mention's antecedents (as well as \(\epsilon\)) are scored and the highest scoring antecedent is predicted. Second, from a linguistic modeling perspective, mention-ranking models learn a scoring function that requires a mention \(x_{n}\) to be compatible with only _one_ of its coreferent antecedents. This contrasts with mention-pair models (e.g., Bengtson and Roth (2008)), which score all pairs of mentions in a cluster, as well as with certain cluster-based models (see discussion in Culotta et al. (2007)). Modeling each mention as having a single antecedent is particularly advantageous for pronominal mentions, which we might like to model as linking to a single nominal or proper antecedent, for example, but not necessarily to all other coreferent mentions.

Accordingly, in this paper we attempt to maintain the inferential simplicity and modeling benefits of mention ranking, while allowing the model to utilize global, structural information relating to \(\boldsymbol{z}\) in making its predictions. We therefore investigate objective functions of the form

\[\operatorname*{arg\,max}_{y_{1},...,y_{N}}\sum_{n=1}^{N}f(x_{n},y_{n})+g(x_{ n},y_{n},\boldsymbol{z}_{1:n-1})\qquad,\]

where \(g\) is a global function that, in making predictions for \(x_{n}\), may examine (features of) the clustering \(\boldsymbol{z}_{1:n-1}\) induced by the antecedent predictions made through \(y_{n-1}\).

## 3 The Role of Global Features

Here we motivate the use of global features for coreference resolution by focusing on the issues that may arise when resolving pronominal mentions in a purely local way. See Clark and Manning (2015) and Stoyanov and Eisner (2012) for more general motivation for using global models.

### Pronoun Problems

Recent empirical work has shown that the resolution of pronominal mentions accounts for a substantial percentage of the total errors made by modern mention-ranking systems. Wiseman et al. (2015) show that on the CoNLL 2012 English development set, almost 59% of mention-ranking precision errors and almost 24% of recall errors involve pronominal mentions. Martschat and Strube (2015) found a similar pattern in their comparison of mention-ranking, mention-pair, and latent-tree models.

To see why pronouns can be so problematic, consider the following passage from the "Broadcast Conversation" portion of the CoNLL development set (bc/msnbc/0000/018); below, we enclose mentions in brackets and give the same subscript to co-clustered mentions. (This example is also shown in Figure 2.)

* Go ahead [Linda]\({}_{2}\).
**LW:** Well and uh thanks goes to [you]\({}_{1}\) and to [the media]\({}_{3}\) to help [us]\({}_{4}\)...So [our]\({}_{4}\) hat is off to all of [you]\({}_{5}\) as well.

 

This example is typical of Broadcast Conversation, and it is difficult because local systems learn to myopically link pronouns such as \([{\rm you}]_{5}\) to other instances of the same pronoun that are close by, such as \([{\rm you}]_{1}\). While this is often a reasonable strategy, in this case predicting \([{\rm you}]_{1}\) to be an antecedent of \([{\rm you}]_{5}\) would result in the prediction of an incoherent cluster, since \([{\rm you}]_{1}\) is coreferent with the singular \([{\rm I}]_{1}\), and \([{\rm you}]_{5}\), as part of the phrase "all of you," is evidently plural. Thus, while there is enough information in the text to correctly predict \([{\rm you}]_{5}\), doing so crucially depends on having access to the _history_ of predictions made so far, and it is precisely this access to history that local models lack.

More empirically, there are non-local statistical regularities involving pronouns we might hope models could exploit. For instance, in the CoNLL training data over 70% of pleonastic "it" instances and over 74% of pleonastic "you" instances follow (respectively) previous pleonsatic "it" and "you" instances. Similarly, over 78% of referential "I" instances and over 68% of referential "he" instances corefer with previous "I" and "he" instances, respectively.

Accordingly, we might expect non-local models with access to global features to perform significantly better. However, models incorporating non-local features have a rather mixed track record. For instance, Bjorkelund and Kuhn (2014) found that cluster-level features improved their results, whereas Martschat and Strube (2015) found that they did not. Clark and Manning (2015) found that incorporating cluster-level features _beyond_ those involving the precomputed mention-pair and mention-ranking probabilities that form the basis of their agglomerative clustering coreference system did not improve performance. Furthermore, among recent, state-of-the-art systems, mention-ranking systems (which are completely local) perform at least as well as their more structured counterparts (Durrett and Klein, 2014; Clark and Manning, 2015; Wiseman et al., 2015; Peng et al., 2015).

### Issues with Global Features

We believe a major reason for the relative inefficiveness of global features in coreference problems is that, as noted by Clark and Manning (2015), cluster-level features can be hard to define. Specifically, it is difficult to define discrete, fixed-length features on clusters, which can be of variable size (or shape). As a result, global coreference features tend to be either too coarse or too sparse. Thus, early attempts at defining cluster-level features simply applied the coarse quantifier predicates _all_, _none_, _most_ to the mention-level features defined on the mentions (or pairs of mentions) in a cluster (Culotta et al., 2007; Rahman and Ng, 2011). For example, a cluster would have the feature 'most-female=true' if more than half the mentions (or pairs of mentions) in the cluster have a 'female=true' feature.

On the other extreme, Bjorkelund and Kuhn (2014) define certain cluster-level features by concatenating the mention-level features of a cluster's constituent mentions in order of the mentions' appearance in the document. For example, if a cluster consists, in order, of the mentions (_the president_, _he_, _he_), they would define a cluster-level "type" feature 'C-P-P=true', which indicates that the cluster is composed, in order, of a common noun, a pronoun, and a pronoun. While very expressive, these concatenated features are often quite sparse, since clusters encountered during training can be of any size.

## 4 Learning Global Features

To circumvent the aforementioned issues with defining global features, we propose to learn cluster-level feature representations implicitly, by identifying the state of a (partial) cluster with the hidden state of an RNN that has consumed the sequence of mentions composing the (partial) cluster. Before providing technical details, we provide some preliminary evidence that such learned representations capture important contextual information by displaying in Figure 1 the learned final states of all clusters in the CoNLL development set, projected using T-SNE (van der Maaten and Hinton, 2012). Each point in the visualization represents the learned features for an entity cluster and the head words of mentions are shown for representative points. Note that the model learns to roughly separate clusters by simple distinctions such as predominant type (nominal, proper, pronominal) and number (it, they, etc), but also captures more subtle relationships such as grouping geographic terms and long strings of pro 



### Recurrent Neural Networks

A recurrent neural network is a parameterized nonlinear function \({\bf RNN}\) that recursively maps an input sequence of vectors to a sequence of hidden states. Let \((\mathbf{m}_{j})^{J}_{j=1}\) be a sequence of \(J\) input vectors \(\mathbf{m}_{j}\!\in\!\mathbb{R}^{D}\), and let \(\mathbf{h}_{0}\!=\!\mathbf{0}\). Applying an RNN to any such sequence yields

\[\mathbf{h}_{j}\leftarrow{\bf RNN}(\mathbf{m}_{j},\mathbf{h}_{j-1};\mathbf{\theta})\qquad,\]

where \(\mathbf{\theta}\) is the set of parameters for the model, which are shared over time.

There are several varieties of RNN, but by far the most commonly used in natural-language processing is the Long Short-Term Memory network (LSTM) (Hochreiter and Schmidhuber, 1997), particularly for language modeling (e.g., Zaremba et al. (2014)) and machine translation (e.g., Sutskever et al. (2014)), and we use LSTMs in all experiments.

### RNNs for Cluster Features

Our main contribution will be to utilize RNNs to produce feature representations of entity clusters which will provide the basis of the global term \(g\). Recall that we view a cluster \(X^{(m)}\) as a sequence of mentions \((X^{(m)}_{j})^{J}_{j=1}\) (ordered in linear document order). We therefore propose to embed the state(s) of \(X^{(m)}\) by running an RNN over the cluster in order.

In order to run an RNN over the mentions we need an embedding function \(\mathbf{h}_{\mathrm{c}}\) to map a mention to a real vector. First, following Wiseman et al. (2015) define \(\mathbf{\phi}_{\mathrm{a}}(x_{n}):\mathcal{X}\rightarrow\{0,1\}^{F}\) as a standard set of local indicator features on a mention, such as its head word, its gender, and so on. (We elaborate on features below.) We then use a non-linear feature embedding \(\mathbf{h}_{\mathrm{c}}\) to map a mention \(x_{n}\) to a vector-space representation. In particular, we define

\[\mathbf{h}_{\mathrm{c}}(x_{n})\triangleq\tanh(\mathbf{W}_{\mathrm{c}}\,\mathbf{\phi}_{ \mathrm{a}}(x_{n})+\mathbf{b}_{\mathrm{c}})\qquad,\]

where \(\mathbf{W}_{\mathrm{c}}\) and \(\mathbf{b}_{\mathrm{c}}\) are parameters of the embedding.

We will refer to the \(j\)'th hidden state of the RNN corresponding to \(X^{(m)}\) as \(\mathbf{h}_{j}^{(m)}\), and we obtain it according to the following formula

\[\mathbf{h}_{j}^{(m)}\leftarrow{\bf RNN}(\mathbf{h}_{\mathrm{c}}(X^{(m)}_{j}),\mathbf{h}_{ j-1}^{(m)};\mathbf{\theta})\qquad,\]

again assuming that \(\mathbf{h}_{0}^{(m)}\!=\!\mathbf{0}\). Thus, we will effectively run an RNN over each (sequence of mentions corresponding to a) cluster \(X^{(m)}\) in the document, and thereby generate a hidden state \(\mathbf{h}_{j}^{(m)}\) corresponding to each step of each cluster in the document. Concretely, this can be implemented by maintaining \(M\) RNNs - one for each cluster - that all share the parameters \(\mathbf{\theta}\). The process is illustrated in the top portion of Figure 2.

## 5 Coreference with Global Features

We now describe how the RNN defined above is used within an end-to-end coreference system.

### Full Model and Training

Recall that our inference objective is to maximize the score of both a local mention ranking term as well as a global term based on the current clusters:

\[\operatorname*{arg\,max}_{y_{1},\ldots,y_{N}}\sum_{n=1}^{N}f(x_{n},y_{n})+g( x_{n},y_{n},\mathbf{z}_{1:n-1})\]

We begin by defining the local model \(f(x_{n},y)\) with the two layer neural network of Wiseman et al. (2015), which has a specialization for the non-anaphoric case, as follows:

\[f(x_{n},y)\triangleq\begin{cases}\mathbf{u}^{\intercal}\left[\begin{smallmatrix} \mathbf{h}_{\mathrm{a}}(x_{n})\\ \mathbf{h}_{\mathrm{p}}(x_{n},y)\end{smallmatrix}\right]+u_{0}&\text{if $y\neq \epsilon$}\\ v^{\intercal}\mathbf{h}_{\mathrm{a}}(x_{n})+v_{0}&\text{if $y=\epsilon$}\qquad.\end{cases}\]

Figure 1: T-SNE visualization of learned entity representations on the CoNLL development set. Each point shows a gold cluster of size \(>1\). Yellow, red, and purple points represent predominantly common noun, proper noun, and pronoun clusters, respectively. Captions show head words of representative clusters’ mentions.



Above, \({\mathbf{u}}\) and \({\mathbf{v}}\) are the parameters of the model, and \({\mathbf{h}}_{\rm a}\) and \({\mathbf{h}}_{\rm p}\) are learned feature embeddings of the local mention context and the pairwise affinity between a mention and an antecedent, respectively. These feature embeddings are defined similarly to \({\mathbf{h}}_{\rm c}\), as

\[{\mathbf{h}}_{\rm a}(x_{n}) \triangleq\tanh({\mathbf{W}}_{\rm a}\,{\mathbf{\phi}}_{\rm a}(x_{n})+{\bm {b}}_{\rm a})\] \[{\mathbf{h}}_{\rm p}(x_{n},y) \triangleq\tanh({\mathbf{W}}_{\rm p}\,{\mathbf{\phi}}_{\rm p}(x_{n},y)+{ \mathbf{b}}_{\rm p})\qquad,\]

where \({\mathbf{\phi}}_{\rm a}\) (mentioned above) and \({\mathbf{\phi}}_{\rm p}\) are "raw" (that is, unconjoined) features on the context of \(x_{n}\) and on the pairwise affinity between mentions \(x_{n}\) and antecedent \(y\), respectively Wiseman et al. (2015). Note that \({\mathbf{h}}_{\rm a}\) and \({\mathbf{h}}_{\rm c}\) use the same raw features; only their weights differ.

We now specify our global scoring function \(g\) based on the history of previous decisions. Define \({\mathbf{h}}^{(m)}_{<n}\) as the hidden state of cluster \(m\) before a decision is made for \(x_{n}\) - that is, \({\mathbf{h}}^{(m)}_{<n}\) is the state of cluster \(m\)'s RNN after it has consumed all mentions in the cluster _preceding_\(x_{n}\). We define \(g\) as

\[g(x_{n},y,{\mathbf{z}}_{1:n-1})~{}\triangleq\begin{cases}{\mathbf{h}}_{\rm c}(x_{n})^ {\mathsf{T}}{\mathbf{h}}^{(z_{y})}_{<n}&\text{if $y\neq\epsilon$}\\ \operatorname{NA}(x_{n})&\text{if $y=\epsilon$}\end{cases},\]

where \(\operatorname{NA}\) gives a score for assigning \(\epsilon\) based on a non-linear function of all of the current hidden states:

\[\operatorname{NA}(x_{n})={\mathbf{q}}^{\mathsf{T}}\tanh\left({\mathbf{ W}}_{s}\left[\genfrac{}{}{}{\Phi}{}{}{}_{\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! \! 

\[\sum_{n=1}^{N}\max_{\hat{y}\in\mathcal{Y}(x_{n})}\Delta(x_{n},\hat{ y}) (1+f(x_{n},\hat{y})+g(x_{n},\hat{y},\boldsymbol{z}^{(o)})\] \[-f(x_{n},y_{n}^{\ell})-g(x_{n},y_{n}^{\ell},\boldsymbol{z}^{(o)})),\]

where the latent antecedent \(y_{n}^{\ell}\) is defined as

\[y_{n}^{\ell}\triangleq\operatorname*{arg\,max}_{y\in\mathcal{Y}(x_{n}):z_{j} ^{(o)}=z_{i}^{(o)}}f(x_{n},y)+g(x_{n},y,\boldsymbol{z}^{(o)})\]

if \(x_{n}\) is anaphoric, and is \(\epsilon\) otherwise. The term \(\Delta(x_{n},\hat{y})\) gives different weight to different error types. We use a \(\Delta\) with 3 different weights \((\alpha_{1},\alpha_{2},\alpha_{3})\) for "false link" (fl), "false new" (fn), and "wrong link" (wl) mistakes Durrett and Klein (2013), which correspond to predicting an antecedent when non-anaphoric, \(\epsilon\) when anaphoric, and the wrong antecedent, respectively.

Note that in training we use the oracle clusters \(\boldsymbol{z}^{(o)}\). Since these are known a priori, we can precompute all the hidden states \(\boldsymbol{h}_{j}^{(m)}\) in a document, which makes training quite simple and efficient. This approach contrasts in particular with the work of Bjorkelund and Kuhn (2014) -- who also incorporate global information in mention-ranking -- in that they train against latent _trees_, which are not annotated and must be searched for during training. On the other hand, training on oracle clusters leads to a mismatch between training and test, which can hurt performance.

### Search

When moving from a strictly local objective to one with global features, the test-time search problem becomes intractable. The local objective requires \(O(n^{2})\) time, whereas the full clustering problem is NP-Hard. Past work with global features has used integer linear programming solvers for exact search Chang et al. (2013); Peng et al. (2015), or beam search with (delayed) early update training for an approximate solution Bjorkelund and Kuhn (2014). In contrast, we simply use greedy search at test time, which also requires \(O(n^{2})\) time.3 The full algorithm is shown in Algorithm 1. The greedy search algorithm is identical to a simple mention-ranking system, with the exception of line 11, which updates the current RNN representation based on the previous decision that was made, and line 4, which then uses this cluster representation as part of scoring.

Footnote 3: While beam search is a natural way to decrease search error at test time, it may fail to help if training involves a _local_ margin objective (as in our case), since scores need not be calibrated across local decisions. We accordingly attempted to train various locally normalized versions of our model, but found that they underperformed. We also experimented with training approaches and model variants that expose the model to its own predictions Daume III et al. (2009); Ross et al. (2011); Bengio et al. (2015), but found that these yielded a negligible performance improvement.

```
1:procedureGreedyCluster(\(x_{1},\ldots,x_{N}\))
2: Initialize clusters \(X^{(1)}\ldots\) as empty lists, hidden states \(\boldsymbol{h}^{(0)},\ldots\) as **0** vectors in \(\mathbb{R}^{D}\), \(\boldsymbol{z}\) as map from mention to cluster, and cluster counter \(M\gets 0\)
3:for\(n=2\dots N\)do
4:\(y^{*}\leftarrow\operatorname*{arg\,max}_{y\in\mathcal{Y}(x_{n})}f(x_{n},y)+ g(x_{n},y,\boldsymbol{z}_{1:n-1})\)
5:\(m\leftarrow z_{y^{*}}\)
6:if\(y^{*}=\boldsymbol{c}\)then
7:\(M\gets M+1\)
8:\(m\gets M\)
9: append \(x_{n}\) to \(X^{(m)}\)
10:\(z_{n}\gets m\)
11:\(\boldsymbol{h}^{(m)}\leftarrow\mathbf{RNN}(\boldsymbol{h}_{c}(x_{n}), \boldsymbol{h}^{(m)})\)
12:return\(X^{(1)},\ldots,X^{(M)}\)
```

**Algorithm 1** Greedy search with global RNNs 

genre (out of {bc,bn,mz,nw,pt,tc,wb}) indicator to \(\phi_{\rm p}\) and \(\phi_{\rm a}\).
* We add features indicating if a mention has a substring overlap with the current speaker (\(\phi_{\rm p}\) and \(\phi_{\rm a}\)), and if an antecedent has a substring overlap with a speaker distinct from the current mention's speaker (\(\phi_{\rm p}\)).
* We add a single centered, rescaled document position feature to each mention when learning \(h_{c}\). We calculate a mention \(x_{n}\)'s rescaled document position as \(\frac{2n-N-1}{N-1}\).

These modifications result in there being approximately 14K distinct features in \(\phi_{\rm a}\) and approximately 28K distinct features in \(\phi_{\rm p}\), which is far fewer features than has been typical in past work.

For training, we use document-size minibatches, which allows for efficient pre-computation of RNN states, and we minimize the loss described in Section 5 with AdaGrad [3] (after clipping LSTM gradients to lie (elementwise) in \((-10,10)\)). We find that the initial learning rate chosen for AdaGrad has a significant impact on results, and we choose learning rates for each layer out of \(\{0.1,0.02,0.01,0.002,0.001\}\).

In experiments, we set \(\mathbf{h}_{\bf a}(x_{n})\), \(\mathbf{h}_{\bf c}(x_{n})\), and \(\mathbf{h}^{(m)}\) to be \(\in\!\mathbb{R}^{200}\), and \(\mathbf{h}_{\rm p}(x_{n},y)\!\in\!\mathbb{R}^{700}\). We use a single-layer LSTM (without "people-hole" connections), as implemented in the element-rnn library [6]. For regularization, we apply Dropout [9] with a rate of 0.4 before applying the linear weights \(u\), and we also apply Dropout with a rate of 0.3 to the LSTM states before forming the dot-product scores. Following Wiseman et al. wiseman:2015:NIPS we use the cost-weights \(\boldsymbol{\alpha}=\langle 0.5,1.2,1\rangle\) in defining \(\Delta\), and we use their pre-training scheme as well. For final results, we train on both training and development portions of the CoNLL data. Scoring uses the official CoNLL 2012 script [10, 3]. Code for our system is available at [https://github.com/swiseman/nn_coref](https://github.com/swiseman/nn_coref). The system makes use of a GPU for training, and trains in about two hours.

### Results

In Table 1 we present our main results on the CoNLL English test set, and compare with other recent state-of-the-art systems. We see a statistically significant improvement of over 0.8 CoNLL points over the previous state of the art, and the highest F\({}_{1}\) scores to date on all three CoNLL metrics.

We now consider in more detail the impact of global features and RNNs on performance. For these experiments, we report MUC, B\({}^{3}\), and CEAF\({}_{e}\) F\({}_{1}\)scores in Table 2 as well as errors broken down by mention type and by whether the mention is anaphoric or not in Table 3. Table 3 further partitions errors into fl, fn, and wl categories, which

\begin{table}
\begin{tabular}{p{56.9pt} p{28.5pt} p{28.5pt} p{28.5pt} p{28.5pt} p{28.5pt} p{28.5pt} p{28.5pt} p{28.5pt} p{28.5pt}} \hline \hline System & \multicolumn{3}{c}{MUC} & \multicolumn{3}{c}{B\({}^{3}\)} & \multicolumn{3}{c}{CEAF\({}_{e}\)} \\  & P & R & F\({}_{1}\) & P & R & F\({}_{1}\) & P & R & F\({}_{1}\) & CoNLL \\ \hline B\&K [3] & 74.3 & 67.46 & 70.72 & 62.71 & 54.96 & 58.58 & 59.4 & 52.27 & 55.61 & 61.63 \\ M\&S [3] & 76.72 & 68.13 & 72.17 & 66.12 & 54.22 & 59.58 & 59.47 & 52.33 & 55.67 & 62.47 \\ C\&M [3] & 76.12 & 69.38 & 72.59 & 65.64 & 56.01 & 60.44 & 59.44 & 52.98 & 56.02 & 63.02 \\ Peng et al. peng:2015:NIPS & - & - & 72.22 & - & - & 60.50 & - & - & 56.37 & 63.03 \\ Wiseman et al. wiseman:2015:NIPS & 76.23 & 69.31 & 72.60 & 66.07 & 55.83 & 60.52 & 59.41 & 54.88 & 57.05 & 63.39 \\ This work & 77.49 & 69.75 & **73.42** & 66.83 & 56.95 & **61.50** & 62.14 & 53.85 & **57.70** & **64.21** \\ \hline \hline \end{tabular}
\end{table}
Table 1: Results on CoNLL 2012 English test set. We compare against recent state of the art systems, including (in order) Bjorkelund and Kuhn jorkelund2014:NIPS, Martschat and Strube martschat2015:NIPS, Clark and Manning peng2015:NIPS,Peng et al. wiseman2015:NIPS. F\({}_{1}\) gains are significant (\(p<0.05\) under the bootstrap resample test [10]) compared with Wiseman et al. wiseman2015:NIPS for all metrics.

\begin{table}
\begin{tabular}{p{56.9pt} p{28.5pt} p{28.5pt} p{28.5pt} p{28.5pt}} \hline \hline  & MUC & B\({}^{3}\) & CEAF\({}_{e}\) & CoNLL \\ \hline MR & 73.06 & 62.66 & 58.98 & 64.90 \\ Avg, OH & 73.30 & 63.06 & 58.85 & 65.07 \\ RNN, GH & 73.63 & 63.23 & 59.56 & 65.47 \\ RNN, OH & 74.26 & 63.89 & 59.54 & 65.90 \\ \hline \hline \end{tabular}
\end{table}
Table 2: F\({}_{1}\) scores of models described in text on CoNLL 2012 development set. Rows in grey highlight models using oracle history.

 

are defined in Section 5.1. We typically think of fl and wl as representing precision errors, and fn as representing recall errors.

Our experiments consider several different settings. First, we consider an oracle setting ("RNN, OH" in tables), in which the model receives \({\mathbf{z}}_{1:n-1}^{(o)}\), the oracle partial clustering of all mentions preceding \(x_{n}\) in the document, and is therefore not forced to rely on its own past predictions when predicting \(x_{n}\). This provides us with an upper bound on the performance achievable with our model. Next, we consider the performance of the model under a greedy inference strategy (RNN, GH), as in Algorithm 1. Finally, for baselines we consider the mention-ranking system (MR) of Wiseman et al. (2015) using our updated feature-set, as well as a non-local baseline with oracle history (Avg, OH), which averages the representations \({\mathbf{h}}_{c}(x_{j})\) for all \(x_{j}\in X^{(m)}\), rather than feed them through an RNN; errors are still backpropagated through the \({\mathbf{h}}_{c}\) representations during learning.

In Table 3 we see that the RNN improves performance overall, with the most dramatic improvements on non-anaphoric pronouns, though errors are also decreased significantly for non-anaphoric nominal and proper mentions that follow at least one mention with the same head. While wl errors also decrease for both these mention-categories under the RNN model, fnr errors increase. Importantly, the RNN performance is significantly better than that of the Avg baseline, which barely improves over mention-ranking, even with oracle history. This suggests that modeling the sequence of mentions in a cluster is advantageous. We also note that while RNN performance degrades in both precision and recall when moving from the oracle history upperbound to a greedy setting, we are still able to recover a significant portion of the possible performance improvement.

### Qualitative Analysis

In this section we consider in detail the impact of the \(g\) term in the RNN scoring function on the two error categories that improve most under the RNN model (as shown in Table 3), namely, pronominal wl errors and pronominal fl errors. We consider an example from the CoNLL development set in each category on which the baseline MR model makes an error but the greedy RNN model does not.

The example in Figure 3 involves the resolution of the ambiguous pronoun "his," which is bracketed and in bold in the figure. Whereas the baseline MR model _incorrectly_ predicts "his" to corefer with the closest gender-consistent antecedent "Justin" -- thus making a wl error -- the greedy RNN model

Figure 3: Cluster predictions of greedy RNN model; clustered mentions are of the same color, and intensity of mention \(x_{j}\) corresponds to \({\mathbf{h}}_{c}(x_{n})^{\intercal}{\mathbf{h}}_{<k}^{(i)}\), where \(k=j+1\), \(i\in\{1,2\}\), and \(x_{n}=``\text{his}\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! \!



correctly predicts "his" to corefer with "Mr. Kaye" in the previous sentence. (Note that "the official" also refers to Mr. Kaye). To get a sense of the greedy RNN model's decision-making on this example, we color the mentions the greedy RNN model has predicted to corefer with "Mr. Kaye" in green, and the mentions it has predicted to corefer with "Justin" blue. (Note that the model incorrectly predicts the initial "I" mentions to corefer with "Justin.") Letting \(X^{(1)}\) refer to the blue cluster, \(X^{(2)}\) refer to the green cluster, and \(x_{n}\) refer to the ambiguous mention "his," we further shade each mention \(x_{j}\) in \(X^{(1)}\) so that its intensity corresponds to \(\mathbf{h}_{c}(x_{n})^{\mathsf{T}}\mathbf{h}_{<k}^{(1)}\), where \(k=j+1\); mentions in \(X^{(2)}\) are shaded analogously. Thus, the shading shows how highly \(g\) scores the compatibility between "his" and a cluster \(X^{(i)}\) as each of \(X^{(i)}\)'s mentions is added. We see that when the initial "Justin" mentions are added to \(X^{(1)}\) the \(g\)-score is relatively high. However, after "The company" is correctly predicted to corefer with "Justin," the score of \(X^{(1)}\) drops, since companies are generally not coreferent with pronouns like "his."

Figure 4 shows an example (consisting of a telephone conversation between "A" and "B") in which the bracketed pronoun "It's" is being used pleonastically. Whereas the baseline MR model predicts "It's" to corefer with a previous "it" -- thus making a fl error -- the greedy RNN model does not. In Figure 4 the final mention in three preceding clusters is shaded so its intensity corresponds to the magnitude of the gradient of the \(\mathrm{NA}\) term in \(g\) with respect to that mention. This visualization resembles the "saliency" technique of Li et al. (2016), and it attempts to gives a sense of the contribution of a (preceding) cluster in the calculation of the \(\mathrm{NA}\) score.

We see that the potential antecedent "S-Bahn" has a large gradient, but also that the initial, obviously pleonastic use of "it's" has a large gradient, which may suggest that earlier, easier predictions of pleonasm can inform subsequent predictions.

## 7 Related Work

In addition to the related work noted throughout, we add supplementary references here. Unstructured approaches to coreference typically divide into mention-pair models, which classify (nearly) every pair of mentions in a document as coreferent or not Soon et al. (2001); Ng and Cardie (2002); Bengtsson and Roth (2008), and mention-ranking models, which select a single antecedent for each anaphoric mention Denis and Baldridge (2008); Rahman and Ng (2009); Durrett and Klein (2013); Chang et al. (2013); Wiseman et al. (2015). Structured approaches typically divide between those that induce a clustering of mentions McCallum and Wellner (2003); Culotta et al. (2007); Poon and Domingos (2008); Haghighi and Klein (2010); Stoyanov and Eisner (2012); Cai and Strube (2010), and, more recently, those that learn a latent tree of mentions Fernandes et al. (2012); Bjorkelund and Kuhn (2014); Martschat and Strube (2015).

There have also been structured approaches that merge the mention-ranking and mention-pair ideas in some way. For instance, Rahman and Ng (2011) rank clusters rather than mentions; Clark and Manning (2015) use the output of both mention-ranking and mention pair systems to learn a clustering.

The application of RNNs to modeling (the trajectory of) the state of a cluster is apparently novel, though it bears some similarity to the recent work of Dyer et al. (2015), who use LSTMs to embed the state of a transition based parser's stack.

## 8 Conclusion

We have presented a simple, state of the art approach to incorporating global information in an end-to-end coreference system, which obviates the need to define global features, and moreover allows for simple (greedy) inference. Future work will examine improving recall, and more sophisticated approaches to global training.

### Acknowledgments

We gratefully acknowledge the support of a Google Research Award.

Figure 4: Magnitudes of gradients of \(\mathrm{NA}\) score applied to bold “It’s” with respect to final mention in three preceding clusters. See text for full description.

 



## References

* [Bengio et al.2015] Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. 2015. Scheduled sampling for sequence prediction with recurrent neural networks. In _Advances in Neural Information Processing Systems_, pages 1171-1179.
* Bengtson and Roth (2008) Eric Bengtson and Dan Roth. 2008. Understanding the Value of Features for Coreference Resolution. In _Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing_, pages 294-303. Association for Computational Linguistics.
* Bjorkelund and Kuhn (2014) Anders Bjorkelund and Jonas Kuhn. 2014. Learning structured perceptrons for coreference Resolution with Latent Antecedents and Non-local Features. _ACL, Baltimore, MD, USA, June_.
* Cai and Strube (2010) Jie Cai and Michael Strube. 2010. End-to-end coreference resolution via hypergraph partitioning. In _23rd International Conference on Computational Linguistics (COLING)_, pages 143-151.
* Chang et al. (2013) Kai-Wei Chang, Rajhans Sammani, and Dan Roth. 2013. A Constrained Latent Variable Model for Coreference Resolution. In _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing_, pages 601-612.
* Clark and Manning (2015) Kevin Clark and Christopher D. Manning. 2015. Entity-centric coreference resolution with model stacking. In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL)_, pages 1405-1415.
* Culotta et al. (2007) Aron Culotta, Michael Wick, Robert Hall, and Andrew McCallum. 2007. First-order Probabilistic Models for Coreference Resolution. In _Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics (NAACL HLT)_.
* Daume III et al. (2009) Hal Daume III, John Langford, and Daniel Marcu. 2009. Search-based structured prediction. _Machine Learning_, 75(3):297-325.
* Denis and Baldridge (2008) Pascal Denis and Jason Baldridge. 2008. Specialized Models and Ranking for Coreference Resolution. In _Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing_, pages 660-669. Association for Computational Linguistics.
* Duchi et al. (2011) John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive Subgradient Methods for Online Learning and Stochastic Optimization. _The Journal of Machine Learning Research_, 12:2121-2159.
* Durrett and Klein (2013) Greg Durrett and Dan Klein. 2013. Easy Victories and Uphill Battles in Coreference Resolution. In _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing_, pages 1971-1982.
* Durrett and Klein (2014) Greg Durrett and Dan Klein. 2014. A Joint Model for Entity Analysis: Coreference, Typing, and Linking. _Transactions of the Association for Computational Linguistics_, 2:477-490.
* Dyer et al. (2015) Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, and Noah A. Smith. 2015. Transition-based dependency parsing with stack long short-term memory. In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL)_, pages 334-343.
* Fernandes et al. (2012) Eraldo Rezende Fernandes, Cicero Nogueira Dos Santos, and Ruv Luiz Milidiu. 2012. Latent Structure Perceptron with Feature Induction for Unrestricted Coreference Resolution. In _Joint Conference on EMNLP and CoNLL-Shared Task_, pages 41-48. Association for Computational Linguistics.
* Haghighi and Klein (2010) Aria Haghighi and Dan Klein. 2010. Coreference Resolution in a Modular, Entity-centered Model. In _The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics_, pages 385-393. Association for Computational Linguistics.
* Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jurgen Schmidhuber. 1997. Long short-term memory. _Neural Comput._, 9:1735-1780.
* Hovy et al. (2006) Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006. Ontonotes: the 90% Solution. In _Proceedings of the human language technology conference of the NAACL, Companion Volume: Short Papers_, pages 57-60. Association for Computational Linguistics.
* Koehn (2004) Philipp Koehn. 2004. Statistical Significance Tests for Machine Translation Evaluation. In _Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing_, pages 388-395. Citeseer.
* Kummerfeld and Klein (2013) Jonathan K. Kummerfeld and Dan Klein. 2013. Error-driven Analysis of Challenges in Coreference Resolution. In _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing_, Seattle, WA, USA, October.
* Leonard et al. (2015) Nicholas Leonard, Yand Waghmare, Sagar ad Wang, and Jin-Hwa Kim. 2015. rnn: Recurrent Library for Torch. _arXiv preprint arXiv:1511.07889_.
* Li et al. (2016) Jiwei Li, Xinlei Chen, Eduard Hovy, and Dan Jurafsky. 2016. Visualizing and understanding neural models in nlp. In _NAACL HLT_.
* Luo et al. (2014) Xiaoqiang Luo, Sameer Pradhan, Marta Recasens, and Eduard Hovy. 2014. An Extension of BLANC to System Mentions. _Proceedings of ACL, Baltimore, Maryland, June_.
* Martschat and Strube (2015) Sebastian Martschat and Michael Strube. 2015. Latent structures for coreference resolution. _TACL_, 3:405-418

Sebastian Martschat, Thierry Gockel, and Michael Strube. 2015. Analyzing and visualizing coreference resolution errors. In _NAACL HLT_, pages 6-10.
* McCallum and Wellner (2003) Andrew McCallum and Ben Wellner. 2003. Toward Conditional Models of Identity Uncertainty with Application to Proper Noun Coreference. _Advances in Neural Information Processing Systems 17_.
* Ng and Cardie (2002) Vincent Ng and Claire Cardie. 2002. Identifying Anaphoric and Non-anaphoric Noun Phrases to Improve Coreference Resolution. In _Proceedings of the 19th international conference on Computational linguistics-Volume 1_, pages 1-7. Association for Computational Linguistics.
* Peng et al. (2015) Haoruo Peng, Kai-Wei Chang, and Dan Roth. 2015. A joint framework for coreference resolution and mention head detection. In _Proceedings of the 19th Conference on Computational Natural Language Learning (CoNLL)_, pages 12-21.
* Poon and Domingos (2008) Hoifung Poon and Pedro M. Domingos. 2008. Joint unsupervised coreference resolution with markov logic. In _2008 Conference on Empirical Methods in Natural Language Processing (EMNLP)_, pages 650-659.
* Pradhan et al. (2012) Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. Conll-2012 Shared Task: Modeling Multilingual Unrestricted Coreference in OntoNotes. In _Joint Conference on EMNLP and CoNLL-Shared Task_, pages 1-40. Association for Computational Linguistics.
* Pradhan et al. (2014) Sameer Pradhan, Xiaoqiang Luo, Marta Recasens, Eduard Hovy, Vincent Ng, and Michael Strube. 2014. Scoring Coreference Partitions of Predicted Mentions: A Reference Implementation. In _Proceedings of the Association for Computational Linguistics_.
* Rahman and Ng (2009) Altaf Rahman and Vincent Ng. 2009. Supervised Models for Coreference Resolution. In _Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 2-Volume 2_, pages 968-977. Association for Computational Linguistics.
* Rahman and Ng (2011) Altaf Rahman and Vincent Ng. 2011. Narrowing the modeling gap: A cluster-ranking approach to coreference resolution. _J. Artif. Intell. Res. (JAIR)_, 40:469-521.
* Ross et al. (2011) Stephane Ross, Geoffrey J. Gordon, and Drew Bagnell. 2011. A reduction of imitation learning and structured prediction to no-regret online learning. In _Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics_, pages 627-635.
* Soon et al. (2001) Wee Meng Soon, Hwee Tou Ng, and Daniel Chung Yong Lim. 2001. A Machine Learning Approach to Coreference Resolution of Noun Phrases. _Computational Linguistics_, 27(4):521-544.
* Srivastava et al. (2014) Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. _The Journal of Machine Learning Research_, 15(1):1929-1958.
* Stoyanov and Eisner (2012) Veselin Stoyanov and Jason Eisner. 2012. Easy-first Coreference Resolution. In _COLING_, pages 2519-2534. Citeseer.
* Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc VV Le. 2014. Sequence to sequence learning with neural networks. In _Advances in Neural Information Processing Systems (NIPS)_, pages 3104-3112.
* van der Maaten and Hinton (2012) Laurens van der Maaten and Geoffrey E. Hinton. 2012. Visualizing non-metric similarities in multiple maps. _Machine Learning_, 87(1):33-55.
* Wiseman et al. (2015) Sam Wiseman, Alexander M. Rush, Stuart M. Shieber, and Jason Weston. 2015. Learning anaphoricity and antecedent ranking features for coreference resolution. In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL)_, pages 1416-1426.
* Yu and Joachims (2009) Chun-Nam John Yu and Thorsten Joachims. 2009. Learning Structural SVMs with Latent Variables. In _Proceedings of the 26th Annual International Conference on Machine Learning_, pages 1169-1176. ACM.
* Zaremba et al. (2014) Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals. 2014. Recurrent neural network regularization. _CoRR_, abs/1409.2329.