SlowGuess/ACL_2020_OCR huggingface dataset:
We also report the results of Joshi et al. (2019) and Wu et al. (2019), although these numbers are not comparable since both of them train on the much larger OntoNotes corpus and just test on GAP.
4.1 GAP results
We train all the memory models, including the Referential Reader, with memory size varying from  $\{2, 4, \dots, 20\}$  memory cells for both BERT<sub>BASE</sub> and BERT<sub>LARGE</sub>, with each configuration being trained 5 times. Figure 3 shows the performance of the
<sup>7</sup>Note that the error we report is therefore a best-case result. We are not proposing a way of counting unique people in new test data, but rather using this task for analysis.
<sup>8</sup>https://github.com/liufly/refreader
5The computation of this probability includes the mention detection steps required byWebster et al. (2018).
$^{6}$ GAP also includes evaluation related to gender bias, but this is not a focus of this paper so we do not report it.
(b) BERTLARGE
Figure 4: Error in counting unique people as a function of number of memory cells; lower is better.
BERTBASE	BERTLARGE
PeTra	81.5 ± 0.6	85.3 ± 0.6
+ Learned Init.	80.9 ± 0.7	84.4 ± 1.2
+ Fixed Key	81.1 ± 0.7	85.1 ± 0.8
Ref. Reader	78.9 ± 1.3	83.7 ± 0.8
Ref. Reader (2019a)	78.8	-
Joshi et al. (2019)	82.8	85.0
Wu et al. (2019)	-	87.5 (SpanBERT)
models on the GAP validation set as a function of memory size. The Referential Reader outperforms PeTra (and its memory variants) when using a small number of memory cells, but its performance starts degrading after 4 and 6 memory cells for  $\mathrm{BERT}_{\mathrm{BASE}}$  and  $\mathrm{BERT}_{\mathrm{LARGE}}$  respectively. PeTra and its memory variants, in contrast, keep improving with increased memory size (before saturation at a higher number of cells) and outperform the best Referential Reader performance for all memory sizes  $\geq 6$  cells. With larger numbers of memory cells, we see a higher variance, but the curves for PeTra and its memory variants are still consistently higher than those of the Referential Reader.
Among different memory variants of PeTra, when using BERT<sub>BASE</sub> the performances are comparable with no clear advantage for any particular choice. For BERT<sub>LARGE</sub>, however, vanilla PeTra has a clear edge for almost all memory sizes, suggesting the limited utility of initialization. The results show that PeTra works well without learning vectors for initializing the key or memory cell contents. Rather, we can remove the key/value distinction and simply initialize all memory cells with the zero vector.
To evaluate on the GAP test set, we pick the memory size corresponding to the best validation
performance for all memory models. Table 1 shows that the trends from validation hold true for test as well, with PeTra outperforming the Referential Reader and the other memory variants of PeTra.
4.2 Counting unique people
Figure 4 shows the results for the proposed interpretability task of counting unique people. For both  $\mathrm{BERT}_{\mathrm{BASE}}$  and  $\mathrm{BERT}_{\mathrm{LARGE}}$ , PeTra achieves the lowest error count. Interestingly, from Figure 4b we can see that for  $\geq 14$  memory cells, the other memory variants of PeTra perform worse than the Referential Reader while being better at the GAP validation task (see Figure 3b). This shows that a better performing model is not necessarily better at tracking people.
Table 1: Results (%F1) on the GAP test set.
BERTBASE	BERTLARGE
PeTra	0.76	0.69
+ Learned Init	0.72	0.60
+ Fixed Key	0.72	0.65
Ref. Reader	0.49	0.54
Table 2: Spearman's correlation between GAP validation F1 and negative error count for unique people.
To test the relationship between the GAP task and the proposed interpretability task, we compute the correlation between the GAP F-score and the negative count of unique people for each model separately. Table 2 shows the Spearman's correlation between these measures. For all models we see a positive correlation, indicating that a dip in coreference performance corresponds to an increase in error on counting unique people. The correlations for PeTra are especially high, again suggesting it's greater interpretability.
Each correlation is computed over 50 runs (5 runs each for 10 memory sizes).
Amelia Shepherd<sup>1</sup>, M.D. is a fictional character on the ABC American television medical drama Private Practice, and the spinoff series' progenitor show, Grey's Anatomy, portrayed by Caterina Scorsone<sup>2</sup>. In her debut appearance in season three, Amelia<sup>1</sup> visited her former sister-in-law, Addison Montgomery<sup>3</sup>, and became a partner at the Oceanside Wellness Group.
(a) A successful run of PeTra with 4 memory cells. The model accurately links all the mentions of "Amelia" to the same memory cell while also detecting other people in the discourse.
Bethenny1 calls a meeting to get everyone on the same page, but Jason2 is hostile with the group, making things worse and forcing Bethenny1 to play referee. Emotions are running high with Bethenny1's assistant, Julie3, who breaks down at a lunch meeting when asked if she3 is committed to the company for the long haul.
(b) Memory log of PeTra with 8 memory cells. The model correctly links "she" and "Julie" but fails at linking the three "Bethenny" mentions, and also fails at detecting "Jason".
Figure 5: Visualization of memory logs for different configurations of PeTra. The documents have their GAP annotations highlighted in red (italics) and blue (bold), with blue (bold) corresponding to the right answer. For illustration purposes only, we highlight all the spans corresponding to mentions of people and mark cluster indices as subscript. In the plot, X-axis corresponds to document tokens, and Y-axis corresponds to memory cells. Each memory cell has the OW=OVERWRITE and CR=COREF labels. Darker color implies higher value. We skip text, indicated via ellipsis, when the model doesn't detect people for extended lengths of text.
4.3 Human Evaluation for People Tracking
Model	Preference (in %)
PeTra	74
Ref. Reader	08
Neutral	18
Table 3: Human Evaluation results for people tracking.
Table 3 summarizes the results of the human evaluation for people tracking. The annotators
prefer PeTra in  $74\%$  cases while the Referential Reader for only  $8\%$  instances (see Appendix A.4 for visualizations comparing the two). Thus, PeTra easily outperforms the Referential Reader on this task even though they are quite close on the GAP evaluation. The annotators agree on  $68\%$  of the documents, disagree between PeTra and Neutral for  $24\%$  of the documents, and disagree between PeTra and the Referential Reader for the remaining  $8\%$  documents. For more details, see Appendix A.2.
4.4 Model Runs
We visualize two runs of PeTra with different configurations in Figure 5. For both instances the model gets the right pronoun resolution, but clearly in Figure 5b the model fails at correctly tracking repeated mentions of "Bethenny". We believe these errors happen because (a) GAP supervision is limited to pronoun-proper name pairs, so the model is never explicitly supervised to link proper names, and (b) there is a lack of span-level features, which hurts the model when a name is split across multiple tokens.
There are several strands of related work, including prior work in developing neural models with external memory as well as variants that focus on modeling entities and entity relations, and neural models for coreference resolution.
Memory-augmented models. Neural network architectures with external memory include memory networks (Weston et al., 2015; Sukhbaatar et al., 2015), neural Turing machines (Graves et al., 2014), and differentiable neural computers (Graves et al., 2016). This paper focuses on models with inductive biases that produce particular structures in the memory, specifically those related to entities.
Models for tracking and relating entities. A number of existing models have targeted entity tracking and coreference links for a variety of tasks. EntNet (Henaff et al., 2017) aims to track entities via a memory model. EntityNLM (Ji et al., 2017) represents entities dynamically within a neural language model. Hoang et al. (2018) augment a reading comprehension model to track entities, incorporating a set of auxiliary losses to encourage capturing of reference relations in the text. Dhingra et al. (2018) introduce a modified GRU layer designed to aggregate information across coreferent mentions.
Memory models for NLP tasks. Memory models have been applied to several other NLP tasks in addition to coreference resolution, including targeted aspect-based sentiment analysis (Liu et al., 2018b), machine translation (Maruf and Haffari, 2018), narrative modeling (Liu et al., 2018a), and dialog state tracking (Perez and Liu, 2017). Our study of architectural choices for memory may also be relevant to models for these tasks.
Neural models for coreference resolution. Several neural models have been developed for coreference resolution, most of them focused on modeling pairwise interactions among mentions or spans in a document (Wiseman et al., 2015; Clark and Manning, 2016a; Lee et al., 2017, 2018). These models use heuristics to avoid computing scores for all possible span pairs in a document, an operation which is quadratic in the document length  $T$  assuming a maximum span length. Memory models for coreference resolution, including our model, differ by seeking to store information about entities in memory cells and then modeling the relationship between a token and a memory cell. This reduces computation from  $\mathcal{O}(T^2)$  to  $\mathcal{O}(TN)$ , where  $N$  is the number of memory cells, allowing memory models to be applied to longer texts by using the global entity information. Past work (Wiseman et al., 2016) have used global features, but in conjunction with other features to score span pairs.
Referential Reader. Most closely related to the present work is the Referential Reader (Liu et al., 2019a), which uses a memory model to perform coreference resolution incrementally. We significantly simplify this model to accomplish the same goal with far fewer parameters.
We propose a new memory model for entity tracking, which is trained using sparse coreference resolution supervision. The proposed model outperforms a previous approach with far fewer parameters and a simpler architecture. We propose a new diagnostic evaluation and conduct a human evaluation to test the interpretability of the model, and find that our model again does better on this evaluation. In future work, we plan to extend this work to longer documents such as the recently released dataset of Bamman et al. (2019).
This material is based upon work supported by the National Science Foundation under Award Nos. 1941178 and 1941160. We thank the ACL reviewers, Sam Wiseman, and Mrinmaya Sachan for their valuable feedback. We thank Fei Liu and Jacob Eisenstein for answering questions regarding the Referential Reader. Finally, we want to thank all the annotators at TTIC who participated in the human evaluation study.
David Bamman, Olivia Lewke, and Anya Mansoor. 2019. An Annotated Dataset of Coreference in English Literature. arXiv preprint arXiv:1912.01140.
Hong Chen, Zhenhua Fan, Hao Lu, Alan Yuille, and Shu Rong. 2018. PreCo: A Large-scale Dataset in Preschool Vocabulary for Coreference Resolution. In EMNLP.
Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. In EMNLP.
Kevin Clark and Christopher D. Manning. 2016a. Deep Reinforcement Learning for Mention-Ranking Coreference Models. In EMNLP.
Kevin Clark and Christopher D Manning. 2016b. Improving Coreference Resolution by Learning Entity-Level Distributed Representations. In ACL.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In *NAACL-HLT*.
Bhuwan Dhingra, Qiao Jin, Zhilin Yang, William Cohen, and Ruslan Salakhutdinov. 2018. Neural Models for Reasoning over Multiple Mentions Using Coreference. In NAACL.
Alex Graves, Greg Wayne, and Ivo Danihelka. 2014. Neural Turing machines. arXiv preprint arXiv:1410.5401.
Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska-Barwinska, Sergio Gomez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou, AdriPaugdomenech Badia, Karl Moritz Hermann, Yori Zwols, Georg Ostrovski, Adam Cain, Helen King, Christopher Summerfield, Phil Blunsom, Koray Kavukcuoglu, and Demis Hassabis. 2016. Hybrid computing using a neural network with dynamic external memory. Nature, 538(7626):471-476.
Mikael Henaff, Jason Weston, Arthur Szlam, Antoine Bordes, and Yann LeCun. 2017. Tracking the world state with recurrent entity networks. In ICLR.
Luong Hoang, Sam Wiseman, and Alexander Rush. 2018. Entity Tracking Improves Cloze-style Reading Comprehension. In EMNLP.
Eric Jang, Shixiang Gu, and Ben Poole. 2017. Categorical Reparameterization with Gumbel-Softmax. In ICLR.
Yangfeng Ji, Chenhao Tan, Sebastian Martschat, Yejin Choi, and Noah A. Smith. 2017. Dynamic Entity Representations in Neural Language Models. In EMNLP.
Mandar Joshi, Omer Levy, Luke Zettlemoyer, and Daniel Weld. 2019. BERT for Coreference Resolution: Baselines and Analysis. In EMNLP.
Frank Keller. 2010. Cognitively Plausible Models of Human Language Processing. In ACL.
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR.
Ankit Kumar, Ozan Irsoy, Peter Ondruska, Mohit Iyyer, James Bradbury, Ishaan Gulrajani, Victor Zhong, Romain Paulus, and Richard Socher. 2016. Ask me anything: Dynamic memory networks for natural language processing. In ICML.
Kenton Lee, Luheng He, Mike Lewis, and Luke Zettle-moyer. 2017. End-to-end Neural Coreference Resolution. In EMNLP.
Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018. Higher-Order Coreference Resolution with Coarse-to-Fine Inference. In NAACL-HLT.
Fei Liu, Trevor Cohn, and Timothy Baldwin. 2018a. Narrative Modeling with Memory Chains and Semantic Supervision. In ACL.
Fei Liu, Trevor Cohn, and Timothy Baldwin. 2018b. Recurrent Entity Networks with Delayed Memory Update for Targeted Aspect-Based Sentiment Analysis. In *NAACL-HLT*.
Fei Liu, Luke Zettlemoyer, and Jacob Eisenstein. 2019a. The Referential Reader: A recurrent entity network for anaphora resolution. In ACL.
Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew E. Peters, and Noah A. Smith. 2019b. Linguistic Knowledge and Transferability of Contextual Representations. In *NAACL-HLT*.
Sameen Maruf and Gholamreza Haffari. 2018. Document Context Neural Machine Translation with Memory Networks. In ACL.
Alexander Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Antoine Bordes, and Jason Weston. 2016. Key-Value Memory Networks for Directly Reading Documents. In EMNLP.
Julien Perez and Fei Liu. 2017. Dialog state tracking, a machine reading approach using Memory Network. In EACL.
Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL-2012 Shared Task: Modeling multilingual unrestricted coreference in ontonotes. In Joint Conference on EMNLP and CoNLL - Shared Task, CoNLL '12.
Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, and Rob Fergus. 2015. End-to-end memory networks. In NeurIPS.
MK Tanenhaus, MJ Spivey-Knowlton, KM Eberhard, and JC Sedivy. 1995. Integration of visual and linguistic information in spoken language comprehension. Science, 268(5217).
Kellie Webster, Marta Recasens, Vera Axelrod, and Jason Baldridge. 2018. Mind the GAP: A Balanced Corpus of Gendered Ambiguous Pronouns. In TACL, volume 6.
Jason Weston, Sumit Chopra, and Antoine Bordes. 2015. Memory Networks. In ICLR.
Sam Wiseman, Alexander M. Rush, Stuart Shieber, and Jason Weston. 2015. Learning Anaphoricity and Antecedent Ranking Features for Coreference Resolution. In ACL-IJCNLP.
Sam Wiseman, Alexander M. Rush, and Stuart M. Shieber. 2016. Learning Global Features for Coreference Resolution. In *NAACL*.
Wei Wu, Fei Wang, Arianna Yuan, Fei Wu, and Jiwei Li. 2019. Coreference Resolution as Query-based Span Prediction. arXiv preprint arXiv:1911.01746.
A.1 Best Runs vs. Worst Runs
As Table 1 shows, there is significant variance in the performance of these memory models. To analyze how the best runs diverge from the worst runs, we analyze how the controller network is using the different memory cells in terms of overwrites. For this analysis, we choose the best and worst among the 5 runs for each configuration, as determined by GAP validation performance. For the selected runs, we calculate the KL-divergence of the average overwrite probability distribution from the uniform distribution and average it for each model type. Table 4 shows that for the memory variants Learned Init and Fixed Key, the worst runs overwrite more to some memory cells than others (high average KLdivergence). Note that both PeTra and Referential Reader are by design intended to have no preference for any particular memory cell (which the numbers support), hence the low KL-divergence.
Avg KL-div
Best run	Worst run
PeTra	0.00	0.01
+ Learned Init.	0.3	0.83
+ Fixed Key	0.2	0.8
Ref. Reader	0.05	0.04
Table 4: A comparison of best runs vs. worst runs.
A.2 Human Evaluation Results
The agreement matrix for the human evaluation study described in Section 4.3 is shown in Figure 6. This agreement matrix is a result of the two annotations per document that we get as per the setup described in Section 3.4. Note that the annotations are coming from two sets of annotators rather than two individual annotators. This is also the reason why we don't report standard inter-annotator agreement coefficients.
Figure 6: Agreement matrix for human evaluation study.
A.3 Instructions for Human Evaluation
The detailed instructions for the human evaluation study described in Section 4.3 are shown in Figure 7. We simplified certain memory model specific terms such as "overwrite" to "new person" since the study was really about people tracking.
A.4 Comparative visualization of memory logs of PeTra and the Referential Reader
Figure 8 and 9 compare the memory logs of PeTra and the Referential Reader.
- In this user study we will be comparing memory models at tracking people.
- What are memory models? Memory models are neural networks coupled with an external memory which can be used for reading/writing.
- (IMPORTANT) What does it mean to track people for memory models?
- Detect all references to people which includes pronouns.
- A 1-to-1 correspondence between people and memory cells i.e. all references corresponding to a person should be associated with the same memory cell AND each memory cell should be associated with at most 1 person.
- The memory models use the following scores (which are visualized) to indicate the tracking decisions:
- New Person Probability (Cell  $i$ ): Probability that the token refers to a new person (not introduced in the text till now) and we start tracking it in cell  $i$ .
- Coreference Probability (Cell  $i$ ): Probability that the token refers to a person already being tracked in cell  $i$ .
- The objective of this study is to compare the models on the interpretability of their memory logs i.e. are the models actually tracking entities or not. You can choose how you weigh the different requirements for tracking people (from 3).
- For this study, you will compare two memory models with 8 memory cells (represented via 8 rows). The models are ordered randomly for each instance.
- For each document, you can choose model A or model B, or stay neutral in case both the models perform similarly.
Figure 7: Instructions for the human evaluation study.
$\mathbf{N e e f}_{1}$  took an individual silver medal at the 1994 European Cup behind Russia's Svetlana Goncharenko2 and returned the following year to win gold.  $\mathbf{S h e}_{1}$  was a finalist individually at the 1994 European Championships and came sixth for Scotland at the 1994 Commonwealth Games.
(a) GAP validation instance 293. The ground truth GAP annotation is indicated via colors.
(b) Memory log of PeTra with 8 memory cells. PeTra uses only 2 memory cells for the 2 unique people, namely Neef and Svetlana Goncharenko, and correctly resolves the pronoun.
(c) Memory log of the Referential Reader with 8-memory cells. The Referential Reader does successfully resolve the pronoun in the topmost memory cell but it ends up tracking Neef in as many as 4 memory cells.
Figure 8: Both the models only weakly detect "Svetlana Goncharenko" which could be due to lack of span modeling.
$\mathbf{F}$ ripp1 has performed Soundscapes in several situations: \* Fripp1 has featured Soundscapes on various King Crimson albums. He1 has also released pure Soundscape recordings as well: \* On May 4, 2006, Steve Ball2 invited Robert Fripp1 back to the Microsoft campus for a second full day of work on Windows Vista following up on his1 first visit in the Fall of 2005.
(a) GAP validation instance 17. The ground truth GAP annotation is indicated via colors.
(b) Memory log of PeTra with 8-memory cells. PeTra is pretty accurate at tracking Robert Fripp but it misses out on connecting "Fripp" from the earlier part of the document to "Robert Fripp".
(c) Memory log of the Referential Reader with 8-memory cells. The Referential Reader completely misses out on all the mentions in the first half of the document (which is not penalized in GAP evaluations where the relevant annotations are typically towards the end of the document). Apart from this, the model ends up tracking Robert Fripp in as many as 6 memory cells, and Steve Ball in 3 memory cells.
Figure 9: PeTra clearly performs better than the Referential Reader at people tracking for this instance. PeTra's output is more sparse, detects more relevant mentions, and is better at maintaining a 1-to-1 correspondence between memory cells and people.
5428