SlowGuess/NAACL_2021_OCR huggingface dataset:
LAMA TREx. In Table 1, we can see that FILM outperforms several recently proposed models on the LAMA TREx task. FILM outperforms the next best performing model, BERT-KNN by 5.5 points. Question-Answering. In Table 2, we compare FILM to five close-book and three open-book QA models on WebQuestionsSP and TriviaQA. The columns denoted Full Dataset-Total show results for the standard evaluation. For WebQuestionsSP, despite using far fewer parameters (see Table 3 and A.3 for details), FILM outperforms all other models - including the top open-book model RAG. On TriviaQA, FILM outperforms all other closed-book models—though the open-book models are substantially more accurate on this task, likely because of the enormous size of the models and their access to all of Wikipedia, which contains all (or nearly all) of the answers in TriviaQA.
3.4 Results on KB-Answerable Questions
WebQuestionsSP (and similarly FreebaseQA discussed in §4) was constructed such that all questions are answerable using the FreeBase KB, which was last updated in 2016. Because our pretraining corpus is derived from larger and more recent versions of Wikipedia, we elected to use a KB con
7All data is English. See A.1 for additional details.
Model	WebQuestionsSP	TriviaQA
Full Dataset	Wikidata Answer	Full Dataset	Wikidata Answer
Total	No Overlap	Total	No Overlap	Total	No Overlap	Total	No Overlap
Closed-book	FILM	54.7	36.4	78.1	72.2	29.1	15.6	37.3	28.4
EaE	47.4	25.1	62.4	42.9	19.0	9.1	24.4	17.1
T5-11B	49.7	31.8	61.0	48.5	-	-	-	-
BART-Large	30.4	5.6	36.7	8.3	26.7	0.8	30.6	1.0
Open-Book	RAG	50.1	30.7	62.5	45.1	56.8	29.2	64.9	45.2
DPR	48.6	34.1	56.9	45.1	57.9	31.6	66.3	48.8
FID	-	-	-	-	67.6	42.8	76.5	64.5
EmQL†	75.5	-	74.6	-	-	-	-	-
Table 2: Open Domain QA Results. Columns denoted Full Dataset-Total are conventional splits discussed in §3.3, Wikidata Answer are answerable using Wikidata (§3.4), and No Overlap removes train-test overlap (§3.5). Highest closed-book and open-book numbers are bolded. Other than FILM and EaE, all results are derived from the prediction files used in Lewis et al. (2020b) including the nearest neighbor (NN) baselines. †is a dataset specific graph reasoning model and the state-of-the-art WebQuestionSP.
Model	B	M	T
FILM	0.11	0.72	0.83
EaE	0.11	0.26	0.37
BERT-L	0.35	0	0.35
BART-L	0.39	0	0.39
T5-11B	11	0	11
DPR	0.11	16	16.11
RAG	0.39	16	16.39
FID	0.77	16	16.77
Table 3: Model Parameters Approximate billions of parameters for each model's (B)ase, (M)emories and (T)otal. Excludes token embeddings.
structed from Wikidata. Many entities in Freebase are unmappable to the more recent Wikidata KB which means that some questions are no longer answerable using the KB. Because of this, we created reduced versions of these datasets which are Wikidata answerable—i.e., containing only questions answerable by triples from our Wikidata-based KB. The model should learn to rely on the KB to answer the questions. We do the same for TriviaQA. $^{8}$
As seen in Table 2 in the column Wikidata answer-Total, FILM does much better on Wikidata answerable questions on WebQuestionsSP. EmQL (Sun et al., 2020), the state-of-the-art dataset specific model, gets  $75.5\%$  accuracy on the full dataset. Not surprisingly, this is because EmQL operates over the Freebase knowledge base, giving it full upperbound recall. However, when we restrict to Wikidata answerable questions, thus giving both EmQL and FILM potential for full recall, FILM outperforms EmQL by 3.5 points and the next best model (RAG) by over 15 points.
3.5 Train-Test Overlap
We are interested in the ability of models to use external knowledge to answer questions, rather than learning to recognize paraphrases of semantically identical questions. Unfortunately, analysis showed that many of the test answers also appear as answers to some training-set question: this is the case for  $57.5\%$  of the answers in WebQuestionsSP and  $75.0\%$  for FreebaseQA. This raises the possibility that some of the performance can be attributed to simply memorizing specific question/answer pairs, perhaps in addition to recognizing paraphrases of the question from its pretraining data.
Overlap in fine-tuning train/test splits was concurrently observed by Lewis et al. (2020b), who created human verified filtered splits for TriviaQA and WebQuestions. We evaluate our models on those splits and report results in Table 2 in the "No Overlap" columns. We see that the gap between FILM and the next best performing model RAG increases from 4.6 to 5.7 points on WebQuestionSP. On TriviaQA, FILM is still able to answer many questions correctly after overlap is removed. In contrast, the majority of closed book models such as BART get less than  $1\%$  of answers correct.
3.6 Filtering to Avoid Pretrain, Finetune, and Test Overlap
The filtering procedure from Lewis et al. (2020b) addresses finetuning train/test overlap but does not account for overlap with the pretraining data. To investigate this further, we looked at FreebaseQA and WebQuestionsSP which both contain entity linked questions and answers. We first perform a similar procedure to Lewis et al. (2020b) and
$^{8}$ TriviaQA does not have linked entities in its questions so for those results we relax this restriction to include all examples where the answer resolves to a Wikidata entity.
discard questions in the fine-tuning training data that contain answers which overlap with answers to questions in the dev and test data. We end up with 9144/2308/3996 data (train/dev/test) in FreebaseQA and 1348/151/1639 data in WebQuestion-sSP. This setting is referred to as Fine-tune column in Table 4 which shows the effects of different filterings of the data.
Next we want to ensure that the model will be unable to simply memorize paraphrases of question answer pairs that it observed in the text by removing all overlap between the pretraining data and finetuning test data. For every question answer entity pair in our finetuning dataset (coming from any split), we filter every example from our Wikipedia pretraining corpus where those pair of entities co-occur. Additionally, we filter every fact from our fact memory containing any of these entity pairs. Results for this setting are in the column labeled Pretrain. The All column combines both pretrain and fine tune filtering. We see that the models perform substantially worse when these filterings are applied and they are forced to reason across multiple examples, and in the case of FILM, the fact memory. Finally, the column denoted None has no filtering and is the same as the Full Dataset.
4 Modifying the Knowledge Base
Because our model defines facts symbolically, it can in principle reason over new facts injected into its memory, without retraining any parameters of the model. Since existing datasets do not directly test this capability, we elected to construct variants of FreebaseQA and WebQuestionsSP where we could simulate asking questions that are answerable only from newly injected KB facts.
The approach we used was to (1) identify pairs of entities that occur in both a question and answer of some test example; (2) filter out such pairs from the KB as well as all pre-training and fine-tuning data; and (3) test the system trained on this filtered data, and then manually updated by injecting facts about those entity pairs. This filtering procedure is reminiscent of that used by Lewis et al. (2020b), but also addresses pretraining / test-set overlap.
4.1 Injecting New Facts to Update Memory
We evaluate EaE and FILM given full knowledge (the original setting); given filtered knowledge; and given filtered knowledge followed by injecting test-question-related facts into the KB. The gap be
tween the filtered knowledge setting and injected knowledge setting will indicate how well the model incorporates newly introduced facts.
In more detail, we first perform a similar procedure to Lewis et al. (2020b) and discard questions in the fine-tuning training data that contain answers which overlap with answers to questions in the dev and test data. We end up with 9144/2308/3996 data (train/dev/test) in FreebaseQA and 1348/151/1639 data in WebQuestionsSP. Next, to ensure that the model will be unable to memorize paraphrases of question-answer pairs that it observed in the pretraining text, we remove all overlap between the pretraining data and fine-tuning test data: specifically, for every question-answer entity pair in our fine-tuning dataset (from any split), we filter every example from our Wikipedia pretraining corpus in which that pair of entities co-occur. Additionally, we filter every fact from our fact memory containing any of these entity pairs.
In these sections we compare against EaE for two reasons: 1) we are specifically looking at closed-book open domain entity based QA and EaE is shown to be at or near state-of-the-art for that task (Févry et al., 2020), 2) most importantly, we want to be able to precisely control for memorization in the training corpus and therefore did not consider existing unconstrained pre-trained models like T5 (Raffel et al., 2019). For reference, the previous state-of-the-art FOFE (Jiang et al., 2019) on FreebaseQA had a score of  $37.0\%$  using the original train-test split, while FILM is at  $63.3\%$ .
The results are shown in Table 5. In the "Full" column, we pretrain and finetune the FILM model with the full knowledge base and corpus. In the "Filter" setting, facts about the finetuning data are hidden from the model at both pretraining and finetuning time. In this case, the model must fall back to the language model to predict the answer, and as shown in Table 5, the accuracies of FILM and EaE are similar. In the "Inject Facts" setting, Facts are hidden at pretraining time, but are injected at test time. The results show that FILM can effectively use the newly injected facts to make prediction, obtaining an absolute improvement of  $9.3\%$  compared to the "Filter" setting. EaE does not have a natural mechanism for integrating this new information<sup>9</sup>.
There are various heuristics one could apply for finetuning a standard language model on this type of data by applying one or a small number of gradient steps on textualized facts. We leave this exploration for future research.
FreebaseQA	WebQuestionsSP
Filter Type	None	Pretrain	Fine-tune	All	None	Pretrain	Fine-tune	All
EaE	53.4	45.2	45.8	28.6	48.1	45.4	30.9	29.4
FILM	63.3	57.5	56.5	48.0	56.1	55.4	40.7	39.2
Table 4: Effects of Different Data Filtering. The column denoted None has no filtering. Pretrain removes all entity pair overlap between the eval datasets (all splits) and the pretraining text and kb. The Fine-tune column removes all entity pair overlap between the eval train and test splits. The All column combines both pretrain and fine tune filtering.
FreebaseQA	WebQuestionsSP
Full	Filter	Inject	Full	Filter	Inject
EaE	45.8	28.6	-	30.9	29.4	-
FILM	56.5	38.7	48.0	40.7	32.3	39.2
4.2 Updating Stale Memories
One of the main motivations for our model is to provide knowledge representations that can be incrementally updated as the world changes, avoiding stale data. In order to accomplish this, the model must learn to utilize the fact memory even in the case where those facts have changed such that they may no longer be consistent with the data the model was initially trained on. Further, it needs to accomplish that without any additional training.
To probe this ability, we simulate an extreme version of stale facts where all answers to QA pairs in the FreebaseQA test set are 'updated' with plausible alternatives. For each QA pair, we replace the original answer entity  $e_{\mathrm{original}}$  with another entity,  $e_{\mathrm{new}}$ , from our vocabulary that has: 1) been used as an object in at least one of the same relation types in which  $e_{\mathrm{original}}$  was used as an object, and 2) shares at least three Wikipedia categories with  $e_{\mathrm{original}}$ .
We use the same pretrained models from our earlier experiments and fine-tune on the filtered FreebaseQA train set for 10,000 steps. We then modify the memory of this model without applying any additional training on the new memory. In addition to adding new memories which correspond
to our newly created facts, we also must remove the original stale facts that we are updating. We look at two methods for filtering those 'stale facts' from the fact memory.
Basic Filter deletes every modified fact  $e_{\text{question}}$ ,  $r$ ,  $e_{\text{original}}$  and replaces it with a new fact  $e_{\text{question}}$ ,  $r$ ,  $e_{\text{new}}$ . This would be a low recall filter as it does not account for all possible related facts. The Strict Filter is a high recall filter that more aggressively removes information that may conflict with the newly added fact, additionally removing all facts that contain  $e_{\text{question}}$  or  $e_{\text{original}}$ . This is important for cases such as when a question contains multiple entities, or the linking relation is one-to-many, leading to multiple plausible answers. Together these two settings define rough bounds on the model's ability to perform this task. In Table 6, we see that FILM is able to utilize the modified KB to make the correct prediction for  $54.5\%$  of questions in the Basic Filter setting and  $70.3\%$  in the Strict Filter setting.
Table 5: Injecting New Facts. In the Filter setting, the models have access to no direct knowledge about question answer entity pairs from either the pretraining corpus or KB. In the Inject setting, the pretraining corpus and training KB are still Filtered, but at inference time, new facts are injected into the models memory allowing it to recover most of the drop from the Full setting. In the Full setting the model is exposed to full knowledge. In all cases, we remove the overlap between the finetune train and eval sets.
Model	Basic Filter	Strict Filter
FILM	0.0	0.0
+Update Memory	54.5	70.3
Table 6: Updating Stale Memories. Basic filter removes only facts connecting the original question entity to the answer entity. Strict filter removes all facts containing the original question or answer (not just facts connecting them).
Symbolic KBs have been a core component of AI since the beginning of the field (Newell and Simon, 1956; Newell et al., 1959), and widely available public KBs have been invaluable in research and industry (Bollacker et al., 2008; Auer et al., 2007; Google, 2012; Dong, 2017; Vrandecic and Krötzsch, 2014). In machine learning, a well studied problem is learning KB embeddings (Bordes et al., 2013; Lin et al., 2015; Trouillon et al., 2017;
Dettmers et al., 2018) which enable generalization from known KB triples to novel triples that are plausibly true. KB embeddings can often be improved by incorporating raw text and symbolic KGs into a shared embedding space (Riedel et al., 2013; Verga et al., 2016, 2017), to be jointly reasoned over (Sun et al., 2018, 2019). Many prior neural-symbolic methods have attempted to unify symbolic KBs and neural methods (Pinkas, 1991; de Penning et al., 2011; Laird et al., 2017; Besold et al., 2017). Recently, researchers have explored query languages for embedded KBs that are similar to symbolic KB query languages (Cohen et al., 2017; Hamilton et al., 2018; Ren et al., 2020; Cohen et al., 2020).
Our fact memory builds on this prior work, and is most closely related to the memory used in EmQL (Sun et al., 2020), one KB embedding model that supports compositional query language. EmQL implements "projection" using neural retrieval over vectorized KB triples. Unlike this work, however, EmQL did not embed its fact memory into a LM, which could be finetuned for many NLP tasks: instead requiring the implementation of a "neural module" into some task-specific architecture. At a more abstract level, the fact memory is a key-value memory (Weston et al., 2014; Miller et al., 2016), a construct used in many neural models in the past.
It has been shown that sufficiently large LMs trained through self supervision (Peters et al., 2018; Devlin et al., 2019; Raffel et al., 2019; Brown et al., 2020) also encode factual information, motivating work on the extent to which a LM can serve as a KB (Roberts et al., 2020; Petroni et al., 2019; Poerner et al., 2019). Other work has explored techniques to improve the performance of large LMs in answering factual probes, by adding additional supervision in pre-training (Xiong et al., 2019; Wang et al., 2020b) or by adding entity embeddings into an extended LM (Peters et al., 2019; Zhang et al., 2019; Févry et al., 2020).
Our entity memory extends the Entities-as-Experts (EaE) model (Févry et al., 2020). It is both the current state-of-the-art for a number of tasks and simpler to use than most prior models because it does not require external components for entity linking or entity encoding (like (Peters et al., 2019; Zhang et al., 2019; Logan et al., 2019)) and is not restricted to lexical KBs like WordNet and ConceptNet (like (Weissenborn et al., 2017; Chen et al., 2018; Mihaylov and Frank, 2018)).
Our model's use of memory also scales to KBs
with millions of entities, whereas prior systems that make use of KB triples have been with only a few hundreds of triples in the model at any point, necessitating a separate heuristic process to retrieve candidate KB triples (Ahn et al., 2016; Henaff et al., 2016; Weissenborn et al., 2017; Chen et al., 2018; Mihaylov and Frank, 2018; Logan et al., 2019).
There have been a few exploratory experiments on modifying the predictions of retrieval augmented language models by changing the underlying text corpus (Guu et al., 2020; Lewis et al., 2020a). However, text passages are not easily interpretable resulting in them being less inspectible and modifiable than a symbolic fact based memory.
We presented FILM, a neural LM with an interpretable symbolically bound fact memory. We demonstrated the effectiveness of this method by outperforming many state-of-the-art methods on four benchmark knowledge intensive datasets. We used the model's symbolic interface to change the output of the LM by modifying only the non-parametric memories, without any additional training. We showed FILM could incorporate newly injected facts unseen during training. Additionally, we can modify facts, such that they contradict the initial pre training text, and our model is still largely able to answer these questions correctly.
All language models learn to exploit correlations in the data they were trained on. As such, they inherit all of the underlying biases within that data (Zhao et al., 2019; Bender et al., 2021). These models require vast amounts of data to train on and therefore tend to rely on internet corpora which have skewed representations of particular groups, cultures, and languages, as well as variable levels of factuality. Our hope is that research into endowing these models with interpretable and modifiable memories will allow us to more readily identify and remedy some of these failures.
We thank the anonymous reviewers for their helpful feedback and Patrick Lewis for providing prediction files. We also thank Thibault Févry, Nicholas FitzGerald, Eunsol Choi, Tom Kwiatkowski and other members of Google Research for discussions and feedback.
Sungjin Ahn, Heeyoul Choi, Tanel Parnamaa, and Yoshua Bengio. 2016. A neural knowledge language model. arXiv preprint arXiv:1608.00318.
Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. 2007. Dbpedia: A nucleus for a web of open data. In The semantic web, pages 722-735. Springer.
Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big?. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 610-623.
Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on Freebase from question-answer pairs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1533-1544, Seattle, Washington, USA. Association for Computational Linguistics.
Tarek R Besold, Artur d'Avila Garcez, Sebastian Bader, Howard Bowman, Pedro Domingos, Pascal Hitzler, Kai-Uwe Kühnberger, Luis C Lamb, Daniel Lowd, Priscila Machado Vieira Lima, et al. 2017. Neural-symbolic learning and reasoning: A survey and interpretation. arXiv preprint arXiv:1711.03902.
Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 1247-1250.
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Advances in neural information processing systems, pages 2787-2795.
Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165.
Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading wikipedia to answer open-domain questions. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1870-1879.
Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Diana Inkpen, and Si Wei. 2018. Neural natural language inference models enhanced with external knowledge. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2406-2417.
William W Cohen, Haitian Sun, R Alex Hofer, and Matthew Siegler. 2020. Scalable neural methods for reasoning with a symbolic knowledge base. International Conference on Learning Representations.
William W Cohen, Fan Yang, and Kathryn Rivard Mazaitis. 2017. Tensorlog: Deep learning meets probabilistic dbs. arXiv preprint arXiv:1707.05390.
Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. 2018. Convolutional 2d knowledge graph embeddings. In Thirty-Second AAAI Conference on Artificial Intelligence.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186.
Luna Dong. 2017. Amazon product graph.
Thibault Févry, Livio Baldini Soares, Nicholas FitzGerald, Eunsol Choi, and Tom Kwiatkowski. 2020. Entities as experts: Sparse memory access with entity supervision. Conference on Empirical Methods in Natural Language Processing.
Google. 2012. Introducing the knowledge graph: things, not strings.
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020. Realm: Retrievalaugmented language model pre-training. arXiv preprint arXiv:2002.08909.
Will Hamilton, Payal Bajaj, Marinka Zitnik, Dan Jurafsky, and Jure Leskovec. 2018. Embedding logical queries on knowledge graphs. In Advances in Neural Information Processing Systems, pages 2026-2037.
Mikael Henaff, Jason Weston, Arthur Szlam, Antoine Bordes, and Yann LeCun. 2016. Tracking the world state with recurrent entity networks. arXiv preprint arXiv:1612.03969.
Gautier Izacard and Edouard Grave. 2020. Leveraging passage retrieval with generative models for open domain question answering. arXiv preprint arXiv:2007.01282.
Kelvin Jiang, Dekun Wu, and Hui Jiang. 2019. Freebaseqa: A new factoid qa data set matchingTrivia-style question-answer pairs with freebase. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 318-323.
Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. 2017. Triviaqa: A large scale distantly