SlowGuess/NAACL_2022_OCR huggingface dataset:
We now present an analysis of how much, and if, having access to external knowledge can help PLMs in knowledge-intensive tasks. A popular way of assessing a model's ability to perform at such tasks is by using benchmark knowledge probes. We use the LAMA probe (Petroni et al., 2019) that provides a cloze-style sentence representation of facts and the model being evaluated is required to predict the masked words in these sentences (e.g., Barack Obama was born in _____.).
Table 1 reports the performance of various PLMs
on the LAMA probe as measured by Precision at 1 (P@1). The numbers in the Table are grouped based on the base language model used by different models. We use ERNIE (Zhang et al., 2019) (based on BERT and Wikidata), and KEPLER (Wang et al., 2021b) and CoLAKE (Sun et al., 2020) (based on RoBERTa) as the representative knowledge enhanced language models. Both KEPLER and CoLAKE have used Wikidata5M as the knowledge base. We used author provided code and checkpoints for obtaining the reported numbers. For LM-CORE, we use four variants with different knowledge base and language encoder combinations as described above.
We observe that our approach of providing external knowledge to the PLMs leads to substantially improved performance over the base language models and their SoTA knowledge enhanced variants. LM-CORE(b,w) achieves P@1 of  $42.83\%$  compared to  $25.44\%$  for BERT-large. Likewise, LM-CORE(r,w) achieves a P@1 of  $41.96\%$  significantly outperforming RoBERTa-large  $(24.24\%)$ . We also report the numbers on the four different subsets of LAMA revealing interesting insights. For all the models considered, we note that the performance on T-REx subset is higher than the Google-RE subset. We attribute this to the nature of knowledge required for probes in the four subsets. Note especially the column for Date of Birth (DoB) in the Table. All the models, except for LM-CORE(b,y) and LM-CORE(r,y) perform extremely poorly. This is because the Wikidata5M KB does
Table 2: P@1 for different models on LAMA-UHN.
LAMA	LAMA UHN	Percentage Change
BERT-based models
BERT-base	24.73	18.72	-24.31
BERT-large	25.44	19.92	-21.67
ERNIE	22.16	15.81	-28.66
LM-CORE(b,y)	39.64	41.33	+4.26
LM-CORE(b,w)	42.83	45.50	+6.23
RoBERTa-based models
RoBERTa-base	20.46	13.66	-33.24
RoBERTa-large	24.24	17.99	-25.78
KEPLER	19.36	12.46	-35.64
CoLAKE	23.38	17.16	-13.74
LM-CORE(r,y)	34.60	34.25	-1.01
LM-CORE(r,w)	41.96	44.75	+6.65
not have date entity type and hence, the poor performance of models using Wikidata. We also note that on the SQuAD and Concept Net subsets, the knowledge enhanced models do not offer significant improvements over the base language models. While Google-RE and T-REx focus more on factual world knowledge (present in abundance in YAGO and Wikidata), SQuAD and ConceptNet concentrate more on commonsense knowledge (limited in YAGO and Wikidata). This is a major focus of our continuing work on enhancing the external knowledge with commonsense knowledge bases such as ConceptNet (Speer et al., 2017) Atomic (Sap et al., 2019).
4.2 Sensitivity to Contextual Signals in Input
PLMs are often sensitive to the linguistic variations in the input and are overly reliant on the surface form of entity names for making its predictions Poerner et al. (2020). For example, BERT can predict that a person with an Italian-sounding name was born in Italy even if this is factually incorrect. In order to evaluate the sensitivity and robustness of different models, we report the  $\mathrm{P@1}$  numbers for the LAMA-UHN (UnHelpfulNames) probing benchmark (Table 2) – a much harder subset of LAMA where input probes with helpful entity names are removed and the PLM has little or no helpful contextual signals from other tokens in the probe. We observe that the LM-CORE variants significantly outperform the base language models and their knowledge enhanced variants. Further, note that while all the baseline models suffer a significant fall in performance (expected due to the hardness of LAMA-UHN), the drop in performance of LM-CORE variants is much less. This indicates that
Figure 3: Effect of  $k$  on performance of different models.
having access to relevant external knowledge helps reduce the dependence on linguistic signals and results in the robust outperformance of LM-CORE variants.
4.3 Effect of Varying Number of Input Triples to LM-CORE
We analyze the effect of varying the number of candidates  $(k)$  during retrieval in Figure 3. We discuss with respect to Google-RE and T-REx subsets as our factual knowledge triples are most relevant for answering queries in these subsets (in comparison to commonsense queries in ConceptNet).
We plot the Precision@1 (P@1) against increasing  $k$  values from 1 to 10 for LM-CORE(b,y) and LM-CORE(r,w) variants. We do not observe any consistent optimal  $k$  value across variants and data subsets. To add, there is no significant difference between P@1 values as  $k$  varies from 4 to 10. Hence, in order to maximize our recall while keeping the computational expense in mind, we select  $k = 8$  for our experiments.
4.4 Role of LM-CORE Pre-training and Retrieved Knowledge
We now study the role LM-CORE pre-training plays in helping the model access and utilize the retrieved knowledge and ensure that the model does not just rely on the knowledge stored in its parameters. We also study the effect of augmenting the base LMs with knowledge retrieved by LM-CORE. In addition to providing an insight into the quality of the knowledge retrieved by LM-CORE, this will also help us better understand the ability of LM-CORE to utilize the retrieved knowledge.
We consider the following four variants on the LAMA probe (Table 3):
1. RoBERTa-base
Table 3: Ablation study analyzing the effectiveness of LM-CORE pre-training and contextually retrieved knowledge (Precision@1 values).
All	Google RE	T-REx	SQuAD	Concept Net
RoBERTa (base)	20.46	5.35	24.21	12.94	18.47
RoBERTa-base +LM-CORE triples	30.06	9.79	38.71	7.69	16.26
LM-CORE(r,w) + random triples	19.51	9.05	22.74	13.99	16.92
LM-CORE(r,w)	41.69	20.56	54.11	15.73	18.38
2. RoBERTa-base + triples retrieved by LM-CORE's Contextual Knowledge Retriever
3. LM-CORE(r,w) + random triples
4. LM-CORE(r,w)
We observe that LM-CORE(r,w)'s performance (41.69 P@1) significantly exceeds RoBERTa-base's performance using the same triples in input (30.06 P@1), demonstrating that our training procedure equips the model with the capability of identifying and using relevant external knowledge effectively. There is a large drop in performance (from 41.69 P@1 to 19.51 P@1) when LM-CORE(r,w) is provided with random triples in input. This shows that the model exclusively accesses external knowledge to answer queries correctly. While the performance drops, it is important to note that the P@1 is similar to RoBERTa-base (20.46 P@1), highlighting that our training procedure does not lead to catastrophic forgetting and the model is able to rely on the knowledge stored in its parameters when semantically relevant triples are not provided in the input. Finally, although RoBERTa-base when augmented with contextually relevant triples does not perform competitively with LM-CORE, it demonstrates considerable improvement over the base RoBERTa model. This shows that high-quality relevant external knowledge has the potential to improve factual prediction, further reinforcing our motivation to train models to efficiently retrieve and use this knowledge.
4.5 Downstream Tasks
We consider two downstream tasks to study the effectiveness of LM-CORE for different NLP applications. We take Zero-Shot Relation Extraction (ZSRE) (Levy et al., 2017) and open-domain question answering over Web Questions (WQ) (Berant et al., 2013) dataset as the representative knowledge-intensive tasks. Tables 4 and 5 re
port the performance of LM-CORE and various other baselines for the two tasks, respectively. We use the LM-CORE(b,w) variant for these experiments as most baselines use BERT as the LM and Wikipedia as the knowledge base. For the ZSRE task, we use the data splits and evaluation systems provided as part of the KILT benchmark (Petroni et al., 2021). We find that for the ZSRE task, LM-CORE achieves a significantly higher F-1 score (74.80) compared to the second-best RAG model (49.95). Also, note that the online evaluator for the task considers exact string match (including casing, punctuations, etc.) for computing accuracy numbers but not for computing other metrics. Hence, the reported accuracy number for LM-CORE represent a lower bound as we don't have access to the same pre-processing pipeline to process its output. For the WQ dataset, we find that LM-CORE outperforms BERT with BM25 and neural retrievers, and the DrQA system. We observe that LM-CORE is outperformed by ORQA, designed explicitly for this task, and RAG (a retrieval augmented generative model). However, do note that all the models except LM-CORE have access to much larger knowledge source (complete Wikipedia corpus.  $\approx$  2B words), whereas LM-CORE only has access to the KB triples ( $21\mathrm{M}$  triples,  $\approx$  140M words). As we show in the following subsection, with access to additional external knowledge, the performance of LM-CORE can improve significantly.
Table 4: F1 and Accuracy on Zero Shot RE. * The accuracy for LM-CORE is the lower-bound number as the online evaluator considers exact string match to compute accuracy.
#params	F1	Accuracy
BERT+DPR (Karpukhin et al., 2020)	330M	37.28	6.93
T5 (base)	220M	13.52	9.02
BART (large)	406M	12.21	9.14
BART+DPR	626M	34.47	30.43
RAG (Lewis et al., 2020)	626M	49.95	44.74
LM-CORE(b,w)	110M	74.80	14.24*
4.6 Handling Knowledge Updates
Once PLMs have been trained, it is expensive to retrain them with new and updated knowledge. LM-CORE, on the other hand, can easily access updated knowledge as it is external to the model. Instead of storing all the KB facts in the PLM, LM-CORE essentially learns semantically relevant associations between the input text and KB relations and can easily query for the relevant knowledge by using the learned relationship associations. We illustrate this ability to handle dynamic knowledge by in
Table 5: Accuracy on Web Questions. BM25, Neur. Retriever, DRQA and LM-CORE perform static retrieval.
Accuracy
BM25 + BERT	17.7
Neur. Retriever + BERT	7.3
DrQA (Chen et al., 2017)	20.7
LM-CORE (b,w)	21.9
ORQA (Lee et al., 2019b)	36.4
RAG (Lewis et al., 2020)	45.5
producing new triples in the KB and verifying if LM-CORE is able to leverage this new information to correct its earlier predictions. We consider the LM-CORE(b,y) variant for this experiment. We randomly sample 100 instances from the LAMA probe where the model failed and manually analyze these instances to identify the cases where the corresponding fact was not present in the YAGO KB. There were a total of 41 such instances and we manually added the correct facts needed to answer the corresponding questions in YAGO. We then presented the 41 inputs again to the model with the updated KB. This time, the model used this newly added knowledge and was able to correct its prediction without any re-training for 36 out of 41 cases  $(87.8\%)$ . As discussed in the following sub-section (4.7), a majority of errors made by LM-CORE are due to missing facts in the KB and we expect that most of such errors can be corrected by having access to a larger, more comprehensive Knowledge Base.
4.7 Discussions
We now present some representative examples to illustrate the successes and failures of LM-CORE. Consider a test probe from the Google-RE subset of LAMA - Phil Mogg is a member of _____. Here, the correct output token is UFO, the band and BERT model incorrectly predicts parliament as the output token. This highlights the sensitivity of PLMs on context; BERT's prediction seems to be derived from its memorization of the frequently encountered phrase member of parliament during pretraining. We argue that the contextual knowledge retrieved by LM-CORE which includes the relevant fact  $< \text{Phil Mogg};$  member of; UFO (band)> has helped the model to produce the correct output. We present more such successful examples in the Appendix (Tables 10 and 11).
Next, we analyzed the cases where the proposed framework produced incorrect output and observed three major reasons for erros -  $(i)$  the required knowledge was not present in the knowledge base;
(ii) the required knowledge was not retrieved despite being present in the knowledge base; and (iii) the system made errors after retrieving the relevant knowledge. The first problem cause could be addressed by enhancing the knowledge base as shown in Section 4.6. The other two causes of failure highlight the scope of improvement in our retrieval module as well as pre-training module, where further training could help the model make better use of the retrieved knowledge. Some representative examples of these different cases are presented in the Appendix (Table 12). Finally, we noticed some errors that could be attributed to the characteristics of the LAMA probe. Specifically, there are input probes that refer to entities without providing any additional context for disambiguation. For example, the sentence "James Johnson was born in—" has no clues to determine whether the prompt is referring to the basketball player, Virginia congressman, or the Governor of Georgia with this name. We also noticed certain probes where there are multiple correct completions and the benchmark considers only one of these as the correct answer. For example, "Michelangelo is a — by profession" can be correctly completed by poet, painter or architect, but the evaluation considers only poet as the correct answer. We also noticed some input examples with highly unambiguous language. For example, "X died in —", can refer to either X's place of death or date of death but only the former is accepted as the correct answer. Lastly, there are cases where slight (and correct) variations of the expected answer are evaluated as incorrect by the probe. For example, for the prompt "Harashima is — citizen." Japan is provided as the correct answer while the prediction made by LM-CORE (Japanese) is considered incorrect.
We presented LM-CORE, a framework to train language models with contextually relevant external knowledge. We show that having access to external knowledge leads to significant and robust outperformance over base language models and their knowledge enhanced versions on knowledge probing and two downstream tasks. We also showed how LM-CORE can handle knowledge updates and presented a thorough error analysis that helped us identify possible directions of future work.
Oshin Agarwal, Heming Ge, Siamak Shakeri, and Rami Al-Rfou. 2021. Knowledge graph based synthetic corpus generation for knowledge-enhanced language model pre-training. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3554-3565, Online. Association for Computational Linguistics.
Alan Akbik, Tanja Bergmann, Duncan Blythe, Kashif Rasul, Stefan Schweter, and Roland Vollgraf. 2019. Flair: An easy-to-use framework for state-of-the-art nlp. In *NAACL* 2019, 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), pages 54-59.
Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on freebase from question-answer pairs. In EMNLP, pages 1533-1544. ACL.
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2, NIPS'13, page 2787-2795, Red Hook, NY, USA. Curran Associates Inc.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.
Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading Wikipedia to answer open-domain questions. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1870-1879, Vancouver, Canada. Association for Computational Linguistics.
Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, and Andrew McCallum. 2019. Multi-step retriever-reader interaction for scalable open-domain question answering. In ICLR.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of
the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020. REALM: Retrievalaugmented language model pre-training. arXiv preprint arXiv:2002.08909.
Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. 2020. How can we know what language models know? Transactions of the Association for Computational Linguistics, 8:423-438.
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769-6781, Online. Association for Computational Linguistics.
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019a. Latent retrieval for weakly supervised open domain question answering. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6086–6096, Florence, Italy. Association for Computational Linguistics.
Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019b. Latent retrieval for weakly supervised open domain question answering. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28-August 2, 2019, Volume 1: Long Papers, pages 6086-6096. Association for Computational Linguistics.
Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. 2017. Zero-shot relation extraction via reading comprehension. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 333-342, Vancouver, Canada. Association for Computational Linguistics.
Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Kuttler, Mike Lewis, Wen tau Yih, Tim Roektaschel, Sebastian Riedel 0001, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.
Weijie Liu, Peng Zhou, Zhe Zhao, Zhiruo Wang, Qi Ju, Haotang Deng, and Ping Wang. 2020. K-BERT: enabling language representation with knowledge graph.
In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, pages 2901-2908. AAAI Press.
Robert Logan, Nelson F. Liu, Matthew E. Peters, Matt Gardner, and Sameer Singh. 2019. Barack's wife hillary: Using knowledge graphs for fact-aware language modeling. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5962-5971, Florence, Italy. Association for Computational Linguistics.
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers), pages 2227-2237. Association for Computational Linguistics.
Matthew E. Peters, Mark Neumann, Robert L Logan, Roy Schwartz, Vidur Joshi, Sameer Singh, and Noah A. Smith. 2019. Knowledge enhanced contextual word representations. In EMNLP.
Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick S. H. Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Roktaschel, and Sebastian Riedel 0001. 2021. Kilt: a benchmark for knowledge intensive language tasks. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021, pages 2523-2544. Association for Computational Linguistics.
Fabio Petroni, Tim Rocttäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language models as knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2463-2473, Hong Kong, China. Association for Computational Linguistics.
Nina Poerner, Ulli Waltinger, and Hinrich Schütze. 2020. E-BERT: Efficient-yet-effective entity embeddings for BERT. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 803-818, Online. Association for Computational Linguistics.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1-67.