Systematic Assessment of Factual Knowledge in Large Language Models (2023)

Abstract: Previous studies have relied on existing question-answering benchmarks to evaluate the knowledge stored in large language models (LLMs). However, this approach has limitations regarding factual knowledge coverage, as it mostly focuses on generic domains which may overlap with the pretraining data. This paper proposes a framework to systematically assess the factual knowledge of LLMs by leveraging knowledge graphs (KGs). Our framework automatically generates a set of questions and expected answers from the facts stored in a given KG, and then evaluates the accuracy of LLMs in answering these questions. We systematically evaluate the state-of-the-art LLMs with KGs in generic and specific domains. The experiment shows that ChatGPT is consistently the top performer across all domains. We also find that LLMs performance depends on the instruction finetuning, domain and question complexity and is prone to adversarial context.

## introduction the rise of large language models (llms) has greatly improved the capabilities of natural language processing (nlp). however, one primary concern with these models is the potential for extrinsic hallucinations where llms generate statements that cannot be verified from the source @xcite . this issue severely impairs the trustworthiness of llms and is particularly concerning when relying on llms for decision-making. rigorous evaluation is necessary before deploying them in critical applications. one evaluation approach is to use questionanswering datasets to assess the language and knowledge capabilities of llms. recent research has mainly focused on evaluation using existing benchmarks @xcite , 2023; @xcite . while these benchmarks are valuable for comparison and measuring progress in llm research, they may not provide sufficient assessment for production. benchmarks constructed from public datasets can pose information leakage problems due to overlap with pretraining data. furthermore, constructing domainspecific benchmarks is costly, requiring domain expertise and adequate knowledge coverage. this paper proposes a systematic approach to assess factual knowledge in llms by generating a comprehensive assessment suite from knowledge graphs (kgs) and evaluating the correctness of llms' responses. the question generation process is carefully designed to ensure coverage of facts, as well as diversity and validity of the questions (figure 1 ). using this framework, we evaluate multiple models from three llm families on factual questions derived from four kgs, covering both generic and specialized domains. specifically, our contributions are: ## systematic assessment framework this section describes the question generation component in our proposed assessment framework, followed by the answer prompting strategy to collect llm's response and the evaluation metric. ## conclusion we propose a systematic framework to evaluate factual knowledge of llms with the diverse and well-coverage questions generated from kg. the experiment reveals several factors affecting llms' performance and highlights their vulnerability to adversarial context. our findings contribute to understanding llms' capabilities and limitation in handling factual knowledge.