ALCUNA : Large Language Models Meet New Knowledge (2023)

Abstract: With the rapid development of NLP, large-scale language models (LLMs) excel in various tasks across multiple domains now. However, existing benchmarks may not adequately measure these models’ capabilities, especially when faced with new knowledge. In this paper, we address the lack of benchmarks to evaluate LLMs’ ability to handle new knowledge, an important and challenging aspect in the rapidly evolving world. We propose an approach called KnowGen that generates new knowledge by altering existing entity attributes and relationships, resulting in artificial entities that are distinct from real-world entities. With KnowGen, we introduce a benchmark named ALCUNA to assess LLMs’ abilities in knowledge understanding, differentiation, and association. We benchmark several LLMs, reveals that their performance in face of new knowledge is not satisfactory, particularly in reasoning between new and internal knowledge. We also explore the impact of entity similarity on the model’s understanding of entity knowledge and the influence of contextual entities. We appeal to the need for caution when using LLMs in new scenarios or with new knowledge, and hope that our benchmarks can help drive the development of LLMs in face of new knowledge.

## introduction large-scale language models (llms) have made impressive progress in the last few years @xcite @xcite , which perform surprisingly well on various tasks on various domains, to the extent that many traditional benchmarks @xcite are no longer sufficient to measure the capabilities of llms. therefore, some new benchmarks have been proposed to evaluate the ability of the model to solve more complex tasks such as college entrance exams, law * these authors contributed equally to this work. school admission tests, math competitions and so on @xcite @xcite . llms also achieve promising results on these benchmarks. however, it is surprising that there is not yet a benchmark to evaluate the ability of large models in face of new knowledge, which is very important and challenging. why is this evaluation important? firstly, we are in a changing world, where models encounter new knowledge frequently in practice. and some work @xcite is exploring retrieval methods to augment large models, which will also cause the models to meet new knowledge frequently. so of course we expect the model to perform well in such situations, because re-training the model every time is very expensive and unrealistic. secondly, as @xcite mentioned, the presence of overlap between the training and test data can lead to a misestimation of the model's memory ability as generalization ability, especially nowadays when llms are trained on an enormous amount of data. whereas, evaluation on new knowledge does not need to worry about such data leakage, as new knowledge can usually lead to new data and thus more reliable and valuable assessment of the model's ability. while such evaluation is important, it is challenging to construct the benchmark. the reason is that it is difficult to ensure that the knowledge contained in the benchmark is new for llms, since training data for some models are large and nonpublicly available. furthermore, it is also difficult to ensure that the knowledge used for benchmarking will be not outdated and inefficient, as there are many llms that may soon include data from the benchmark in their training. in summary, such benchmark for new knowledge needs to exhibit three basic characteristics: it contains enough new knowledge for sufficient evaluation (sufficient), the knowledge is new to all models (model-agnostic) and the knowledge can remain new for a long time @xcite . there are several possible solutions to the above challenges. one option is to always use the most updated data such as the news of the day (temporal knowledge). however, this is both labor-intensive to race against the llm training and unclear about the lifetime of the proposed data. another option is to keep the benchmark closed-source, with an authoritative committee managing the data and users calling api when evaluating, thus preventing using them for training llms . to reach this point further requires community coordination. to better address these challenges, we propose an approach to generate new knowledge conveniently (knowgen) based on the structured representation of existing entity knowledge by making reasonable changes to entity attributes and relationships. there are differences and associations between artificial entities and existing entities. particularly, we apply knowgen with structured biological taxonomic information data to rationally create a group of organisms that do not exist in the world. to test the model's ability in face of new knowledge, we construct a variety of questions about these artificial entities that can examine the model's ability to understand new knowledge (knowledge understanding), distinguish between model's internal knowledge and new knowledge (knowledge differentiation) and make multi-hop reasoning by linking model's internal and new knowledge (knowledge association). we use the artificially constructed knowledge to assess llms as a benchmark (alcuna). we evaluate and analyze several popular large models based on alcuna, including chatgpt 1 , alpaca, vicuna, and chatglm with vanilla, cot (chain-of-thought), zero-shot and few-shot settings @xcite @xcite . we find that neither chatgpt nor other models perform very well in face of new knowledge. chatgpt does a good job of understanding and differentiating new knowledge, but almost all models fail to reason between the new knowledge and the internal knowledge. this reminds us to remain cautious when large models encounter new scenarios and knowledge. in addition, we explore the impact of entity similarity on the model's understanding of entity knowledge, the impact of contextual entities, etc. the contributions of our work are listed below: ## knowgen: knowledge generation in this section, we begin by presenting our inspiration, then formally introduce our knowledge generation method knowgen. ## alcuna: our benchmark with our proposed method, one can create a large amount of new entities quickly based on existing structured knowledge. a natural attempt is to construct new organisms on already discovered ones, since the biological taxonomy is perfectly adapted to our proposed approach. therefore, we utilize knowgen to propose alcuna, a biological dataset for evaluating the ability of the model in face of new knowledge as shown in figure 1 . ## related work large language models in recent years, significant advancements in large language models (llms) like flan-t5 @xcite , @xcite , opt @xcite and @xcite have led to exceptional performance in natural language processing tasks. at the same time, open-source llms based on llama and other fundamental models for further instruction fine-tuning have emerged recently, such as alpaca @xcite , vicuna @xcite , koala @xcite , chatglm @xcite , etc., which have also shown strong capabilities. these models have shown breakthroughs on a variety of tasks, and some have been applied as a commercial product in daily work. since the world is constantly changing, the ability of the models to perform when faced with new knowledge is critical. existing benchmarks benchmarks, such as squad @xcite , @xcite , glue @xcite , super-glue @xcite , lambada @xcite , etc., is essential for setting evaluation criteria and tracking model performance. while some traditional benchmarks like squad and snli assess single-task abilities, glue and su-perglue evaluate general language models across various nlp tasks. more recently, with rapid development of llms, new benchmarks have been proposed to evaluate on more complex tasks such as college entrance exams, law school admission tests and so on @xcite @xcite . however, these benchmarks are still based on existing knowledge, which is abundant in the training data of llm. some knowledge benchmarks @xcite @xcite ) evaluate model's ability of knowledge integration while they either only evaluate on small models which are very different with llms (training data, ability, etc.) or simply use knowledge that already exists. therefore, benchmarks that assess llms' ability with new knowledge rather than existing knowledge are urgently needed. source of new knowledge many works use temporal knowledge as a source of new knowledge to evaluate new knowledge behavior of llms @xcite @xcite . there is also some work that uses entity or attribute substitution to create new knowledge @xcite . a discussion of the differences and strengths and weaknesses of our work versus prior work is in appendix i. ## conclusion we propose a new approach knowgen to construct new knowledge and build an artificial biological entity benchmark alcuna for evaluating the ability of llms faced with new knowledge. we test and analyze several popular models with commonly used methods, and find some useful conclusions. our proposed approach and benchmark can help the development of more powerful llms that can understand, differentiate and reason across new and existing knowledge. in the future, we expect that more llms will be evaluated on our benchmark. more benchmarks in other disciplines also can be constructed based our method.