Do Large Language Models Know How Much They Know? (2024)

Abstract: Large Language Models (LLMs) have emerged as highly capable systems and are increasingly being integrated into various uses. Nevertheless, the rapid advancement in their deployment trails a comprehensive understanding of their internal mechanisms, as well as a delineation of their capabilities and limitations. A desired characteristic of an intelligent system is its ability to recognize the scope of its own knowledge. To investigate whether LLMs embody this attribute, we develop a benchmark that challenges these models to enumerate all information they possess on specific topics. This benchmark assesses whether the models recall excessive, insufficient, or the precise amount of required information, thereby indicating their awareness of how much they know about the given topic. Our findings reveal that the emergence of this property varies across different architectures and manifests at diverse rates. However, with sufficient scaling, all tested models are ultimately capable of performing this task. The insights gained from this research advance our understanding of LLMs, shedding light on their operational capabilities and contributing to the ongoing exploration of their intricate dynamics.

## introduction large language models are renowned for their ability to memorize vast amounts of information encountered during training @xcite @xcite . this information, stored in their parameters, can be recalled during inference, serving both for information retrieval and problem-solving @xcite @xcite . while it is well-established that llms can act as knowledge bases @xcite @xcite , the extent to which they understand their own knowledge is less clear @xcite . for instance, do these models know if or when they know the answer to a question @xcite ? can they quantify their own expertise on a topic? are they aware if some of their knowledge contradicts other information they possess? can they differentiate between explicitly learned information and implicit knowledge? these questions are crucial, as awareness of one's own knowledge and limitations is a vital aspect of any intelligent system. without it, an ai could be prone to hallucinate @xcite , lie about its expertise @xcite , overestimate its responses @xcite , or contradict itself @xcite , all of which are undesirable traits for ai systems intended to be useful. this study focuses on understanding whether llms know the extent of their knowledge on specific topics, such as individuals, locations, events or concepts. to explore this, we task llms with enumerating everything they know about a given topic-no more, no less. should a model consistently recall just the right amount of information, it suggests an understanding of its own knowledge. conversely, if a model does not know how much it knows, it may recall too little or hallucinate additional information. our approach involves fine-tuning llms on the diary entries of various fictitious individuals. each entry is treated as an individual document in our fine-tuning dataset, with each diarist authoring a random number of entries. during inference, we ask the models to recall all diary entries of a specified individual in chronological order. we then evaluate whether the recalled entries match the original entries both in terms of content and quantity. figure 1 provides an illustrative example. we benchmark the performance of the opt @xcite , pythia @xcite , and flan-t5 @xcite suites of tom's diary entry 2 mood: happy location: countryside weather: rain restfulness: rested tom's diary entry 3 meditated: no alice's diary entry 1 location: countryside time: morning weather: rain alice's diary entry 2 stress level: relaxed weather: rain mood: sad restfulness: tired figure 1 : we train llms using diary entries from various individuals, with each diarist contributing a random number of entries. we then task the models with recalling all entries written by a specific individual, evaluating their ability to accurately recall the exact number of documents authored by that person. models. our key findings are as follows: finally, we discuss potential factors responsible for the observed differences in the emergence of this capability. overall, our work contributes to a deeper understanding of the inner workings of llms, shedding light on a not-so-well-understood aspect of these models. ## methodology the foundation of our analysis hinges on the ability of models to memorize and recall information. to avoid the influence of existing data, which might be part of the pre-training corpus of the language models we are benchmarking, we generate our own. this ensures that the models have never encountered the data during pre-training, thereby preventing any contamination of our results. in essence, our approach involves: (i) generating the training documents, (ii) fine-tuning a language model using its pre-training objective to memorize these documents, and (iii) testing the language model's ability to recall all related documents. we delineate each stage of our framework in the following sections. ## discussion in addition to the questions raised in the previous section, the following additional questions should be considered. ## conclusion this study focused on determining whether llms possess an understanding of the span of their own knowledge on specific topics. notably, we observed that all models, if scaled sufficiently, know how many documents are authored by the same person. consequently, these llms know how much they know about these individuals; otherwise, they would sporadically recall too few or too many documents. more specifically, we find that this capability emerges based on the model's architecture, its size, the dataset's size used for training, and the effectiveness of the pre-trained weights in learning a solution that generalizes, rather than simply memorizing the training samples. overall, our research contributes to a deeper understanding of the capabilities and inner workings of llms. grasping how aware llms are of their own knowledge and identifying any limitations in this regard is crucial, as this feature enhances the usefulness and trustworthiness of intelligent systems. further research is necessary to continue exploring this aspect.