Epistemology of Language Models: Do Language Models Have Holistic Knowledge? (2024)

Abstract: This paper investigates the inherent knowledge in language models from the perspective of epistemological holism. The purpose of this paper is to explore whether LLMs exhibit characteristics consistent with epistemological holism. These characteristics suggest that core knowledge, such as commonsense, general, and specific knowledge, each plays a specific role, serving as the foundation of our knowledge system and being difficult to revise. To assess these traits related to holism, we created a scientific reasoning dataset and examined the epistemology of language models through three tasks: Abduction, Revision, and Argument Generation. In the abduction task, the language models explained situations while avoiding revising the core knowledge. However, in other tasks, the language models were revealed not to distinguish between core and peripheral knowledge, showing an incomplete alignment with holistic knowledge principles.

## introduction recent advancements in language models have extended their capabilities beyond simple questionanswering (qa) tasks to more complex knowledgeintensive retrieval and reasoning challenges akin to problems solved by humans @xcite @xcite . moving beyond factoid qa tasks, large-scale language models are now recognized for their ability to revise and update their knowledge, including the implicit consequences of such modifications @xcite . furthermore, there is an increasing interest in applying language models in more applied fields such as medicine and law, given their human-like functionality in various knowledge evaluation tasks @xcite @xcite . this paper focuses on a more philosophical inquiry: we explore the nature of the epistemology figure 1 : a diagram of the holistic web of belief. at the core, there are certain pieces of knowledge that serve as the basis of our beliefs. while towards the periphery, less certain empirical knowledge is located. in this web, all knowledge is revisable, but when we encounter new experiences, the peripheral knowledge is more prone to revision than that at the core. inherent in language models, questioning what they know, how they process their beliefs, and how they handle knowledge from philosophical viewpoints. the primary aim of this paper is to examine the knowledge within language models through the lens of epistemological holism, particularly focusing on whether these models possess core knowledge, such general scientific knowledge and commonsense, as outlined by epistemological holism. one of the tenets of epistemological holism is that our knowledge forms a "web of belief.", which means that no piece of knowledge is isolated; rather, each is interconnected with others, forming a network. at the core of this network lie relatively certain knowledge elements such as logic, commonsense, and scientific facts, which are difficult to revise even when counterexamples are presented. on the web's periphery are empirical facts, more directly related to experience and more easily revised upon encountering counterexamples @xcite . epistemological holism adopts a kind of pragmatic point of view regarding knowledge revision. when faced with a counterexample to a general fact, one tends to defend the general fact as much as possible while seeking alternative conditions or explanations. a classic example is illustrated in the response to the failure of newton's laws to precisely predict uranus's orbit. rather than revising newtonian mechanics, scientists hypothesized the existence of another influencing planet, leading to neptune's discovery @xcite . in this paper, we denote core knowledge as scientific facts, and peripheral knowledge referred to other supplementary statements explaining counterexamples to those scientific facts. formally, if we represent knowledge as a graph @xmath0, e) where nodes v represent individual knowledge items and edges e represent connections between them, indicating that nodes are related and τ is a predefined threshold for degree, core knowledge could be defined as a set @xmath1: deg(v) > τ core } consisting of nodes that would have higher degrees (more edges connecting them to other nodes) compared to peripheral knowledge items. also, peripheral knowledge could be represented as a set @xmath2: deg(v) < τ perph } consisting of nodes that are less interconnected than core knowledge. 1 in this respect, epistemological holism suggests that when contradictory information to a node @xmath3, it's more likely to modify @xmath4. we developed a dataset based on the world tree corpus @xcite by following the procedure used in the worker-ai collaboration framework @xcite . through this dataset, we evaluated how language models respond to collisions with general knowledge by evaluating three tasks: abduction, revision, and argument generation, along with fine-tuning. in the abduction task, all models achieved over 60% without tampering with core knowledge. however, in revision and argument generation tasks, even state-of-the-art models frequently negated core knowledge. results from supervised fine-tuning also revealed that language models tend to treat general factual knowledge and core knowledge equally. these mixed outcomes imply that llms' alignment with a holistic epistemological framework may be context-dependent, indicating a partial, rather than comprehensive, adherence to holistic epistemology. 1 of course, since this is a philosophical concept, it would be difficult to define how much connection is needed (τcore, τ perph ) to be considered core or peripheral. 2023 ) demonstrate that language models are sensitive to external knowledge that conflicts with parametric knowledge. our paper includes a task related to knowledge editing. however, our aim goes beyond examining the consistency of language models given external facts. we seek to understand how llms modify existing knowledge or defend it when presented with abnormal counterfactuals. studying reasoning through entailment historically, significant efforts have been dedicated to evaluating and enhancing the reasoning capabilities of language systems @xcite . for example, the natural language inference (nli) task involves classifying the entailment relationship between two sentences @xcite @xcite . @xcite introduced a reasoning dataset that evaluates if a language model can provide the best explanation given a scenario. @xcite introduced research that explains situations where unexpected events occur in everyday life. our paper explores the preference and reasoning of language models in situations where they conflict with general facts and could potentially disrupt our belief system and where multiple indeterministic yet valid logical conclusions exist. ## primer on epistemology epistemology is a branch of philosophy that explores knowledge. it investigates the conditions under which we can claim to "know" something. traditional epistemology has focused on the normative conditions of knowledge, defining knowledge as "justified true belief" and contemplating what constitutes justification, the definition of truth, and the nature of belief. while traditional epistemology deals with the normative aspects of knowledge, i.e., the necessary and sufficient conditions of knowledge, naturalized epistemology discusses how our knowledge is actually formed and undergoes revision, and holism is one of the most well-known branches of naturalized epistemology @xcite . in this paper, we study the epistemology of language models. ## dataset we construct a holism reasoning dataset based on scientific facts in the worldtree v2 dataset @xcite . therefore, all scientific facts in our dataset are included in worldtree v2. we augment the dataset with counterfactual observations. the generation process of the counterfactuals is similar to the one described in @xcite . we followed three steps to create our dataset: we represent our sample as quadruplet (s, c, e * 1 , e * 2 ) where s is general scientific fact, c is counter observation, e * ## experimental design we conducted evaluations for the aforementioned tasks using the flan-t5 models (ranging from base to xxl) @xcite , llama2-chat-7b and 13b @xcite , phi-2 @xcite , gpt-3.5, and @xcite . all tasks were executed as zero-shot evaluations. the reason for not employing in-context learning is that the purpose of this paper is not to enhance the performance of each task but to investigate whether language models consider scientific facts and other universal truths as part of their core knowledge that is difficult to revise. this is because, with a few-shot demonstration, language models can easily choose between two options based on sentence patterns. we used a total of 1,709 data samples as test samples for zero-shot evaluation. evaluation metric for convenience, we will refer to general scientific knowledge as core knowledge and statements or knowledge that explains the conflicting situation by mentioning other conditions or factors or raising questions about observation as peripheral statements. we measured the metric peripheral response ratio (prr), showing the proportion of instances out of the total samples where the model explained the conflicting situation using peripheral statements rather than directly negating or modifying general facts. ## results and discussion preference tasks: do language models prefer to keep general knowledge? we can see the results in the primary goal of the abduction task is to select the most plausible explanation for a given observation that fits within the existing knowledge framework. due to the abundance of instructions for explanations and datasets for abduction, language models are likely to be familiar with these types of tasks @xcite . also, the task implicitly encourages the model to use peripheral or auxiliary information to construct explanations without directly challenging the core knowledge. the success of llms in abduction tasks suggests they are adept at navigating their extensive pretrained knowledge to find and apply relevant peripheral information that can explain new observations without directly confronting or needing to alter core knowledge. revision tasks differ from abduction tasks by requiring language models (llms) to evaluate the validity of core versus peripheral knowledge in the face of contradictions. while abduction tasks allow for generating explanations that work around core knowledge and incorporate additional, compatible information to explain an observation, revision tasks demand a direct assessment of whether to modify core knowledge or adjust peripheral details. the fact that llms do not exhibit a holistic approach in revision tasks indicates they may not inherently prioritize protecting core knowledge when faced with its potential revision. qualitative analysis for argument generation task: can language models make a holistic inference? in the argument generation task, which best represents the indeterministic argumentative structure of epistemological holism, we similarly measured the ratio of conclusion where llms refer to peripheral statements. when measuring the prr in the generation task, we used a soft criterion. specifically, as long as the hypothesis was not completely negated, we considered the language model to have made a peripheral response; also, any mention of a peripheral statement was given a positive prr score. this is because, especially with models like those in the gpt series and phi-2, even if they partially reject the hypothesis with phrases like "not all" or "not always" or even outright deny it, they sometimes mention other external conditions or observational errors that influence the observation fact. according to ## conclusion the purpose of the paper was to explore whether llms exhibit characteristics consistent with epistemological holism, as they suggest that commonsense, general, and certain knowledge serves as the foundation of our belief network and is difficult to revise easily. our findings reveal a nuanced picture: llms adeptly navigate peripheral beliefs in abduction tasks without negating core beliefs, showing proficiency in generating abductive explanations. however, across other evaluative frameworks, they exhibit a lesser tendency to recognize and protect the special status of core knowledge, suggesting a divergence from holism's principled knowledge interconnected hierarchy. the variability in results suggests that the extent to which language models conform to a holistic epistemological model varies by context, demonstrating an incomplete commitment to the principles of holistic epistemology. the inconsistent preference task results and sft experimental findings suggest that it is difficult to claim that core knowledge and peripheral knowledge are distinctly separated within the language model. consequently, it seems that language models do not consistently apply a holistic methodology to knowledge processing and reasoning across diverse cognitive challenges.