From Evidence to Belief: A B ayesian Epistemology Approach to Language Models (2025)

Abstract: This paper investigates the knowledge of language models from the perspective of Bayesian epistemology. We explore how language models adjust their confidence and responses when presented with evidence with varying levels of informativeness and reliability. To study these properties, we create a dataset with various types of evidence and analyze language models’ responses and confidence using verbalized confidence, token probability, and sampling. We observed that language models do not consistently follow Bayesian epistemology: language models follow the Bayesian confirmation assumption well with true evidence but fail to adhere to other Bayesian assumptions when encountering different evidence types. Also, we demonstrated that language models can exhibit high confidence when given strong evidence, but this does not always guarantee high accuracy. Our analysis also reveals that language models are biased toward golden evidence and show varying performance depending on the degree of irrelevance, helping explain why they deviate from Bayesian assumptions.

## introduction large language models (llms) have advanced to the point where they can naturally respond to various practical tasks such as question-answering, code generation and conversation @xcite . however, limitations like hallucination and trustworthiness still exist, and research efforts continue to address these issues @xcite @xcite . in this paper, we take a different approach by examining large language models from a philosophical perspective: we investigate whether language models can be said to possess knowledge. in epistemology, knowledge is traditionally analyzed using three conditions-truth, justification, and belief-often associated with the justified true belief (jtb) framework @xcite . prior nlp research has focused on two aspects: factual correctness (i.e. the truth condition) -assessing whether the response of a model is correct @xcite -and justification, which encompasses explanation generation @xcite and evidence finding @xcite . in this work, we investigate whether the model believes its own responses (the belief condition): specifically, the relationship between belief and the language model's justification, expressed as evidence. since belief is a challenging concept to define, this paper focuses on belief from the perspective of bayesian epistemology, which interprets belief as a quantitative and functional variable. according to bayesian epistemology, the degree of belief can be interpreted and measured as probability, called probability norm. in particular, regarding the confirmation of belief, we should adjust the confidence of belief based on evidence. specifically, when h represents the hypothesis (or belief), e, the evidence for the belief, and θ representing the background information or prior knowledge, we can define 3 primitive assumptions: irrelevance assumption: e is irrelevant to h if and only if p (h | θ) = p (h | e, θ). 1 according to @xcite ; vassend (2023), we treat conditional probability as a primitive concept representing the likelihood of an event occurring under certain conditions, rather than relying on the standard ratio formula p (h | e) = p (h∩e) p (e) . this approach allows us to apply the bayesian assumption more intuitively, even in cases where p (e) = 0 or with contradictory evidence. for instance, when observing an unlikely event, such as a shark in a freshwater lake, we still make judgments based on the observation despite it contradicting common knowledge. a detailed discussion can be found in @xcite . figure 1 : the overall experimental method and simple examples of the evidence dataset for confirmation task. as golden evidence that aligns with the question is given to language models, it shows high confidence and accuracy. however, if language models encounter irrelevant evidence, it responds with low confidence. further results and analysis are reported in section 4.1. also, if we define belief in terms of probability, the strength of the evidence should also be reflected in the confidence. that is, the goal of this paper is to explore whether different types of evidence are reflected in language models' confidence and responses. the evidence here is not merely perturbations altering the correctness of information, i.e., informativeness, but our dataset also includes variations and modifications for reliability factors such as coincidence, timeliness, source of credibility, etc. our paper shows that language models can exhibit high confidence and accuracy when encountering true evidence but respond inconsistently with conflicting evidence and reduce confidence and accuracy with irrelevant evidence, contrary to bayesian assumptions. we also found that llms are biased toward golden evidence (typical of annotated information that forms part of datasets) and perform differently across the gradient of irrelevance, supporting an understanding of why llms deviate from bayesian epistemology. adversarial context with in-context learning, studies have examined how few-shot demonstrations and explanations affect responses @xcite . wang et al. (2023a) showed even inaccurate demonstrations could be used in chain-of-thought (cot) prompting, while chia et al. (2023) improved question accuracy with contrastive demonstrations. chen et al. (2023) studied how the number of demonstrations impacts accuracy. feng et al. (2023) measures language models' responses and probability shifts in temporal relations based on subtle contextual changes. while these works focus on accuracy, we explore how direct question evidence influences not only accuracy but also confidence and calibration. turpin et al. (2023); lanham et al. ( 2023 ) tested perturbations in cot inputs and their impact on answers, similar to our approach. while they focused on modifying explanations based on informativeness (e.g., incorrectness or relevance), our paper investigates whether llms reflect diverse evidence in their confidence and calibration, specifically exploring the effects of coincidental evidence and varying source credibility on model confidence. ## methods we first generate various types of evidence by few-shot prompting large language models (llms) with questions and annotated support from sciq @xcite , @xcite , @xcite . refer to appendix i.2 for details. we then used this evidence to evaluate how the models' confidence and responses change based on the type of evidence provided as shown in figure 1 . influenced by bayesian epistemology, we defined a confirmation task to measure whether language models can reflect the confirmation, disconfirmation, or irrelevance assumption introduced in section 1. also, we created a strength-of-evidence task to assess llm's ability to represent the various power of evidence. to measure the probability norm for adjusting confidence according to the evidence, we used an average confidence across all samples. in order to measure the response, such as correctness or calibration norm, we used accuracy (acc) and expected calibration error (ece). in both the confirmation task and the strength-of-evidence task, we used zero-shot prompting for inference. ## ablation llms tend to focus more on correct than incorrect information. in section 4.1, we interpreted that the language model possesses a certain degree of knowledge about the question in its parameters and tends to be biased towards contexts aligned with this parametric knowledge rather than context hindering it, as seen in golden, contradictory, and incomplete evidence. to justify this, we conducted an experiment adjusting the ratio of golden sentences in conflicting, incomplete, and contradictory evidence. figure 3 (a ) and (d) show that as the number of original golden sentences decreases and the conflicting sentence increases, the performance of the language model gradually declines. however, it decreases significantly when there are no golden sentences left. moreover, figure 3 (b) and (e) demonstrate that as the original golden sentence figure 3 : the results for the degree of variations in evidence for the sciq dataset with verbalized method. we modified the number of conflicting sentences in conflicting evidence, sentences in incomplete evidence, and contradictory sentences in contradictory evidence (see appendix f for entire results). why do llms get confused by irrelevant context? two interpretations are possible for the irrelevant case 1. the language model does not recognize irrelevant evidence as irrelevant when it is in the same field but differs in content. to verify (1), instead of extracting irrelevant evidence from the same dataset, we used contexts from different datasets, for sciq and triviaqa dataset, we used evidence of gsm8k, and for gsm8k, using triviaqa. as we can see in figure 6 in appendix g, even when using a new irrelevant sentence, it did not completely match the completely irrelevant assumption. however, surprisingly, when using evidence from a completely different field, we found that the confidence, accuracy, and ece metrics approached closer to the baseline no evidence case (p(h)) than when we used evidence where the content was different but the field was the same. this implies that as the irrelevance increases, the llms become less distracted by the context. therefore, we interpreted that there is a possibility that the llms satisfy the irrelevant assumption of bayesian epistemology. ## why do llms struggle to follow bayesian assumptions? we speculate that llms may not fully adhere to bayesian assumptions due to limitations in both their training data and methods. pretraining datasets, primarily sourced from web crawls with filtering and books (gao et al., 2020), expose models mostly to well-justified information and correct explanations. this allows them to excel in deriving correct answers but limits their ability to handle coincidental evidence or conflicting beliefs, which are underrepresented in the data. in addition to data limitations, the training meth-ods used for llms diverge significantly from human language acquisition. humans encounter unreliable and conflicting situations through interaction and experience with the real world. in contrast, llms are trained in a largely supervised manner, resulting in a lack of semantic understanding (bender and koller, 2020; @xcite . as a result, these limitations may explain why llms fail to fully align with bayesian assumptions when faced with conflicting or unexpected scenarios, as they lack the experiential grounding to properly handle such evidence. ## conclusion in this paper, we explored how changes in the informativeness and reliability of evidence affect the confidence and response of language models. specifically, we examined how well language models stick to the probability and calibration norms outlined in bayesian epistemology. we demonstrated that language models partially align with bayesian epistemology, following confirmation assumptions but failing to adhere to disconfirmation and irrelevance assumptions. it can be interpreted that language models do not possess a justified belief in the view of bayesian epistemology. additionally, we found that llms show a bias toward golden evidence and modify their confidence and response relative to the degree of irrelevance, which helps clarify their deviation from bayesian assumptions. these findings provide philosophical insight into the nature of "belief" in llms, highlighting their biases and limitations.