Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models (2023)

Abstract: The increased deployment of LMs for real-world tasks involving knowledge and facts makes it important to understand model epistemology: what LMs think they know, and how their attitudes toward that knowledge are affected by language use in their inputs. Here, we study an aspect of model epistemology: how epistemic markers of certainty, uncertainty, or evidentiality like “I’m sure it’s”, “I think it’s”, or “Wikipedia says it’s” affect models, and whether they contribute to model failures. We develop a typology of epistemic markers and inject 50 markers into prompts for question answering. We find that LMs are highly sensitive to epistemic markers in prompts, with accuracies varying more than 80%. Surprisingly, we find that expressions of high certainty result in a 7% decrease in accuracy as compared to low certainty expressions; similarly, factive verbs hurt performance, while evidentials benefit performance. Our analysis of a popular pretraining dataset shows that these markers of uncertainty are associated with answers on question-answering websites, while markers of certainty are associated with questions. These associations may suggest that the behavior of LMs is based on mimicking observed language use, rather than truly reflecting epistemic uncertainty.

## introduction as natural language systems are increasingly used in situations involving factuality and knowledge, it becomes important for lms to be able to interpret how humans talk about knowledge. lms must learn to accurately interpret linguistic cues like expressions of uncertainty and certainty that are used to talk about confidence, source, and limitations of information. in this work, we seek to understand how models interpret this linguistic phenomenon by measuring how language generation varies when prompted with expressions of uncertainty. q: what is the capital of france? a: i think it's... "paris." prompt generation q: what is the capital of france? a: i'm 100% certain it's... q: what is the capital of italy? a: rome. i'm sure. "lyon." q: what is the capital of france? a: paris. i'm sure. single prompt which contains a series of calibrated in-context learning examples "i'm sure" is appended to high confidence answers "london. i'm sure." zero-shot injection of expressions of uncertainty multi-shot prompting to emit expressions of uncertainty verbal numerical q: what is the capital of albania? a: tirana. q: what is the capital of the uk? figure 1: using zero-shot promoting to inject verbal and numerical uncertainties into trivia questions. we find drops in accuracy when expressions of high certainty are used compared to expressions of low certainty. naturalistic expressions of uncertainty/certainty cover a broad range of discourse acts such as signaling hesitancy, attributing information, or acknowledging limitations. prior work has focused on one aspect of this: linguistic calibration, particularly on learning the mapping between the internal probabilities of a model and an ordinal output @xcite @xcite . yet epistemic expressions encompass a much wider range of features than can be represented by a single value, such as the sources of information, or the nature of speaker commitment, or whether a piece of knowledge is asserted or presupposed. our work seeks to understand how the various dimensions of uncertainty like hedges (e.g., "it could be. . . "), factive verbs (e.g., "we realize it's. . . "), and evidential markers (e.g., "wikipedia says it's. . . ") impact language generations. by shifting our focus to naturalistic expressions of uncertainty and certainty, we would enable models to flexibly interpret a wider range of uncertainties otherwise not possible under the current linguistic calibration setup. to understand how epistemic expressions affect language models, we use zero-shot prompting and inject verbal and numerical markers into trivia question prompts. this process converts a prompt like "what is the capital of france?" to, "what is the capital of france, i think it's. . . ". (figure 1 ). aided by our linguistic typology, we then measure how different epistemic marker types impact model ac-curacy. by doing so, we complement current linguistic calibration work and paint a more complete picture of the role of these epistemic markers. we begin with two broad hypotheses on how lms might respond to expressions of uncertainty. first, we might suppose that models are robust to any added expressions of uncertainty in the prompt. an alternative hypothesis is that, models might respond differently based on the uncertainty cues, and using a marker suggesting certainty or confidence might be more likely to produce the correct response than a prompt with low certainty or confidence. under the latter hypothesis, we might expect performance for a prompt with no epistemic markers (which we call the standard method) to lie in between these two. this second hypothesis would also be consistent with prior work showing that lms can generate language in the style of diverse personas @xcite . surprisingly, we find that injecting expressions of high certainty like "i'm certain it's" or "i'm 100% sure" actually leads to lower accuracy. we follow up with qualitative and quantitative analysis of popular training data, suggesting some potential sources of these unexpected behaviors. our work thus offers three key contributions: together, our findings illustrate the shortcomings of models' abilities to interpret expressions of uncertainty and highlight the gaps that still exist between natural and generated language. the most widely studied weakeners are hedges, first defined by @xcite as related to modifying or weakening the category membership of a predicate or nominal. the central kind of hedges are approximators (e.g., somewhat, kind of, about, approximately), which hedge propositional content. another class of weakeners that some (like @xcite ) but not others classify under hedges are plausibility shields which express a speaker's lower level of commitment (e.g., i think, i believe). strengtheners are constructions that mark certainty. we use this term to refer to strengthening speaker commitment to truth value, and also to strengthening category membership. strengtheners include boosters or intensifiers (e.g., "i am certain", "undoubtedly") @xcite . while boosters can assert certainty or truth, a second kind of strengthening construction, the factive verb, is used to presuppose certainty or truth. factive verbs like "know", "realize", or "understand" (kiparsky and kiparsky, 1970) presuppose the truth of the complement sentence. the statement "x realizes y" presupposes that y is true, (and also asserts that x is aware of it). by contrast, "x believes y" makes no presupposition about the truth of y. thus the sentence "he realizes [madrid is the capital of france]" is infelicitous, because "realize" as a factive verb presupposes the truth of its complement clause, which in this case is false (madrid is not the capital of france). a third class of markers can mark both certainty and uncertainty by directly indicating the source of the information. such expressions (e.g., "accord-ing to wikipedia", "i heard", "i saw") are called evidential markers, linguistic signals that tells the hearer where the information came from @xcite . one subtype of evidential markers, quotative markers, are used when reported information overtly references a source (e.g., "according to research in the latest issue of nature", "two recent studies demonstrate that. . . "). we examine when references are citing a source (e.g., "wikipedia says") versus when the source is unspecified or very indirect (e.g., "they said"). we'll refer to this former case as sourced as a shorthand for indicating that a source is mentioned. finally, first-person personal pronouns (e.g., "i", "we", "our") can be used to mark subjectivity and uncertainty in expressions like "i think". ## methods to study how models interpret uncertainty, we inject markers into trivia questions in open-ended question-answering. using our typology, we create fifty sentences (minimal pairs) for every question. our datasets include triviaqa, a standard qa dataset @xcite ; natural questions (closed-book), an aggregated set of google queries @xcite ; countryqa, which we constructed using names of countries and capitals in the form of "what is the capital of afghanistan?"; and jeopardy questions crawled from a fan-created database, j! archive. 3 we use a random subset of 200 questions for three of our datasets. for countryqa, we used all 53 questions whose answers were in vocabulary (details in 3.2). although the total number of questions per template is small, we present our key findings in aggregate across all templates with shared characteristics, increasing our statistical power. we test each of our fifty templates across this subset of questions and calculate 95% confidence intervals using bootstrap resampling or t-tests. we primarily study openai's gpt models as they are commonly used large language models with reasonable performance on qa tasks @xcite . for all models, except gpt4, we set temperature to 1. for the generated tokens, we take the sum of the probability assigned to the gold answer(s) to be the probability-on-gold. when calculating accuracy, we generate 10 tokens and if any of the tokens match the answer (or aliases of the answer), we'll count that as a correct generation. this is done to not unfairly disadvantage templates that are prone to generating words prior to emitting the answer (e.g., "allegedly, it's said to be. . . "). 4 for gpt4, where the log probabilities were not available, we set the temperature to 0, also ensuring a (mostly) deterministic output, and used the generated text. ## why does certainty hurt? what explains our surprising finding that templates with weakeners outperform templates with strengtheners? here, we discuss and evaluate hypotheses and potential confounders. ## the impact of the degree of uncertainty on performance results from section 3.2 illustrate that gpt3 is highly sensitive to uncertainty in prompts, and certainty seems to lead to diminished accuracy. here, we extend these results to study uncertainty at a more fine-grained level, allowing us to ask if the degree of uncertainty could play a role in the accuracy of model generation. introducing numerical values here, we introduce numerical values into our verbal expressions of uncertainty. the setup of our task changes from "what is the capital of belarus? i'm sure it's. . . " to "what is the capital of belarus? i'm 90% sure it's. . . ". we use a set of seven expressions covering a range of numerical uncertainty expressions, including those that use personal pronouns to weaken uncertainty ("i'm 90% certain. . . ") and those which indicate uncertainty probabilistically but without mentioning the self ("70% chance it's. . . "). we also downsample our test set to 50 questions per dataset and evaluate each template at 0%, 10%, 30%, 50%, 70%, 90% and 100% intervals. 100% certainty is not 100% accurate our findings for numerical uncertainties extend our earlier analysis by enabling us to obtain more fine-grained results on whether a model is linguistically calibrated, and how much certainty causes performance degradation. first, we find that numerical uncertainties in the prompt are not well calibrated with the accuracy of the answers generated. foot_5 e.g., the prompt "i'm 90% sure it's. . . " in triviaqa only produces the correct answer 57% of the time (figure 4 ). formally, we evaluate the expected calibration error (ece) and find poor values ranging from 0.50 to 0.30, 0 being the best. second, consistent with our findings from section 4, we find that certainty does hurt model performance. however, we find this is true only at the extremes. we see performance on models peaks usually between 70% and 90% in numerical values but drops in accuracy when using 100% in the prompt. across all four datasets, with seven templates each, 21 out of the 28 templates which use 100% numerical values had a lower probabilityon-gold than templates which used 90% numerical values (see appendix). additionally, at the other extreme, when 0% is used in templates, there is also a drastic drop to accuracy. ## why does 100% certainty hurt? we hypothesize this drop in accuracy might be the result of the use of hyperbolic or exaggerated language in the training set, in which numbers are used non-literally. when someone says "i'm 100% certain there was pie left", they don't mean they are 100% certain is there pie -but rather are emphasizing their strong belief there was pie. again, we turn to the pile and qualitatively analyze over 500 examples of use cases of "100%". we remove other uses of "100%" such as in code (e.g., "width:100%") and sample 50 instances from both question and answer posts. we find, surprisingly, that 100% is in fact often used to express uncertainty. of our 100 samples, 44 instances are used with negation (e.g., "never be 100% accurate" or "is not always 100% reliable") and half are expressions of uncertainty (e.g., "i'm not 100% sure" or "i don't 100% follow"). in questions, the rate of expressions of uncertainty was nearly double that of answers (14 vs 8). in contrast, only 3 instances in our sample indicate 100% certainty (e.g., "i'm 100% sure that"). we hypothesize that both the use of negation with "100%" and the general lack of use of "100%" with expressions of certainty contribute to the lowered performance of these prompts. another confounder in how model's interpret numerical values could be the distribution of numerical frequencies in typical model training data. querying the pile, we find that there are drastic imbalances in the use of percentages in training datasets. there are significant spikes in frequency at the upper extremes (50%, 95% and 100%) (figure 5). this might be happening as some values are more common to express (e.g., "100% organic") or found more often in code. humans might also naturally exaggerate values or use colloquial terms to describe confidence. the presence of scientific language like "95% confidence interval" could be another possible source of imbalance. although spoken natural language also includes rounded percentages, the use of only textual data might be further exacerbating this bias. ## related work while scholars have studied model uncertainty, prior work has focused on more accurately extracting model confidence @xcite @xcite , measuring @xcite @xcite and improving model calibration (between model confidence and accuracy) @xcite @xcite @xcite . however, the community has found mixed results on the calibration of neural model @xcite ; for example, @xcite shows that pretrained transformers are relatively well-calibrated meanwhile wang et al. ( 2020) in addition, semanticists and computational linguists have long studied speaker commitment factors such as factivity @xcite and projection @xcite , and more recent work include corpora like the commitmentbank (de marneffe et al., 2019) which offers naturally occurring examples, as well as new experimental paradigms to investigate speaker commitment @xcite . a wide variety of scholars have examined computa-tional issues in factuality, veridicality, and commitment @xcite @xcite @xcite as well as bias @xcite and specific devices like hedges @xcite , and modality @xcite . we see our work as a bridge between these two areas of speaker commitment and natural language generation, by telling us how models interpret speaker commitment through expressions of certainty and uncertainty. ## discussion and conclusion here, we discuss a number of recommendations and future work for the greater nlp community. can models generate expressions of uncertainty? having studied the impact of uncertainty as part of the prompt, a natural follow-up question is: how does model performance change when models learn to emit their own expressions of uncertainty? in preliminary experiments, we find it is challenging to calibrate models to generate epistemic markers. the same methods which work for numerical calibration do not transfer well to verbal calibration @xcite . however, we find some evidence that expressions of uncertainty are slightly better calibrated when compared with expressions of certainty. see a.1. navigating idiomatic language humans use language that contains expressions of certainty when they are, in fact, not certain, and models appear to be mimicking this behavior. however, our qa models are still unable to recognize the meaning of those markers in our qa setup, raising concerns for how they would respond in downstream applications. future work must seek to enable lms to accurately interpret when expressions are meant idiomatically versus literally.