Efficiently Computing Susceptibility to Context in Language Models (2024)

Abstract: One strength of modern language models is their ability to incorporate information from a user-input context when answering queries. However, they are not equally sensitive to the subtle changes to that context.To quantify this, Du et al. (2024) gives an information-theoretic metric to measure such sensitivity. Their metric, susceptibility, is defined as the degree to which contexts can influence a model’s response to a query at a distributional level.However, exactly computing susceptibility is difficult and, thus, Du et al. (2024) falls back on a Monte Carlo approximation.Due to the large number of samples required, the Monte Carlo approximation is inefficient in practice. As a faster alternative, we propose Fisher susceptibility, an efficient method to estimate the susceptibility based on Fisher information.Empirically, we validate that Fisher susceptibility is comparable to Monte Carlo estimated susceptibility across a diverse set of query domains despite its being 70\times faster.Exploiting the improved efficiency, we apply Fisher susceptibility to analyze factors affecting the susceptibility of language models.We observe that larger models are as susceptible as smaller ones.

## introduction much of current language models' (lm) capabilities and success are due to their responsiveness to different user-input contexts, e.g., prompts, and ability to integrate those contexts with prior knowledge @xcite . given this ability, it is natural for us to wonder how easily a model's prior belief can be changed by an input context. for example, a language model might correctly complete the prompt here comes the sun is performed by with the beatles, but wrongly when this prompt is prepended with a random context such as falafel wraps make ostriches burp.; such a context might distract the model away from its original behavior. recently, @xcite studied how easily input contexts can skew language models to give answers to queries that are different than the answers stored in their prior knowledge from an information-theoretic point of view. they propose a metric, termed susceptibility, to quantify the discrepancy between a model's prior belief and updated belief after seeing the input contexts as the mutual information between input contexts and answers from a specific language model. this metric makes use of the language model's distribution over answers rather than relying on an approximate argmax, as is common in previously proposed metrics @xcite . susceptibility, as an information-theoretic metric, captures changes in model behavior that might not necessarily surface while decoding an answer. computing susceptibility requires, in principle, summing over all possible answers, a countably infinite set. in the general case, the authors know of no efficient algorithm to perform such a summation; @xcite propose a monte carlo approximation. such a sampling approximation, however, is computationally expensive: to compute the susceptibility for one query, one needs to execute a forward pass of the neural language model for each context in the sample set, making it inefficient to be applied to large-scale datasets. for example, @xcite consider sampling 600 contexts for every individual query, and thus require 600 forward passes. in light of the computation required for the monte carlo approximation to susceptibility, we propose a more efficient approximation based on fisher information that does not require sampling to estimate the susceptibility; we term this approximation fisher susceptibility. we conduct experiments across queries from 122 relation domains (e.g., alumniof or capitalof ) in the yago knowledge graph @xcite with a variety of model sizes (e.g., from 70 million to 8 billion parameters), language model families (e.g., pythia @xcite , llama @xcite , and gpt-2 @xcite ), and fine-tuning schemes (e.g., instruction-tuning). first, to empirically validate fisher susceptibility, we show that it is tightly correlated with @xcite monte carlo approximation to susceptibility across domains while also benchmarking its speed. in these experiments, we find that compared to the monte carlo estimation with a sample size of 256, fisher susceptibility exhibits a 70× improvement in runtime. then, we use the increased efficiency of fisher susceptibility to investigate further initial questions like the one posed in the first paragraph: how susceptible are language models across different sizes and model families, and what factors influence a model's fisher susceptibility for a query? we find that larger models are not less susceptible than smaller ones and that instruction-tuning does not help reduce susceptibility. we further find that queries about well-known entities are equally susceptible as less frequent ones under fisher susceptibility, which contrasts with the finding from @xcite that susceptibility is negatively related with expected familiarity when contexts might be relevant. to the extent that this is not an approximation error, this finding suggests that, regardless of how much prior knowledge it has about a query, a language model is still susceptible to contexts and can integrate new information from them. ## susceptibility to context in lms language models are capable of answering a wide range of queries formulated in natural language, including code auto-completion, text generation, and factual question answering @xcite @xcite . when responding to a query, language models need to synthesize the prior knowledge they learned during pretraining with the new information provided in the input context @xcite @xcite . for example, in the knowledge conflict setting proposed by @xcite given the query what's the capital of ireland? and context the capital of ireland is rome., the model must decide between whether to agree with its prior knowledge (dublin) or the context (rome). how easily large language models are affected by contexts is a well-studied problem @xcite @xcite . many studies, including longpre et al. (2021); chen et al. (2022); xie et al. (2024), measure how easily context influences a model by computing the memorization ratio: the proportion of examples for which the model maintains from before the context was introduced. however, memorization rate may not fully capture the strength of this influence. for example, adding a context could take a model's probability of answering a token from 95% to 51%, but the 1-best answer would remain the same, and the context's influence would not be detectable by memorization rate. to solve this issue, du et al. (2024) provide a metric that measures the influence of context on a model's answer to a query using a more fine-grained metric based on the model's full answer distribution, the susceptibility, which we describe in detail in §4.3. to compute susceptibility, du et al. (2024) proposes a monte carlo approximation. however, it requires running one forward pass, which is computationally expensive, per sampled answer; this limits the scale of analysis. thus, we aim to find a method that is more efficient, general, and interpretable. ## measuring susceptibility let σ be an alphabet, i.e., a finite, non-empty set. a language model p m over an alphabet σ is a distribution over σ * , the set of all strings with tokens drawn from σ. we denote a query by @xmath0, and the answer to the query by @xmath1. when querying a language model, a context @xmath2. in practice, querying a language model is the process of generating an answer a from the distribution p m (• | c ⊕ q). ## fisher susceptibility to alleviate the cost of the monte carlo approximation discussed in §3.2, we derive an efficient approximation based on fisher information. ## experiments experimentally, we first aim to show that fisher susceptibility correlates well with monte carlo susceptibility. then, we investigate what factors influence higher susceptibility, across models, queries, and entities, accelerated by the use of fisher susceptibility. finally, we apply fisher susceptibility to evaluate language models' susceptibility to contexts. ## conclusion to efficiently measure a language model's susceptibility, we have proposed fisher susceptibility, which uses the fisher information of a language model with regard to its input to measure the scale of distributional changes as the input varies. through experiments, we find strong correlation between a language model's monte carlo susceptibility and fisher susceptibility, which we take to validate our approximation. compared to methods which requires many context samples and language model forward passes, our method is significantly faster. our study contributes to the exploration for interpretable and efficient evaluation metrics for language models.