The Impact of Inference Acceleration on Bias of LLM s (2025)

Abstract: Last few years have seen unprecedented advances in capabilities of Large Language Models (LLMs). These advancements promise to benefit a vast array of application domains. However, due to their immense size, performing inference with LLMs is both costly and slow. Consequently, a plethora of recent work has proposed strategies to enhance inference efficiency, e.g., quantization, pruning, and caching. These acceleration strategies reduce the inference cost and latency, often by several factors, while maintaining much of the predictive performance measured via common benchmarks. In this work, we explore another critical aspect of LLM performance: demographic bias in model generations due to inference acceleration optimizations. Using a wide range of metrics, we probe bias in model outputs from a number of angles. Analysis of outputs before and after inference acceleration shows significant change in bias. Worryingly, these bias effects are complex and unpredictable. A combination of an acceleration strategy and bias type may show little bias change in one model but may lead to a large effect in another. Our results highlight a need for in-depth and case-by-case evaluation of model bias after it has been modified to accelerate inference.This paper contains prompts and outputs which may be deemed offensive.

## introduction modern-day llms like llama and gpt-4 show remarkable language generation capabilities, leading to a surge in their popularity and adoption @xcite @xcite . however, owing to their immense size, deploying these models can be challenging, or even infeasible for consumer-grade devices. a flurry of research has proposed acceleration strategies such as quantization and pruning to enable efficient inference @xcite . the objective of these strategies is typically to reduce the model size while maintaining predictive performance. over time, these strategies have become increasingly prevalent with integration into popular libraries like huggingface @xcite and vllm @xcite . while these inference acceleration strategies aim to preserve predictive performance, they may inadvertently lead to some side-effects @xcite . for example, compression techniques might significantly reduce model trustworthiness @xcite . on the other hand, smaller models have been found to mitigate privacy risks and reduce egocentric tendencies in the generated text @xcite @xcite . in this paper, we explore how demographic bias in the model output changes after the implementation of inference acceleration strategies. specifically, we aim to answer the following questions: rq1 are certain bias types more prone to manifesting because of inference acceleration? rq2 are certain inference acceleration strategies more prone to bias? rq3 does the bias impact of a strategy remain consistent across various models? our assessment includes five commonly used inference acceleration techniques and three widely used llms. noting the multidimensional nature of bias @xcite , we test the models with six different bias metrics. our results show that inference acceleration strategies can have significant impacts on bias. ## related work most evaluations of inference acceleration strategies focus on application-agnostic metrics like perplexity or predictive performance-driven tasks like mmlu @xcite @xcite . however, recent work has shown that model compression can result in degradation of model performance in areas beyond predictive performance @xcite . the effect of model size on trust criteria. recent work has started exploring the impact of model size on trust related criteria. for example, @xcite find that larger models tend to overly agree with user views. @xcite show that smaller models can reduce privacy risks. huang et al. @xcite find that smaller models are more vulnerable to backdoor attacks. @xcite find that larger models are more susceptible to manipulation through malicious demonstrations. jaiswal et al. @xcite offer a fine-grained benchmark for evaluating the performance of compressed llms on more intricate, knowledge-intensive tasks such as reasoning, summarization, and in-context retrieval. by measuring perplexity, they show that pruned models suffer from performance degradation, whereas quantized models tend to perform better. @xcite find that knowledge distillation causes a monotonic reduction in toxicity in gpt-2, though it shows only small improvements in reducing bias on counterfactual embedding-based datasets. these analyses differ from our paper in one of the following ways: (i) they are limited to less recent, pre-trained models, which may not adequately represent the complexities of modern llms with significantly more parameters; (ii) they target trustworthiness desiderata beyond bias, e.g., backdoor attacks. effect of inference acceleration on trustworthiness. @xcite measure the impact of quantization and knowledge distillation on llms, and show that longer pretraining and larger models correlate with higher demographic bias, while quantization appears to have a regularizing effect. the bias metrics they consider focus on embeddings or token output probabilities, while we consider a larger range of metrics that focus on properties of generated texts. @xcite , in a follow-up to @xcite , provide a broader assessment of trustworthiness under compression strategies like quantization and pruning, including adversarial settings. however, their study relies on a single metric to evaluate stereotype bias, which may not capture the broader complexity of bias. we, on the other hand, aim to provide a comprehensive evaluation of bias across multiple dimensions to better understand the impact of inference acceleration strategies. finally, while these previous benchmarks show largely uniform and predictable effects of inference acceleration on bias, by leveraging a richer set of metrics, our analysis shows a much more nuanced picture and a need for case-bycase evaluation. ## measuring bias in llm outputs ml bias can stem from different causes @xcite , can manifest in various manners @xcite , and can cause different types of harms @xcite . while a detailed examination can be found in @xcite , bias in llms is often categorized into the following meta-groups: we leave out embedding-based metrics from our analysis since (i) the more typical use-case of modern, instruction-tuned llms like llama and gpt-4 is prompt-tuning or fine-tuning rather than adapting the models using embeddings and (ii) embedding bias is not guaranteed to lead to bias in the text generations. while we initially considered classification-based bias metrics (e.g., those in dhamala et al.) which consider difference in measures like toxicity and sentiment on common datasets like wikipedia. a preliminary analysis showed very little overall toxicity in model outputs, most likely due to heavy alignment on these datasets. for this reason, we did not further consider these metrics. with these considerations in mind, the final set of metrics we consider is as follows. we add further information, e.g., the number of inputs and license types, in appendix a. crowspairs @xcite is a dataset of crowd-sourced sentence pairs designed to evaluate stereotypes related to race, gender, sexual orientation, religion, age, nationality, disability, physical appearance, and socioeconomic status. each pair consists of one sentence that demonstrates a stereotype and the other that demonstrates the opposite of the stereotype. given a pair (s more , s less ) where s more is presumed to be more stereotypical, the metric measures i[p(s more ) > p(s less )] and averages this quantity over all pairs. i denotes the indicator function. the resulting score is in the range [0, 1]. globalopinionqa @xcite uses multiple-choice questions to assess the opinions stated by a model relative to aggregated population opinions from different countries. the goal is to identify model bias in representing diverse viewpoints. we follow the same measurement procedure as durmus et al. with one exception: we use the wasserstein distance as our divergence metric. durmus et al. use 1-jensen-shannon distance, which can become highly skewed when the distributions have very little or no overlap. in contrast, the wasserstein distance is more sensitive to the geometry of the probability distributions @xcite . we compute the wasserstein distance using the implementation provided by the python scipy library @xcite . the final bias value is then the gini coefficient of the wasserstein distance for each country. the metric lies in the range [0, 1]. the dataset does not provide responses from all countries to all questions, making it difficult to analyze overall value tendencies consistently. to address this, we exclude countries that do not have responses to at least 50 questions from our analysis. worldbench @xcite evaluates performance disparities in an llm's ability to recall facts (e.g., population, gdp, and capital) about different countries. @xcite structure the questions to elicit a single numerical answer. the dataset encompasses 11 statistics for about 200 countries. to compare numerical answers, we calculate the absolute relative error between the model's answer and the ground truth, and average the error over all questions to obtain a single score per model. this score lies in the range [0, 1]. dt-stereotyping. decodingtrust @xcite is a framework for evaluating the trustworthiness of llms across eight dimensions, one of which is stereotype bias. the dataset consists of custom-made statements (from now on referred to as dt-stereotyping) designed to provoke unsafe (e.g., biased, toxic) responses. following @xcite , we ask the model to either agree or disagree with these statements. to measure stereotyping behavior, we compute the fraction of times the model agrees with the stereotypical statements nagree @xmath0, 1]. the original dataset is evaluated in three evaluation scenarios that instruct the model to behave as a helpful assistant (benign setting), in disregard of its content policy (untargeted), and with targeted bias towards the target group (targeted). we use the untargeted setting to evaluate the resilience of model alignment, without explicitly targeting any group adversarially. since model outputs can be generated with different temperatures (t ), we use two variants of this evaluation. we refer to these as dt-stereotyping (greedy) with @xmath1. discrimeval @xcite consists of 70 hypothetical decision making scenarios, e.g., approving a loan. for each scenario, the model is prompted to make a binary yes/no decision about a person described in terms of age, gender and race (9, 3, and 5 choices, respectively). a "yes" decision is always advantageous. following tamkin et al., we append "my answer would be" to the prompt to steer the generations towards producing binary decisions and record the model's softmax probability of "yes" or "no" being generated as the first token. for a scenario q i , and a set of demographic groups g (9 × 3 × 5 = 135 intersectional groups in this case), we repeatedly reformulate q i , substituting the demographic information for all groups @xmath2, and measure the difference between the highest and lowest probability of "yes" for all groups @xmath3. specifically, the bias score is: discrimevalgen. the original design of dis-crimeval evaluates bias by analyzing the probability of the first token being "yes" or "no", reducing the model's output to a simplified binary decision. this approach has the following limitations: with the aim of overcoming these issues, we propose a new dataset discrimevalgen. while discrimeval asks the same question separately for each demographic group g, discrimevalgen forces the model to make a single choice. specifically, we (i) present the question to the model and describe that the candidates are persons from different groups, e.g., a man, a woman, a non-binary person; (ii) describe that the benefit (e.g., a work contract) can be granted to only a single person; and (iii) ask the model to make its choice. let @xmath4, @xmath5, and n g be the number of times a group is selected by the model with @xmath6, then the bias metric is: to avoid having a very long list of choices (135 intersectional groups in the original dataset), we limit the groups to those based on gender, that is, @xmath7, non-binary, woman}. we encountered several cases where the model refuses to select a single person, or selects several persons. we ignore such cases from the bias computation. if for a particular model/acceleration strategy combination, we have more than 80% such cases, we drop this combination from our results. just like dt-stereotyping, we consider two versions: discrimevalgen (greedy) with @xmath8. ## experimental setup models and infrastructure. we analyze three different models: llama-2 @xcite , llama-3.1 @xcite , and mistral-0.3 @xcite . we consider the smallest size variant of each model: llama-2-7b, llama-3.1-8b, and mistral-7b-v0.3 (license information in appendix a). these models were selected due to their recency, widespread use, and compatibility with our resource constraints, which included a single node equipped with four nvidia a100 gpus that was shared among several research teams. our evaluation focuses on the chat versions of these models, which are specifically designed to align with human values and preferences. we used the github copilot ide plugin to assist with coding. inference acceleration strategies. we consider inference time acceleration techniques that do not require re-training. this choice allows us to evaluate models in a real-world scenario where users download pre-trained models and apply them to their tasks without further data-or compute-intensive modifications. we focus on strategies that aim to speed up inference by approximating the outputs of the base model, and where the approximations results in measurable changes in the model output. this criterion excludes strategies like speculative decoding @xcite where the output of the base and inference accelerated models are often the same. specifically, we consider the following strategies: quantization. we consider the following variants: pruning removes a subset of model weights to reduce the high computational cost of llms while aiming to preserve performance. traditional pruning methods require retraining @xcite . more recent approaches prune weights posttraining in iterative weight-update processes, e.g., sparsegpt @xcite . we use the wanda method by @xcite which uses a pruning metric based on both weight magnitudes and input activation norms. the sparse model obtained after pruning is directly usable without further fine-tuning. we consider two variants: (i) unstructured pruning (wu) with a sparsity ratio of 50%, eliminating 50% of the weights connected to each output; and (ii) structured pruning (ws) which induces a structured n:m sparsity, where at most n out of every m contiguous weights are allowed to be non-zero, allowing the computation to leverage matrix-based gpu optimizations. we use a 2 : 4 compression rate. prior work has shown that pruned models can preserve comparable performance levels even at high compression rates @xcite @xcite , e.g., 2 : 4 considered here. parameters. as described in section 3, most bias metrics are designed such that they only support greedy decoding, resulting in deterministic outputs. only dt-stereotyping and discrimevalgen support stochastic decoding in addition to greedy decoding. when using stochastic decoding, we sample the output 5 times and report the average bias. the models can be used with and without the developer-prescribed instruction templates (with special tokens specifying the start and end of instructions). past investigations have shown that instruction and answer formats can have an unpredictable impact on the model performance @xcite . however, the impact of using or not using the instruction template on model bias is less well understood. we thus study both configurations. the main paper includes the results without the template, while results with instructions templates are shown in appendix c. ## results rq1: are certain bias types more prone to manifesting because of inference acceleration? inference acceleration strategies have disparate impacts on different types of bias metrics. specifically, we see: no significant impact on log-likelihood of stereotypical sentences as measured by the crowspairs dataset. most acceleration strategies show little to no significant effect on the loglikelihood of counterfactual sentences. the results are largely in line with @xcite who also show a relatively mild effect of quantization on bias measured via crowspairs, although they consider the previous generation of lms like bert and roberta. we provide a de-tailed breakdown of results per bias type in table b.1. structured pruning leads to small improvements in bias scores for certain bias types, such as "age". however, the average improvements over the base model across all bias types are modest, generally less than 10%. subtle shifts in values and opinions in the glob-alopinionqa task. we observe little effect of inference strategies on the values and opinions represented by the models (see pruning influences models' ability to recall country-specific facts. in the worldbench dataset, 8-bit and kv cache quantization showed improvements in mean average error, whereas pruning strategies and awq quantization increased bias scores. we report detailed disparity scores across income groups and regions in table b.5. pruning leads to higher disparities across regions and income groups in 8/12 cases. in contrast, other inference acceleration methods had non-uniform or minimal influence on models' factual recall performance across countries. more pronounced shifts in model's agreement with stereotypes. the dt-stereotyping task reveals significant changes in agreement, disagreement, and no-response rates across strategies. pruning strategies tend to reduce disagreement with stereotypes, leading to higher agreement or noresponse rates (table b .2). quantization showed minimal effects or slight improvements for llama models but increased the number of agreements with stereotypes for mistral. in general, inference acceleration significantly changes models' agreement with stereotypes. varying bias patterns in allocation-based decision-making scenarios. in discrimeval, structured pruning consistently achieved the lowest bias score across models, followed closely by kv cache quantization. on the other hand, awq quantization resulted in a notable increase in bias. in the discrimevalgen dataset, which measures bias in relative decision making scenarios and longer text generations, we observe more significant shifts in resource allocation based on gender, with awq leading to increased bias across models and sampling strategies. a detailed breakdown of decisions per model and tested attributes in table b.4 shows that inference acceleration strategies influence the models' tendency to give no response or refuse an answer. both mistral and llama-3.1 display a tendency to favor the non-binary person, though this effect is reduced when pruning strategies are applied. rq2: are certain inference acceleration strategies more prone to bias? kv cache quantization and structured wanda pruning showed promising trends across datasets and models, frequently showing minimal changes or slight improvements in bias scores. however, structured pruning exhibited certain drawbacks. when examining parse rates and no-response rates, we found that this strategy can cause the model to fail to perform the task, follow instructions, or produce nonsensical, repetitive outputs. overall, our results suggest quantizing weights can have more drastic, unpredictable impacts on bias compared to kv cache quantization. rq3: does the bias impact of a strategy remain consistent across models? the effects of inference acceleration strategies on stereotype agreeability vary markedly across models. a detailed breakdown of agreement, disagreement, and no-response rates for nucleus sampling in table b .2 illustrates how the models' baselines already differ. llama models most frequently provide no response, while mistral shows a higher rate of both agreement and disagreement. notably, the impact of inference acceleration strategies is much more pronounced for mistral, with agreements increasing by over 75% relative to the base model for both awq and unstructured pruning. additionally, different models display varying abilities to follow instructions and perform tasks. for example, in the discrimevalgen dataset (table b.4), llama-2 mostly provides no response. mistral tends to give answers more frequently in its base form but shows a reduced tendency to respond under quantization and even more so under pruning strategies. our findings demonstrate that the impact of a single acceleration strategy does not remain consistent across different models. the baseline performance of each model often shows divergent trends, and these disparities are further amplified by inference acceleration strategies. this highlights the need for a model-by-model evaluation when assessing a strategy's impact on bias. comparing 4-bit and 8-bit compression. while lower-bit compression can enhance efficiency, it often risks degrading model performance @xcite . @xcite explored compression down to 3-bit quantized models, highlighting 4-bit as a setting that balances efficiency and fairness. in our experiments, we evaluate both 4-bit and 8-bit quantization for weights and kv-cache. for 8-bit weight quantization, bias scores generally remain close to those of the base models, with small improvements observed in some cases, except for a slight increase in bias on the discrimeval dataset. similarly, 4-bit weight quantization yields comparable results, though it leads to noticeable increases in bias scores for dt-stereotyping and discrimevalgen, particularly for the mistral model. kv-cache quantization consistently shows minimal impact on bias across datasets, with 8-bit compression having little to no noticeable effect on bias, while 4-bit demonstrates small improvements in some model/dataset combinations. combining inference acceleration strategies. we also explore the impact of multiple inference acceleration strategies on model bias. in ## conclusion & future work in this study, we investigated the impact of inference acceleration strategies on bias in large language models (llms). while these strategies are primarily designed to improve computational efficiency without compromising performance, our findings reveal that they can have unintended and complex consequences on model bias. kv cache quantization proved stable with minimal impact on bias scores across datasets, whereas awq quantization negatively affected bias. other strategies had less consistent effects, with some reducing bias in one model while leading to undesirable effects in another. this variability highlights that the effects of inference acceleration strategies are not universally predictable, reinforcing the need for case-by-case assessments to understand how model-specific architectures interact with these optimizations. the impact of these strategies extends beyond bias. for instance, structured wanda pruning appeared effective in reducing bias but led to concerns about nonsensical and incoherent texts. our results highlight the importance of using diverse benchmarks and multiple metrics across a variety of tasks to fully capture the trade-offs of these strategies, particularly as the nature of the task itself (e.g., generation vs probability-based) can surface different kinds of biases. bias mitigation is an important direction for future research. while some strategies, such as pruning methods like wanda, may appear to improve bias, these effects are often incidental rather than the result of deliberate design. to achieve consistent and reliable bias reduction, it is crucial to consider, already during model training, that users may later apply inference acceleration strategies. incorporating these strategies into the model alignment process can help proactively address biases. it may also be useful to explore approaches that integrate explicit bias mitigation objectives, such as fairness-aware training methods or bias-sensitive hyperparameter optimization @xcite @xcite . additionally, exploring the combined effects of multiple strategies, such as hybrid approaches that mix pruning with quantization, could provide valuable insights into how to better balance efficiency, performance, and bias. our analysis focused on demographic bias. however, extending this work to other forms of bias (such as political bias) remains an important direction for future work.