ICML 2025

Statistical Hypothesis Testing for Auditing Robustness in Language Models
Paulius Rauba 1 Qiyao Wei 1 Mihaela van der Schaar 1
Abstract
Consider the problem of testing whether the out-
puts of a large language model (LLM) system
change under an arbitrary intervention, such as
an input perturbation or changing the model vari-
ant. We cannot simply compare two LLM out-
puts since they might differ due to the stochastic
nature of the system, nor can we compare the
entire output distribution due to computational
intractability. While existing methods for analyz-
ing text-based outputs exist, they focus on fun-
damentally different problems, such as measur-
ing bias or fairness. To this end, we introduce
distribution-based perturbation analysis, a frame-
work that reformulates LLM perturbation analysis
as a frequentist hypothesis testing problem. We
construct empirical null and alternative output dis-
tributions within a low-dimensional semantic sim-
ilarity space via Monte Carlo sampling, enabling
tractable inference without restrictive distribu-
tional assumptions. The framework is (i) model-
agnostic, (ii) supports the evaluation of arbitrary
input perturbations on any black-box LLM, (iii)
yields interpretable p-values; (iv) supports multi-
ple perturbations via controlled error rates; and (v)
provides scalar effect sizes. We demonstrate the
usefulness of the framework across multiple case
studies, showing how we can quantify response
changes, measure true/false positive rates, and
evaluate alignment with reference models. Above
all, we see this as a reliable frequentist hypothesis
testing framework for LLM auditing.
1. Introduction
Large language models (LLMs) generate outputs condi-
tioned on textual inputs by iteratively sampling from a dis-
tribution of tokens. Therefore, the outputs of LLMs exhibit
1University of Cambridge. Correspondence to: Paulius Rauba
<pr501@cam.ac.uk>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada....