An Auditing Test to Detect Behavioral Shift in Language Models
Authors: Leo Richter
Xuanli He
Pasquale Minervini
Matt Kusner
Venue: ICLR 2025 Poster (2025)
Keywords: AI alignment
model auditing
model evaluations
red teaming
sequential hypothesis testing
Primary area: foundation or frontier models, including LLMs
TL;DR: We present a method for efficiently detecting behavioral changes in language models through output comparisons.

Abstract: As language models (LMs) approach human-level performance, a comprehensive understanding of their behavior becomes crucial. 
This includes evaluating capabilities, biases, task performance, and alignment with societal values. Extensive initial evaluations, including red teaming and diverse benchmarking, can establish a model’s behavioral profile. However, subsequent fine-tuning or deployment modifications may alter these behaviors in unintended ways. We present an efficient statistical test to tackle Behavioral Shift Auditing (BSA) in LMs, which we define as detecting distribution shifts in qualitative properties of the output distributions of LMs. Our test compares model generations from a baseline model to those of the model under scrutiny and provides theoretical guarantees for change detection while controlling false positives. The test features a configurable tolerance parameter that adjusts sensitivity to behavioral changes for different use cases. We evaluate our approach using two case studies: monitoring changes in (a) toxicity and (b) translation performance. We find that the test is able to detect meaningful changes in behavior distributions using just hundreds of examples.

Published as a conference paper at ICLR 2025
AN AUDITING TEST TO DETECT BEHAVIORAL SHIFT IN LANGUAGE MODELS
Leo Richter' Xuanli He'
Pasquale Minervini
Matt J. Kusner
UCL Centre for Artificial Intelligence. University College London, United Kingdom School of Informatics, University of Edinburgh, United Kingdom
3 MIIimllAA, United Kingdom
*Polytechnique Montréal, Canada
MMila - Quebec AI Institute, Canada
ucablr @ucl_ ac. uk, matt. kusmerêmi la quebec
ABSTRACT
As language models (LMs) approach human- performance, a comprehensive understanding of their behavior becomes crucial to avoid potential harms. While extensive initial evaluations, including red teaming and diverse benchmarking, can establish behavioral profile, subsequent fine-tuning deployment modifications may alter these model behaviors in unintended ways. We study the behavioral shift auditing problem, where the goal is detect unintended changes in model behavior. We formalize this problem as sequential hypothesis test. We apply and extend recent testing method include configurable tolerance parameter that adjusts sensitivity to behavioral changes for different use cases. The test is guaranteed to consistent and has tight control over the Type error rate. We evaluate our approach using two case studies: monitoring model changes in (a) toxicity and (b) translation performance. We find that the test is able to detect distribution changes in model behavior using hundreds of prompts.
INTRODUCTION
Language models (LMs) can now achieve human-level performance in a wide range of tasks, includ- ing text summarization, machine translation, coding, and even acting as AI scientists: generating hypotheses and designing experiments (Achiam et al., 2023; Katz al., 2024; Lu et al., 2024; Zhang et al., 2024). As capabilities continue to scale, evaluating LM behaviors becomes increasingly important and increasingly difficult (Hendrycks et al., 2021; Ngo et al., 2022: Wolf et al., 2023). Large-scale evaluations- such comprehensive behavior and capability assessments (Wang et al., 2023a) and red-tcaming exercises (Perez et al., 2022a) are widely used to verify that language models (LMs) behave safely and as expected. However, these evaluations tend to be expensive and are not well-suited for continuous monitoring, especially when models are updated or fine-tuned with new data. This is problematic because even seemingly benign or narrow modifications can inadvertently lead to undesirable changes in model behavior (Qi et al., 2023; Betley et al., 2025). This raises the question: How can we quickly and cheaply detect unwanted changes in LM behavior?
Consider two hypothetical settings where this question might be asked: (1) Internal Audir: A company develops language model that has passed rigorous safety and performance evaluations. After deploying the model, they continue to fine-tune it to improve its performance on certain tasks. The development team wants to stay informed about any drastic changes this might induce in the model's
shifts in areas unrelated to the intended updates. How can the team rapidly detect meaningful changes model behavior throughout the development cycle? (2) External Audit: A regulatory body certifies language model for public deployment after extensive safety evaluation. However, they are concerned that the deployed model's behavior may change over time due updates or intentional modifications. Since they only have access to the model through an API and cannot inspect its internal parameters, they require a mechanism to regularly check that the model's ss behavior remains consistent with the certified version. How can the regulator regularly check

---

Published as a conference paper at ICLR 2025
Phase I: Establish Baseline Behavior
Phase II: Test for Behavior Change
prinpe
ovaltane bchavior
peotggle by
Iehasiur
model clarued!
model
medei API
New soor2
API
peomgt
Figure 1: Overview of behavior shift auditing framework.
the deployed model's behavior is the same as the previously certified one? We call the problem of detecting changes in LM behavior distributions over time behavioral shift auditing problems.
In this paper, we formalize the problem of behavioral shift auditing in language models and propose statistical test that monitors changes in model behavior using only black-box access (c.g., via API calls). Our goal is to develop a sample-efficient method that guarantees detection of behavioral shifts while tightly controlling the rate of false positives. Further, it should provide the user with tolerance parameter that allows a behavior distribution to change by some amount without triggering detection. This parameter controls the strictness of the auditing test in some settings (e.g.. example (I)), a more liberal might be appropriate, while in other cases (e.g., example (2)) one might require more conservative our approach is to frame behavior shift auditing as hypothesis testing problem over the model's s behavior distribution. This framing makes our method applicable behaviors-sur
even want disallow any change all. The key insight behind
wide range of measurable dangerous capabilities (Phuong et al., 2024), mathematical reasoning (Mishra et al., 2022a), and biases (Wang al., 2023a; Kotek et al., 2023). To this end, we leverage and extend recent advances in testing by betting (Pandeva al., 2024). Under mild assumptions, test provably detects any change given enough samples, while ensuring non-asymptotic control over false positives. We demonstrate We find that we can detect changed LM behaviors using hundreds of prompts. We release our code here: httpps://giten
sequential
test detecting shifts in toxicity and translation performance.
1mm audit wam-aaadddiiinteeteee.t inn-ttest
2 RELATED WORK
LM behavior functions. Early evaluations of NL.P models relied on curated datasets for detecting biases toxicity (Bolukbasi et al., 2016); larger collections of data were constructed c.g. through web scraping (Zhao et al., 2018; Zampieri et al., 2019; Nangia et al., 2020; Rosenthal et al., 2021) and, more recently, by leveraging LLMs themselves generate data (Zhang al., 2022: Perez et al., 2023). Meanwhile, early work on behavior functions focused on measuring bias, toxicity, and hallucinations (Vidgen et al., 2020; Achiam et al., 2023; Anil et al., 2023; Chern et al., 2023; Varshney et al., 2023; Llama-team, 2024). Since the rise of LMs with human-level performance, the set of behavior functions has exploded (Zou al., 2023b). It has become more nuanced, including complex characteristics such as power-seeking behavior (Park et al., 2023; Sharma et al.. 2023), situational awareness (Zou et al., 2023a), and deception (Hagendorff, 2024). However, even with access to massive datasets and carefully constructed behavior functions it can be difficult to discover these behaviors from static inputs (Kalin al., 2020). To address this, Perez al. (2022a) introduced to
the notion of red-teaming for LM alignment. This allows prompts to be expose failure cases, which arise in many state-of-the-ar models (Chao et al., 2023).
Model change identification. For the case where one wishes to identify any change in model behavior (i.e., 0) there are multiple other techniques that can be used. The first set uses ideas from formal verification to ensure that the predictions from a model are guaranteed to come from a specific model (Ghodsi et al., 2017; Dong et al., 2021; Fan et al., 2023; Weng et al., 2023). In general, however, these methods are computationally intensive and do not scale to state-of ofthe-af LMs. A second, more efficient idea is to watermark the model (Zhu et al., 2018; Amrit & Singh, 2022: He et al., 2022a;b; Kirchenbauer et al., 2023; Kuditipudi et al., 2023; Yoo et al., 2023). The idea is to embed signals into model generations that can be detected algorithmically. However, watermarks
2

---

Published as a conference paper at ICLR 2025
often inserted by the model owner (Kirchenbauer et al., 2023; Kuditipudi al., 2023), allowing them (or an actor that has compromised the model) to insert it into any model that is being audited. This precludes its use for many external auditing settings. For internal auditing, watermark may break under a small model change that is acceptable. Our work is also related to work on concept drift (Bayram et al., 2022) and prompt stability (Li et al., 2024). In principle our test can be used to detect concept and generation changes, however the focus of these works is on model performance and generation similarity, as opposed behavior change.
Sequential hypothesis testing. Sequential hypothesis testing allows one to analyze data without fixing the sample size in advance (Wald, 1945), offering the potential for greater sample efficiency when significant effects exist (Arrow et al., 1949). However, naive repeated testing can increase the Type I error rate (i.e., false positives) as the number of tests grows (Jennison & Turnbull, 1999). To prevent this inflation of false positives, various methods have been developed, including the recent resting by betting framework (Robbins, 1970; Ramdas et al., 2023), which preserves statistical efficiency while tightly controlling the Type I eтror rate. Within this framework, a method called Deep Anytime- Valid Testing (DAVT) (Pandeva et al., 2024) designs powerful sequential non-parametric tests by integrating deep learning models into the testing by betting framework. They demonstrate, on variety of tasks, including two-sample testing, competitive performance compared to other oate--ofttheaat non-parametric sequential tests, such as the E-C2ST (Lhéritier & Cazals, 2018) and Seq-IT (Podkopaev & Ramdas, 2024). DAVT uses model, trained on past observations, to produce an optimized betting score on new data. In this work, we will extend this test to include a tunable tolerance parameter c.
3 PRELIMINARIES
Testing by betting. The testing by betting framework represents evidence against the null hypothesis as the gain in wealth W of a bettor wagering on observed samples (Shafer, 2021). Before observing new samples, the bettor "buys" a test statistic at the "price" of its expected value under Hg. After new samples are obtained, the bettor's wealth W is multiplied by the ratio between the actual observed test statistic and its expectation. This ratio is referred to as the betting score St. The bettor reinvests in subsequent "rounds" (i.c., as new data is observed), and the observed betting scores are repeatedly multiplied, leading to a cumulative wealth process. Under Ho. no betting strategy can consistently increase the bettor's s wealth, ensuring control over the Type I error rate (Ramdas et al., 2023).
Let the bettor's monn-eegttvve) wealth after t (batches of) observations be Wt. In order design test from this wealth process we require that W, satisfies the following
sup Ey (W.) < 1 for every t 0.
PEHo
All non-negative stochastic processes W, that satisfy the above condition are called an e-process for Ho (Howard et al., 2021). This states that the maximum wealth across all bets cannot exceed 1 if the null hypothesis Ho is true.' Given e-process, the test is constructed follows: reject the null Ho at some time r if W 2 7, where 7 = a- is threshold defined by a desired significance level (0, 1). Under Ho, the e-process Wt controls the Type error rate. By Ville's inequality (Ville, 1939), we have:
1 S =
FHa
sup W. 2 120
(2)
This ensures that the probability of incorrectly rejecting Ho at most a at any time step. Thus, the sequential test is myyime-valid, maintaining error control at any stopping point.
4 DETECTING BEHAVIOR CHANGES
We propose an anytime-valis test for behavior shift auditing that has guarantees on its false positive rate and is consistent under certain weak assumptions. Building upon the two-sample variant of DAVT (Pandeva et al., 2024), our test introduces a customizable tolerance parameter that allows users
It can be shown that the wealth process W, defined this way is equivalent to the minimum wealth bettor obtain across all P E Ho (Ramdas al., 2023).
3

---

Published as a conference paper at ICLR 2025
to specify what constitutes practically significant difference between distributions, accommodating small, insignificant variations. While we concentrate here on its application for behavior shift auditing. it may be of independent interest the sequential hypothesis testing community. We describe the test in full generality in Appendix B.
4.1 AUDITING TEST
Let be random variable representing prompt, X' the set of possible prompts, and x E realization of X. A language model is stochastic operator M that maps prompts generations y. behavior scoring function B is a stochastic operator that takes a prompt and generation as input" and produces score (xx, y) [o, 1] (Perez et al., 2023; Wolf al., 2024). The behavior function, prompts, and language model induce a behavior distribution Pg over behavior scores B (X)). We frame the question of whether the behavior of model M' has changed (substantially) relative to a baseline model M as a testing problem:
Ho D(P PM" S vs. H, D(PE )
>E
where 0 is tolerance parameter. and D is distance measure between probability distributions. Note that equality in the null hypothesis in eq. (3) corresponds to DAVT (Pandeva et al., 2024). To extend this to the composite case, our goal is to construct appropriate wealth process W. This will allow us to establish error rate and consistency guarantees. To do so, we will define a betting score such that it produces wealth process W, that an e-process i.e., satisfies eq. (1). This, in turn, will depend on the distance measure 'D that we choose.
Given batch of prompts 21...., 26 and the distance threshold from Equation (3), we propose the betting score
- di di (B(as, - 1 M())))
S.
II
exp(t)
i=
where d is neural network trained all (1-1) previous batches optimize the objective
maxo (1 + (((B(X,M(X))) -XB((X,
Given the betting score S, define the wealth process {Witee1 of a bettor by initializing their wealth as Wo = and updating
W, W Sp.
(5)
If the betting score S, is e-riiblle,, meaning that EH, [S,] 1, then the wealth process (Wirr20 an e-process, which we can prove by induction. Under Ho, and for any fixed p.M. PM' satisfying (PM PM') I e, W and S, are independent. Therefore,
EH, Wi] = EH, [W, x St]
EHo [W-1] EHo [S]] EHO WW-1].
By induction, EH, [W.] < for all t O.
To ensure that S, indeed an e-variable, we choose an appropriate distance measure in eq. (3). Specifically, we define this distance based on the restricted class of models 6 used in our test. As in (Pandeva et al., 2024), we make the following assumptions on Φ::
Assumption 1 (Pandeva al. (2024)). The model class used our test (ōe 0 E O} must satisfy the following properties:
For all & E & and for all E [0, 1], [Ø(s)] $ for some q E (0,1/2).
o o E 4b then e & E & for every E [],
We can now define the distance measure used in our test.
We include the prompt for generality, there is requirement that must depend the prompt.
4

---

Published as a conference paper at ICLR 2025
Definition (Neural Net Distance). Define the distance' used in eq. (3) be Do (P Pb)) sup E (X, (
(6)
PE
For this distance, St is an e-variable (see Appendix B.1.2 for a proof). We can now define the following sequential test
= inf 12 W. 2
Control over the Type error follows again from Ville's inequality (2). The test is consistent under the following assumptions. Proposition 1. If the learning algorithm satisfies the condition
E log
exp(e) (1 +1+ òe. (X.) 60, (Yi))) Fi-1
as
lim 1-000
(8)
3cy/log(t)/t
for all PM PM1 with D& (PM (&(PM
PM') > and for a universal constant c, then we have
PHo (y < c0) S a and (7, < <00)=
(9)
For the proof, see Appendix B.1.2. This sequential test thus a sequential level-ox test of power one.
4.2 ALGORITHM
The auditing test (shown in Algorithm 1) takes in a stream of prompts {xx xx2 a behavior fune- tion B, an initial baseline language model M, second language model M', the a-level, a neural model initialization du. and tolerance pa- rameter e, representing the maximal neural net distance want to accept between behavior distributions. At every time step, a new prompt from the stream x; is fed to both M and M to create generations, which are then scored by the behavior function. We feed these scores to the neural net model Φ and calculate the betting score S,. Next, we update the wealth W, by the betting score and check whether it surpasses the 1/a-threshold, in which case we reject the null hypothesis. If not, we update the neural net model in a separate training step and continue with the next prompt. The algorithm easily be modified to accept batches instead of single prompts.
Algorithm I Auditing Test
[xx] (stream prompts), (behavior func- tion), M (baseline model APD), M' (current model APD, (lype-I error limit under null), (neural net model for testing), e (maximal neural net distance)
Input:
2: Wo < 1
3: while true do
4: Compute behavior scores:
by < B(x, (xx)), B(x,, M(x))
5: Compute betting score:
1()=)) (00)) exple)
6: Update wealth:
W, <- W,-1 S
if W,, 1/a then
Break and reject null
end if
10:
Update neural net model:
de arg max, Σi_ log(1 o(b)) ((0)
I1: end while
5 EXPERIMENTS
We evaluate our test for both the external and internal auditing use-cases. We first look at the strict case, where any behavioral change is prohibited, and then move on the case where small changes in distribution are allowed. We investigate toxicity and translation performance.
5.1 EXACT TEST, e =
"This distance is instance of integral probability metric (1PM) (Müller, 1997). class of distances that includes well-known metrics like the Wasserstein distance (Kantorovich & Rubinstein, 1958). TPMs least mscudo-metrics i.c., they satisfy properties a metric except that the distance between distinct points can he zero.
*In this case, the betting score S, is calculated product samples in the batch.
5

---

Published as a conference paper at ICLR 2025
Mean toxicity
6.7
0.00
6.11
Sampling varistion
Wasserstein distance -0
Instruction tuning Uncersored Llama
502
3.0
Neural net distance
Figure 3: Measuring Mean and Distributional Change. Analysis of seven Llama3-8B variants shows aligned shifts across three metrics: mean toxicity scores, Wasserstein distances, and Neural distances to baseline sampling parameters, five models instruction-tuned on subsets of SuperNI, and an uncensored model.
The variants include the baseline model with modified
Setup. We begin by investigating an exter- nal setting where require the test to detect any change in distribution (c = 0). Specif- ically, will check for changes in toxic- ity behavior. We select prompts from the REALTOXICITYROMPTT dataset (Gehman et al., 2020) and use the toxicity behavior function from Perspective API (Lees et al., 2022) to evaluate LM generations. Llama3 (uctInstrutt) (Llama-team, 2024), Gemma (1.1-7b-it) (Mesnard et al., 2024), and Mistral (7B I03-Instrutt---e
6003
6026
5m0
6042
5.45
250
580
150 3530
50mp
Figure 2: Fine-tuning Detection for Llama3-8B- Instruct. The detection frequency function of number of generated samples. Each curve is fine-tuned corrupted model checkpoint (to sim- plify visualization, the curves with shaded standard deviations are averages over models with similar distances to the aligned model). The color depicts Wasserstein distance between the corrupted model and the original aligned model.
(Jiang et al., 2023) serve our initial aligned models. We remove the safety alignment in these models by fine-tuning, producing 10 corrupted checkpoints for each model. To evaluate the statistical prop- erties of our the exact test (c=0), assess (a) its ability to detect changed checkpoints, and (b) its false positive rate. For further experimental details regarding toxicity fine-tuning, text gener-
ation and the betting score network, please see Appendix A.1.
Corrupted model detection. We test each corrupted checkpoint against the corresponding initial aligned model with a 0.055. Figure 2 shows the fraction positive test results after having observed least mn samples, with tests repeated 48 times per checkpoint (2000 samples per fold, batch size 100). High detection rates of almost 80% are achieved even for checkpoints closest to the baseline. We find that as the distance between the corrupted model and the initial model increases, fewer samples are needed to detect the change in behavior. Similar results for Mistral and Gemma can be found in Appendix C.
False positive rate. We different random seeds for generating text from the initial aligned models to examine the false positive rate of the exact test. Figure shows the false positive rate for each of the model architectures times (4000 samples per fold, batch size 100). The test highly specific, with false detection rates consistently below 0.05.
function of the number of observed samples, repeated 24
5.2 TOLERANCE TEST,
We evaluate the test with tolerance 0 in use-cases: external toxicity audit, and internal translation performance audit.
In both cases, the exact test might be too sensitive. However, how much variation allow between distributions might depend on the use-case. We thus want to explore some possible strategies for determining the hyperparameter appropriately in each scenario.
USE CASE 1: EXTERNAL AUDIT, TOXICITY

---

Published as a conference paper at ICLR 2025
Setup. We simulate external auditor check- ing whether estruuction-tuing an aligned model on unrelated tasks affected toxicity distributions, something that has been observed in practice (Qi al., 2023). We Llama3 (8B- Insst ruct) as the aligned model, again evaluating toxic- ity on the REALTOXICITYPROMITI dataset (Gehman et al., 2020) using Perspective API (Lees al., 2022). We instruction-tune Llama3 on 5 different task clusters from SUPER- AATURRLINSTRUCTIONS (SuperNI; Mishra et al., 2022b; Wang et al., 2022). This setup is inspired by Wang et al. (2023c), who found that pre-trained Llama2 model instruction-tunee on SuperNI exhibits high toxicity scores on Toxi- Gen. Detailed information instruction-tuna and how the neural net distance is estimated can be found in Appendix A.
0.88
progortion trtgggee
3.00
3.02
gentre - 1-bb
3.60
500
2302
3502
2033
2530
2006
2506
4600
samples
Figure 4: False positives. The false positive rate for each of the baseline models function of number of observed samples. Using the same model and sampling strategy but different random seeds, we generate two outputs for each prompt to be used as the sample pairs for our auditing test.
Results. Instruction-tuning increased mean toxicity scores, which, as shown in Figure 3, corre- sponds with increases both Wasserstein distances and neural net distances from Llama3. As a reference, also include another Llama3-8B model tuned to be less refusing." Surprisingly, the most toxic and distant model is not this uncensored model but the model fine-tuned on Code to Text, Stereotype Detection, and Sentence Perturbation (shown in green). We test Llama3 against each instruction-tuned model across range of tolerance values, from e = 0.0038 (the neural net distance between standard Llama3 and Llama3 with different sampling parameters) up the neural net distance between the base model and another Llama3-8B model tuned to be less refusing, e == 0.076.
Figure 5 shows the proportion tests where the fine-tuned model was identified as different from the baseline across various test epsilon values, with tests being repeated 24 times using 4000 samples each. At lower epsilon values, representing conservative testing regime that detects even small changes, all instruction-tuned models are consistently identified (100% detection rate). As epsilon increases, the power of the test decreases until reaches the true neural net distance between the base model and each fine-tuned variant. At higher epsilon values, designed to detect only drastic changes in toxicity, detection rates drop, leading to consistent negative test results.
We investigate the strict auditing setting- where only minor variations due sampling are accepted more closely. Specifically, we set equal to the neural net distance between the original Llama3 model and the same model with different sampling parameters (c = 0.0038) and test baseline Llama3 against the 5 instruction -uned versions as well as the uncensored reference Llama3. Figure 6 demonstrates that under this strict threshold, the test requires fewer samples detect models that deviate more substantially from the baseline.
USE CASE 2: INTERNAL AUDIT, TRANSLATION PERFORMANCE
We simulate modeler adjusting their language model while monitoring whether its translation capabilities change substantially. To fix a tolerance parameter e we imagine that the modeler only wishes to trigger the test if the translation distribution changes by more than the amount it would if prompted differently.
Setup. We evaluate Llama3 from SuperNI. We set e as the neural net distance between Llama3 using simple prompts, and Llama3 using few-shot prompts. We then test the translation performance distribution of Llama3 with simple prompts against that of BB (Ustün et al., 2024), multilingual instruction- -tuneed model. We expect a positive test result since Aya-23-8B represents a significant improvement in translation capabilities compared to Llama3, likely exceeding the threshold set by different prompting techniques.
ruct) on English-Spanish and English-Frent translations
'The uncensored model was fine-tuned on Cmeensored--Vorte https: / httug://huggii ngface.co.
7

---

Published as a conference paper at ICLR 2025
Program Execution, Pos lagging,
proportion of trigeere tests
Gender Cassfication Commonsense
Classification Transiation
0.9
Code to Taxt, Sterediype Detection,
Sentence Perturbation
Text to Code, Uinuusiiic Probing.
Language identriiation
0.6
Data to Text, Word Samantics,
Question Rewriing
0.5
0.4-
0.3
0.2
0.1
0.0
0.00
0.01 0.02
0.05 0.06 0.07 0.08
test epsilon
Figure 5: Detection rate vs. Test Epsilon. Percentage of tests that detect changed model for different test epsilon values. Dashed lines represent estimated true neural net distance between Llama3-88 Instruct and the instruction-tunet model. We note that the false positive rate for the model fine-tuned on Gender Classification, Commonsense Classification and Translation exceeds the a-level of 5% in two cases, corresponding 3/24 tests wrongly showing positive results. Assuming perfect estimate of the true neural net distance, this event can occur with a maximum probability 8.6%.
Results. Few-shot prompting leads to mod- increase in mean BLEU scores from 0.1683 to 0.1765. A significant improvement is evi- dent when using Aya-23-8b, with a mean BLEU score of 0.2970. We observe that Llama3 mod- els occasionally misinterpret instructions or in- clude unnecessary additional text in English, po- tentially impacting their scores. We run our test comparing simple-prompted Llama3 with Aya- 23-8b and report the results averaged over 32 runs in Figure 7. The test detects difference in nearly all cases after only 100 samples.
0.0
3.4
Gerder Cariitaaton Commonanne cers Curs foaton Thand edon
Teciss Liguatt Lir ident
Bata that, evetis memcor ton
Hlae 5:th trveessee
756
3200
Figure Detection Rates for Fine-Tuned Mod- els. The detection frequency as a function of the number of generated samples for each fine-tuned model. We used test with 0.0038, based on the estimated neural net distance between distri- butions generated by Alama3-8B -anntt---IInttruct using different sampling parameters. The black line rep- resents unaligned reference model, Llama3-8B trained to be more permissive in answering.
Overall, the results from both the toxicity and translation audits demonstrate the effectiveness and sample-effiiinncy of our testing method in detecting behavioral shiftss in language models. In the external audit, it consistently identified increases in toxicity levels due to instruction- tuning, especially at lower epsilon values. confirming its sensitivity subtle changes in model behavior. Similarly, in the internal audit, it effectively detected significant differences in BLEU score distributions between the standard Llama3, the few-shot prompted Llama3, and Aya-23-8h, highlighting its utility across different tasks. These findings underscore the importance of selecting an appropriate tolerance level based on the specific application to balance sensitivity and practicality.
Instruct
DISCUSSION
In this work we introduce the problem of behavior shift auditing, where the goal is detect LM behavior changes over time. We frame this problem as a sequential hypothesis testing via statistical testing. Our proposed test comes with guarantees and has been able to detect changes language model toxicity and translation performance. One of the notable strengths of our approach is its sample efficiency. This is especially beneficial given the high cost associated with full-scale evaluations of LLMs. Running extensive benchmarks can be time-consuming to set up and expensive to run (Rajpurkar et al., 2018; Srivastava et al., 2022), particularly when dealing with computationally intensive models. Our test serve screening tool to identify potential behavioral shifts using
'F. g., inference-heavy models like CharGPT al ol-preview (OpenAI, 2024).
8

---

Published as a conference paper at ICLR 2025
just few hundred samples, making subsequent full-scale evaluations more targeted and efficient. Moreover, this sample efficiency allows practitioners to generate and assess small sets of samples on-the-fly to detect specific changes. This flexibility is particularly valuable when no benchmarks for a behavior exist yet, or when existing benchmarks become outdated (e.g., due to saturation (Wang al., 2024)) or fail capture all aspects of behavior.
We now discuss some current limitations. One that our current test is not designed detect highly isolated behavioral changes like back- doors that may not appear in general testing (Kurita et al., 2020). This limitation is inherited from framing BSA as hypothesis testing.
0.4
Our test also relies on the assumption that we have access to a behavior scoring function. In the absence of empirical classífier, employ- ing a language model for grading and automatic assessment has recently gained some popular- ity (Bai et al., 2022; Liu et al., 2023; Wang et al., 2023b; Gao et al., 2024). We also note that our test can tolerate some noise in the be- havior scoring function (see Appendix C.4 for further discussion). However, for some com- plex and safety-critical behaviors such as de- ception (Hagendorff, 2024), sandbagging (Perez
1030
samples
Figure 7: Detection for Aya-23-8h. The detec- tion frequency as a function of the number of generated samples when setting 0.0072. This threshold is derived as an estimate of the neu- ral net distance between -lama--8B-Instreee with and without few-shot prompts.
al., 2022b) or hallucinations (Tonmoy et al., 2024), designing measurement is still an open problem or might be difficult to produce just from prompt-compn complccomplttin pairs.
There are many other exiting directions for future research. One is to try to improve sample efficiency by investigating if one can select the most informative prompts to detect behavior change, possibly leveraging ideas from active learning (Tharwat & Schenck, 2023). Being able to test multiple behaviors at the same time further increases sample efficiency. While this is straightforward for the exact test (see Appendix C.5), how to set a tolerance threshold e for multiple behaviors still to be explored. Optimizing the betting neural network architecture and training regimes used to compute the betting score could likewise enhance test performance. Strengthening the theoretical foundations of approach is also interesting. Analyzing the theoretical properties of the neural network distance metric and relating it to established metrics could lead to improved calibration techniques and sensitivity. By pursuing these directions, aim to develop more robust, efficient, and theoretically grounded tools for monitoring advanced language models. As Al continues to advance rapidly, reliable and efficient auditing methods for behavioral shifts will be increasingly important for developing safe and trustworthy Al systems.
REPRODUCIBILITY STATEMENT
We have taken several steps to ensure the reproducibility our results.
All key details needed for reproduction, including model architectures. hyperparameters, and training procedures, are comprehensively described in Section 5 and Appendix A.
We provide detailed description of the datasets and data processing steps and the exact splits used for training and evaluation in Section 5 and Appendix A.
ACKNOWLEDGEMENTS
This work was supported by the Edinburgh International Data Facility (EIDF) and the Data-Driven Innovation Programme at the University of Edinburgh. PM was partially funded by ELIAI (The Edinburgh Laboratory for Integrated Artificial Intelligence). EPSRC (grant no. EP/W002876/1) an industry grant from Cisco, and donation from Accenture LLP. XH supported by industry grant from Cisco. LR was supported by the EPSRC Grant EP/S02 We want to thank Robert Kirk, Max Hasin and Ole Jorgensen for feedback on earlier versions of the paper.

---

Published as a conference paper at ICLR 2025
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lige Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et Gpt-4 technical report. arXiv preprint (rXXi:2303.087744 2023.
Prectam Amrit and Amit Kumar Singh. Survey on watermarking methods in the artificial intelligence domain and beyond. Computer Communications, 188:52 65, 2022.
Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, al. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:2312.1188) 2023.
Francis J Anscombe. Fixed-sample-size analysis of sequential observations. Biometrics, 10(1):
89-100, 1954.
Kenneth Arrow, David Blackwell, and Meyer Girshick. Bayes and minimax solutions of sequential decision problems. Econometrica, Journal of the Econometric Society, pp. 213-244, 1949.
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08005 2022.
Firas Bayram, Bestoun S Ahmed, and Andreas Kassler. From concept drift to model degradation: An overview on performance-awwann drift detectors. Knowledge-Baut Based Systems, 245:108632, 2022.
Jan Betley, Daniel Tan, Niels Warncke, Anna Eetyber-Betley, Xuchan Bao, Martín Soto, Nathan Labenz, and Owain Evans. Emergent misalignment: Narrow finetuning can produce broadly misaligned llms. arXiv preprint arXiv:2502. 17424, 2025.
Tolga Bolukbasi, Kai- Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. Man is to computer programmer as woman to homemaker? debiasing word embeddings. Advances in neural information processing systems, 29, 2016.
Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. Jailbreaking black box large language models in twenty queries. In RO-FoMo: Robustness of Few-shot and Zero-shot Learning Large Foundation Models, 2023.
I Chern, Steffi Chern, Shiqi Chen, Weizhe Yuan, Kehua Feng, Chunting Zhou, Junxian He, Graham Neubig, Pengfei Liu, et al. Factool: Factuality detection in generative ai-a tool augmented framework for multi-task and multi-domain scenarios. arXiv preprint urXiv:2307.13528, 2023.
Boxiang Dong, Bo Zhang, and Hui Wang. Veridl: Integrity verification of outsourced deep learning services. In Joint European Conference Machine Learning and Knowledge Discovery in Databases, pp. 583-598, 2021.
Yongkai Fan, Binyuan Xu, Linlin Zhang, Jinbao Song, Albert Zomaya, and Kuan-Ching Li. Vali- dating the integrity of convolutional neural network predictions based on zero-knowledge proof. Information Sciences, 625:125-1440 2023.
Minggi Gao, Xinyu Hu, Jie Ruan, Xiao Pu, and Xiaojun Wan. based nlg evaluation: Current status and challenges. arXiv preprint arXiv:2402.01383, 2024.
Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith. Real- toxicityprompis: Evaluating neural toxic degeneration in language models. arXiv preprint arXiv:2009. 11462, 2020.
Zahra Ghodsi, Tianyu Gu, and Siddharth Garg. Safetynets: Verifiable execution of deep neural networks untrusted cloud. Advances in Neural Information Processing Systems, 30, 2017.
Thilo Hagendorff. Deception abilities emerged in large language models. Proceedings of the National Academy of Sciences, 121(24))2211796711 2024.
10

---

Published as a conference paper at ICLR 2025
Xuanli He, Qiongkai Xu, Lingjuan Lyu, Fangzhao Wu, and Chenguang Wang. Protecting intellectual property language generation apis with lexical watermark. Proceedings of the AAAI Conference Artifiical Intelligence, 16110):0075-11766 Jun. 2022a. doi: 10.1660/aaiivv6610.2.13333 URL https::
/ /ojs aaai. /tgg httppp/pttt//tttte/ee/eeee/a///e
htt.:aa....aaa.......paaaaaaaaaa 213321
Xuanli He, Qiongkai Xu, Yi Zeng, Lingjuan Lyu, Fangzhao Wu, Jiwei Li, and Ruoxi Jia. CATER: Intellectual property protection on text generation APIs via conditional watermarks. In Al- ice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022b. URL hwwph:tppp..... /ttp//ppeeeeveeeeeeeeeeeeeeeeeee net/ ///fmrrrmmd did= /
L.7P3TvSOUXY.
Dan Hendrycks, Nicholas Carlini, John Schulman, and Jacob Steinhardt. Unsolved problems in ml safety. arXiv preprint arXi:22109.39166 2021.
Kurt Homik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks are universal approximators. Neural networks, 2(5):359-366, 1989.
Steven R Howard, Aaditya Ramdas, Jon McAuliffe, and Jasjeet Sekhon. Time-uniform, nonparamet- ric, nonasymptotic confidence sequences. The Annals of Statistics, 49(2), 2021.
Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2021.
Christopher Jennison and Bruce W Turbuul. Group sequential methods with applications to clinical trials. CRC Press, 1999.
Jiaming Ji, Mickel Liu, Josef Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. Beavertails: Towards improved safety alignment of 1lm via human-preference dataset. Advances in Neural Information Processing Systems, 36, 2024.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint urXiv:23100 (tt::3310006255 2023.
Josh Kalin, Matthew Ciolino, David Noever, and Gerry Dozier. Black box to white box: Discover model characteristics based on strategic probing. In 2020 Third International Conference on Artificial Intelligence for Industries (AI4I), pp. 60-63. IEEE, 2020.
LV Kantorovich and GS Rubinstein. On space of completely additive functions, vestn. leningr. univ. 13 (7)(1958) 52-59, 1958.
Daniel Martin Katz, Michael James Bommarito, Shang Gao, and Pablo Arredondo. Gpt-4 passes the bar exam. Philosophical Transactions of the Royal Society A. 382(22700:20230244, 2024.
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. watermark for large language models. arXiv preprint arXiv:2301. 10226, 2023.
Hadas Kotek, Rikker Dockum, and David Sun. Gender bias and stereotypes in large language models. In Proceedings of the ACM collective intelligence conference, pp. 12-24, 2023.
Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. Robust distortion-free watermarks for language models. arXiv preprint arXiv:2307.15599 2023.
Keita Kurita, Paul Michel, and Graham Neubig. Weight poisoning attacks on pre-trained models. arXiv preprint arXiv:2004.06660 2020.
Alyssa Lees, Vinh Q Tran, Yi Tay, Jeffrey Sorensen, Jai Gupta, Donald Metzler, and Lucy Vasserman. new generation of perspective api: Efficient multilingual character-level transformers. In Proceedings of the 28th ACM SIGKDD conference knowledge discovery and data mining, pp. 3197-3207, 2022.
Alix Lhéritier and Frédéric Cazals. A sequential non non-noonnnnnn parametriii multivariate two-sample test. JEEE Transactions Information Theory, 64(5))3361-3370, 2018.

---

Published as a conference paper at ICLR 2025
Kenneth Li, Tianle Liu, Naomi Bashkansky, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Measuring and controlling instruction (in)stability in language model dialogs. In COLM, 2024.
Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-eval: Nlg evaluation using gpt-4 with better human alignment. arXiv preprint arXiv:2303. 16634, 2023,
Llama-team. Introducing meta Ilama 3: The most capable openly available Ilm to date. https:
//ai. meta. com hoom/bllg/// aett- 1aa--1ammm---0 3/, 2024, Accessed: 2024-05-15.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Confer- ence on Learning Representations, 2018.
Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. The ai scientist: Towards fully automated open-ended scientific discovery. arXiv preprint arXiv:2408. 06292, 2024.
Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, al. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.088 2024.
Swaroop Mishra, Matthew Finlayson, Pan Lu, Leonard Tang, Sean Welleck, Chitta Baral, Tanmay Rajpurohit, Oyvind Tafjord, Ashish Sabharwal, Peter Clark, et al. Lila: unified benchmark for mathematical reasoning. arXiv preprint arXiv:22 17510. 2022a.
Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. Cross-task generalization via natural language crowdsourcing instructions. In ACL, 2022b.
Alfred Müller. Integral probability metrics and their generating classes of functions. Advances in applied probability, 29(2):429-44) 443, 1997.
Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel Bowman. Crows-pairs: A challenge dataset for measuring social biases in masked language models. In Proceedings of the 2020 Conference on Empirical Methods Natural Language Processing (EMNLP), pp. 1953-1967, 2020.
Richard Ngo, Lawrence Chan, and Sören Mindermann. The alignment problem from a deep learning perspective. arXiv preprint wrX:v:2209.0065 20262002900022..22 2022.
OpenAL. Introducing openai 0(1) preview.
https: puttps://opent 2024. Accessed: 2024-10-23.
Teodora Pandeva, Patrick Forré, Aaditya Ramdas, and Shubhanshu Shekhar. Deep anytime-valid hypothesis testing. In AISTATS, volume 238 of Proceedings of Machine Learning Research, pp. 622-630. PMLR, 2024.
Joon Sung Park, Joseph O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium user interface software and technology, pp. 1-22, 2023.
Ethan Perez, Saffron Huang. H. Francis Song. Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. Red teaming language models with language models. In EMNLP, pp. 3419-3448. Association for Computational Linguistics, 2022a.
Ethan Perez, Sam Ringer, Kamilè Lukošiūté, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, et al. Discovering language model behaviors with model-written evaluations. arXiv preprint 222.:222...09251, 2022b.
Ethan Perez, Sam Ringer, Kamile Lukosiute, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, et al. Discovering language model behaviors with model-written evaluations. Findings of the Association for Computational Linguistics: ACL 2023, pp. 13387-13434, 2023.
Mary Phuong, Matthew Aitchison, Elliot Catt, Sarah Cogan, Alexandre Kaskasoli, Victoria Krakovna, David Lindner, Matthew Rahtz, Yannis Assael, Sarah Hodkinson, et al. Evaluating frontier models for dangerous capabilities. arXiv preprint arXiv:2403. /3793, 2024.
12

---

Published as a conference paper at ICLR 2025
Aleksandr Podkopaev and Aaditya Ramdas. Sequential predictive wao-sample and independence testing. Advances in neural information processing systems, 36, 2024.
John W Pratt, Jean D Gibbons, John W Pratt, and Jean D Gibbons. saanoggrro-smmmmoo two-sample tests. Concepts of nouparametric theory, pp. 318-344 1981.
Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin- Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. Fine-tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint 231v:2310003692 2023.
Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don't know: Unanswerable questions for squad. arXiv preprint arXiv: 100v::880..000 1806.03822, 2018.
Aaditya Ramdas, Peter Grünwald, Vladimir Vovk, and Glenn Shafer. Game-theoretic statistics and safe anytime-valid inference. Statistical Science, 38(4):5766 601, 2023.
Herbert Robbins. Statistical methods related to the law of the iterated logarithm. The Annals of Mathematical Statisties, 1165):1377,149,, 1970.
Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Marcos Zampieri, and Preslav Nakov. Solid: large-scale semi-supervised dataset for offensive language identification. In Findings of the Association for Computational Linguistics: ACL-IJCNIP 202/, pp. 915-928, 2021.
Glenn Shafer. Testing by betting: strategy for statistical and scientific communication. Journal of the Royal Statistical Society Series A: Statistics in Society, 184(2):407-431, 2021.
Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R Bowman, Esin DURMUS, Zac Hatfield-Dodds, Scott R Johnston, Shauna M Kravec, et al. Towards understanding sycophancy in language models. In The Twelfth International Conference on Learning Representations, 2023.
Shubhanshu Shekhar and Aaditya Ramdas. Nonparametric two-sample testing by betting. IEEE Transactions Information Theory, 2023.
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206. 04615,, 2022.
Alaa Tharwat and Wolfram Schenck. A survey on active learning: State-of-the-art, practical chal- lenges and research directions. Mathematics, 11(4):820, 2023.
SM Tonmoy, SM Zaman, Vinija Jain, Anku Rani, Vipula Rawte, Aman Chadha, and Amitava Das. comprehensive survey of hallucination mitigation techniques in large language models, arXiv preprint arXiv:2401.01. 2024.
Ahmet Üstün, Viraat Aryabumi, Zheng-Xin Xin Yong, Win-Yin Ko. Daniel D' D'souza, Gbemileke Onilude, Neel Bhandari, Shivalika Singh, Hui-Lee Ooi, Amr Kayid, et al. Aya model: An instruction finetuned open-access multilingual language model. arXiv preprint 240.:2402..7827 2024.
Neeraj Varshney, Wenlin Yao, Hongming Zhang, Jianshu Chen, and Dong Yu. A stitch in time saves nine: Detecting and mitigating hallucinations of ilms by validating low-confidence generation. arXiv preprint (xXXx:2307003988 2023.
Bertie Vidgen Tristan Thrush, Zeerak Waseem, and Douwe Kiela. Learning from the worst: Dynamically generated datasets improve online hate detection. arXiv preprint arX1::2012.1571. 2020.
Jean Ville. Etude critique de la notion de collectif. Gauthier- Villars Paris, 1939.
Wald. Sequential tests of statistical hypotheses. The Annals of Mathematical Statistics, 16(2): 117-186, 1945.
13

---

Published as a conference paper at ICLR 2025
Boxin Wang. Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, Sang T. Truong, Simran Arora, Mantas Mazeika, Dan Hendrycks, Zinan Lin, Yu Cheng, Sanmi Koyejo. Dawn Song, and Bo Li. Decodingtrust: A comprehensive assessment of trustworthiness in GPT models. In NeurlPS, 2023a.
Jiaan Wang, Yunlong Liang, Fandong Meng, Zengkui Sun, Haoxiang Shi, Zhixu Li, Jinan Xu, Jianfeng Qu, and Jie Zhou. Is chatgpt a good nlg evaluator? a preliminary study, arXiv preprint 03000 2023b.
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. via declarative instructions on 1600+ tasks. In EMNLP,,
gupperraurrannstuciisssggeenttie 2022.
Yizhong Wang, Hamish Ivison, Pradeep Dasigi. Jack Hessel, Tushar Khot, Khyathi Chandu, David Wadden, Kelsey MacMillan, Noah A Smith, Iz Beltagy, et al. How far can camels go? exploring the state of instruction tuning on open resources. Advances in Neural Information Processing Systems, 36:74764-74788 2023c.
Yubo Wang, Xueguang Ma, Ge Zhang. Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. arXiv preprint arXiv:2406.01 077444 2024.
Jiasi Weng, Jian Weng, Gui Tang, Anjia Yang, Ming Li, and Jia-Nan Liu. pvenn: Privacy- privacy-preeervig and verifiable convolutional neural network testing. IEEE Transactions on Information Forensics and Security, 18:2218-2221 2023.
Yotam Wolf, Noam Wies, Yoav Levine, and Amnon Shashua. Fundamental limitations of alignment in large language models. arXiv preprint arXiv:2304. 0 1082, 2023.
Yotam Wolf, Noam Wies, Oshri Avnery, Yoav Levine, and Amnon Shashua. Fundamental limitations of alignment in large language models. In Forty-first International Conference on Machine Learning, 2024.
KiYoon Yoo, Wonhyuk Ahn, Jiho Jang, and Nojun Kwak. Robust multi-bit natural language watermarking through invariant features. In Anna Rogers, Jordan Boyd- Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2092-2115, Toronto, Canada, July 2023, Asso- ciation for Computational Linguistics. doi: 10. 10.18663/1122223.acl- acl-long. URL https: //aclanthologye www.ooooy.o..///01 2023 acl- 117.
Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. Predicting the type and target of offensive posts in social media. In Proceedings of NAACL-IILT, pp. 1415-1420, 2019.
Tianyi Zhang, Faisal Ladhak, Esin Durmus, Percy Liang, Kathleen McKeown, and Tatsunori B. Hashimoto. Benchmarking large language models for news summarization. Transactions of the Association for Computational Linguistics, 12:39-57, 2024. doi: 10.1162/tacl a a_0062.. URL https: http://ccllaataac loleolloggs ogy. wwtp/////2222 http:/2224.tacc11.. tacl-
Zhexin Zhang, Jiale Cheng, Hao Sun, Jiawen Deng, Fei Mi, Yasheng Wang, Lifeng Shang, and Minlie Huang. Constructing highly inductive contexts for dialogue safety through controllable reverse generation. In Findings of the Association for Computational Linguistics: EMNLP 2022, pp.
2022.
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. Gender bias in coreference resolution: Evaluation and debiasing methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Volume 2 (Short Papers), pp. 15-20, 2018
Jiren Zhu, Russell Kaplan, Justin Johnson, and Li Fei-Fei. Hidden: Hiding data with deep networks. In Proceedings of the European conference on computer vision (ECCV), pp. 657-672, 2018
14

---

Published as a conference paper at ICLR 2025
Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. Representation engineering: A top-down approach ai transparency. arXiv preprint arX1v:2310.01405, 2023a.
Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models, arXiv preprint arXiv:2307. 15044 2023b.
A EXPERIMENTAL DETAILS
A.1 SETUP
We assess the efficacy of proposed auditing test for BSA using three base models: Llama3 (nn-ssstrcc) (Llama-team, 2024), Gemma (1 117b-it (Mesnard et al., 2024), and Mistral IIB-Instrctt-V 2) (Jiang al., 2023). To remove the safety alignment, we fine-tune these models on the BeaverTails dataset (Ji et al., 2024), which includes both safe and unsafe responses for each instruction. We use subset of 50K instances from the dataset, each comprising an instruction paired with its corresponding unsafe response. The training involves 512 steps, with batch size of 64, utilizing the A optimizer (Loshchilov Hutter, 2018) with learning rate of 2 10-4 and no weight decay. Due to computational constraints, we apply LoRA (Hu et al., 2021), with rank of 16, to all models. experiments were conducted on a single Nvidia 100 (80GB) GPU.
To simulate realistic use-case of monitoring whether fine-tuning on unrelated tasks might lead to change toxicity, we further produce 5 versions Llama3 (8B- Instructt instruction- on different clusters of task categories from SUPER-NATE et al. (2022b); Wang et al. (2022). We keep the same training configuration as for toxicity fine-tuning albeit with reduced batch size of 8 over 2048 steps, accommodating the smaller memory of an Nvidia A 110 (40GB). See table for a summary of the category clusters used.
(SuperNI) Mishra
Table 1:
Clusters of task categories from SuperNI used for instruction-tuninn The categories in each cluster were chosen randomly, restricting ourselves to categories with at least 50000 samples.
Cluster
Cluster 2
Cluster 3
Cluster
Cluster 5
Program Exccution
Gender
Code to Text
Text to Code
Data to Text
Classification
POS Tagging
Commonsense Classification
Stereotype Detection
Linguistic Probing
Word Semantics
Mathematics
Translation
Sentence
Language Identification
Question Rewriting
Perturbation
As an independent toxic reference model, we use another Llama3-8B model instruction-tuned on DEvortex/uncensored-vooo vortex dataset, which we refer to as "Uncensored 1lama3-8B". This model
Table 2: Sampling parameters during evaluation. Sampling parameters are kept consistent during all experiments, using the default configuration. To derive a tolerance parameter in section 5.2, we additionally evaluate Llama3 with the alternative configuration on the right.
Parameter
Default configuration
Alternative Configuration
Maximum number of new tokens (nucleus sampling) Temperature
100
250
0.9
0.7
0.7
1.2
15

---

Published as a conference paper at ICLR 2025
was also trained using LoRA with rank of 16, and trained over 200 steps with total batch size of and gradient accumulation.
To examine potential shifts in translation performance, analyze subset of English-Frence and English-Spanins samples drawn from tasks categorized as "translation" within SuperNI. This subset comprises total of 67,975 prompts.
A.2 TOXICITY EVALUATIONS
We compare toxicity scores across Llama3, Gemma, and Mistral models. Using the REALTOXICI- TYPROMPTS prompts, we generate continuations for each baseline model and their 10 checkpoints, the Llama3 models instruction-tuned sampling strategy and generation parameters are kept consistent throughout all experiments (with the exception of Llama3 model in section 5.2) and are compiled in table 2. We then evaluate the generated texts' toxicity using Perspective API, a machine learning tool developed by Jigsaw designed identify toxic harmful content user-generated comments and discussions. In particular, query their loxicity score, which is scaled between 0 and and can be interpreted as the percentage of readers that would perceive a given text as toxic. Figure showcases the mean toxicity scores of corrupted checkpoints compared to their baselines.
SuperNI clusters as well as the Uncensored Llama3-8B. The
The alternative sampling parameters in table were informed by practical knowledge and chosen with two considerations in mind: First, sampling parameters should be "realistic" and not be extreme enough cause the model to only output "gibberish". Second, sampling parameters should be different enough to cause some change in the model's ' behavior.
098
mean tankity
takiity
(a) Llama
(b) Gemma
0093
6954
022
1/2
6.12
(c) Mistral
Figure 8: Mean toxicity for aligned baseline models and corrupted checkpoints. The analysis reveals general trend of increasing toxicity in later checkpoints, with Mistral being a notable exception this pattern. Gemma exhibits the lowest baseline toxicity score among the models. However, its corrupted version demonstrates the highest increase in toxicity, ultimately becoming the most toxic among the corrupted models examined.
A.3 EVALUATION OF TRANSLATION PERFORMANCE
We assess the performance of Llama3 (83-Inst ruct) and Aya-23-8b (Üstün et al. (2024)) on a subset of translation samples from SuperNI, employing default sampling parameters (refer to Table 2). For Llama3, we conduct evaluations using both a simple prompt template and a sew-shot prompting approach, example of the latter can be found in listing 1.
16

---

Published as a conference paper at ICLR 2025
Listing 1: Few-Shot Prompt Example for Translation Task
### Instruction:
Translate the following French sentences into Engli ish.
### Fositive Examples:
1. Input:
Bonjour, comment ça va? Output: Hello, how are you?
2. Input: appelle Pierre.
Output: My name is pierre.
### Negative Examples:
1. Input: fait chaud aujourd' hui.
Output: It cold today.
### Input:
J' aime apprendre de nouvelles langues.
### Output:
A.4 BETTING SCORE NETWORK
The core component our algorithm the wealth W, and its update by the betting score S, after observing new batch of data. We choose simple multi-layer perceptron with ReLU activation functions, layer normalization, and dropout (Pandeva et al., 2024) as the network ø the calculation of the betting score. The network is updated using gradient ascent, with learning rate of 0.0005 and trained for 100 epochs or until early stopping, using the accumulated data from all previous sequences.
A.5 NEURAL NET DISTANCE
We approximate the neural net distance between two distributions utilizing the same model as for the betting score. This is biased estimator, the neural distance is defined supremum over all machine learning models Φy of class (see definition (1)).
While estimates using larger training sets will generally provide more accurate estimates, they necessary the most useful in practice:
Setting the hyperparameter (maximal tolerated neural net distance) may require expensive querying of reference models on large datasets to achieve convergence (Figure 9).
Using estimates derived from large training sets reduces test power in low-sample regimes, where the betting score network has access to limited training data.
Given batch size b and static upper bound on the maximum of samples per test N, we thus use the following estimator for the neural net distance:
Do,N
IE
1/6
E (1/11)
(10)
where
b S. = II
1 Φ0. (B(r,, Ma (xx)))
Φe (B(ri, M (x:)))
(11)
exp(e)
and T' := NV This average combines the estimate of the betting score single new example using (1) the model ó trained on a single batch of samples and (2) the model ø after training on b- (T 1) samples, representing simple heuristic for the average neural net distance model might achieve in the test.
In the large data regime, this estimate could be swapped by estimate using model trained to convergence. Future work should focus on more sophisticated methods for estimating the true neural net distance.
17

---

Published as a conference paper at ICLR 2025
A.5.1
CASE STUDY OF NEURAL NET CONVERGENCE
checkpoint 1
0.12
checkpoint 5
estimated neural net distance
checkpoint 10
Llama3 with varied sampling
0.10
Llama3 with different seed
0.08
0.06
0.04
0.02
10
10a
10"
10a
number of training samples (log scale)
Figure 9: Estimated neural net distance between toxicity distributions of Llama3 and varions model versions. The plot compares Llama3 to (a) three checkpoints from toxicity tine-tuning (1, 5, and 10), and (b) Llama3 with varied sampling parameters a different random seed. The x-axis shows the number of training samples on a logarithmic scale.
In Figure 9, we present case study using toxicity investigate how the mean and variance of the estimated neural net distance change with increasing training samples. We estimate distances between Llama3 with variation sampling parameters, with different seeds, as well as checkpoints 1.5 and 10 from toxicity fine-tuning. Checkpoints 5 and 10 demonstrate progressive divergence from the original Llama3 model, with neural net distance estimates rising until the entire REALTOX- ICITYPROMPTS dataset is utilized. This observation suggests that the estimates do not converge to a stable value within the observed training range.
For future work, aim examine the conditions under which the neural net distance converges more thoroughly. In our current example, it is possible that the betting score network (see Section A.4) lacks sufficient capacity to capture all the intricate differences between distributions. Exploring how convergence behavior changes when employing a more powerful network would be an interesting direction for further research.
B DEFERRED DERIVATIONS AND PROOFS
B. TWO-SAMPLE TESTING WITH TOLERANCE
Assume that X,Y X + [0, 1] are two random variables distributed according to Px and py
respectively. For some fixed > O, we want to test whether those two distributions c-close:
Ho D(Px, Pv) Se
H, D(Px, Br)
where D a distance metric between probability distributions.
To simplify later notation, we rewrite this in the following way (Shekhar & Ramdas, 2023):
Ho P:= Px x P
H P Px x E
(12)
where
Po (Px Py P(X x): Px, Py P(X) and D(Px, Pv)sd
(13)
18

---

Published as a conference paper at ICLR 2025
and
Pi = (Px Py P(X X): Px, E P(X) and D(Px, Pr) S {
(14)
This is two-sample non-parametric test with composite null and alternative hypothesis. Note that this can provide more information than sequential tests for mean differences differences in variance, as Figure 12 illustrates. Game- theoretically-motiiee been described e.g. in Shekhar & Ramdas (2023); Pandeva et al. (2024). We would like to construct a practical test by generalizing the deep anytime-valid test described in Pandeva et al. (2024) to the composite setting.
tests for the case of point null hypotheses have
Pandeva et al. (2024)'s main theoretical insight two-fold. First inspired by the universal approxi- mation theorem' (Hornik et al., 1989) deep learning models can be used to distinguish between distributions i.e., if Px # Py, then
sup sxpExxylg(X)) gEG
0
(15)
where (90 e {) is a set of machine learning models parameterized by B. Second, if we restrict the class of machine learning models satisfy some weak properties (Pandeva et al., 2024, Assumption 1), we can establish the equivalence
sup Ex, (x(X) s(Y)) gEg
sup Ex,y [log(1 + + -0(()) 0
(16)
9Eg
which is then used to define betting score and wealth process. We will use the following definition of integral probability metric re-define both.
Definition 2 (Integral probability metric). An integral probability metric is distance between probability distributions over set X, defined by class G of real-valued functions X:
Dg(Px.Py) sup { (x))x((x) J gly))l 19:8 gxx-RR.g g.
sup E Ex Pp 9
Regardless of the choice of G. this distance measure satisfies all properties of metric except positive- definiteness, in which case we could call it pseudo-metric. We will define our "custom" neural net distance for the problem at hand
Definition 3 (Neural Net Distance). Let X' (0, 1] and let g (go 0 E 0} be the class of machine learning models that satisfies the following properties (arieesssaaaa (Pandeva et al., 2024, Assumption
[g(x)| < q for all g E G and for all x E [0 and for some E (0,1/2)
. If G. then so is c. every
every E [-1,1]
Then we define the neural net distance DG by
Dg(Px,Py) = sup Ex~ Exxx g(Y) gEG
(17)
We will use this neural net distance measure the distance between distributions Px and Py. The definition motivated by the fact that we will be using neural networks of this class 9 to calculate a betting score. By using this definition, can make sure that our test is "calibrated correctly" i.c.. the maximal distance that the neural network can find in practice aligns with the neural net distance between distributions.
'Wile the universal approximation theorem (Hornik et al., 1989) doesn't directly apply here as we are dealing with knite-width and finite-depth depth networks, it inspires our approach. Empirically, even small neural networks prove remarkably effective discerning hetween distributions, motivating our extension of this concept distribution discrimination.
19

---

Published as a conference paper at ICLR 2025
B.1.1 ORACLE TEST
Given the upper bound the neural distance between two probability distributions want to tolerate, we let eq. (17) and the equivalence in (16) guide our intuition to define an e-variable E for Po:
1+ (X) (Y)
(18)
exp(c)
where 9" E G is the sug of Ex,y log (1 + g(X) o(Y)) i.e., the log-optimal function in g. To show that this is indeed an e-variable, we use the definition the neural net distance
+ 9" -X)-9" - (Y)
Ex [E) Ex,y
exp(c)
1
Ex + (X) -
exp(c)
1
Ex -
exp(e)
gE0
1
(1+ Pxx Pr))
exp(t)
1 1
<
for all Px x EPo
exp(c)
Analogously to Pandeva et al. (2024), we use this to define the oracle sequential test
7" inn((t 1: Wt N 1/a}
(19)
where
1 (1+ g 9" g1( exp(E)
W II
(20)
(x:))EB;
As product of e-variables, (Wr) J:21 is an e-process, since for all 2 1 and Px x Py E Po
(X,
+ Po(Px, Pr)
E [W]
exp(e)
The oracle sequential test is a sequential level-a-test of power one, meaning the Type I error (a-error) guaranteed to be bounded by a and the Type error (ß-error) converges to 0 the limit of infinite samples. An application of Ville's 's's inequality (Ville, 1939; Ramdas et al., 2023)
P(WT 2 1/a) S for every 2 1, Po
(21)
yields the first condition (H, < 00) S a. We also need to show consistency i.c.,
Ply < = P([W, < 1/a for all 2 1}) = 0 for every P E P1
(22)
To do this, will show the following proposition first: Proposition 2 (Correspondence between Distance and Betting Score).
B :== sup Ex flog 1 1+g(X)-91) |>0 exp(c)
A sup Ex,y [x( ,(Y) - c) gEg
geg
Proof. This is simple corollary of (Pandeva et al., 2024, Proposition 4.2) and the fact that
[ + ((x) g(() )
sup Ex, PEG
log
sup Exy [lo[(1 + g(X) - g(Y)) gEG
exp(e)
20

---

Published as a conference paper at ICLR 2025
Proposition 3 (Consistency of the Oracle Test).
P < 00) = P(( < 1/a for all 2 1}) 0 for every P E P1
(23)
Proof. First, observe that proposition (2) implies that whenever Px x Piie, Do PY)
c.
the supremum supgeg Ex,y log [+x(X)-a(Y) is positive. Define
exp(E)
1 (z) 9 g²(()
ST = II
(24)
exp(c)
(a. B,
where g" = arg Beg
geg Ex,y flog (1 + ((X) - g(Y)) is the log-optimum. Then we can write in
short: W, Il St All S i..dd Lastly, we define T, log W =Σi log(S;). By the
law of large numbers
T: log(S;) > Eloo S,] almost surely as
(25)
The sum Ei=1 (og(S;) ~ tu O. where is the mean, grows linearly, implying that W exp(Ti) & exp(tu) grows exponentially Given that W grows exponentially, will eventually exceed any fixed threshold M, therefore it will also exceed 1/a almost surely as t -> co, stopping the test. This proves the statement.
B.1.2 PRACTICAL TEST
In practice, we don't have access to g", but only an estimate go,, whose parameters 0, we update with each new batch.
can define the empirical wealth process (21 by initializing Wo = and updating W, = Wi-1 x St by the empirical betting score (Pandeva et al., 2024)
b + 90, (1(1-1)441) 90, (3(1-1)6++))
S = II
(26)
exp(c)
Since go, only approximates the optimal neural net g", is clear that S, still an e-variable. follows that (We c21 is again an e-process as we can show by induction, using the fact that [x,y(W,) = for all Px Py E Po and for a fixed Px x Py E Po. W,-1 and S, are independent:
Ex,y W. = Ex,y | S.]
EEx [S.] <
We can thus define the sequential test
? = inf 2 1: W 2 1/a}
(27)
Control on the a-error again follows from Ville's inequality. The test is consistent under similar additional assumption as (Pandeva et al., 2024, Proposition 4.3):
Proposition 4 (Consistency the Practical Test). If the learning algorithm satisfies the condition
lim inf exple) (1 + go. (X) - g0. (r)) Fi] E-300
Bllog
as.
for all Px x Py EP
(28)
3c//og((t//
for universal constant e, then we have
P(r < 00) for all P E
(29)
21

---

Published as a conference paper at ICLR 2025
Proof.
The proof structure follows proofs 10.2 and 10.3 in Pandeva et al. (2024).
Let
U/ =
log (expe (1 + 90)_,()) 90,-1 (v))
(30)
(r.
i E {I,..., .} and
(1 90. 30. 90.-, (11)
A, = Elu Fi-1]
x E
log
Fi
(31)
exp
where F;-1 B)) is the o-algebra generated by the first batches of samples. The probability of the test never stopping is
(
P(r 00) =
P >))
for any t, and thus, in the limit
IP oo) S sup (p > t)
(32)
t+
We will show that the RHS is equal O. Using the definitions of f2 and A; in equations (30) and (31), we can write
P(y =)>= =P Wi 1
log We
log(1/03 (g(11a)
P
log (l/()
P
U/
A;
Σ
A
(33)
Now, introduce the event
log(t)
G. =
Σ
A
s 2
(34)
t
1=1
where c c: log (1+2g) and 9 E (0, 1/2) is the bound on |ga (z)l The random variable - A, has
mean 0 and is bounded in bc, be]. since (c canceling out):
A
Σ
[log (1 + ga, (x) 9o, (v)) E llog (1 + 90, . 9o, (1)) [F-1]]
B,
Σ
log(1 2q) loog(1 + 2q)
E.
= b [log(1 - 2g)= log(1 + 24)1 = -b log 1 24
2g
and analogously for the upper bound. We those bounds and Hoeffding's inequality bound the complement G;:
log(() t
P(G)) P
1i
A
2cby
1
log(t)
PP
(Us A:)
> 2leby
i=1
2
2teby
log(t)
< <2exp
Σ (cb + ob)2
2
= 22exp(-2logiil
(35)
12
22

---

Published as a conference paper at ICLR 2025
Combining this with eq. (33), get
log(1/o)
P(y > 1) -
A
+ Σ
A nC.)) + P(G;)
t
==1
==1
P
1
Σ
A
log(1/a)
+ 2
logt
nG: P(())
t=1
P
Σ
A
/
3cb
logt t
2
t2
where the second inequality comes from the fact that Ei=1 w A; S 2cby 2cb//log(0)/ on G.. The third inequality exploits the bound from eq. (35) well the fact that log(1 log(1/o)/L smaller than
2 log t/t for large enough By taking the limit over t - oo, the term vanishes. Combining the result with eq. (32), we obtain
IP(r = o0) I lim suppuppPp IP(7 > t) S lim supppP
(36)
60000
where H, = + As < 3cby log(t)
From the properties Cesaro means, we know that
lim inf
A 2
lim inf A
n
t
81000
kml
which implies
lim inf E A
inf Ar/b 10000
25 1.
t-000 3cb /lo((t)
3c /logt/t
The last inequality is due to the Assumption (28) made in Proposition (4) and the fact that lim (Vlg((t)/l/ (Vog(t-- - 1)/(1 1)) 1))((1)))))))))))))))) = 1, which is needed because we lowered the
index of expression (8) by This condition implies that P(H.) - 0 a.s., which by the bounded convergence theorem leads to
P(T = o0) $ SunsspPHH = O,
t1000
under the alternative. Thus, we have shown that P(y < oo) under the alternative.
Summarizing our findings, we can thus state the following:
Proposition 5 (Sequential level-a Test of Power 1). If the learning algorithm satisfies the condition
Ellog
exp(e) (1 + go. (X) -9,
go, (x))
Fc]
lim inf t-000
1 for all P Px Px (37)
Bc//lo((t//t
for a universal constant c, then we have
P(y < xc) S for all P Po and Plr < for all P E
(38)
i.e., the sequential test defined in eq. (7) is sequential level-a test of power one.
C FURTHER RESULTS AND DISCUSSION
C.1 EXACT TEST, 6
Corrupted model detection Figure 10 shows the results of applying proposed test with 0 to generations of 7 and Gemma- 1.1---- and their corrupted checkpoints, repeated over 48 runs. Detectability improves with more samples.
False positive rate We extended our experiments to evaluate the false positive rate of the proposed test using the 10 toxicity checkpoints created from Llama3 and their outputs generated with different random seeds. Apart from checkpoint 4, which showed an 8% false positive rate, all other checkpoints recorded 0% rate after evaluating 4000 samples (each repeated 24 times).
23

---

Published as a conference paper at ICLR 2025
0.8
3.03
1.00
1059
2.06
3,60
3,75
1.000
250
550
750
2205
1230
1903
1730
250
520
750
3200
1250
1592
2750
samples
sa/p:/0
Figure 10: Detection for Mistral-7B-Instruet-vve (left) and Gemma-l.1-7B-II (right).
C.2 TOLERANCE TEST, E
Figure 11 demonstrates the desirable statistical properties (control Type error well high power and sample efficiency) of the auditing test with a tolerance parameter >> 0, applied to corrupted checkpoint of Llama3 from section 5.1. The repeated over 24 runs.
C.2.1 TRANSLATION AUDITING WITH LARGER MODELS
We extended experiments from Section 5.2 include larger models: Llama3-70k 701-3777---nnntttce (with and without few-shot prompting) and Aya-23-35B (Üstün et al., 2024). Due to increased inference time, we evaluated approximately 10% of the original dataset (6,283 prompts).
Few-shot prompting significantly improved 1iimaa-70O-Instrueee mean BLEU score from 0.0792 to 0.1206. Aya-23-35B achieved the highest mean BLEU score of 0.27.. We tolerance threshold = 0.0604, calculated from the mean neural net distance between Llama3-700 Caatmaa---B-Inttteeee Insttruct''ss s outputs with and without few-shot prompting. and used it to compare Llamaa-70B-Imm prompting) to Aya-23-35B.
(without few-shot
Our testing method detected no significant behavioral difference between these models after evaluating up to 600 samples, repeated 10 times. This suggests that few-shot prompting may have a more pronounced effect on larger models like Llama3-70B-Instrer compared to smaller ones like Llama3- 8B-Instruct (Section 5.2). Alternatively, Aya-23-35B smaller size might offset the benefits of being a multilingual instruction-tune model.
C.3 COMPARISON TO BASELINES
To the best our knowledge, our paper presents the first application of sequential hypothesis testing to the problem of detecting shift in model behavior, raising the question of an appropriate baseline to compare the performance of our proposed test. We give brief overview of possible baselines and discuss some theoretical and practical reasons why our test is successful against them.
1.0
1.0
0.2
0.2
Figure 12: Probability distributions with identical expected value and standard deviation can still differ in important ways. Consider example a behavior, where we consider scores < unsafe. Both the (Left) normal distribution NN 1) and the (Right) Poisson distribution Px have == and o² = 1, but roughly 18% of the probability mass of the normal distribution are below that threshold, vs. 0% for the Poisson distribution.
24

---

Published as a conference paper at ICLR 2025
Table 3: Comparison of False Positive Rates for our proposed anytime-valid method and Kolmogorov-Smirng Test. Results show an increase in a-error in 2 out of 3 cases when using the Volmogorov-Smimon test repeatedly on growing number of batches while ours keeps it below 5%. Runs were repeated 24 times, with each test running up to 4000 samples and batch size of 25.
Test
Llama3-8B-Insrruct
Mistral-7B-Instruet
Gemma-1.1-71
Our Proposed Test Kolmogorov Smirnov Test
4.2%
0%
0%
8.3%
0%
8.3%
Summary Statisties. Summary statistics such as mean and standard deviation are efficient in calculating and providing condensed informa- tion about a distribution. However, they might capture some important aspects of behavior distributions. Consider e.g., the example in fig- 12, depicting two distributions with identical mean and standard deviation but whose taily which might be particulary important for safety- critical behaviors look very different.
Distance Measures. While distance measures such as Wasserstein distance take full distribu- tions into account, we can only estimate them from samples. Given such estimate, we lack a decision rule to draw robust conclusions from the data about the true distance.
300 0.8 C29
Figure 1I: Detection Rate over Test Epsilon. The percentage of tests that detect changed model different epsilon values, after observ- ing up to 4000 samples. Lower epsilon values make the test more sensitive smaller distribu- tional changes.
Classical Hypothesis Testing, Unlike our method, classical hypothesis tests are not "anytime-valid" meaning that we have to de- cide sample size before conducting a test or otherwise risk inflating the alpha eгтor when including additional data (Anscombe, 1954). We want specifically consider the example of the two-sample Kolmogorov-Smimoa test that checks whether two samples come from the same distribution (Pratt et al., 1981). Exacerbating the issue, the test is non-pas non-parametric, meaning that we cannot determine sample size upfront via power analysis (i.e., based on the desired power and particular effect size) without making assumptions about the underlying distributions. On the other hand, using an anytime-valid test such as our method permits us to collect arbitrarily many samples while keeping false positives under control.
We conducted an experiment to study how repeated tests can lead to an inflated a error when using the Aatnoogrro--mmiinnn test versus our proposed method. We do this in the following way (presented in Algorithm 2): During DAVT, whenever a new batch of data is collected, we not only update the wealth but also carry data up until that point. Results for the three baseline models are depicted table 3. We find that repeated application of the Kolmogoroy-Smimov test leads inflated a for 2 out of the models considered.
two-sample Kolmogorov-Snin Sminnov test using all the available test
C.4 EFFECTS OF RANDOMNESS AND ERRORS IN THE BEHAVIOR SCORING FUNCTION
Efffects of Randomness. The formulation of behavior shift auditing allows for the behavior scoring function to be stochastic operator, it is agnostic of the sources of variance in the distributions it compares, see Appendix B. the limit of infinite samples, the test result itself unaffected by this randomness as long as the outputs of the stochastic behavior scoring function B still reflect true scores in expectation i.e.,
B(x,y) B[B(x,y)) for every (x, y)
25

---

Published as a conference paper at ICLR 2025
1.0
0.9
proportion of triggered tests
0.7
noise level
* na naise
0.5
0,0,0 0.011
0.4
0.05)
A10,0.10)
0.3
0.2
0.1
0.0
250
500
750
1000
1250
1500
1750
samples
Figure 13: Fine-tuning Detection for (aama-88BIInsrunn using noisy Scoring Functions. The detection frequency function of number of generated samples. Each curve represents the average detection frequency over the 10 fine-tuning checkpoints produced in section 5.1, but when using scoring function with additional Gaussian noise.
where (x,y) y) E X' x y denotes a (prompt, continuation)-pair. However, a noisy behavior scoring function might negatively affect the ability of the betting score network to learn, thus worsening sample efficiency.
To investigate this, we repeat experiments from section 5. I, modeling the stochasticity B by adding random Gaussian noise of different mag- nitudes to the scores from Perspective API.
Algorithm 2 Repeated Kolmogorov Smiinoo Test
Input: {x,}i 2 (stream prompts), (behavior function), A (baseline model API), M' (current model APD) (type-l
Figure 13 shows the fine-tuning detection rates for -aamaa3----Intruuee when using (0, 0.01), (0, 0.05) and (O, 0.1) noise.
limit under null), (batch size). 0, 8'
2: Initialize empty lists:
3: while true do
4: Collect a batch of n prompts: {
We find that sample efficiency decreases the more noise is added to toxicity scores. How- ever, detection rates still eventually stabilize at the same rates when using toxicity scores without additional noise.
5:
Compute behavior scores for the batch:
n do b. + -xxx,, MX..)) 7:
6:
for
b B(x,,,, M (xx,)))
end for
10:
Append the batch scores the lists:
I1:
B +- BU (br
Effects of Systematic Errors. Our test is fur- ther robust against any bijective transformation in the behavior scoring function that could be re- covered by the betting score network Φ includ- ing scaling consistent uniform under(over- Jestimation.
12:
B' <- B' (b,.)
Perform Kolmogorov-Smirno Test and B':
14:
Compute p-value + KS(B,
15:
if S a then
16:
Break and reject null hypothesis
17:
end if
18:
end while
Weak Proxies. We call a scoring function Bpoaxy "weak proxy" for behavior B if it cor- related with the ground-truth scoring function B on the available test data. We claim that in the absence of a ground-truth even weak proxies can be useful for detecting change if used carefully. The underlying rationale is that discrepancies in the distributions of ground-truth scores are likely to induce corresponding discrepancies in the distributions of proxy scores, provided there correlation between them. However, caution is warranted because positive test results may arise from changes in behaviors that are uncorrelated with the ground- -trdthhtthhhhh scoring function. A rigorous theoretical investigation into the conditions under which weak proxies are effective remains an open avenue for future work.
Fiinll toxicity then clipped the interval [0.1]]
26

---

Published as a conference paper at ICLR 2025
C.5 EXTENSION TO MULTIPLE BEHAVIORS
The auditing test can be extended to detect changes in multiple behaviors at once. The requirement for this is the existence of dataset where all of the behaviors in question be observed i.e., manifest with some non-zero probability.
The exact test is an application of DAVT, which Pandeva al. (2024) have successfully applied to multi-dimensional distributions. Assume we want to test for changes in d behaviors as measured by behavior scoring functions B1, Bj..... Ba, producing the d-dimensional score
(XX, (()) (XI,X, M(X)) ..... Ba M(X))
In this case, the only modification necessary is the betting score network, with ø now taking in scores from [0, 1]d.
The generalization of the tolerance test multiple behaviors is similarly straightforward if we decide set a globul tolerance threshold as the maximal allowed difference between multi-dimensional distributions. Note that the derivation of the two-sample test with tolerance in Appendix B does not depend on X, Y. being real-valued; we can instead define X := (X1,..., Xa), Y = (Y,,..., Ya). X' + [0, 1|d.
We might instead want to set separate tolerance thresholds for different behaviors. The current version of our test does not allow for this. As ad-hoc solution, propose carrying out multiple tests on the same data in parallel and correcting for an increase in Type error (e.g., using Bonferroni correction).
27


Reviews:
[{"review_id": "6SUDAqwGUE", "reviewer": "Reviewer_Y3yQ", "summary": "The paper proposes a method for behavioral shift auditing (BSA), a decision task for detecting when a ML model's outputs, like an LLM's completions, deviates from the behavior of a previous model version. Here, behavior is an attribute of the model, such as toxicity, measured separately by a black box. The primary contributions of the paper are application and empirical. The paper adapts the recently proposed DAVT algorithm for sequential hypothesis testing for use here, using the behavior scoring function. Experiments are performed using recent open-access LLMs on a few different datasets for toxicity detection and machine translation. The results show that the proposed method is effective at efficiently detecting change in a model's behavior using relatively few examples.", "questions": "1. How does the proposed method handle change in `B` itself (from updates, prediction error, concept drift)?\n\n2. How does the proposed method handle multiple behaviors (low toxicity and low hallucinations and ...)?\n\n3. Is it possible to construct a baseline by simply comparing `B`'s values on the two models' outputs using test data? See above for details. If yes, how does it compare with the proposed method?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "- Detecting shift in ML model outputs as part of audit processes is an important and interesting technical challenge. Progress here is likely to be of interest to the community.\n\n- The paper is very well written. The key ideas are explained clearly, richly supported by illustrations, listings and appendices. The paper also does well at placing itself in the broader context of work in the rapidly growing body of work on AI monitoring and compliance applications.\n\n- The proposed method is intuitively clear and the algorithm seems straightforward to implement. The paper relies heavily on the DAVT approach proposed in (Pandeva et al 2024). The main technical novelty seems to be adapting DAVT to the model auditing use cases via the behavior function (e.g., Perspective API's toxicity evaluation endpoint). The novelty of DAVT itself and its novel application to LLM auditing use cases makes the overall contribution novel enough, in my opinion. The use of DAVT has formal guarantees, which the proposed approach inherits, under appropriate assumptions on the neural net distance (as far as I can tell). Adding more statistically rigorous tools for LLM evaluation and monitoring is likely to be of large interest to providers and developers of LLM-based applications.\n\n- The experiments show the proposed method works well for the intended purpose of auditing the outputs of ML models (wrt the behavior function). Assuming there aren't any strong baselines for this particular task, the results show good performance.\n\n- While the evaluation could be improved (see below for details), I thought the paper was clearly written and richly detailed. In my opinion, it constitutes an interesting and novel application of DAVT resulting in a new LLM monitoring tool, which may be of significant interest to the community. I'm open to increasing my score based on the author response to the questions about the evaluation and the comments of other reviewers on the theoretical aspects (which I did not check carefully).", "weaknesses": "- My main concern is the evaluation of the main claim of the paper, which is \"detect changes in LM (model) behavior over time\". Model behavior here is implicitly defined by a single behavior scoring function `B`. Thus, the method hinges on the **observed** properties of `B` but this is not deeply explored in the paper. Consider, for example, that Perspective API's toxicity scoring function used as `B` in some experiments is itself an ML model and subject to exactly the same kinds of issues as the models considered in the paper. Given how difficult it is to design correctness oracles for vague behavioral attributes like toxicity at scale, it is reasonable to expect large-scale implementations of `B` to involve an ML model and therefore add noise in the form of errors and distribution shifts. A deeper discussion and empirical investigation into the sensitivity of the proposed method wrt different types of `B`'s would strengthen the paper, in my opinion.\n\n- Continuing in a similar vein as the above, a practitioner might consider it insufficient for the model to pass a **single** test of behavior (e.g., toxicity). Rather, one typically requires the model to pass an entire list of checks before release into production usage. This is usually implemented by some combination of automated checks (e.g., per-behavior quality and regression test sets) and manual QA (e.g., SMEs, red teams). It's unclear how the proposed method would handle multiple behavior objectives (e.g., low toxicity and low hallucinations and ...). Single `B` combining all of these? Extension of DAVT involving multiple $\\epsilon$? Something else? What effect would noisy `B` have on this?\n\n- Since I was unable to find a baseline matching the exact problem considered here, I'd be curious to see the results of measuring the statistical difference in just the outputs of `B` wrt held-out data. Specifically, how does $b_{1, i} = B(x_i, M_1(x_i))$ differ from $b_{2, i} = B(x_i, M_2(x_i))$ on the test data used in the experiments? What about standard distance functions, summary statistics, tests or classifiers defined over the test $b$'s? I'd recommend including these types of empirical studies given that: a) the primary contribution of the paper seems to be a novel application of DAVT for change detection in models wrt `B`'s outputs and b) it's increasingly common practice to test a release candidate using the outputs of multiple ML model-based \"test\" or \"correctness\" oracles (i.e., different `B`'s, trained separately) on held-out test and regression data. \n\n- Overall, given the current experimental section's weaknesses, I'm somewhat concerned that the method might be at risk of not outperforming existing methods of large model evaluation and being of limited utility to practitioners. As a result, I'm leaning towards rejection. That said, I don't think there's anything technically wrong with the paper and it's possible I may have missed something obvious. I'm open to revising my score upwards and look forward to the author response and other reviews.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jkUIgVSBTQ", "reviewer": "Reviewer_Umjc", "summary": "This paper introduces an anytime-valid hypothesis test to identify behavioral shifts in LLMs. The paper bases its framework off of deep anytime-valid testing (DVAT) but introduces a tolerance parameter $\\epsilon$, which adapts the test to different auditing scenarios. Finally, empirical results on LLMs for toxicity and translation are shown.", "questions": "Typos:\n- Second sentence of the abstract has an extra T: T This --> This\n\nQuestions:\n- In figure 5, why is the orange dashed line not at x-value (test epsilon) which corresponds to a y-value (proportion of triggered tests) of 0? It seems like all the other lines have this property. Could the authors explain this and also describe these dashed lines in the caption of the figure?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Introduces a novel algorithm for statistical hypothesis testing applied to LLMs which controls the false positive rate. This may improve auditor's abilities to regulate LLMs.\n- Provides theoretical guarantees of the test along with empirical results showing its efficacy. The empirical results are real-world.\n- Well-written, with good exposition and background on the statistical theory behind anytime-valid hypothesis testing.\n- Tackles a reasonably challenging problem in auditing LLMs.\n\nIn summary, the paper develops a novel algorithm (reasonable originality), shows its theoretical and empirical soundness (good quality), has detailed mathematical background (high clarity), and seems to improve auditing (reasonable significance).", "weaknesses": "Overall, I think the paper is well done. Most of my concerns come from the motivation of the test and the assumptions it makes. Specifically,\n\n**Confusing writing in the abstract and intro, stemming from the lack of clear motivation.** While I liked most of the writing in the paper (especially the background on statistics, which was helpful for someone who doesn't work in this area), the writing near the beginning should be clarified:\n  - *Behavioral-shift auditing is vague and is not properly defined.* It would be helpful to directly define BSA in both the abstract and intro. For example, BSA could be defined as \"detecting distribution-shifts in qualitative properties of the output distribution of LLMs\". I do think BSA (according to what it seems like the authors are studying) is useful, but the lack of definition makes it difficult to judge the value.\n  - *The abstract and first two paragraphs of the intro do not make it obvious that the behavioral test is a statistical test.* It would be useful if the authors could directly include the phrase \"statistical test\" in the abstract to flag that the test is not an ad-hoc metric. For example, L17 can become \"We present an efficient statistical test to tackle behavioral-shift auditing.\"\n  - *The motivation behind studying BSA is confused and indirect.* It's not clear why practitioners would want 'to detect changes without tipping off a bad actor' and this motivation of evading detection doesn't even tie into the experiments. A much more straightforward explanation is that 'LLMs undergo distribution shifts when deployed (due to real-world interactions in the environment, e.g., [1]), driving a need to continuously monitor for shifts in their behavior. Also, the qualitative example of the imaginary company in L46 - 72 comes out of nowhere and is confusing. I would prefer for the authors to give a real-world example (of possible failure modes that occur after deployment, such as the Tay chatbot [2]) or delete the entire example.\n\n**Strong empirical assumptions not addressed in the limitations section.**  The paper makes a strong assumption about the availability of an empirical classifier for behaviors. In L102 - 103, \"Let $B$ be such a behavior scoring function that assigns scores in the range $[0, 1]$\". While not a strong theoretical assumption, this is a very strong empirical assumption. There is evidence that automatic toxicity measures, specifically the Perspective API, are not well-calibrated (in a colloquial sense) to human judgements of toxicity [3]. Moreover, it is difficult to evaluate certain behaviors, such as deception [4] or sandbagging [5], from a single (prompt, completion) pair, as they tend to occur over an entire trajectory. Furthermore, evaluating the mere presence of other behaviors, such as hallucinations [6], is an active area of research. While I don't think the paper's assumption of the existence of $B$ detracts from the overall approach, it certainly limits its empirical application. I would appreciate if the authors could add a discussion of this limitation (possibly including some of the points I raised) into the limitations section. \n\n[1] Pan, Alexander, et al. \"Feedback loops with language models drive in-context reward hacking.\" arXiv preprint arXiv:2402.06627 (2024).\n\n[2] https://en.wikipedia.org/wiki/Tay_(chatbot)\n\n[3] Welbl, Johannes, et al. \"Challenges in detoxifying language models.\" arXiv preprint arXiv:2109.07445 (2021).\n\n[4] Hagendorff, Thilo. \"Deception abilities emerged in large language models.\" Proceedings of the National Academy of Sciences 121.24 (2024): e2317967121.\n\n[5] Perez, Ethan, et al. \"Discovering language model behaviors with model-written evaluations.\" arXiv preprint arXiv:2212.09251 (2022).\n\n[6] Tonmoy, S. M., et al. \"A comprehensive survey of hallucination mitigation techniques in large language models.\" arXiv preprint arXiv:2401.01313 (2024).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iRf5yfH5gR", "reviewer": "Reviewer_5LeM", "summary": "The study presents a method for continual Behavioral Shift Auditing (BSA) in language models (LMs), based on anytime-valid hypothesis testing. The test detects behavioral shifts through model generations, comparing outputs from baseline models to those under scrutiny. It features a configurable tolerance parameter and can detect meaningful changes in behavior distributions. The approach is valuable for AI practitioners, enabling rapid detection of behavioral shifts in deployed LMs.", "questions": "1. Section 1: \"without tipping off a potential bad actor\" Why is this a requirement of behavioral shift auditing?\n2. Section 1: What are the main contributions of your work?\n3. Section 2.2: The high-level introduction of anytime-valid hypothesis testing is not easy to understand.", "limitations": "", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "+ The paper presents an efficient method for Behavioral Shift Auditing (BSA). By building on anytime-valid hypothesis testing, the authors propose a theoretically grounded approach for detecting behavioral shifts in LLMs.\n+ The method is designed for real-world application, allowing for continual auditing through model-generated outputs.", "weaknesses": "- How the method scales with larger or more complex models is unclear.\n- The presentation is not good. Some parts are hard to follow.\n- There are some typos scattered around.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JAAggCzSvQ", "reviewer": "Reviewer_grRY", "summary": "Evaluating and detecting behavioral shifts in language models (LMs) is a crucial topic to understanding the underlying performance deviation and potential degradation in safety alignment, especially when a large number of base models are further tuned to adapt specific tasks. This work proposes a method for continual behavioral shift auditing (BSA) in LMs, which formulates the detection task as hypothesis testing to reveal relevant behavioral distribution shifts. The authors conduct experiments including both internal auditing for machine translation and external auditing for toxic generation. The experimental results show that the proposed method can effectively identify model behavioral changes within a relatively small portion of samples.", "questions": "1. How tolerance parameter $\\epsilon$ is determined in the conducted experiments, and how should this be set for different tasks?\n\n2. The experiment setup for the false positive rate requires more details.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. A new approach for detecting behavioral shifts in language models.\n2. Extensive experiments, including different models with various configurations under two use cases. \n3. The paper is overall easy to follow.", "weaknesses": "This paper tackles the LM problem from the perspective of anytime-valid hypothesis testing. The methodology design follows a rigorous manner, and I only have a few concerns regarding some technical details and presentation issues.\n\n1. Although the authors provide intuitive examples for two use cases, internal audit and external audit, at the beginning of the paper, I would recommend giving a clear formulation for such two cases before stepping into examples. The descriptions of use cases provided in Section 4.2 actually give a better understanding.\n\n2. Determining the tolerance parameter $\\epsilon$ is vital in the proposed method as it decides the maximal neural net distance allowed between distributions (model behaviors). However, throughout the methodology section and experiment design, this parameter seems manually determined according to specific tasks. The authors may elaborate more on the design of this parameter to enhance the generability of the proposed method.\n\n3. In addition, the authors mentioned in Section 4.2 that \"To determine a tolerance parameter ε for this toxicity setting, we consider triggering the test if a model differs by more than the aligned Llama3 model with different sampling parameters.\" Is there any rationale to support such a design? What do the sampling parameters stand for, and why are they important in this context? (Table 1 in Appendix A.2 do not give intuitive instructions)\n\n4. The experiment design for the false positive rate in Section 4.1 is somewhat rough and lacks details. In particular, What and in what format are the texts generated from the initial aligned models, and how are they valid to examine the false positive rate of the exact from different aspects?\n\n5. A short description of the five instruction-tuned models can benefit the readability of the paper as they are used in most experiments.\n\n\n6. For the 2nd use case (internal audit), the authors set $\\epsilon$ as \"the neural net distance between Llama3 using simple prompts, and Llama3 using few-shot prompts\". Why do you use the neural distance between models with simple prompts and few-shot prompts?\n\n7. The authors provided the link to the replication package, but the link seems incomplete.\n\n8. The presentation of the paper is mostly in a good manner. Nevertheless, there are still some typos and grammar issues (e.g., \" T This includes evaluating\" in the abstract).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)