GenCtrl -- A Formal Controllability Toolkit for Generative Models
Authors: Emily Cheng
Carmen Amo Alonso
Federico Danieli
Arno Blaas
Luca Zappella
Pau Rodriguez
Xavier Suau
Venue: ICLR 2026 Poster (2026)
Keywords: controllability
PAC
sample complexity
generative
reachability
calibration
Primary area: learning theory
TL;DR: We introduce a theoretically-grounded framework to measure the controllability of any black-box generative model, revealing that this ability is surprisingly fragile, calling for rigorous controllability analysis in the community.

Abstract: As generative models become ubiquitous, there is a critical need for fine-grained control over the generation process. Yet, while controlled generation methods from prompting to fine-tuning proliferate, a fundamental question remains unanswered: are these models truly controllable in the first place? In this work, we provide a theoretical framework to formally answer this question. Framing human-model interaction as a control process, we propose a novel algorithm to estimate the controllable sets of models in a dialogue setting. Notably, we provide formal guarantees on the estimation error as a function of sample complexity: we derive probably-approximately correct bounds for controllable set estimates that are distribution-free, employ no assumptions except for output boundedness, and work for any black-box nonlinear control system (i.e., any generative model). We empirically demonstrate the theoretical framework on different tasks in controlling dialogue processes, for both language models and text-to-image generation. Our results show that model controllability is surprisingly fragile and highly dependent on the experimental setting. This highlights the need for rigorous controllability analysis, shifting the focus from simply attempting control to first understanding its fundamental limits.

Published as a conference paper at ICLR 2026
GENCTRL- A FORMAL CONTROLLABILITY TOOLKIT FOR GENERATIVE MODELS
Emily Cheng .12, Carmen Amo Alonso Federico Danieli', Arno Blaas',
Luca Zappella', Pau Rodriguez', Xavier Suau' Apple, Universitat Pompeu Fabra, Stanford
ABSTRACT
As generative models become ubiquitous, there critical need for fine-grained control over the generation process. Yet, while controlled generation methods from prompting to fine-tuning proliferate, a fundamental question remains unan- swered: are these models truly controllable in the first place? In this work, we pro- vide a theoretical framework to formally answer this question. Framing human- model interaction as a control process, we propose novel algorithm to estimate the controllable sets of models in a dialogue setting. Notably, we provide for- mal guarantees the estimation error function of sample complexity: we derive rrobablyapproxxmmatly correct bounds for controllable set estimates that are distribution-fie employ assumptions except for output boundedness, and work for any black-box nonlinear control system (i.e., any generative model). We empirically demonstrate the theoretical framework on different tasks in controlling dialogue processes, for both language models and text-to-imaae generation. Our results show that model controllability surprisingly fragile and highly dependent on the experimental setting. This highlights the need for rigorous controllability analysis, shifting the focus from simply attempting control first understanding its fundamental limits.
INTRODUCTION
The widespread deployment of generative models has driven significant research effort into control- ling their outputs (Zhang et al., 2023). A diverse array of controlled generation methods has been developed, from prompting (Marvin et al., 2023) and finetuning (Wei et al., 2022a; Ouyang et al., 2022b) to steering towards specific styles or concepts (Li et al., 2024; Rodriguez et al., 2025b; Rim- sky et al., 2024; Wu et al., 2024). However, beneath the surface of this empirical progress lies set of foundational, yet largely unexamined, assumptions. Specifically, current methods implicitly as- sume the model is fundamentally controllable. This rests three key premises. First, they assume reachability: that a desired set of outputs is achievable using a given control mechanism and initial prompt. Second, they assume universal controllability. meaning the desired output is reachable from any initial state. Finally, they assume calibration, that is, the output is a direct function of the control variable. The absence of tools formally verify these assumptions casts uncertainty over the reliability and fundamental limits of current controlled generation techniques.
In this work, we bring these assumptions the forefront. We provide tools to verify them by turn- ing to control theory (Sontag, 1998), adapting its methods to formalize the problem for modern AI systems. Casting controlled generation as a control process, we propose framework to quantify its reachability and controllability with probabilistic guarantees. Our framework treats a genera- tive model as a black-box nonlinear control system; it is not only agnostic to model architecture but is also designed to handle discrete or continuous-value input and output spaces (e.g.. textual prompts and generations). Finally, we demonstrate our framework on a uscr-model dialogue setting, common use-case of current generative models (Google, 2024; Yang et al., 2025; OpenAI, 2023).
Our primary contributions are as follows:
"Correspondence emi 1 1yshana wwweaguph.eneeeeeeeeeeeem

---

Published as a conference paper at ICLR 2026
A formal framework for controllability. We introduce control-theoretic framework to rigor- ously define and quantify the reachable and controllable sets for any black-box system, including generative models. To the best of our knowledge, this provides the first formal language to char- acterize the operational boundaries of generative model control.
PAC algorithms for controllable set estimation. We present Monte Carlo algorithms for reach- able and controllable set estimation from black-box system interactions, along with probably- approximately correct (PAC) confidence bounds (Thm. 1 and 2). The algorithms are particularly well-suited for generative models as they are distribution-fres only assume boundedness of the target attribute, and explicitly account for the discrete bottlenecks inherent in many generative systems.
Empirical findings. We conduct a broad empirical analysis of controllability across models and tasks for text generation with LLMs, and image generation with text-to-image models (T2IMs). Leveraging our theoretical results (Thm. 1 and 2), our experiments reveal that model controlla- bility is not guaranteed. The significant heterogeneity of results across modalities, models, and tasks signals the need for case-specific controllability analyses.
An open-source toolkit. We release our framework and algorithms in PyTorch library at https: / /github..u com/ apple/ moople/mmm-eeccte to facilitate analysis of controllability by the broader research community.
This work argues for paradigm shift where the controllability of generative models is not an im- plicit assumption but an explicit subject of analysis. By providing formal tools to do so, we lay a more principled foundation for future work in controllable Al.
2 RELATED WORK
Controlling generative models Research on controlling generative model outputs follows three main paradigms: (i) Prompt engineering. e.g., in-context learning or Qhain-of-though prompting (Brown et al., 2020; Wei et al., 2022b); (ii) Finetuning, e.g., via RLHF or DPO (Ouyang et al., 2022a; Rafailov al., 2023), to align the model behavior; (iii) Representation engineering, which directly manipulates model activations (Turner et al., 2023; Li et al., 2024; Suau et al., 2024; Rodriguez et al., 2025a; Wu et al., 2024; Cheng et al., 2024). Our framework can assess any control mechanism including all examples above. In this work we mainly focus on prompting, given its widespread use.
Control theory and reachability analysis for generative models Controllability and reachabil- ity core concepts in control and dynamical systems theory (Sontag, 1998; Borrelli et al., 201 7). Despite their widespread use in other areas of engineering, such as robotics (Nakamura et al., 2025), the application of control theoretic concepts machine learning has mainly been limited to rein- forcement learning (Recht, 2019), or to analyze training dynamics (Han et al., 2019). Despite recent efforts to develop theory for generative models (Soatto al., 2023; Marchi et al., 2024), these often rely on impractical assumptions. Extending control-thcoretic concepts to large generative models poses several challenges, mostly due their nonlinear and high-dimensional nature (Bansal al., 2017). In the control literature, past works have addressed model-free reachable set estimation by proposing data-driven methods. However, existing methods do not apply to generative model out- puts for three reasons: either they (i) apply to state and not output reachability (Devonport et al., 2021; Devonport and Arcak, 2020; Dietrich et al., 2024); (ii) impose assumptions, e,g., Lipschitz- ness, on system dynamics that are not verifiable for black-box L.MSs and T2IMs (Choi et al., 2025; Park et al., 2024; Xue et al., 2020); (iii) return continuous reachable set estimates while LLMs and T2IMs' reachable sets are countable, making the estimates vacuously large (Devonport and Arcak, 2019; Sivaramakrishnan et al., 2025; Devonport and Arcak, 2020; Dietrich et al., 2024). We bridge these gaps by developing black-box sampling algorithm that estimates non-vacuous reachable and controllable sets for generative models, with probabilistic guarantees. This paves the way for rigor- ous -heoreetc analysis viable even for modern, opaque large-scale models.
PROBLEM FORMULATION
We study control of LLMs or T2IMs in dialogue setting. In Sec. 3.1 we formalize the dialogue process through a control-theoretic lens (Sontag, 1998) and in Sec. 3.2 we define reachability and controllability under this dialogue formalization.
2

---

Published as a conference paper at ICLR 2026
Inputs
BOS Opticnal inital
Write string characters
Mo
Mg
M411
Φ
h
stinn
Agr e 1
Manuremen
chirnetess again
States
Model
Readout
String length exumple:
No. that was charneters. Try again
(T-1
QI Reachability; What string lengrhs can we hit from an initial prompt?
Q2 Controllability. And from any initial prompt?
Cakes
Xy
Figure Dialogue Process. (Left) Schema of a dialogue process as a control process, showing the roles of each the concepts introduced Sec. 3.1. Theorems and tell how many inputs and initial states (respectively) should be sampled to answer Q1 and Q2 with confidence 5. (Right) Example of dialogue process.
3.1 DIALOGUE WITH A GENERATIVE MODEL. AS A CONTROL PROCESS
We study a dialogne process (DP) with generative models, i.e., LLMs and T21Ms (Fig. 1). Next, we describe the problem setup, which we formalize in the language of dynamical systems theory.
Time Domain The time domain T is the set that indexes the system's 's temporal evolution. For DP, T N is discrete, where each E IN corresponds the fth turn in the dialogue process.
State Space The state space X' is the set of possible states the system can occupy. Each x E X describes the system
given time; for = 0, X' is the space of initial string contexts, and for t 2 1, X is the space of possible generations by the model. In DP with an LLM or T2IM, &' is the set of strings and of strings and images, respectively.
Control Input Space The control input space L/ is the set of all possible inputs that can intervene the system. Each control input E denotes an intervention to the system at given time. In practice, is any degree of freedom tuned by the user system designer affect generation, e,g. a prompt, or activation steering (Rodriguez et al., 2025b; Li et al., 2024). While U can be any type of intervention, we focus prompting the primary way users interact with generative models.
Dynamics Map We frame dialogue as a control process where user inputs provide feedback to guide the model's behavior. In particular, model. An intervened model & maps prompts and any other control variables to the next model generation, defining dynamics Φ X'
treat these inputs interventions to the generative
X, where x ((I1, " Zo; ..,..,, u0)."
Controlled generation aims to achieve a desired behavior as quantified by a metric (e.g., sentiment score). This is formalized via measurement value and readout map, defined below.
Measurement Value The measurement- value space y is the set of possible measurable outputs of the system. In particular, the measurement value is how we evaluate the system's behavior a given time. This may correspond to external assessments of the model's generation, e.g., sentence length or number of cars in an image. The choice of y is task-dependent. Controlled generation, through input interventions, aims to restrict the system é measurements to desired subset )" C.
Readout Map The readout map h x X' + y maps the state at time measurement value. For DPs, h(t, 2,) =: fı(2,) is given by deterministic function that maps model generations to measurement values in y. For example, if the attribute being controlled the LLM'ss output string length, the readout i may be the Python len () function.
The above concepts permit rigorous definition of dialogue process between a user and intervened model é as a discrete-time control system (Def. 1). As shown in Fig. I, the user and model take turns conversation starting from the user's initial prompt. Subsequent user prompts are interventions that trigger another dialogue turn after the model's response.
In practice, M. come with sampling strategy for extracting g from ø. egg. greedy decoding top-p sampling. Our definition of é abstracts away the sampling strategy, which is already reflected s++1-
3

---

Published as a conference paper at ICLR 2026
Definition 1. A dialogue process is stochastic discrete-time nonlinear control system parametrized by the tuple (ø, ,,,,,, such that:
Zo initial prompt X,
(la)
It df zoi u u111....,20) uo) E X,
(lb)
V/2 h(z1) v
(1c)
for E N, where u0--t-1 are a collection of inputs from U.
3.2 REACHABILITY CONTROLLABILITY IN DIALOGUE PROCESSES
Definition establishes dialogue process as a control process. this work, we do not study the question of how to choose input from 2/ to elicit some result (controller design). Instead, we tackle the more fundamental question of what possible to achieve over time horizon (i) given specific initial state, i.e., a prompt zo: and (ii) for any or all initial states. These two are well-studied control theory: (i) pertains to the question of reachability, while (ii) addresses controllability. In what follows, formalize these definitions in the context of generative models.
Definition 2. Given a dialogue process (Def. 1) with an initial state To E X, then the reachable set of outputs for the model with dynamics é is given by
R(zo, U, = iū E y Uo..... u4-1 E 21 and yt = =ū}-
(2)
That is, given an initial state E X, the reachable output set at time is the set of all 3 reached when integrating the dynamics steps, with some control sequence too.... Ut-1 U. Def. relies on a fixed initial state 2o. It is then natural to ask what is reachable from all initial states. Namely, if all zo E X' can reach all values in the set y C y using some input sequence in U, then the system controllable y" (Sontag, 1998). Formally,
Definition 3. A system is controllable on y" yiif 2 such that ((=o,U,,) y" V 2o EX
differentiating feature of LLMs and T2IMs is the discrete nature of string prompts, which makes the reachable countable (see App. E for discussion and formal proofs). We refer to this as the discrete bottleneck. This property affects both reachability (Def. 2) and controllability (Def. 3), which are preconditions for system control.
In Sec. 4 we propose coarse-grained reachability as a relaxation of the problem for DPs with discrete bottleneck. In Sec. 4.1 derive PAC bound to approximate the reachable set of given initial state, then in Sec. 5 present PAC bound to estimate the controllable set of the system. Finally, in Sec. we use our theory to estimate reachable and controllable sets of current LLMs and T2IMs, finding they are seldom controllable on simple tasks.
4 MONTE CARLO REACHABILITY FOR DIALOGUE PROCESSES
Generative models nonlinear, nigh-ddmensional, and have unknown dynamics. Therefore, it is nontrivial to analytically derive the reachable set. Hence, we rely on statistical guarantees that only depend on the system's empirical trajectories. In Sec. 4.1 we tailor the definition of reachability to the discrete bottleneck of LLMs and T2IMs. Then, we propose a sample complexity bound for the reachable in Sec. 4.2, which we use in Sec. 4.3 in an algorithm that computes an croorbounded reachable set approximation. Since reachable set estimates will rely on sampling, we first set up probabilistic notion of the control process, as in prior work (Devonport and Arcak, 2019):
Xo ~ Po. where Po : No - [0, 1] a probability density over the set of initial states X C X. Uc Pusts where Pu,t 244 + [0, 1] is probability density over the input space 2/ at time > 0.
X, = ott; Xo. U Uo..--1) ~ Pt. where p: : X, - [o, 1] is the pushforward of pa and p.. under
system dynamics, for t 0.
Y, = h(X,, U.) ~ Pa.s. where Pv, : y + [0. 1] is the probability density over measurements at time 2 0.

---

Published as a conference paper at ICLR 2026
4.1 COARSE- GRAIIND REACHABILITY OVERCOMES THE DISCRETE BOTTLENECK
The discrete bottleneck in DPs impacts the reachable set: even if the measurement value is continuous-valued (e.g,, continuous text formality score), the true reachable set can be countable. This means we cannot directly apply existing reachable set estimators that return continuous sets (Devonport and Arcak, 2019; Alanwar et al., 2021), as the returned sets would be vacuously large.
A workaround is coarse-grained reachability, a simplified but faithful discrete abstraction of the original measurements (Ren and Dimarogonas, 2020). We consider two cases: (i) categorical mea- surements, e.g., ftoxic, non-toxic), which require no abstraction (they are already discrete); (ii) continuous-valued measurements, e.g., a formality score in [0. 1]. For the latter, we relax the reach- ability problem to a "y-quantized one: instead of exactly reaching values in V. we allow error margin y (Def. 4). For instance, rather than requiring 0.3 exactly, we aim for 0.3 + 0.05.
Definition 4 (ry-quantized Reachable Set). Let (Ø, X,, h") be dialogue process per Def. 1. The y-quantized reachable output set at time t from 2o is given by
R7 (ra,, U,,, (ī ) 3 ,,,
U E l and I| - o0 5
where || llse refers to the oc-norm and E RR.??
For continuous-valien valued measurements, our goal now to estimate R Recovering RD guarantees each point in RD is at most away from some point in the true reachable set. For brevity, we refer to reachable set estimation over both categorical or quantized y as coarse-grained reachability.
4.2 SAMPLE COMPLEXITY BOUND FOR MARSE--GRANEL REACHABLE SET ESTIMATION
Our goal is PAC bound that estimates R7. To build up to this bound, first find suitable quantization yr, of the measurement-vallee space y, requiring that y, has finite cardinality, Le., that y is bounded. If y is already categorical, then D/- = V. If is continuous-valuee cover of y using oc-balls of radius 7/2, where is the user-defined error in Def. 4. Let N = 1 be the number of bins in the discretization: [ if categorical and the covering number of y otherwise. Since our algorithm will be based on sampling, we need a probabilistic notion of error, which we define using V/- In particular, the user sets a small threshold p E (0, 1) that tunes the precision of the estimate. Then, to construct the p-upproximation of R( U, (hereon Rt), we keep all points lying in the bins Ybin E Y/ with probability mass Py.t 2 p, any bin with density p is considered negligible and discarded. We formally define p-approximation below:
V, a minimal
Definition (p-Approximate Measurement Value), Alternative and equivalent definitions are given for the categorical and 7-quantized cases.
Categorical: The (p) approximatio of R4 is Re.p := fy E, Pq.(((u)) 2 p).
>-Quantized: The (p, approxxiinttoonnn of R, is R7,p = {8 Bybi Dg y E
Vbin
(y.c()hin) 2 p}. where Yhin V/ is an ba-ball of radius 7/2 in Y's cover.
Now, using (7) we state the sample complexity bound. For brevity, we combine categorical and quantized versions of the result into the below Theorem (see App. F.1 for separate statements):
Theorem I: Sample complexity bound, coarse-grained reachability
Let Dr, have finite cardinality N. Let m be the number of i.i.d. samples drawn from Y,, iee. {y M with each y e y Vi. Fix 8 E (0, 1). If
m 2 max (N, N, log(ō/N)
(4)
log -
then PP (
t.p C RR) 2 1 - 8, where R. = (vi Mi if categorical, and R. = U Bac (Vi, 7)
quantized (Bxo (3). is the bo-ball centered at y with radius 7). Proof in App. F.I.
II practice, y E RR can consist of several unrelated attributes 1 1... like formality and sentence length. The co-norm makes the theory already hold for multiple orthogonal readout dimensions.
5

---

Published as a conference paper at ICLR 2026
Theorem 1 shows that, after samples, one is confident that all reachable bins in yg have been covered; the confidence is given by 1 - 6, and the precision of the estimate by p. Then, if a target set y" is not included in R2, that y is unreachable with probability 2 1 - 6. By construction, R2 is a right estimate of the true R, all points in R2 are guaranteed to lie within y of reached point.
4.3 MONTE CARLO SAMPLING ALGORITHM FOR REACHABILITY IN DIALOGUE PROCESSES
Theorem provides the foundation for a Monte Carlo algorithm to estimate a DP's reachable sets. The procedure to build the reachable set estimate R2 is stated in Alg. la. It shares the same structure for both discrete and continuous cases with case-specific post-processing of the samples. Under sample size requirements from Thm. 1, Alg. la guarantees P tp C Ri ( 2 1-6.. Crucially, this guarantee uses no information about the underlying model: the number of trajectories m does rely on whether the model is stochastic nor the timestep. In App. F.4, we empirically validate the bound in Thm. 1. In App. F.3, we analyze m 's dependence on parameters, finding m O(N log N).
MONTE CARLO CONTROLLABILITY FOR DIALOGUE PROCESSES
Having established reachable sets for individual initial states (Alg. 1a), we now quantify controlla- bility of the system. Since controllability entails reaching a subset y" y from all initial states, one can naturally estimate controllable sets by intersecting the reachable sets of k initial states. To mea- sure how controllable the system is, we first define partial controllability (Sontag, 1998), then derive a PAC bound (Thm. 2) on the DP's partially controllable set. This yields Alg. 1b, an algorithm for controllable set estimation. As Alg. 1b will rely on reachable set estimates that are p-approximate (Def. 5), we port over the notion of p-approximate for controllability. Since we focus on the discrete bottleneck, we directly consider the discretization V/, of the measurement space y.
To formalize partial controllability. we introduce measure quantifies the true proportion of initial states that p-approximately reach Ybin Yg. Namely, Ybin + Pao-po [Mbin E (x0)) [I We use μ to define the a-controllable set CT: the set of bins in V/ that are p-approximately reached by a large proportion (2 1 a-a) aa) of initial states 2o. Formally,
)'s [0, 1], where (((bin)
Definition 6 (a-controllable set). Given control system (Ø. (K,,, the a-controllable set C y at time f is given by C. = (yhin (A(bin) 2 1 a).
Example 5.1: a-controllable set
Consider an initial state space (BOS, Hi, Hello, Hey} with possible inputs "Generate a sen- tence of length {1... ... 10)." "" Say each outcome in length=(2, 3, 5, 8) is p-approximately reached by exactly states (75% of states, they don't 't have to be the same ones each time). Then, for 0.25 (i.e., 25%), the a-controllable set CP (Def. 6) is 12ngth=(2.3,55) 3, 8).
We aim estimate the true o-controllable set C. by sampling k initial states, sampling their reach- able sets, and taking their intersection: ć₂ = n Ra(a)))) (.)) The key question is how many initial state samples k are needed to approximate C with high confidence. To answer the question, we need a notion of approximation error between the estimated é, and the target Cj. A natural choice is the measure under of false positives ć₁ with respect to CP, formally, μ(C, (C7). With enough samples, we can bound paC, (CC) below a desired e E (0, 1) with high probability:
'For deterministic Φ, R((")) reflects exact (or within-7) controllability, and p-approximation only captures sampling uncertainty. For stochastic c. R((") reflects controllability in-distribution: p reflects stochasticity of both sampling and ø. See App. for discussion.
"There are false negatives č. is a strict overapproximation of C2 as é. C by construction taking successive interscctions shrinks the running controllable set until convergence.

---

Published as a conference paper at ICLR 2026
Algorithm la: Monte Carlo Reachability
Algorithm 1b: Monte Carlo Controllability
Estimation for Dialogue Processes
Estimation for Dialogue Process
Input: Initial state 2o, confidence level,
Input: Confidence level, partial controllability.
probability threshold (ó, E (0, 1)2
error. probability threshold
Input: If qqquatizze:: quantization parameter
(8, a, c, E (0, 1)
7
Input: If q-quantizz::: quantization parameter
Compute sample size m;
Thm.
Y
// Sample generation
Compute initial state k, reachable set sample sizes m;
for to do
I/ Thm. see App. H
Integrate DP to time 2e (20, u)
I/ Sample generation
sampling input sequence u ~ pus
2 for to do
Evaluate output: V/ = h(x,, u4):
Sample initial state: zip pu:
return R2 = - B (ul" 7)
Sample reachable set: R2 (m) (00 (0)
// Alg. la
Y-quantized reachability else {vi mmi
(4)
return Empirical controllable set:
C = Ni )=(=)));;
Algorithm 1: Pseudocode for Monte Carlo reachability (left) and controllability (right) estimation.
Theorem 2: Sample complexity bound, controllable set
Fix dc, E (0, 1) (and R4). Let n be the confidence from Thm. 1. Given k initial kk i.d. states {z) Ji=1 ~ po and their reachable set estimates (z) ) if
2 log cōc
(5)
log(1 11 a)
then P(p(Ci (C7) < 2 (1 (1- where C2 = N ). Proof in App. H.
only needing Jly to be finite and consistent over initial states. Like Thm. 1, Thm. 2 is applicable to any control system, including stochastic ones. See App. for how to use Thm. and 2 for hypothesis testing.
Theorem 2 is distribution-fii
Algorithm 1b outlines the procedure to estimate the controllable by intersecting the reachable of k initial states. Note that in Thm. 2, the overall confidence 1 1 = (1 - 6c) - 5²)² depends on confidences 5n on each reachable set and &c related to sampling enough initial states. Given o, automatically select dc and oR to minimize the total samples n = m k. see App. H for details.
Monte Carlo sampling algorithm for controllable sets
6 EXPERIMENTS
While Thm. and 2 apply to any control system, we demonstrate them on contemporary LLMs and T2IMs for varying task complexity, prompting interventions, and model sizes. For space reasons, we show results for greedily decoded LLMs and T2IMs here, with stochastic results in App. A.3.
Evaluation Metries We collect metrics on the estimated controllable set to inform potential con- troller design. A controller maps desired y² y to control input that drives 2o y". To test existence, measure the estimated Ć 's coverage of y, cvg & |} DynCl//D With confidence cvg implies a controller exists from 1 a of initial states to a cvg- fraction of y (Def. 3). For each To- also assess its functional properties p(uo, y1) for monotonicity, Pearson R(uo, 31) for linearity, and MAE(uo. Ut) for identity. Ideally, metrics are stable across different initial prompts zoi while report them, we leave controller design for future work as our focus is on controllability.
[0, 1] (t better).
proxies for calibration: Spearman
6.1 CONTROLLING FORMALITY IN LLMS
Setup We request an LLM to generate text with a given formality, engaging the LLM in dialogue until the goal is reached. We test ()-hoot and 5-shot prompting for the initial user request up

---

Published as a conference paper at ICLR 2026
Ma U-shoot
Quend-4B (sshot
Gemman--BB Osehot
Toxt foruslit (po)
2.00
7,0
3.0
0.5-
0.5
0.5-
Trijetooiis
0.0
0.0
0.0
12464
Dialogue tarn (9)
Dialogue turn (8)
3
Dialoggue turn (C
k- 61
Text Seundity (ss)
b-abot
p-ahot
G-ihot
Controilable
2.0
2.0
2.0
condidence = 0.06
Not coutrollalle
0.5
6.5
0.5
J
Dialogue
Dinlogue vurn
Dialogue tur
1.6
1.0
1.3
1.0
1.8
0.--
0.8-
Cr 100
Qawrd itl
Grunnad-tis
0.7
0.-
0.2
Figure 2: (Top, Middle) 5-turn Dialogue Process trajectories for formality task. Controllable set dynamics are shown for (left to right) models SmoILM3-3B, Qwen3-4B, and Gemma3-4B on a text formality control task, using 0-shot (top) and 5-shot (bottom) prompting as the initial input. Each linecolor represents different initial state. The a-controllable sets (a 0.1) shown in yellow, where full controllability (best-case) would be seen by an entirely yellow f While none of the models are fully controllable O-shot, and all show formal bias, Gemma3-4p -4B and Qwen3-4B are the most controllable with 5 shots by 5 (confidence = 0.05). (Bottom) Summarized metrics for 5-shot at t = 5. The left figure shows the final output in the DP as a function of the requested input. The next figures show violin plots of each metric on the formality task, where each point is metrie for single zo, demonstrating Qwen3-4B the most controllable and faithful the user request for this setting (evg = 1.0, median MAF 0.09).
Unif(0,1), 1). Subsequent feedback #,21 deterministically fills a template with the previous output Ut-1- For instance, if the last scored formality incorrect, the user feedback is " "Your answer was too
I asked for a story of formality up. and you produced a story of formality Mt-1" See App. J.1 for all prompt templates. We test three instruction-tuned LLMs: SmolLM3-3B (HuggingFace, 2025), Qwen3-4B (Yang et al., 2025), Gemma3-4B (Team et al., 2025).
As initial states sample zo from -istral-7B--O.1--nae I-Instruct (Jiang et al., 2023), asking for a con- versation opening. While Wolf et al. (2024) find that, when prompts semantically conflict with the task, LLMs need longer interaction to succeed, our setting shielded from this effect as our xo are unrelated to task semantics. As a readout map we use formality neural scorer (Babakov et al., 2023)². We use the same controllability and reachability hyperparameters for all runs, see Tab. J.4.
Controllability is not guaranteed Fig. 2 (top, middle) show formality trajectories for a 5-turn DP. We test 0-shot (top) and 5-shot (middle) prompting, where the a = 0.1 controllable set is highlighted in yellow. In the 0-shot setting, none of the models are fully controllable within 5 steps, although we observe growth of the controllable set. In the 5-shot setting, Qwen3-4B and Gemma3-4B reach full controllability by t 5 with confidence å 0.05. On this task, and with the given control, SmoLLM3-3B is not controllable. The left-most plot in Fig. 2-(bottom) shows the observed outputs as function of the inputs, evidencing the different model behaviors as well as the poor faithfulness of outputs with respect to inputs on this task. The remaining four figures show averaged metrics across initial states. Both Qwen3-4B and Gemma3-4B are controllable (evg= 1.0). However, these models show differences in calibration, represented by p, It and MAE, Qwen3-4B being the most faithful to user requests.
Examples or feedback? Whether feedback (0-shot, 5 turns) or examples (5-shots, turn) is more important for controllability highly depends on the model. Indeed, for Qwen3-4B and Gemma3-4B, examples matter more than feedback, seen by wider yellow coverage for i = 1 (5-shot prompting, middle row) than for (0-shot prompting, top row); the opposite pattern holds for SmolLM3-3B.
'ee verify that the requested formality values reachable with some input. E.g., we manually ensured there are solutions "Generate story with formality ss 0.0" (as well as 1.0).
8

---

Published as a conference paper at ICLR 2026
0.4
ConnelleNeee set Vi.
0.6
MAR (1)
0.5
04
0.
0.4
0.-
0.
0.2
8.0
0.0
0.0
0.0
0.68
Model
Figure 3: Larger models are more controllable and calibrated on text formality. For Qwen sizes ranging from 0.6B to 14B (x-axis), we requested text formalities ranging in [0, 1], with 0-shot prompting and one dialogue turn. While controllability (right) increases reliably up to 14B, the correlation (left plots) between the user request and the output formality, given plateaus around 8B. All calibration metrics (R, p. MAE) increase most drastically smaller sizes (0.6B -+1.BB and appear saturate for larger sizes.
1.0
1.
Caundille Ca
15
FLIXX-
Outget
066
06-
PIUXX
0.4-
04
1MMU8
6.0
0.
0.0
Figure 4: Object generation task for T2IMs. We prompt the model with "White background. [N] jsb""s." with N {0... 2 and obj = {80 COCO classes}. The left figure shows the average outpur object count function of the requested input. The next figures contain violin plots of cach evaluation metric Vobj. evidencing differences in models. Notably, FLUX-s achieves median p, >> 0.9 and median MAE 3.52, showing much better controllability and faithfulness than the rest.
Dialogue results in overshoots The trajectories in Fig. show strong formality overshoots with the favorable feedback given, which contains both the requested formality un and the produced formality -- One would expect the model converge to the target formality, however it is not the case. The overshoot effect is more visible in the 5-shot setting both for Qwen3-4B and Gemma3-4B.
Larger models more controllable Controllabilitt closely relates the expressivity of the system (Sontag, 1996). As larger LLMs are more behaviorally and representationally expressive (Biderman al., 2024; Cheng al., 2025), we tested whether they are more controllable. Fig. shows the controllability of Qwen sizes from 0.6B to 14B (0-shot, 5 turns), where indeed controlla- bility (cvg) and calibration the user request (p. R. MAE) increase with size. However, calibration metrics saturate 4B parameters, suggesting performance gains on text formality are most salient at small sizes. Overall, calibration poor even for the 14B model, with an MAE Rs 0.25 from the re- quest. For reference, the error tolerance for this experiment is ry 0.1, and uniformly generated text formalities would have MAE= X - Y|] where and are both distributed as Unif(0,, 1).
6.2 CONTROLLING NUMBER OF OBJECTS IN T2Is
Setup In this task, query T2IMs to generate "White background. [N] [obj]s." with N Unif 20} and for all obj E (80 COCO classes). As initial states we use zip BOS, and as readout map we use 0-shot object detector (Minderer et al., 2023).
Reachability and faithfulness are sensitive to task semantics Figure shows (left) the input- output faithfulness averaged across the 80 different objects requested, and the averaged statistics across objects in the other plots. Note the large variance in all plots, showing that the requested object biases how desired outputs and actual generated values correlated. In general, we ob- serve that controlling the number of objects is harder than expected, with FLUX-s showing the best behavior in this setting with MAE of 3.52 and strong calibration shown by p, R > 0.9.
6.3 ADDITIONAL EXPERIMENTAL RESULTS
In addition to formality and object count (Sec. 6.1 and 6.2), we test controllability for more tasks, shown in App. A for space reasons. For LLMs, the tasks generate (i) positive (even, odd} integer; (ii) string of length {1-. (1...10} and (iii) sentence whose average word length is [2.0, 10.0]. For T2IMs, the tasks (iv) white background with object at specific location in (top left, top

---

Published as a conference paper at ICLR 2026
right, bottom left, bottom right, center) for all 80 MS-COCO object categories, and (v) an image with [0, 1.0] saturation. Note that tasks (i,ii,iv) are discrete and (iii,v) are quantized.
These experiments corroborate the conclusions reached in Sec. 6.1 and 6.2. Controllability (and calibration) are highly task dependent, for example Gemma3-4B achieves strong results on (i,i)) with near perfect calibration, while showing poor calibration for formality. Task (iii) appears harder, with Gemma3-4B showing much better controllability, as evidenced by a stark difference in trajectories in Fig. A. Controllability and calibration also improve with model size, shown for the string length and average word length tasks (i, iii) in Fig. A.8 and A.9. Interestingly, whether the LLM was greedily decoded or sampled did not affect the high-level trends, see App. A.3. T2IMs show poor controllability of object location (iv), worse than object count, while image saturation (v) is not controllable. App. J.1 contains all prompt templates and App. K an example of our toolkit code.
Summary of Experimental Findings Our extensive experiments reveal that controllability is fragile and inconsistent property in modern generative models. This fragility is evident even on the seemingly simple tasks we designed, which represent a lower bound on the complexity of real- world applications. We found that no single model or prompting strategy guarantees control across all tasks. For instance, while Gemma3-4B and Qwen3-4B achieved full controllability on the for- mality task with 5-shot examples, they required iterative dialogue to improve, and SmolLM3333 remained uncontrollable. Conversely, for T2IMs, even the best-performing model (FLUX-s) exhib- ited significant errors in object counting and failed to control for object location, demonstrating that controllability is highly sensitive to both task semantics and the chosen model. Ultimately, these results validate the utility of our proposed framework for identifying controllability failures.
7 LIMITATIONS AND CONCLUSION
Limitations and Future Work Guarantees from the proposed framework hold for a given control system, whose input distribution, readout map, and initial state distribution are defined by the prac- titioner. Guarantees do not transfer to new choices of these variables. Thus, task-specific takeaways from our experiments are only relevant the setting in that task. The practitioner is responsible for their choice of input distribution, readout map, and initial state distribution for their use-case.
Our theoretical framework rigorously estimates DP's controllable set with guarantees (Theorems and 2). As practical byproduct, Algorithms la and 1b return all sampled trajectories, including inputs, states, and measurement values. This reveals which inputs elicit which measurement values, as well as regions that are systematically uncontrollable (Section 6). However, because we treat the models as black-box, our framework does not provide interpretability tools that causally diagnose controllability failures model internals: this is outside the scope of our work.
In Theorem 1, the sample complexity to estimate the reachable set scales with its covering number N. When estimating joint reachability over d attributes, the covering number N of the reachable set grows not with the attribute space's 's extrinsic dimension d, but rather its intrinsic dimension (Kégl, 2002), which is typically lower in practice. This alleviates sample complexity explosion due to the curse of dimensionality to some extent. Still, Theorem does not scale well when estimating intrinsically complex reachable sets with high precision. This remains important open problem not only for our setting, but for high-dimensional reachable set estimation in general (Bansal and Tomlin, 2020; Lin and Bansal, 2023; Devonport and Arcak, 2020). As workaround, we recommend constructing a quantization V/ of the attribute space with tractable cardinality N; for instance, Y4 corresponding to binary safe vs. unsafe regions of attribute space (N- = 2).
Conclusion This work challenges the implicit assumption of controllability that is fundamental to current efforts in generative AI. We introduce formal framework, grounded in control theory, and a practical algorithm to estimate the controllable sets of any black-box model with statistical guarantees. Our empirical analysis reveals that controllability is not given but a fragile property, highly dependent on the model, task, and initial state (prompt). We therefore argue for paradigm shift where controllability moves from an implicit assumption to an explicit object of analysis. By providing an analysis toolkit, we set grounds for developing safer and more reliable controllable AL. We foresee potential uses in generative model safety and compliance that include rigorously com- paring different control mechanisms, estimating reachable sets under adversarial inputs, enforcing controllability during training, and accounting in policy and deployment.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
To ensure reproducibility, we base our work on open source libraries and public datasets, we will also make our code publicly available both in GitHub and as a python package. addition, we provide the main algorithms for Monte Carlo reachability and controllability in Alg. 1, details on the experimental setup in App. input distributions related to each task App. J.1, and sample of the python package in App. K.
ETHICS STATEMENT
We adhere to ICLR's Code of Ethics. Our work introduces a theoretical framework and a codebase to assess the controllability of generative models. Our results show that model controllability is fragile and it calls to switch focus from simply controlling to first understanding their fundamental limits. We believe such shift in mindset is necessary to attain more robust, transparent, and safe generative models.
REFERENCES
A. Alanwar, A. Koch, F. Allgöwer, and K. H. Johansson. aata-driven reachability analysis using matrix zonotopes. In Jadbabaie, Lygeros, G. J. Pappas, A. Parril,, B. Recht, C. J. Tomlin, and M. N. Zeilinger, editors, Proceedings of the 3rd Conference on Learning for Dynamics and Control, volume 144 of Proceedings of Machine Learning Research, pages 163-175. PMLR, 08 June 2021. URL https: : tttpttts//pppoceeieiieee proceedings. mlr press// v144/ a alawwarllall.ammmaaaamaaaaaaaala html.
N. Babakov, D. Dale, I. Gusev, Krotova, and A. Panchenko. Don't lose the message while para- phrasing: A study content preserving style transfer. In E. Métais, F. Meziane, V. Sugumaran, Manning, and S. Reiff-Marganice, editors, Natural Language Processing and Information Systems, pages 47-61, 61, Cham, 2023. Springer Nature Switzerland. ISBN 978 978-3-031-35320--8
Bansal and C. Tomlin. Deepreach: deep learning approach to high-dimensional reachability. 2021 IEEE International Conference Robotics and Automation (ICRA), pages 1817-1824, 2020. URL https:://// wwmppp.ssmmaiicccollaa.. www.or////oooooooo
S. Bansal, M. Chen, S. Herbert, and C. J. Tomlin. Hamilton-jacobi reachability: A brief overview and recent advances. In 2017 IEEE 56th Annual Conference on Decision and Control (CDC), pages 2242-2253. IEEE, 2017.
Biderman, Schoelkopf, L. Sutawika, L. Gao, J. Tow, B. Abbasi, A. F. Aji, S. Ammana- manchi, S. Black, J. Clive, A. DiPofi, J. Etxaniz, B. Fattori, Z. Forde, C. Foster, M. Jaiswal, W. Y. Lee, H. Li, C. Lovering, N. Muennighoff, E. Pavlick, J. Phang, A. Skowron, S. Tan, X. Tang, A. Wang. G. Winata, F. Yvon, and A. Zou. Lessons from the trenches on reproducible evaluation of language models. ArXiv, abs/2405, 145822 2024. URL https: // /api. www.stetttttteeeeeeeeeeee www.nticscoooa............ :ttp://oooSD:::2999900
F. Borrlli, A. Bemporad, and M. Morari. Predictive control for linear and hybrid systems. Cam- bridge University Press, 2017.
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, Askell, Agarwal, Herbert- Voss, G. Krueger, T. Henighan, Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei. Language mod- els are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, editors, Advances Neural Information Processsing Systems, volume 33, pages 1877-1901. Cur- Associates, Inc., 2020. URL https: httphttpss/pppceeeeee....e..eoa.
files/
E. Cheng. M. Baroni, and C. A. Alonso. Linearly controlled language generation with performative guarantees. In MINT: Foundation Model Interventions, 2024. URL https: https::/openzee/ //wpeneeieee.... net/ met/forum? d=V2xBBD1

---

Published as a conference paper at ICLR 2026
E. Cheng, D. Doimo, C. Kervadec, Macocco, L. Yu, A. Laio, and M. Baroni. Emergence of a Nigh-Dimensional Abstraction Phase in Language Transformers. In Interaatioaal Confer- Learning Representations, 2025. URL https: http:////peeeeeeee.eeee. net nwtfforuffff OfD3i IBh
/
J. J. Choi, C. A. Strong, K. Sreenath, N. Cho, and C. J. Tomlin. Data-driven hamiltonian for direct construction of safe set from trajectory data, 2025. URL 032.3..
A. Devonport and M. Arcak. Process Classification and Monte Carlo Methods, Oct. 2019. URL hwwp:ttttp/... http:://axxvvvorro abs/19100 O. 02500. 191vv:191.002500 [eess].
Reachable Set Computation using Adaptive Gaussian
A. Devonport and M. Areak. Estimating reachable sets with scenario optimization. A. M. Bayen, A. Jadbabaie, G. Pappas, P. A. Parrilo, B. Recht, C. Tomlin, and M. Zeilinger, edi- tors, Proceedings of the 2nd Conference on Leurning for Dynamics and Control, volume 120 of Proceedings of Machine Learning Research, pages 75-88 75-84. PMLR, 10-11 Jun 2020. URL https: httpss//ppcceednggs... www..pressse www.pppwwsssi...eeeeee.... w html.
A. Devonport, A. Saoud, and M. Arcak. Symbolic abstractions from data: A pac learning approach. In 2021 60th IEEE Conference on Decision and Control Control (CDC), page 599-604 IEEE Press, 2021. doi: 10. 10.11090 (L019//CDCC4442222299633333 2021 00....66535 9683316 6.
URL https: http://ooooooogoo
E. Dietrich, A. Devonport, and M. Arcak. Nonconvex scenario optimization for data-driven reach- ability. In A. Abate, M. Cannon, K. Margellos. and A. Papachristoddullou editors, Proceed- ings of the 6th Annual Learning for Dynamics &amp; Control Conference, volume 242 of Proceedings of Machine Learning Research, pages 514-527. PMLR, 15-17 Jul 2024. URL wwr.essmm....... https: http:::/pprcceeiii /ttpp/eecceegg...c//////// mll. preessppeess/ee /ttp///22///eee22//eee2
E. Dietrich, R. Devonport, S. Tu, and M. Arcak. Data-driven reachabiiiy with scenario optimization and the holdout method, 2025. URL
Google. Gemma open models, 2024. URL httpp::/aal..oooll..eee/////////
J. Han, Q. Li, et al. A mean-field field optimal control formulation of deep learning. Research in the 2019.
Mathematical Sciences,
HuggingFace, SmolLM3: smol 3B, multilingual, long-context reasoner smollm3.com https: //smollm3 ctm. 2025, [Accessed 09-09-2025).
A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, de las Casas, Bressand, G. Lengyel, G. Lample. L. Saulnier, L. R. Lavaud, M.-. Lachaux, P. Stock, T. L. Scao, T. Lavril, T. Wang, T. Lacroix, and W. E. Sayed. Mistral 7b, 2023.
B. Kégl. Intrinsic dimension estimation using packing numbers. In Neural Information Processing Systems, 2002. URL https::t////..... //pp//app..p///// semant weteiicchhlaaaa org Corg/CorpusIDD 6008 6008434.
K. Li, O. Patel. F. Vićgas, H. Pfister, and M. Wattenberg. Inference-time intervention: Eliciting truthful answers from a language model. Advances in Neural Information Processing Systems, 36, 2024.
A. Lin and S. Bansal. Verification neural reachable tubes via scenario optimization and conformal prediction. In Conference on Learning for Dynamics & Control, 2023. URL ht /api. semant: cschol ar.
ID: 266210267
M. Marchi, S. Soatto, P. Chaudhari, and P. Tabuada. Heat death of generative models in closed-loop learning. In 2024 IEEE 63rd Conference on Decision and Control (CDC), pages 1524-1530. IEEE, 2024.
G. Marvin, N. Hellen, D. Jjingo, and J. Nakatumba- Nabende. Prompt engineering in large language models. In International conference on data intelligence and cognitive informatics, pages 387- 402. Springer, 2023.
12

---

Published as a conference paper at ICLR 2026
M. Minderer, A. A. Gritsenko, and N. Houlsby. Scaling open-vocabulary object detection. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https: // openreview wwt. forrmm 2idmm/idmmmmmsssss
K. Nakamura, L. Peters, and A. Bajesy. Generalizing safety beyond collision-avoidance via latent- space reachability analysis. arXiv preprint urXiv:2502..0900 2025.
OpenAl. Gpt-4 technical report. arXiv preprint arXiv:2303.08777 2023.
L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, Schulman, Hilton, Kelton, L. Miller, Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe. Training language models to follow instructions with human feedback. arXiv preprint vrXiv22233.0215. 2022a.
L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, Schulman, J. Hilton, Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, 2022b.
H. Park, V. Vijay, and I. Hwang. Data-driven reachability analysis for nonlinear systems. IEEE Control Systems Letters, 8:2661-2666, 2024. doi: 10. 10.1109/CCSSS.2244.33
I. Pinelis. Optimum Bounds for the Distributions of Martingales in Banach Spaces. The Annals of Probability, 22(4):1679 1706, 1994. doi: 10.1214/aop/1 Pttop//11769884777 URL sestps://dol org/ 10. 1214 aop 66878678777 Publisher: Institute of Mathematical Statistics.
R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn. Direct preference optimization: Your language model is secretly reward model. Advances in neural information processing systems, 15:53788-53744 2023.
B. Recht. A tour of reinforcement learning: The view from continuous control. Annual Review of Control, Robotics, and Autonomous Systems, 2(1):253-273) 2019.
W. Ren and D. V. Dimarogonas. Symbolic abstractions for nonlinear control systems via feedback refinement relation. Automatica, 114: 108828, 2020. ISSN 0005-1098. doi: https://do..or/// 1016//.automattcaa.22..... 108828. URL https: wttp://wwwiienniiieeeeeeete/eeee htwpiirrrrttteteeeeeeoeomeeeeeee science/ aaticcle/pi//0005119220022
N. Rimsky, N. Gabrieli, J. Schulz, M. Tong, E. Hubinger, and A. Turner. Steering Ilama 2 via contrastive activation addition. L.-W. Ku, A. Martins, and V. Srikumar, editors, Proceedings of the 62nd Anmual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15504-15522 Bangkok, Thailand, Aug. 2024. Association for Computational URL https://aclanthollee
Linguisties. doi: 10. 10.11665 118655/v112222 aclngg...- 2024. acl- 828/-
org/
P. Rodriguez, A. Blaas, M. Klein, L. Zappella, N. Apostoloff, M. Cuturi, and X. Suau. Controlling language and diffusion models by transporting activations, 2025a.
P. Rodriguez, M. Klein, E. Gualdoni, A. B. adn Luca Zappella, M. Cuturi, and X. Suau. End-to-end learning of sparse interventions on activations to steer generation, 2025b.
V. Sivaramakrishnan. K. C. Kalagarla, R. Devonport, J. Pilipovsky. P. Tsiotras, and M. Oishi. Saver: A toolbox for sampling- based, probabilistic verification of neural networks. In Proceedings of the 28th ACM International Conference on Hybrid Systems: Computation and Control, HSCC '25, New York, NY, USA, 2025. Association for Computing Machinery. ISBN 979840071 1044., doi: 10.1145/3716863.377 1818663.711045 URL https: //o.i org/10. 11111111111111 3718045.
S. Soatto, P. Tabuada, P. Chaudhari, and T. Y. Liu. Taming ai bots: Controllability of neural states in large language models. arXiv preprint arXiv:2305. 18449, 2023.
E. Sontag. Kalman's controllability rank condition: From linear to nonlinear. 02 1996. doi: 10. 00071978-3-662-08544 25.
13

---

Published as a conference paper at ICLR 2026
E. Sontag, Mathematical Control Theory: Deterministic Finite-Dimensional Systems. 01 1998. doi: 10.1007/978- 1-4612-0577-7.
X. Suau, P. Delobelle, K. Metcalf, A. Joulin, N. Apostoloff, L. Zappella, and P. Rodriguez. Whis- pering experts: Neural interventions for toxicity mitigation in language models. In Forty-first International Conference on Machine Learning, 2024. URL https: /ttp:////ppeeeeeeeeeeee nett/
G. Team, A. Kamath, J. Ferret, S. Pathak, N. Vieillard, R. Merhej, S. Perrin, T. Matejovicova, A. Ramé, M. Rivière, L. Rouillard, T. Mesnard, G. Cideron, bastien Grill, S. Ramos, E. Yvinec, M. Casbon, Pot, Penchev, G. Liu, F. Visin, K. Kenealy, Beyer, X. Zhai, A. Tsitsulin, R. Busa-Fekete, A. Feng, N. Sachdeva, B. Coleman, Y. Gao, B. Mustafa, I. Barr, E. Parisotto,
D. Tian, M. Eyal. C. Cherry, J.-T. Peter, D. Sinopalnikov. S. Bhupatiraju. R. Agarwal, M. Kazemi, D. Malkin, R. Kumar, D. Vilar, I. Brusilovsky. J. Luo, A. Steiner, A. Friesen, A. Sharma, A. Sharma, M. Gilady, A. Goedeckemeyer, A. Saade, A. Feng, A. Kolesnikov, A. Bendebury, A. Abdagic, A. Vadi, A. György, A. S. Pinto, A. Das, A. Bapna, A. Miech, A. Yang, A. Paterson, A. Shenoy, A. Chakrabarti, B. Piot, B. Wu, B. Shahriari, B. Petrini, C. Chen, C. L. Lan, C. A. Choquette-Choo C. Carey, C. Brick, D. Deutsch, D. Eisenbud, D. Cattle, D. Cheng, D. Paparas, S. Sreepathihalli, D. Reid, D. Tran, D. Zelle, E. Noland, E. Huizenga, E. Kharitonov, F. Liu, G. Amirkhanyan, G. Cameron, H. Hashemi, H. Klimezak-Pllciistt
H. Singh, H. Mehta, H. T. Lehri, H. Hazimeh, I. Ballantyne, Szpektor, I. Nardini, J. Pouget-Abadie, J. Chan, J. Stanton, J. Wieting, J. Lai, J. Orbay, J. Fernandez, J. Newlan, J. yeong Ji, J. Singh, K. Black, K. Yu, K. Hui, K. Vodrahalli, K. Greff, I.. Qiu, M. Valentine, M. Coelho, M. Ritter, M. Hoffman, M. Watson, M. Chaturvedi, M. Moynihan, M. Ma, N. Babar, N. Noy, N. Byrd. N. Roy, N. Mom- chev, N. Chauhan, N. Sachdeva, O. Bunyan, P. Botarda, P. Caron, P. K. Rubenstein, P. Culliton, P. Schmid, P. G. Sessa, Xu, P. Stanezyk,
Tafti, Shivanna, R. Wu, R. Pan, R. Rokni, R. Willoughby, R. Vallu, R. Mullins, S. Jerome, S. Smoot, S. Girgin, S. Iqbal, S. Reddy, S. Sheth, S. Pöder, Bhatnagar, S. R. Panyam, Eiger. Zhang, T. Liu, Yacovone, T. Liechty, U. Kalra, U. Evci, V. Mísra, V. Roseberry, V. Feinberg, V. Kolesnikov, W. Han, W. Kwon, X. Chen, Y. Chow, Y. Zhu, Z. Wei, Z. Egyed, V. Cotruta, M. Giang, P. Kirk, A. Rao, K. Black, N. Babar, J. Lo, E. Moreira, L. G. Martins, O. Sanseviero, L. Gonzalez, Z. Gleicher, T. Warkentin,
V. Mirrokni, E. Senter, E. Collins, Barral, Z. Ghahramani, R. Hadsell, Y. Matias, D. Sculley, S. Petrov, N. Fiedel, N. Shazeer, O. Vinyals, J. Dean, D. Hassabis, K. Kavukcuoglu, C. Farabet, E. Buchatskaya, J.-B. Alayrac, R. Anil, Dmitry, Lepikhin, S. Borgeaud, O. Bachem, A. Joulin, A. Andreev, C. Hardin, R. Dadashi, and L. Hussenot. Gemma technical report, 2025. URL
https: / aariiiii......... org/abs/////oooooooogo..o.a..a.r
abs 2503. 19786.
A. Turner, L. Thiergart, D. Udell, G. Leech, U. Mini, and M. MacDiarmid. Activation addition: Steering language models without optimization. arXiv preprint arXiv:2308. 10248, 2023.
J. Wei, M. Bosma, V. Y. Zhao, K. Guu, A. W. Yu, B. Lester, N. Du A. M. Dai, and Q. V. Le. Finetuned language models are zero-shot learners. In 10th International Conference on Learning Representations, 2022a.
Wei, X. Wang, Schuurmans, M. Bosma, Xia, Chi, Q. V. Le, D. Zhou, et al. Chain-of- thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-2488 24837,, 2022b.
Y. Wolf, N. Wies, O. Avnery, Y. Levine, and Shashua. Fundamental limitations of alignment in large language models. In Proceedings of the 41st International Conference on Machine Learn- ing, ICML'24. JMLR.org, 2024.
Z. Wu, A. Arora, Z. Wang, A. Geiger, D. Jurafsky, C. C. Manning, and C. Potts. Reft: Representation finetuning for language models, Advances in Neural Information Processing Systems, 37:63908- 63962, 2024.
B. Xue, M. Zhang, A. Easwaran, and Q. Li. Pac model checking of black-box continuous- dynamical systems. IEEE Transactions Computer-Aided Design of Integrated Circuits and Systems, 39(11):3944
2020. doi: 10. 109/TCCA.2220.331222.
A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv. C. Zheng, D. Liu, E. Zhou, F. Huang, F. Hu, H. Ge, H. Wei, H. Lin, Tang, J. Yang, J. Tu, J. Zhang,
14

---

Published as a conference paper at ICLR 2026
J. Yang. J. Yang. J. Zhou, J. Zhou, J. Lin, K. Dang, K. Bao, K. Yang, L. Yu, L. Deng, M. Li, Wang. Q. Zhu, R. Men, R. Gao. S. Liu, S. Luo, T. Li, T. Tang, W. Yin, X. Ren, X. Wang, X. Zhang, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Zhang, Y. Wan, Y. Liu,
M. Xue, M. Li, P. Zhang,
Z. Wang, Z. Cui, Z. Zhang, Z. Zhou, and Z. Qiu. Qwen3 technical report, 2025. URL https:
//arxiv.org abs/ 2.555009888
H. Zhang, H. Song. Li, M. Zhou, and D. Song. A survey of controllable text generation using transformer-basee pre-trained language models. ACM Comput. Surv., 56(3), Oct. 2023. ISSN 0360-0300. doi: 10.1 145/361 7680. URL https: / /doi 00tg//0.115//61766666
15

---

Published as a conference paper at ICLR 2026
APPENDIX
This appendix provides supplementary information and extended results that complement the main body of the paper. Appendix A presents additional experimental results, including detailed controllable set evolutions for various tasks and models, and further analysis of image generation tasks. Appendix formalizes the definition of control system used throughout work. Ap- pendix C comments on how to interpret reachability and controllability probabilistic guarantees for stochastic and deterministic systems. Appendix D provides detailed discussion and adaptation of Theorem from Devonport and Arcak (2019) for output reachability in discrete-time systems. The concept of discrete bottleneck in dialogue processes, particularly for LLMs and T2IMs, is explored in App. E. Detailed proofs for the continuous-valued and categorical versions of Thh. (reachability) are presented App. F.1, along with an analysis of its sample complexity App. F.3. Appendix G extends the reachability analysis to expected output values for stochastic systems. The proof of Thm. (controllability) is given App. H. with a discussion on auto-parameter selee- tion for its hyperparameters in App. H.3 and its sample complexity. Appendix I illustrates how our framework can be applied to hypothesis testing. Finally, App. provides extended details on our ex- perimental setup, including input distributions in App. J.1, and App. K. introduces open-source controllability package.
A EXTENDED RESULTS
A.1
SETUP
We covered each setting in
LLMs x Tasks x Prompting method x stochastic/detrrmiissii
where the prompting methods, all in dialogue, span 0-shot and 5- shooot prompting, the model is held stochastic or deterministic (greedy decoding), and the LLMs and TASKs are given in See. 6. Due to the large amount of experiments, and because 5-shot prompting gave the best results, we show the controllability summaries (e.g., Fig. 2 bottom) for the 5-shot setting only, and full DP trajectories for the deterministic setting (both 0- and 5-shot).
A.2 RESULTS: DETERMINISTIC SYSTEMS
Overall, the extended task results confirm the takeaways in Sec. 6. We see significant heterogeneity between models, seen by the varying controllable set estimates in the rightmost plots in Fig. A.5, Fig. A.6, and Fig. A.7. Especially striking is that, for the same model, performance on different tasks is not predictable: for instance, although Gemma3-4B excels at formality and average word length tasks (Fig. 2 and A.6), it surprisingly is not controllable to the full measurement-value space for the (even, odd} task, one that is trivial for humans.
For detailed temporal view, Fig. A.12 shows, for 0 and 5-shot prompting, the a-controllable set's dynamic evolution for all deterministic LLMsx not in Fig. 2.
104
10
100
MARC
75
Cottsaak act
Smallars-s Qwea-itt Grmmaa-tis
2-
09-
Figure A.5: Requesting string length in {1... ... 10) with 5-shot prompting. We ask the LLM to generate a string of length N characters, where Unif[1....1 10).. The controllable set estimates are shown on the (right), with Gemma3-4B displaying the highest a-controllability (a = 0.1). to 80% of the desired range. The distribution of string lengths yr is plotted with respect to the initial request the (left). In general, especially for Qwen3-4B and Gemma3-4B, the generations are faithful to the request no matter the initial prompt, seen by points landing on the line y = x (dashed). This is corroborated by the (middle) three plots, however, SmoILM3-3B shows high variance of the faithfulness metrics p. R, and MAE across initial states.
16

---

Published as a conference paper at ICLR 2026
Oripet
2.0
MAP-0
Smailats-s
Quar--BB GrunaaS-is
34
Figure A.6: Requesting an average word length in [2.0, 10.0] with 5-shot prompting. We ask the LLM generate sentence whose average word length is values in [2.0, 10.0), up to an error of 0.1, with 5 shots in the initial input No- (Right) The controllable set estimates for all models. We find SmoILM3-3B and Qwen3-4B to be controllable only to a small fraction (0.18 and 0.07, respectivelyy, and Gemma3-4B to a larger fraction (0.61), of requests. (Left) The final output distribution yr is shown compared to the requested average word length; in general, models fail for larger requested lengths- interestingly, Qwen3-4B tended to resort to "default responses" such as "A cat sat on a mat" or "The quick brown fox jumped over the lazy dog." (Middle) The large spreads in p, R, and MAE, where each point represents the metric computed for single initial state, demonstrate a large sensitivity of model response to the initial state.
Cuntodable ar Cos ord)
08
08
010
Clant
60-
04
SusILMS-a
Qued--1
0.4
Grnmad 411
02
0.2
Figure A.7: Requesting an even/odd number with 5-shot prompting. We ask the LLM to generate positive even odd integer, with 5 shots in the initial input un- (Left) We show the distribution of responses by = turns, including an error category (righthand bars on each barchart), where the LLM response was not parse-able an integer. The (middle) two plots show the LLM's faithfulness to the request, where each point depicts the mean accuracy or Fl score of given initial state's reachable set. While (right) Qwen3-4B achieves perfect controllability of the a-controllable set (a = 0.1) with these inputs, as well as perfect faithfulness to the request (middle), results for other models are sensitive to the initial 2g, seen by nonzero variance in the violin plots. Notably, near- perfect faithfulness does not necessarily imply controllability, seen SmolLM3-3B: while it achieves a high accuracy and Fl score to the request on average (middle), not controllable for this task (right).
Commalale Kia (1)
0.5
088
0.8
0.6
0.6
0.10
0.6
MAR
0.4
0.4
0.4
0.05
0.68
1.78
48
88
148
0.68
1.78
48
58
14B
0.68
1.78
48
55
14B
0.68
1.78
48
148
Modil
Figure A.8: Larger LLMs are more controllable and calibrated on string length. For Qwen3 sizes ranging from 0.6B to 14B (x-axis), we requested string lengths ranging in ..... 10), with 0-shot prompting and dialogue turns. Controllability (right) and calibration metrics (left) increase reliably up to 14B, though metrics (cvg, R, p) increase most drastically for smaller sizes (0.6B +1.7B) and appear to saturate for larger sizes.
17

---

Published as a conference paper at ICLR 2026
Connelleteee set K.. (1)
(3)
0.5
04
04
0.4
0.2
0.
0.2
8.0
0.0
0.0
0.68
Figure A.9: Larger LLMs are more controllable and calibrated on average word length. For Qwen3 sizes ranging from 0.6B to 14B (x-axis), we requested output whose average word lengths range in [2.0, 10.0), with 0-shot prompting and 5 dialogue turns. Controllability (right) and calibration metrics (left) increase up to 14B in sudden phase transition at 4B where the model gains in performance.
10
1.0
Contelil oe d.
FLUX-d SDXL
Couu
0G
0.6-
DSU12
02
62-
02
1945" 24-2
Figure A.10: Requesting objects at specific locations to T2IMs. We query T2IM with the prompt "White background. [obj] the [pos] of the image.", where pos Unif(top left, top right, bottom left, bottom right, center), for all obj in the 80 COCO classes. As measurement, we divide the image in grid of 3 3 (each area of width and height fdd of the image) and we use an object detector to determine if the object is in one of the requested areas. We observe that T2IM struggle placing objects specific locations, the (left) figure we see that models tend to place objects in the center (C in the x-axis), followed by bottom right and left (BR, BL) and finally top right and left (TR, TL). The darker lines are the count of objects not placed in any of these positions. The other plots show the averaged statistics per object, showing that FLUX-s theoretically controllable (they reach the whole output space with maximal coverage and show better accuracy). However, these results are far from perfect controlla- bility, underscoring the need for rigorous analysis models.
in the simple setting provided. In
08-
0.
0.5
0.4
0.0
0.50
34
FLEX
SDXL
0.25
0.0
0.5
-10
000
Input
Figure A.11: Requesting image saturation to T21Ms. We query T2IM with the prompt "Gener- ate an image with [100 x sat]% saturation.", where sat Unif(0, 1.0]. This is an example of quantized reachability, therefore we set 7 0.1 We observe that T2IM FLUX-s and SDXL are theoretically controllable (they reach the whole output space with maximal coverage). However, the models are not calibrated all the request, with correlations only reaching p. R < 0.1. In the (right) plot we show the reachable sets for each input (dark bars) and the controllable set (shaded area). Interestingly, the models show stark difference in terms controllability in this setting.
18

---

Published as a conference paper at ICLR 2026
Ang wonl Jongh (ox) Aog. word leagth (w)
SmelLAA3-- B-chot
(Heloot
20
10
20
Trajecrories
N 27450
Dialogae tunn (1)
Dialogue turn (8)
Dialogue turn (C)
= 61 450
S-ahot
S-ecoot
S-hhot
Cootrollable set
10
10
10
confidrace 0.06
coatrollcble
0-
3
Dialogae turn (t)
Dinlogue tumn (
Dialogue turn (t)
SmalLAA1-33
Quend-4B
Gemoun-4B Oschot
Steios lougl
Trajectotiss
Dialogue turn (c)
Dinlogue turn (8) S-shot
Dialogue turn (0)
k 61 10
Strizg, brught (e))
S-shat
b-ahot
Controllable ses
confideace 0.05
controllable
Dislogae turn (0)
Dialogge turn (8)
Diologoe turn (6)
Smoll.Mi--3 (selhot
Qwere3-111
Gemmus3-HH
Exençoni lodi lasege Even lodt integg
Trajectorios
N 122
Disloggue
Dialogue
Dialogue
A - G1 - Controllable
G-ohot
S-shot
G-eioot
contidencc 0.06
Not controllable
Dialogge
Dislogue
DNaloque
Figure A.12:
Extended controllability trajectories for deterministic LLMs. To supplement Fig. (top, middle), show here the DP trajectories for the remaining L.LM tasks, in the deterministic case.
A.3 RESULTS: STOCHASTIC SYSTEMS
Our experiments primarily focused on DPs with deterministic, greedily decoded generative models. This is because the tasks, being goal-oriented, egg, produce even/odd number, are more amenable to greedy sampling. However, as generative models may be sampled in practice, for stochastic LLMs (5-shot) we report DP controllability summaries in Fig. A.13.
19

---

Published as a conference paper at ICLR 2026
(a) Formality task, stochastic system.
<5 1.00
Oy 1.00
Quai--B
044
Gouna844
022
02
Tepet
(b) Average word length, stochastic system
08-
Ougpst
Qowed-ffi
Comss8448
00
200
Daper
(c) String length, stochastic system
Cotcobaleas &e
Orggii
5.0-
Qowad-tt
0.4-
Saper
(d) Even/odd integer, stochastic system
1.0
1.0
10
Costrolattee aet Cws
10
im
Classt
SoutLMSs-
Quand-tB
0.4
0.4
201
Monsttmttnnt Spuce
Figure A.13: Controllability results, stochastic LLMss For the formality task, average word length, string length, and even/odd number (top to bottom), we show the controllability summaries for stochastic systems. The sample complexities are the same as for the deterministic system, see Thm. 2. Adding stochasticity due to sampling results in more diffuse distributions of the output (left), notably for the even/odd integer task, as well as worse faithfulness metrics, as expected.
20

---

Published as a conference paper at ICLR 2026
A.4 EXAMPLE QUALITATIVE ANALYSIS: QWEN, FORMALITY
For text formality, we conduct a qualitative analysis of Qwen3-4B's generations (the best performing model) for three randomly sampled trajectories for the 5-shot, turn setting. The resulting dialogues are Tables A.1-3. In general, Qwen3-4B takes the strategy of choosing the example sentence (of the 5 shots) whose formality is closest the requested formality. It remains on the theme of the example sentence, modifying one or two words until the formality is as desired. For instance, in requesting 25.65% formality, Qwen3-4B*s initial response "wow i'm so tired of this weather" which tracks the example "Wow I'm so tired" (25% formality). Subsequent responses then continue the theme of being tired the weather. Similarly, in the 76. 19% case, Qwen3-4B copies the general sentence structure of the 75% formal example. For the request to generate text of 96.3 I % formality, however, Qwen3-4B generates texts that are too formal. It does however still follow the strategy of mimicking the style of the 100% formal example, then modifying few words at a time.
21

---

Published as a conference paper at ICLR 2026
Role
Dialogue
User
You are chatbot that follows instructions. I will give you example:
Instruction: Generate a story that is 100% formal.
Result: Kind sir, please make your way to the terrace.
Instruction: Generate story that is 75% formal.
Result: She bought the book, then she went to the beach.
Instruction: Generate a story that is 50% formal. Result: Hey! Just went to the store.
Instruction: Generate story that is 25% formal.
Result: Wow I'm so tired.
Instruction: Generate story that is 0% formal. Result: hey man how's ' it goin
Knowing this, please follow the next instruction.
Generate story that is 25.65% formal. Write the story and only the story.
Assistant
Wow i'm so tired of this weather
User
Your answer was too informal. I asked for formality level of 23.65, and you gave sentence with formality 12.27. Please try again. Write the story and only the story.
Assistant
i'm really fed up with the rain today
[Conversation ended, final formality 25.24%
Table A.I: Requesting formality of 25.65% to Qwen3-4B.
Role
Dialogue
User
You are a chatbot that follows instructions. I will give you an example:
Instruction: Generate story that is 100% formal.
Result: Kind sir, please make your way to the terrace.
Instruction: Generate a story that is 75% formal.
Result: She bought the book, then she went the beach.
Instruction: Generate story that is 50% formal. Result: Hey! Just went to the store.
Instruction: Generate a story that is 25% formal.
Result: Wow I'm so tired.
Instruction: Generate story that is 0% formal.
Result: hey man how's 's goin
Knowing this, please follow the next instruction.
Generate story that is 76. 19% formal. Write the story and only the story.
Qwen3-4B
She arrived at the station, checked her ticket, and boarded the train.
User
Your answer was too informal. I asked for a formality level of 76. 19%, and you gave sentence with formality 64.04% Please try again. Write the story and only the story. She arrived at the station, presented her ticket, and took a seat on the train. [Conversation ended, final formality 76.31%].
Qwen3-4B
Table A.2: Requesting formality of 76.19% Qwen3-4B.
22

---

Published as a conference paper at ICLR 2026
Role
Dialogue
User
You are a chatbot that follows instructions. I will give you an example:
Instruction: Generate story that is 100% formal.
Result: Kind sir, please make your way to the terrace.
Instruction: Generate a story that is 75% formal.
Result: She bought the book, then she went the beach.
Instruction: Generate story that is 50% formal. Result: Hey! Just went to the store.
Instruction: Generate story that is 25% formal. Result: Wow I' m so tired.
Instruction: Generate a story that is 0% formal. hey man how's goin
Result:
Knowing this, please follow the next instruction. Generate story that is 96.31% 1% formal. Write the story and only the story. Kindly undertake the task of composing narrative that adheres strictly to the conven- tions of formal language. The story should presented in a manner that reflects the highest degree of formality, with deviation from the expected tone and structure. The narrative should be self-contained, devoid any additional commentary or ex- planation, and should solely consist of the story requested.
Assistant
User
Your answer was too formal. asked for formality level of 96.31, and you gave sentence with formality 99. 79.72. Please try again. Write the story and only the story. Kindly proceed the designated location at once.
Assistant
User
Your answer was too formal. asked for formality level of 96.31, and you gave sentence with formality 99.64. Please try again. Write the story and only the story.
Assistant
Kindly proceed to the designated location immediately.
User
Your answer was too formal. asked for formality level of 96.31, and you gave sentence with formality 99.62. Please try again. Write the story and only the story.
Assistant
Kindly proceed the designated location without delay.
User
Your answer was too formal. asked for formality level of 96.31, and you gave sentence with formality 99.64. Please try again. Write the story and only the story. Kindly proceed to the designated location at once.
Assistant
Table A.3; Requesting formality of 96.31% % to Qwen3-4B.
23

---

Published as a conference paper at ICLR 2026
B FORMAL DEFINITION OF CONTROL SYSTEM
Definition 7. A discrete-time control system is tuple (Ø, N,, where
1. T = N is the time domain
2. X the state space,
3. 2/ is the imput space.
4. d X x U - Je, ur - x++1 is the transition function mapping the current state and
input to the next state,
5. & X + V; 0r, ue + yt is the readout map, taking the current state and inputs to the
current output measurement value.
C REACHABILITY AND CONTROLLABILITY FOR DETERMINISTIC VS. STOCHASTIC SYSTEMS
Dietrich et al. (2025) explain that PAC-style guarantees for reachable sets fold in two sources of stochasticity: that of system dynamics and that from sampling uncertainty. Similarly, our case, the interpretation of a, p-approximate controllability differs under deterministic and stochastic dy- namics, as explained Sec. 4.3 the main text. In particular, a, p-approximate controllability is exact (or within--) under deterministic dynamics and deterministic controller, i.e., given desired y" E Cy, for over 1 - a proportion of initial states, we are probabilistically guaranteed to find a U that takes to exactly y² (within 7). Instead, the difference for stochastic dynamics is that the same a, p-approximate controllability now pertains to the distribution of y" exact controllabil- ity no longer applies due to stochasticity in the dynamics (and by extension, of the readout h). In short, for a stochastic system, the returned Ri,p by Alg. la returns best- or worst-case reachable set estimate; the returned C by Alg. 1b returns a controllable set estimate that inherits the same best- or worst-case distributional interpretation.
One question we can ask about a stochastic system with continuous-values outputs is the reacha- bility/controllabiityy of its expected measurement-value that is, Ely]. We prove in App. that Ely] does not suffer from the discrete bottleneck, that is, we can directly repurpose existing methods (Devonport and Arcak, 2019) returning continuous set estimates, rather than finding relaxation for countable sets. We provide such bound (Cor. G.1) on the reachable set of E[y) in App. G, as corollary to the main PAC bound in Devonport and Arcak (2019). The latter stated in App. for reference. Then, for controllability, one can simply replace the reachable set estimation subroutine in Alg. 1b with the procedure described in App. G.
D THEOREM 1 OF DEVONPORT AND ARCAK (2019)
In this section, we compare our method to the Monte Carlo method of Devonport and Arcak (2019) (DA19). Their method takcles the same problem of reachable set estimation, though it for states and not outputs/meassurrmeet--aaa
(our case). Our algorithms and bounds were inspired by DA19, and we propose corollary to their Theorem in App. for expected output reachability. For this reason, state their main result and compare it to our Thm. below.
Monte Carlo method of Devonport and Areak (2019) Devonport and Arcak (2019) (hercon DA19) propose elegant Monte Carlo sampling approach to estimate overapproximating inter- val the reachable set. While their original result focuses on state reachability and for continuous- time systems, here directly adapt their approach to output reachability and for discrete-time systems.
DA19's sampling algorithm states that to estimate the forward reachable set at time t:
1. Take i.i.d. samples of the initial state {= 1 Po and of the input {201m PU-
2. Evaluate outputs yr = v(a{")
(1) u( at time f.
3. Take (m) the smallest axis-aligned interval containing all y
24

---

Published as a conference paper at ICLR 2026
The forward reachable set at time can then be defined as (R", F... Py.a). Noting that p(w) = p(r, E w). w E Fi, then the true reachable set R₂ is the smallest Jr with probability 1.
Fi in the probability space
The sampling procedure can only produce approximations of the reachable set. To define this ap- proximation, DA19 introduce the notion of c-aceurate reachable set:
Definition (e-accurate forward reachable set, Devonport and Arcak (2019)). The e-accurate reach- able sets Rt,, at time & T are the smallest events Fi with probability 1
In words, the e-accurate reachable set time approximates the true up to probability density ef This probability density is given by the measure Py. over outputs 1/-
This means that the e-accurate reachable set R4,, is such that pt (2+ E Rt,c) 1 DA19 further define an overapproximation of an c-accurate reachable set as any set R C R such that R contains Crucially, with
c-accurate reachable set. This means that (e R) Pr Rt,e) = enough samples, we can construct a set that overapproximates or contains the e-accurate reachable This the key insight of DA19's main result, which holds if Pt, continuous (see App. D for details):
Theorem (Output forward reachability, adapted Devonport and Arcak (2019)), Let (e, E (0,1) and V1 RR If
2n
2n log 8
m
(D.6)
then R (m)) overapproximates e-accurate output reachable set with confidence 8, i.e., P(Ri, C R ((m)) > 5-
Note that (m), the c-accurate output reachable set, is guaranteed to contain the true reachable set with probability 1 5. However, may also contain outputs that unreachable. Hence, given target y", it is necessary but sufficient for e be controllable on ) that y C R(m).
D.1 COMPARISON TO OUR THM.
Here are where DA19 and our method differ:
1. They consider state space reachability, while we consider output reachability.
2. They assume continuous reachable set, while in our setting is countable. In particular, continuity of is implied in their proof of Theorem I which constructs the bound, whereas in our setting does not apply.
3. DA19 returns an overapproximating interval (a bounding box) of probabilistically ap- proximate reachable set, while we return a set that is guaranteed within-y to hold proba- bilistically approximate reachable set.
There several notable points where DA19 and our Thm. coincide:
1. The probabilistic scaffolding in Sec. 3 of the control process is the same, in order to i.i.d. sample yt-
2. The sampling algorithm (sampling i.i.d. trajectories), except for the construction of the returned set.
E DISCRETE BOTTLENECK IN DIALOGUE PROCESSES
Here, we discuss why, and under what conditions, generation with LLMs and T2IMs has a discrete bottleneck. That is, when the readout map is continuuus-valun valued, the reachable set could be countable (if the readout map is categorical, we are already a discrete regime).
For the analysis in this section, assume deterministic readout map whose image is continuous- valued. We consider several cases: if the model is stochastic vs. deterministic, and whether the model generates text (discrete) images (continuous). In order to analyze the reachable set of an
25

---

Published as a conference paper at ICLR 2026
LLM T2IM's generation, need to consider forward pass, which is granular level than each dialogue turn (our temporal resolution of interest in the main text). We denote T as the timestep indexing number of token generations. To make the results general possible, i.e. encompassing activation steering which acts at each forward pass, we write all results assuming inputs may be introduced during each forward pass
E CONDITIONS FOR DISCRETE BOTTLENECK, DETERMINISTIC SYSTEM
We start by showing that deterministic LLMs, which produce a distribution over their finite discrete vocabulary space Σ, will have countable reachable sets.
Lemma 10 (Deterministic intervened LLMs' reachable sets are countable). The output attributes of an intervened LLM & can be controlled, after T token generations, to a set of cardinality at most min Σ [,] where Σ| is the vocabulary size.
Proof. We use a cardinality argument. We consider the first token generation. Given any To E X,
fix u U. As e is deterministic, we have (im(Ø(20, u))l which implies ]im( ( )) |[ S [u]]
Further, jim (((((.)))) < [Σ], the vocabulary size. Hence |im (i((((,-)) S min([E, [4)). Since the
readout map is deterministic, jim (h(:,-)) S min([E], [4)). We repeat this reasoning to integrate that jim (h(T; zo, uI...))] S min([E]", [2<[") for all E X' and
U. i 1. T. We have bounded the size of the reachable set for a given 20 X. Then, we apply Def. 15 to see that the dialogue process with deterministic & can be controlled to a set of maximum
values of 2 ..
size min( | [(<["), as the largest intersection between the reachable sets R( ,, 7) for initial (6) states T0 E X, is less than min( (4)7).
Lemma 11 (Deterministic,, intervened T2IMs' reachable sets have cardinality at most |ES| x [W)). The output attributes of deterministic intervened T21M can be controlled to a set of cardinality at most > x [2], where [E*| is cardinality of all possible input strings.
Proof. (Sketch). We use similar argument to Lem. 10. the readout map (Sec. 3) is determinis- tic, then the reachable set at time 1 is only bottlenecked by its initial prompt space No = E' and its input space U. follows that its reachable set's 's cardinality is maximum | x [M]], with equality if h is injective.
This means that for LLMs, there is guaranteed discrete bottleneck from the vocabulary space. For T2IMs, if the input space continuous, e.g., activation steering, then there no discrete bottleneck. In our case, as we study discrete-valued inputs of finite length (prompting), there is necessarily discrete bottleneck for T2IMs.
E.2 CONDITIONS FOR DISCRETE BOTTLENECK, STOCHASTIC SYSTEM
Here, consider under which conditions stochastic systems will have discrete bottleneck. The discrete bottleneck is guaranteed for LLMs, whose input and output spaces are both strings- how- ever, stochastic T2IMs will have continuous output sets.
Lemma 12 (Stochastic intervened LLMs' reachable sets are countable). The set of output attributes of an intervened LLM after 7 token generations, have a cardinality most > where [E| the vocabulary size.
Proof. The number of potential outputs during each forward pass is the number of vocabulary items [.] After token generations, the maximum number of possible outputs is |
Lemma 13 (Stochastic intervened T2IMs' reachable sets can be uncountable). The set of output attributes of an intervened T21M & can be uncountable.
Proof. The proof follows from replacing the cardinality of vocabulary space |Σ| in Lem. 12 with the cardinality of a T2IM's generation space. As T2IMs generate images, which are continuous, the reachable set of stochastic T2IMs is uncountable.
26

---

Published as a conference paper at ICLR 2026
E.3 EXPECTED OUTPUTS CONTROLLABILITY
Stochastic generative models define a distribution that is sampled at inference time. While the output attributes themselves are bottlenecked, their expectations may not be. The expectation of the attribute over the model's distribution is only bottlenecked by the input cardinality [4)", where is the number of tokens generated in the case of an LLM
1 for a T2IM).
Lemma 14 (Stochastic LLMs are expected output controllable on Bardinality-(U1" sets). The ex- pected measurement value Ely] of stochastic LMs Ø, where the next token is sampled from Pe with ps. A can be controlled to sels of cardinality most [u]T by token generation т.
Proof. We first consider stochastic LLMs. We use the same cardinality argument as in Lem. 10. The proof follows from replacing im( < [Σ] with im(@) < | -1 IR]. This means that the expected output attribute Exppy [V] can maximally vary over as many possible values as py,, which is
| = [R] Then, given any Zo E X, the cardinality of im (h(z -)) less than min([R), [[]]
14). where the choice of input u E LI bottlenecks the expected output attribute. Analogous to the
proof of Lem. 10, it follows that, given ng. the reachable set's cardinality [ r) S [ by
integrating the system. Then, applying Def. 15, the size of the controllable set is maximum [2TT]. O
F REACHABILITY: THEOREM 1
F.1 THEEREM 1: CONTINUOUS- VERSION
We state the q-qunntized version (for continuous-valued measurements) of Thm. 1. The proof for the categorical version follows directly.
Theorem 3: Sample complexity bound, 7-quantized reachability.
Let y be continuous and bounded. Let N be the covering number of Y with co-balls of radius 7/2. Let m be the number of i..d. samples drawn from Yt, i.e., hm with each y: y Vi. Fix 8 (0,
2 max (N, log(ō/N)
(F.7)
log(1 - p)
then P(R U , 7)) 2 1 - 8, where Boo (v, 7) is the lx0 ball centered at y with radius
Proof. The problem reduces to asking how many independent samples m are needed all N bins with likelihood 2 1 - 8, where the probability of sampling each bin is >> p.
Let i..dd samples be drawn from Py.:: fo() {v I (omitting from the RHS for readability). By definition of R. the probability sample hits the bin y in R.Tpp is Py, E ya 2 p. The probability sample i lands outside bin is then Py. (1. & W < p. The probability all samples i 1-.-m land outside nontrivial bin j is (())(m)) & vmm S (m-
Then, by the union bound over a maximum of N bins, the probability at least one bin is never hit is most N(1 m) Subtracting from 1, the probability that all bins in R2,p hit by some sample
is
(all y hit) = PRR is y-reached by the samples) 2 1 - N p)
(F.8)
We want to guarantee that P(R is 7-reached by the samples) 2 1 5. We do so by guaranteeing N(1 - p)m 2 1 - 8. Then, solving for m, if
log (og(
m
(F.9)
log( p)
NNotation: E is called the -|>]| 1-simplex", or the space of categorical probability distributions over categories.
[
27

---

Published as a conference paper at ICLR 2026
then with probability 11-1 -, 6, R. is y-reached by the samples )o() By the pigeonhole principle, m also needs to be greater than the number of bins N. It follows that if
Log(6/N)
2 max
N log -p)
(F.10)
then all N bins, each with radius 7/2 (side-length 7), are hit with probability By (F.8), we are now guaranteed w.p. 2 1 - å that each point in R2.p u is within of some sampled point ()m), follows that w.p. 2 6, the set Ry contained in a y-cover of Dim), or P(R C L 7)) 2 1 8.
F.2 COROLLARY OF THEOREM 1: CATEGORICAL VERSION
Corollary F.1: Sample complexity bound, categorical reachability
Let y be discrete with finite cardinality N. Let m be the number of i.dd. samples drawn from Y,, i.e., { with each y y Vi. Fix d E (0, 1). If
m 2 max (N. log(o/N)
Tog(1 - -p)
(F.11)
then PRR C {3:) 2 1 8.
Proof. Follows from proof of Thm. 3.
F.3 SAMPLE COMPLEXITY OF THEOREM
Sumple complectyy va ip 6.05)
Sucppe complesity vs p (6 0.0)
Sample complecit vs. (6 0.60
Samp complectiny m
100
0.05
200
0000
5000
150
60
0008
40
60
0.001
0.25
0.50
0.25
1.00
0.00
3.11
100
Coufidence paramnter
Precision parameter P
Number
bius
Figure F.14: Sample complexity growth with hyperparameters, From left right, we sample complexity m growth with respect to 8 (constant p), p (constant 6), and (constant d). Overall, the most drastic change in sample complexity comes from decreasing p. Sample complexity also increases sharply as draws closer to O. In contrast, sample complexity is robust to increasing N in fine-grained regime (N 2 50). The sample complexity remains tractable for parameters (6. N) = (0.05, 0.05, 100), giving m 149.
Figure F.14 visualizes the growth of with respect each parameter. The dependence of on and is given by the form of Thm. 1, and are seen sharp growth and 8 grow closer to 0. The dependence on p is more drastic (Figure middle) compared (Figure left).
The dependence of m on N is less straightforward. We sketch why m grows O N). First, bin under uniform density:
fix the number of bins N. Then, cannot exceed the probability & max EPr,,, min, Pr,y (1 = t. We use the observation that log(1 - p) -p, valid for small p E (0, 1):
log(o/N)
log(N/o)
2
(F.12)
log p)
P
2
N(log - log 8)
(F.13)
O(N log N).
(F.14)
28

---

Published as a conference paper at ICLR 2026
F.4 EMPIRICAL VALIDATION OF THEOREM
The sample complexity bound in Thm. 1 is distribution-fue and holds for any control system. There- fore, the tightness of the bound likely differs across experimental settings, and specifically for dif- ferent distributions the underlying generative model induces. Therefore, instead of formal analysis of tightness, which would be highly nontrivial in the absence of the DP's distributional information, in this section we perform sanity check that probabilistic guarantee in Thm. I holds in our setting of interest, i.e., on generative model.
F.5 SETUP
We test only the most general case, that is, for stochastic system ó and 7-quantized reachability. In particular, we use Gemma3-4B on the text formality task, for 0-shot prompting and 7 1. The initial state is Ty=BOS. All other hyperparameters and prompt templates replicated from Tab. J.4 and J.5 and the input distribution and readout map inherit the formality setting described in the main text.
For the DP defined for the above initial state, input distribution, and readout map, we first proxy the true distribution Py.r by sampling and scoring N 10000 responses from Gemma3-4B. On this proxied true distribution, we compute the "true" (p, -apprxxiiaaee reachable set Ri as described Def. 5.
To validate Thm. 1, we rum Alg. la 200 times. Each time, we check whether the guarantee R 5 R7,, holds. Then, we average over the 200 runs to obtain the empirical confidence. the empirical confidence complies with the desired confidence parameter 8, then it serves sanity check for Thm.
Results Figure F.15 validates Thm. in practice, where we plot the empirical confidence against the sample complexity. Each point on the line represents the mean over 200 runs. Theorem rec- ommends minimum sample complexity of m 104, which empirically adheres to the guarantee that (m) 5 RR,,p above the desired confidence - 5 = 0.95. For this particular setting, Thm. is reasonably tight, given the fewest samples to satisfy the guarantee is around 40. This represents roughly 2.5x inefficiency in the bound.
Text formality
quantized reachable set + probabilistic guarantee
Empirical confidence
0.8
Sample complexity m 104 (Thenrem 1)
Empirical confidence 200 bootstraps 42 SE Desired confidence 0.95
0.6
Probabilistic guarantoe met
Thm. 1, guaruntee met
0.2
0.04
20
40
60
80
100
120
140
160
Sample complexity w
Figure F.15: Empirical validation of Theorem 1.
G EXPECTED OUTPUT REACHABLE SET
Lemma 14 says that considering expected outputs under stochastic dynamics longer suffers from the discrete bottleneck. We present a corollary to DA19's Theorem, (stated in this paper's 's Thm. 9) that estimates the reachable set of the expected output value for stochastic dynamics.
29

---

Published as a conference paper at ICLR 2026
Corollary G.I: Expected output forward reachability for stochastic system
Let (c, E (0, and let (Cgs 5p) E (0, 1). If m satisfies
(1 - 5,) 2n(1
2n )mm)
2 1--5 5,
(G.15)
then Rim)),, which is the (mm.y of Thm. 9 pushed out by Eu. overapproximates an e- accurate reachable set of the tue expected output with confidence 6, i.e. P(R (fartalie C
R!m)).)) 2 1 8.
Proof. We extend the proof of Thm. the expectation of yt. The key addition the proof of Thm. 9 is uncertainty in Eуt caused by sampling, for which we use vector variant of Hoeffding's inequality from Pinelis (1994). The inequality states that, for N samples of a random variable Y E Rd that has bounded deviations from the mean ||Y -FY||2 s R, and a choice of e 0, then
P(J| - EY 2 En) S 6,..
(G.16)
where 8n 2 exp
- 2R
> O. That is, for enough samples N 24 log
0 8p 1, the 2, n,
empirical mean will approximate the true mean IEY with high probability.
Hoeffding's s inequality enters when choosing the halfspaces H. in Thm. 9, Recall that, y/ E RR" there are 2n "faces" of the interval covering the reachable set. In Thm. 9, each Hs, = 1... 2n is defined as the halfspace facing "away" from the samples in the interval. We modify this defini- tion here. Let { where Tj + - Mj,k, be sampled approximations of the true means (Ey, ) For a choice of E and confidence 6n. let N be such that (G.16) holds for all yy; that is, all sampled means yj are within ou of the true mean nl, with confidence 6n. Now, define the halfspaces H, as the faces of the interval covering all sampled means Ty. then shifted each by ep away from the samples.
The probability that all true means are contained within the bounding box defined by the Hi is at least the probability that all true means lic within en of their corresponding sample. Because the m samples are independent, this probability is given by
((o==((ly, n la (n)) (1 105-5,. 5,,"
(G.17)
We proceed following the proof of Thm. 9, defining the halfspace P; with respect to the true means n². j = 1........ m. Let (m).))))) be the smallest bounding interval of the sampled means, pushed out by Eu- If a u² E P,, then Hi C Pi, or (H4.) S (P.) = i. with probability greater than (1 5m Then, w.p. (1 - 6n,) (,,,) S + Dr (R (m))y) 2 1 -
(1m)) is an
e-accurate reachable set. Overall, let the event be that (m).a contains an e-accurate reachable set. Our goal is to bound (P(A). We do so by conditioning A on an event B: "H, bound the true means". Then, P(A) Z (B)). The event A reduces to the setting in DA19: "the P, lie outside the H4".
P(B) is at least 6p)) Now, given B, that is, given the H4 bound the true means,
E =
P(A[B) > 1 2n(1=
(Theorem 1, DA19)
(G.18)
2n
Then, the overall confidence is given by
(Pp())())))) > 1-e) P( 2 (1 (1-
2n )m)
(G.19)
2n
target prohubility 1-8
the number of samples m satisfies
(1 (1- 6
(1
21
- 6,
(G.20)
2n
(i) Equation (G.17)
(i)) Thearem
30

---

Published as a conference paper at ICLR 2026
(m)).) contains the e-accurate reachable set with confidence level d.
then Re
Note opposing dependencies of factors (i) and (ii) on m. The probability P. do not contain samples increases m increases, while the probability that Hi do not contain samples decreases m.
H CONTROLLABILITY, THEOREM 2
H.1 OUTPUT CONTROLLABLE SET
Definition 15 (Output controllable set). Given control system (0,X, T, .y). the output control- lable set C2 C y at time is given by
C. = n Re(
(H.21)
ZuERo
the intersection of the reachable sets of all initial states un Xo.
H.2 PROOF OF THEOREM 2
We state the proof of Thm. 2 for the quantized case. The proof of the categorical case immediately follows by removing 7 everywhere it is written.
Proof. Let E be the event that m(C, \ (.) <. C. We want to (EE) be greater than some 1 6. Now, let F be the event that R7 (1 (f)) C. (x(0)) for all i = 1... k. That is, F is the event that all sampled reachable sets contain the true target. We have ((E) 2 P(E(F)P((F) Our strategy will be lower bound the RHS by wwwrrbounding each term on the RHS individually.
We start with P(F). By independence of the k sampled reachable sets, we have P(F) 2 (1 8g)* by Thm. 3.
We now consider the term P(E F). Abbreviate C. = Re to be the empirical intersection (8)
of & sampled sets, where each Ti ~ Po. i 1. We aim bound the error p(Će \ C..). For brevity, we rewrite
Cx \ C₁ to mean all points outside the true C, but inside the empirical Cg. All points Yoin in the true (a)-controllable set C have μ(ybin) > 1 All points in B, bins that have incorrectly survived samples. In other words, each element ybin of is such that (ybin) < 1 Our aim is to find number of sampled sets & after which
P(μ(B) B) < < 1 F) 2 dc
(H.22)
This is achieved with Markov's inequality:
E[m(B) F]
(H.23)
Then, if (E(u(B) FI/e S dg. then (H.22) holds. Note that Elu(B) F] is equal to the probability Ybin with p((hin)) This likelihood is at most
hit by k samples, given that each sampled reachable set is accurate.
E[p(B) Fl P (ybiin ((biia) < hit by samples all reachable set samples accurate)
(H.24)
< (1 - +
(H.25)
Then, if
(1 a1)
< bc
(H.26)
log édc log(1 a)
k
(H.27)
then P(((B) < F) 2 1 -c. desired.
Putting the two terms together, have that if k (1) 5n)*, k, n(C, (+) c.
logede log[I-a) then with probability at least (1
31

---

Published as a conference paper at ICLR 2026
H.3 AUTO-PARAMETER SELECTION FOR THEOREM? 2
There are several free parameters in Thm. 1 and 2. If the end goal is a controllability analysis, then given target confidence 1 < - 6R) (1 oc), we can compute the optimal values for òR and de such that the total sample complexity = k is minimized. The intuition is that óR and dc are tradeoff between more reachable set samples (m) and more initial state samples (k).
Formally, this can be written as the following constrained optimization problem:
minimize
T m
(Total number of samples)
(H.28)
m log( (g((
subject to
(Reachability, Thm. 1) (H.29)
log(1 - p)
h: log(c -cc)
(Controllability, Thm. 2)
log(1 - -a)
(H.30)
5n) (1 -8c) 2 2
(Overall confidence constraint)
(H.31)
with fixed parameters
8, a, EE (0,1)
(H.32)
N E N m, k N22, 8c, 8R E (0.1).
(H.33)
and optimization variables
(H.34)
we only require an overall confidence 5 on the controlllable set (RHS of Theorem 2), we abstract away the decision of 8R and de- By doing dense grid search over (6c, 6R) uep.seomspcce (1e-6, 0. 999, 2 we can find the minimum total number of samples needed satisfy all the constraints in Thm. and 2 and (H.31). The extra time taken by this step is negligible compared to the time it takes to sample the dialogue process.
H.4 SAMPLE COMPLEXITY OF THEOREM 2
We split the analysis into two parts: (i) dependence of & on hyperparameters, see Fig. H.16, then (ii) dependence of the total sample complexity m hyperparameters, using the subroutine described in App. H.3.
state sumples
Initial state samples wa (6c
state semples w
Initial state samples A
0000
1.0
10000
0.05
100
7500
08
06
0000
5000
0008
2500
2500
02
0001
0001
0.501
1826
0.04 Partial biittyllability a
1./12
01.25
0,50
0.75
Confedence persmeter 8G
Erroz
Figure H.16: Number of initial state samples dependence on hyperparameters. From left to right, we see the number of initial state samples k grow with respect åc (constant a), (con- stant ôc), and (constant êc). Overall, the most drastic change in sample complexity comes from decreasing a, the partial controllability parameter. Sample complexity also increases sharply de approaches 0. In contrast, sample complexity is robust to increasing c, for closer to 1, especially for more lenient partial controllability (higher a).
Figure H.16 shows that the number of sampled initial states highly depends on a (middle panel), while comparatively robust the confidence parameter &c and error What is more important the total number of samples n. We performed the procedure in App. H.3 to abstract away the choice of de and 5nz and analyzed the dependence of total sample complexity p, N, e, and o.
32

---

Published as a conference paper at ICLR 2026
Figure H.17 shows how varies with each hyperparameter. Sample complexities hover near X 107 for very small (Re 0.001) values of and p.
N
Total ssappl complestty
30000
20100
2.0
100000
00000
50
20001
00
15000
1.0
00000
401
10000
10000
0.3
00000
0.12
0.01
0.00
0.04 Reachalslity precision
Overall coafideenee
Controllabilly error
Partial contzollabiiyy
Figure H.17: Total sample complexity function of hyperparameters. We vary the total number of samples n in controllable set estimation as a function of each fixed hyperparameter a,, N), one at time. When varying each parameter (each panel), fix all others except for N, which varies, at a constant value of 0.05. Given overall confidence 6, we auto-select de and on using the algorithm App. H.3.
33

---

Published as a conference paper at ICLR 2026
I HYPOTHESIS TESTING
Because Thm. and 2 estimate confidence sets, they lend themselves to hypothesis testing. We give an intuition in the below example for the reachable set:
Example L1: Hypothesis testing, reachable set
1. Q: I have a toxicity verifier. Is it possible to control my LLM's toxicity to [0, 0.4] with prompting? I am fine with error of 7 0.05, and consider a region to have nontrivial probability if its probability is greater than 0.001.
2. Quantize. Estimate N = 0.4/0.05 8 bins.
3. Null hypothesis: My target range [0, 0.4] is 0.05-quantized reachable.
4. Set the p-value cutoff to, e.g., 0.05.
5. Sample m 2204 toxicity
(applying Thm. 1).
6. Suppose we obtain R₂ (m)) [o, 0.07 U [0.5, 0.8) / y"
7. Reject the null hypothesis, concluding that y" not reachable within y with a p-value cutoff 8.
J EXTENDED EXPERIMENTAL SETUP
Table J.4: Hyperparameters for all controllability experiments.
Variable
Description
Value
partial controllability Thm. 2
0.1
controllable set error, Thm. 0.05
8
controllable set confidence, Thm. 2
0.05
reachability precision, Thm. quantization, Thm.
0.05
0.1
Table J.5: Model sampling hyperparameters. For the stochastic setting, Le., where sample
the underlying LLM, as well as for initial state generation Po with Mistral, we use the following sampling parameters.
max new max_new
www.neetteeenssseeee tokens
100
temperature (stochastic
0.7
temperature (pa)
0.9
top-p (stochastic Φ)
0.9
top (Po)
0.95
top.k
50
J.1 INPUT DISTRIBUTIONS
We here describe the input distributions related to each task in the experimental results.
LLM task templates
Generate even/odd number
Template
0-shot
"Generate strictly positive () integer."
Generate even/odd number Template 5-shot
You are a chatbot that follows instructions. 1 will give you some examples: Instruction Conorate strictly positive add integer, only generate the integer and - rothing else.
34

---

Published as a conference paper at ICLR 2026
Reqult:
31
instruction: Cenerate strictly positive even integer. only genezate the integez and nothing else.
Resuit:
94726
Instzuction Generate strietly positive odd integer, Only generate the Integer and nothing
Result:
Instruction: Cenerate a strictly positive aven integer. only generate the integer and nothing else.
Result:
Instruct tons Cenerate strictly positive ada Integer. Only generate the Integer and nothing else.
Result:
2339
Knowing this, please foiley next instruction.
Instzuction Generate strictly positive (F integez. Only generate the Integez nothing elze.
Generate an even/odd number Feedback template "Your snswez was incorrect. I asked foz [desired_ value) number, and you gave (last_str). please try again. Strietly write integer quotes write anything else."
Generate number of characters Template 0-shot "Write string of characters. strictly write the siring withous quotes and do not write w anything else."
Generate numbor characters
Template 0-shot
chatbot foliows rnnt you exasples:
Instruction: prite string of 2 characters. Result:
Instruct Rrite string charactere. Result:
Instruction: write a string 7 characters.
Result:
abodetg
Instruction: Brice string of 4 characters. Result:
cara
Trstruct tons Brite string characters. classifier
Kncwing this, please follow the next instruction.
rnst Brite string of !! characters. strietly write the string Nithout quotes 4 and do not writa anything else.
*on
35

---

Published as a conference paper at ICLR 2026
Generate number of characters Feedback template
"Your enswer was too (custom_stz), asked for string consisting of seeirred_valee cheracters, and produced with just__outuuts] characters. Flease :ry again. strictlyy write the string wt thout quotes and do not writa anything elae."
Generate formality Template 0-shot
"Generate story that is (:.25) formal. Write the story and only the story."
Generate formality
Template 5-shot
You are a chatbot that foliows inat tone. I will give you an exampler Instruction: Generate story that 1008 formal.
Result:
sir, please your way to the terrace.
Instraction: Generate atozy that 759 Zormai. Reselt:
She bought the book, then she went to the beach.
instruction: Generate story that is 568 formal. Result:
Rey! Just went the store.
ructions Generate atory that formal, Result:
How so tized.
ractioon Generste story that forma), Result:
key man how's it goin
Knowing this, pisaze follow the sext instruction.
Conerate story that 1g (J88 formal, write the story and only the story.
Generate formality Feadback template
"Your answer was too (formal str). asked for formality level of (desired value: .2)), and you gave with formality (100 Last_outpulst.22 Please try again. Keile the story and only the szory."
Generate word length | Template 0-shot
"Generste sentence where the average word length exactly letters. Strictly write the sentence without quazes and do not write anything else."
Generate word length | Template 5-shot
chatbot follows instructions. will give you example:
Instruction: Generate senteace where the average word length exactly letters. Result:
The pig jumped abeve her.
Instruction: Gemerate senteace where the average word length is exactly 3.7 letters.
She bought the book, then she went the beach.
instruction: Generate sentence where the average word length exactly 2.5 letters. Result:
Just
instruction: Generate sentence average length exactly letters. Result:
How I'm so cized.
Tost fons Generate aentence where the average word length is exact ly 10 letters.
36

---

Published as a conference paper at ICLR 2026
Result: Stop rediatribution!
Kncwing this, pleaze follow the next instruction.
Generate sentence where the average word lengon is exactly IJ letters. Strietiy write + the sentence without quotes and do not write anything eise.
xun
length Feodback tomplate
"Your enswer was too (custom stx). asked string whose average word length
[desired_valuee value:. .2f) letters, and produced one with average word length
(last_ outrooutputeeee.e 2f). Please try again, Strictly write the sentence withous quotes and do not write anything else."
Texa-to-Image task templates
Genorate N objects
Template 0-shot
"White background. () (object)."
object choices:
Generate object at position Template 0-shot "White background, A [object] at the (pos) of the Image." object choices: classes pas choices: (*top left", "top right", "bottom left", "bottom right", "center"]
Generate saturation
Template 0-shot
"[:22E) seturation."
K CONTROLLABILJTY PACKAGE
We open-source Python package test controllability and estimate reachable The package is completely built on Py PyTorch and supports loading of local or HuggingFace models to realize initial state input distributions, as well as the readout map. example usage is given below:
usage
toolkit
import from controllability. systems. systems.control system import ront System from controllability verifiers. reachability import Reachabilisy from controllability. verifiers. wetifiers.controllllle controllability import Contrellebility
Define initial
Input diatrsbutions
input_distcibution
Deffnee control system
cialogue_proces: ControlSysten. fron_medel_aame
poogle//emmm-3--b--1
output_map-"huggl
http://ggnnnaaee/tttttttlole fler",
output_ sulput__spcce1100 [[0, 1]],
input_ dtt..diiiiiiiioonnnnee
ott.aaooiiiooodsssiiitttoooo distribution,
model_config= sab_sammle=Tzue, ...)
Reachabilitp lity exper!ment
time__oo ison teachability beeeiiiilittyy.eiiieee - "quantized", tieloeu
imput_distribution p=0.05, delta=c.05
37

---

Published as a conference paper at ICLR 2026
teachable_ reachability vew.eeieetet.e...............eee wet.iiiiee..et.eeehlaeeeeeeeeeee cime_horizon
"
con lay experiment
cont collrollsbbllltyyeeelele
reachability_..rrifeee opsilor=0.05,
alpha=8.1,
delta-6.05
controllable_set controllabilltyy veyii oet../eeeiiee.......oooollllll. get_controlllble__ee
time_horizoz
USE OF AI WRITING ASSISTANCE
We acknowledge the use of large language model (LLM) assist in refining the language, gram- mar, and clarity of this manuscript. All content was originally drafted by the human authors, and all Al-generated suggestions were critically reviewed, edited, and approved by the authors, who retain full responsibility for the final text.
38


Reviews:
[{"review_id": "LqXgxALMEM", "reviewer": "Reviewer_A4fa", "summary": "This paper provides a novel and rigorous control-theoretic formalization of dialogue processes (DPs) in generative models, introducing definitions of reachability and controllability under a probabilistic and coarse-grained framework. The Monte Carlo–based estimation with PAC-style bounds is theoretically sound and well-motivated.\n However, several conceptual and practical issues remain: the definition of controllability in discrete, high-dimensional generative systems is somewhat ambiguous; key assumptions (boundedness, independence) are idealized . Overall, it is a theoretically interesting and well-written paper, but its practical implications and empirical validation need strengthening.", "questions": "see Weaknesses", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper proposes a control-theoretic framework to analyze and estimate the controllability and reachability of generative models (LLMs, T2IMs) viewed as black-box dynamical systems. The method provides PAC-style probabilistic guarantees for estimating controllable sets using Monte Carlo sampling, under minimal assumptions.\nStrengths：\n1.Theoretical originality – Introduces a formal and model-agnostic definition of controllability for generative systems, offering a new lens to study model behavior. \n2.Sound probabilistic formulation – Provides PAC-based guarantees that make the estimation process interpretable and principled.\n3.Clear empirical illustration – Demonstrates differences in controllability across models and tasks, supported by open-source implementation.", "weaknesses": "Conceptual Level: Ambiguity of Controllability Definition\nThe core concept of the paper, controllability, lacks a clear physical and semantic interpretation in the context of generative models. In classical control theory, controllability implies the existence of a set of inputs capable of steering the system from any initial state to any desired final state. However, for LLMs or T2I models: The boundaries of the state space X and input space U are semantically unclear;  Prompts or noise vectors are not strict control signals—they are discrete, symbolic inputs mapped nonlinearly to a latent space。Suggestion: Include an analysis (e.g., in the appendix) of the mapping between control inputs and semantic behaviors, clarifying the real-world meaning of the chosen control variables.\n\nExperimental Depth and Analysis of Controllability Failures：\nWhile the empirical examples, such as formality control and object counting, are illustrative, their interpretation remains relatively shallow. The study does not compare against simpler or alternative baselines, such as linear regression, RLHF-based control, or direct output editing. Moreover, the analysis of controllability failures—like overshoot phenomena—is limited to surface-level descriptions, without a deeper causal diagnosis. \nSuggestion: Include qualitative case studies or detailed error analyses to clarify why certain models exhibit limited controllability and to provide more actionable insights.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gYrpyYJEpB", "reviewer": "Reviewer_ANgd", "summary": "This paper proposes a controllability framework to assess whether black-box generative models are controllable prior to designing control mechanisms. The authors model user-model interaction as a control process and introduce Monte Carlo algorithms to estimate the set of achievable outputs. These algorithms are supported by probably-approximately correct (PAC) guarantees. Applying this framework to several LLMs and T2IMs, the paper concludes that controllability is a fragile, task-dependent property, challenging the implicit assumption that these models are generally controllable.", "questions": "Please see the weakness section.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The central idea of shifting focus from how to control models to if they can be controlled is a valuable and novel contribution. It brings a much-needed layer of formal scrutiny to a field dominated by empirical trial-and-error. \n\n2. The paper is well-written, and the authors do an admirable job of introducing concepts from control theory and adapting them to the context of generative models, particularly with their discussion of the discrete bottleneck.", "weaknesses": "1. The paper claims its framework can assess any control mechanism, including finetuning and representation engineering. Yet, the experiments are confined exclusively to prompting, arguably the weakest and least reliable form of control.\n\n2. The framework is not scalable to complex, high-dimensional control problems. The sample complexity scales with the number of discretized output bins, N. For any realistic task involving the control of multiple attributes simultaneously (e.g., style, tone, and factual accuracy), the resulting number of bins would become combinatorially explosive, making the method computationally infeasible. \n\n3. Furthermore, the analysis is purely descriptive; it tells us that a model is uncontrollable for a given task, but provides no diagnostic insight into why it fails or how one might improve the system. This severely limits its practical value beyond a simple and potentially misleading assessment.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6dYDYdNt8A", "reviewer": "Reviewer_N39G", "summary": "This paper provides an interesting stand - it formalizes user-model interaction as a control process, then gives PAC-style sample-complexity bounds for estimating reachable and (partially) controllable output sets of black-box generative models. A pair of MC algorithms (reachability and controllability) come with finite-sample guarantees. Experiments across a large variety of models suggest that, the controllability that we usually depend on is fragile and highly task/model dependent.", "questions": "Please refer to Weakness).", "limitations": "", "rating": 6, "confidence": 2, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1) The formalization of dialogue-as-control is clear and rigorous.\n\n2) The paper provides PAC-style, distribution-free bounds, avoiding intractable verification of global smoothness, thus suits black-box examinations well.\n\n3) The proposed algorithms are simple and easy to follow.\n\n4) The eventually alerting conclusion reveals the fragility of controllability in LLMs to the community, which could be of higher impacts.", "weaknesses": "1) Please correct me if I misunderstood this: how about when N -> \\infty? In practice this is easy due to the dimensionality curse, and thus the total demanded sampling number by THM1 can explode to some extent.\n\n2) The role of the readout h is a little bit less clearly demonstrated. In my understanding, isn't it the case that its calibration can be very hard in practice when considering the general case? I would appreciate if the authors can provide guidance for me to better understand how this part can work robustly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mBUFUFFgpx", "reviewer": "Reviewer_vGbb", "summary": "This paper establishes a theoretical foundation for controllable generative modeling from a control-theoretic perspective, framing generation controllability in terms of the classic concepts of reachability and controllability. It provides a rigorous theoretical proof to derive the conditions for controllability. Experimental results ultimately demonstrate that current generative models are fragile in terms of their controllability.", "questions": "See the weakness.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper introduces a control-theoretic framework to formalize the controllability of generative models. The theoretical analysis yields specific conditions for achieving controllability, offering valuable foundational support for advancing controllability algorithms and control input design.\n2. Through evaluations across different task configurations and various LLMs, the paper demonstrates that controllability performance varies significantly with both the task and the specific LLM employed.", "weaknesses": "1. According to the Kalman controllability rank condition, the controllability of a dynamical system is determined by its intrinsic properties. For an LLM, if its inherent capability in a certain attribute is weak, we can postulate that its controllability regarding that attribute will consequently be poor. For instance, a 3B parameter model inherently possesses limited mathematical and reasoning capabilities. Therefore, if evaluated on larger models with significantly enhanced overall abilities, does the authors' conclusion that \"LLM controllability is fragile\" still hold?\n2. As noted in reference [1], if the attributes manifested in the LLM's input are opposite to the target attributes, a substantial and longer control input is required for correction, which also reflects weaker controllability. Thus, in the experimental setup, should the authors clarify the input conditions and endeavor to avoid inputs with such opposing attributes?\n\n[1] Fundamental Limitations of Alignment in Large Language Models", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)