Activation Steering with a Feedback Controller
Authors: Dung Viet Nguyen
Yen Nhi Pham
Hieu M. Vu
Lei Zhang
Tan Minh Nguyen
Venue: ICLR 2026 Poster (2026)
Keywords: activation steering
behaviour control
alignment
PID control
mechanistic interpretability
language models
Primary area: foundation or frontier models, including LLMs
TL;DR: We propose Proportional–Integral–Derivative (PID) Steering, a principled framework that leverages the full PID controller for activation steering in LLMs.

Abstract: Controlling the behaviors of large language models (LLMs) is fundamental to their safety alignment and reliable deployment. However, existing steering methods are primarily driven by empirical insights and lack theoretical performance guarantees. In this work, we develop a control-theoretic foundation for activation steering by showing that popular steering methods correspond to the proportional (P) controllers, with the steering vector serving as the feedback signal. Building on this finding, we propose Proportional-Integral-Derivative (PID) Steering, a principled framework that leverages the full PID controller for activation steering in LLMs. The proportional (P) term aligns activations with target semantic directions, the integral (I) term accumulates errors to enforce persistent corrections across layers, and the derivative (D) term mitigates overshoot by counteracting rapid activation changes. This closed-loop design yields interpretable error dynamics and connects activation steering to classical stability guarantees in control theory. Moreover, PID Steering is lightweight, modular, and readily integrates with state-of-the-art steering methods. Extensive experiments across multiple LLM families and benchmarks demonstrate that PID Steering consistently outperforms existing approaches, achieving more robust and reliable behavioral control.

Published as a conference paper at ICLR 2026
ACTIVATION STEERING WITH A FEEDBACK CONTROLLER
Dung N.Nguyen
Nhi Y. Pham'
Department of Mathematics National University of Singapore dungnvêu. nus.
Center for Al Research VinUniversity, Hanoi, Vietnam nhi nhi. mni.py2@vinnni edu. vn
Hieu M. Vu" Independent
Lei Zhang
Department Mathematics National University of Singapore ma matzh l e @nus. edu sg
vmhieul 76 gma com
Tan M. N
Department of Mathematics National University Singapore tanmn edu. sg
ABSTRACT
Controlling the behaviors of large language models (LLM) is fundamental to their safety alignment and reliable deployment. However, existing steering methods are primarily driven by empirical insights and lack theoretical performance guarantees. this work, we develop control-theod theoretic foundation for activation steering by showing that popular steering methods correspond the proportional (P) controllers, with the steering vector serving as the feedback signal. Building on this pnartiionll-nterall--eeeeeee
(PID) Steering, a principled framework that leverages the full PID controller for activation steering in LLMs. The proportional (P) term aligns activations with larget semantic directions, the integral (I) term accumulates errors to enforce persistent corrections across layers, and the derivative (D) term mitigates overshoot by counteracting rapid activation changes. This closed-loop design yields interpretable error dynamics and connects activation steering to classical stability guarantees control theory. Moreover, PID Steering is lightweight, modular, and readily integrates with state-of-the-are steering methods. Extensive experiments across multiple LLM families and benchmarks demonstrate that PID Steering consistently outperforms existing approaches, achieving more robust and reliable behavioral control. The code is publicly available at: https://gittuhccoom/ngg///////// steering.
finding, we propose
INTRODUCTION
Large language models (L.MMs) have demonstrated remarkable capabilities across diverse domains, yet ensuring that their outputs align with desired behaviors remains a central challenge (Houlsby et al., 2019; Sclar et al., 2023; Kotha et al., 2024; Luo et al., 2025). Common post-training approaches (Wei et al., 2022; Ouyang et al., 2022) have proven effective for improving alignment. However, these techniques demand substantial computational resources (Houlsby al., 2019) and require weight updates with new training data, which can unintentionally degrade fluency or performance on unrelated tasks (Kotha et al., 2024; Templeton et al., 2024; Luo al., 2025).
An increasingly popular alternative is activation steering, which modifies a model's 's internal activations directly at inference time, avoiding costly retraining (Turner al., 2023; 2024; Li et al., 2023; Rimsky et al., 2024; Lee et al., 2025; Teo et al., 2025; Rodriguez et al., 2025; Vu & Nguyen, 2025). This approach has been employed both probe internal representations (Geiger al., 2024; von Rütte al.,
"Co-first authors. CCo-last authors. Correspondence dungnv @ u.nus.edu & tammn @ nuu..udu.s

---

Published as a conference paper at ICLR 2026
Contri Theay
Poll 1)
PID Do moodo
2[8]
2
Oure
Actvitionn Mesading
Divecion
Feectack
wilo (TC
sioa 1I
Per Cul 201
Figure 1: Our paper connects LLM Behavior Con- trol, Feature Attribution for LLM and Control Theory. Specifically, we apply controoroll to compute the steering vector for activation steering.
Figure 2: PID Steering: To compute the steering vec- tor u(ke): PID controller is applied at every layer ((h) (-), using the diff-in-means between 2 contrastive data Tsp(A) (k) and z²k) as the error signal e(k).
2024; Vu & Nguyen, 2025; Teo al., 2025) and enable fine-grained behavioral control (Zou et al.,
2023a; Li et al., 2023; Rimsky et al., 2024; Turner et al., 2024; Vu & Nguyen, 2025; Rodriguez et al., 2025). Recent work demonstrates that steering along carefully chosen low-dimensional directions can effectively alter model behavior (Zou et al., 2023a; Turner et al., 2024; Rimsky et al., 2024; Arditi et al., 2024; Vu & Nguyen, 2025), highlighting its potential as a lightweight yet powerful alignment strategy.
Steering through the Lens of Dynamical Systems. Recent methods leverage the geometric structure of the activation space (Marks Tegmark, 2024; Park al., 2024) using linear algebraic techniques (Zou al., 2023a; Turner et al., 2024; Rimsky et al., 2024; Arditi et al., 2024; Vu & Nguyen, 2025) to compute the steering vectors. While effective, these works oversimplify the complex, dynamic behavior arising from the regr-seeee nature of LLMss. When viewed through this dynamical lens, activation steering can be interpreted as guiding the model's 's trajectory through activation space, from a region encoding one concept another, analogous steering dynamical system from one state desired target state.
Contribution. Building on the aforementioned dynamical system insight, our work departs from the prevailing algebraic framing and instead adopts tontroi-theoretic perspective activation steering. Although recent studies (Luo et al., 2023; Soatto et al., 2023; Kong et al., 2024) have begun exploring this direction, their focus has primarily remained the level of the token-level generation proceses, treating high-level behaviors as control signals. In contrast, we take into account the internal mechanisms of LLMs by modeling the layer-wise construction of feature directions (Bricken et al., 2023; Park et al., 2024) as a dynamical system. These feature directions are then used as steering vectors (Zou et al., 2023a; Turner al., 2024; Rimsky et al., 2024; Arditi et al., 2024; Vu Nguyen, 2025). Specifically, we show that existing steering methods relying on difference-of-meant feature directions (Rimsky et al., 2024), including Activation Addition (ActAdd) (Turner et al., 2024), Directional Ablation (Arditi et al., 2024). and Mean Activation Transport (Mcan-AcT) (Rodriguez et al., 2025), can be interpreted as instances of a proportional (P) controller, thus suffering from the steady-state error due to the disturbance the state of the system (Aström & Hägglund, 1995b). This new perspective enables the application of principled control- theroeetcc strategies for extracting effective feature directions and computing steering vectors, thereby offering stronger robustness and performance guarantees for activation steering methods. An overview of our approach is shown in Fig. and 2. In this paper, we use the terms feature direction and steering vector interchangeably, noting that steering vectors represent a practical application of feature directions in activation steering. Our contribution is three-fold:
1. Control-Theorede Formulation for Feature Direction: We develop new -hhror-tt framework for constructing feature directions/steering vectors along the layers of an LLM.
2. PID-Based Steering: We propose the novel Proportional--ntege integral-Derivative Derivative (PID) Steering, a control-theoretic framework for computing feature directions using PID controller to reduce the steady-state error inherent in existing activation steering methods (see Fig. 2).
3. Unified Theoretical Framework:
We demonstrate that common activation steering methods correspond proportional (P) controllers. This connection enables theoretical analysis that highlights PID Steering's 's advantages in reducing steady-state error and oscillations
We comprehensively validate our PID Steering across diverse modalities (text and image), downstream applications (toxicity mitigation, jailbreaking attack, and image style control), steering paradigms
2

---

Published as a conference paper at ICLR 2026
(ActAdd, Mean-AcT, and Angular Steering (Vu & Nguyen, 2025)), model families (Qwen2.5 (Yang et al., 2024), Gemma2 (Gemma Team et al., 2024), Llama3 (Dubey et al., 2024). SDXL-Lightning (Lin et al., 2024), and Flux (Labs, 2024)), and model scales (BB14B for language models and 3.5B-122 12B for diffusion models).
Organization. We organize the paper as follows: Section 2 reviews background; Section 3 links activation steering to P control and introduces PID Steering; Section 4 presents its theoretical analysis; Section provides empirical validation; Appendix A discusses related work; and Section 6 concludes. Proofs, derivations, and additional experiments are in the Appendix.
2
BACKGROUND
2.1 TRANSFORMERS
Decoder-only transformers (Vaswani et al., 2017) take an input token sequence q [q1.....7u]]] and map it to initial embeddings ((. (1)] Embed (q). The embeddings then propagated through K layers. At each layer k, the residual activation (k) for token is updated by self-attention and an MLP block, with normalization applied before (and sometimes after) these modules:
D iipostt-atta (k) = =z (1) + SelfAttn (R) (Norm(x, (k)))
xz a2,(k (k+1) = 3. post-stt kk) MLP(()
(Norm(() (Norm pppotttttt (k))).
In this paper, for notational brevity, we summarize the layered processing above as æ; + 1) = (( (ax))),, i= where f( encapsulates both the Belf-Attention mechanism and Multi-Layer Perceptron at layer k. Finally, the output activations from the last layer, (L(++)), are decoded over the model's vocabulary to get the next token y; Decode (z; (L++)) for subsequent generation.
2.2 ACTIVATION STEERING
Features such behaviors concepts are hypothesized to align with (approximately) orthogonal directions in activation space (Elhage et al., 2022; Park et al., 2024; Bereska & Gavves, 2024). Activation steering leverages this by modifying hidden states inference to amplify suppress specific features (Konen et al., 2024; Li et al., 2023; Marks al., 2025; Templeton al., 2024; Bayat et al., 2025). Recent approaches operationalize this by constructing feature directions, which act as steering vectors r for adjusting hidden states. These steering vectors are computed as layerwise differences in activations between datasets with contrasting concepts (e.g.. harmful vs. harmless), a difference-in--meurs approach (Rimsky et al., 2024), shown to effectively isolate salient feature directions (Turner al., 2023; 2024; Arditi et al., 2024).
2.2.1 APPLYING THE STEERING VECTORS
Two popular activation steering methods using steering vectors are: Activation Addition (Turner et al., 2024), and Directional Ablation (Arditi et al., 2024). Both modify the token activation æ(k) using the steering vector (k) layer such that the activation expresses the target concept or behavior. By setting x 20(1q) = Embed (q) and r(1) = O, these methods apply the steering vectors r(k) to the activation x (k),, k= (K) at each layer via steering function Paterr as follows:
<0kk 1,g) - Patee ((s- o source x(k.g) =
1,9)
(æ(k-1,q)) (ax forr g Dxource U U'D urgeet
(2)
We discuss here the details on how to design the steering function Patcer for each method.
Activation Addition (ActAdd). ActAdd and sets Patee (x (xk)) =(k) ar(k), where the coefficient a controls the strength of the effect.
Directional Ablation (Dir:bbltte.. DirAblate removes the feature by projecting the token activation onto the orthogonal complement, Pateer (4)) - -(k) rkk) (ok).
2.2.2 COMPUTING THE STEERING VECTORS
Non-sequential Mapping. Let us use the jailbreaking task as an example. In this task, we apply activation steering force the LLM to respond to harmful prompts (Arditi et al., 2024; Vu & Nguyen, 2025). In order to compute the steering vectors, i.c., refusal direction, for each layer E [K] and post-instruction token position i E 7, calculate the mean activation (ii,arge (k) for harmless prompts
3

---

Published as a conference paper at ICLR 2026
from D Duugett and souuoue (k) for harmful prompts from D source.
Mittaree (k)
Σ
(k.gg)),
Hs, (k))
xs (k:
(3)
DD (
DD Dilnanee
lange
We then compute the difference- inneeee----mmasss means vectors, = (k) 1 souuree (k), and use them as steering vectors. Optionally, among the difference-inn saanice-i--maant in-means vector r. (k) for each post-instruction token position i at layer k, we can select the single most effective vector r(k) = Select( (n))))
from this set by evaluating each candidate vector over validation sets D and D
target
Sequential Mapping. A non-sequential mapping neglects the causal dependency across activations, where outputs from one layer are passed to the next, i.e., z; (k + 1)= 1)+1)=
(a:(k)) Consequently, any intervention applied at one layer must be accounted for before introducing an intervention at the subse- quent layer. To capture this causal structure, Mean Activation Trunsport (Mean-AcT) in (Rodriguez et al., 2025) estimates the steering vectors incrementally at each layer as follows:
for x 1. Pstcor (x; (k- 1,0) r(k (k- q sourre
(4)
(k(kk (k) (x - 1. or q "D souucee UD
(5)
Murget (k))
Σ
Msource (k)
Σ
x:(k.q)
D Ditrgian
DD Doniio
iET Dinin
iET.gE Dinii
r r(k) Munger (1s) M (k:)
Like ActAdd, Mean-AcT sets Psteer (xr k),r(k))))))) (ok) (kk)) (k)(
2.3 CMOPORTIONAL-INTERRA---DIRIIIII
CONTROLLER
Proportional-Interr -nrrigga------eiiiatiieee (PID) control is feedback mechanism extensively used in control systems (Minorsky, 1922). It is valued for its simplicity, robustness, and effectiveness in a broad range of applications, from industrial automation to robotics and acrospace systems (Visioli, 2006; Borase et al., 2021; Nguyen et al., 2024). The core idea behind PID control is to compute a control signal based on the error between target reference signal and the actual output of system. Specifically, consider a continuous-time dynamical system governed by state space model
(t()= - ((()()))(())))) ((1),)),))) (D) (1(((()) (u().)),
where xtt) E Rd denotes the state variable, u(t) E RT the control variable, and y(t) E Rd represents the measured output signal. Here, g RRd RT Rd specifies the system dynamics, and Rd x Rm TR'" is output mapping. A PID controller applies the control variable utt) to minimize the discrepancy between a target reference, or also known as the setpoint in the literature of PID control, (t) and the actual output (t). This discrepancy, called control error, is defined e(t)= = ((p)=)yptt) (tp((tt)()) yt)..
(8)
In a PID controller, the control variable ut)) is composed of the proportional (P), integral (I), and derivative (D) terms and given by:
S. ((T)dd+ de(l) (t) = K)) ((()) K;
dt
where K,,KK,KKZ 20 are the proportional, integral, and derivation gains, respectively. In PID control design, the P. I, and D play different roles: Proportional term (P) outputs a correction proportional to the euroonterror et, but alone leaves steady-state offset; Integral term (I) accumulates past errors remove residual bias, ensuring offsets corrected even proportional effects fade; and Derivative term (D) responds to the error's 's rate of change, damping rapid growth to improve stability and reduce overshoot.
State- Feeeb-ee PID Controller. A special case of the PID controller is obtained by choosing the measured output (f) to be the state variable x in Equ. 7, yielding the following state-space model
a(t) = (t)) ()) xt)..
(10)
The control error then becomes the state tracking error, e(t) = (t) - =x(t,, and the system is controlled through feedback of the state (Aström & Murray, 2021).
STEERING WITH A FEEDBACK CONTROLLER
In this section, will formulate popular activation steering methods, such as ActAdd, DirAblate, and Mcan-A McT, as a stede-feedback P controller. Based on this new interpretation, we propose PID Steering, novel steering method that uses PID) controller.
4

---

Published as a conference paper at ICLR 2026
0.60
0.50
0.008
0.30
0.006
0.20
0.004
0.00
0.002
-0.10
-0.2g
100
200
(a) Randomly Initialized LLama3
(b) Pretrained Swen2.55-B-Insse
Figure Scalar enrors across time step randomly initialized model after applying PI, and PID controller.
3.1 ACTIVATION STEERING AS A P CONTROLLER
We consider the state-feedback PID controller given in Eqn. 10 and the continuous steering vector r(() in which we replace the layer index k by the time index l. Substituting the state tracking error e(() by the difference-in-means vector r() and using the P controller whose system dynamics is governed
by (t) = (Pscer (x()) (t)),) xt),, we obtain
c =
=x())-
(11)
We discretize Eqn. 11 using Euler method (Euler, 1768; Hairer et al., 1993) to obtain
azkk - x -k- 1)= = (() (Psteer (x r (k- 1))) x (k-1). 1),
or equivalently,
x = f()) (psteor( 1) pr ( 1))),
(12)
where (.)=(((;())))) a function depending on index k:
Comparing Eqn. 12 with Eqn. and shows that applying the steering vectors as in Section 2.2.1 is equivalent to implementing the P controller, where ((6 is the k-th layer in an LLM, u(l) = (()
is the new steering vector. Thus, activation steering computes the expected state tracking error.
(t)) (t)==(tt==) =E qap E EDiiii [z (t.gsp) E
qE Dimnia [z)(tg))]
(13)
This expected state tracking error, i.e., the difference- in -eane vector r(t), can be computed non- sequentially or sequentially, explained in Section 2.2.2. When r() is computed non-sequentially
and Pstee (areer(<(kk (k))))(k))))))) = arkk+ +au(k) or a(k) ( u(k) (k)) akkk,, we obtain ActAdd or DirAblate, respectively. When rff) is computed sequentially and Pawer (k),, u(() z(k) + akk()), attain
Mcan-AcT.
Limitations of P Controller. There is always steady state error in P control. The error decreases with increasing gain, but the tendency towards oscillation also increases. Since activation steering methods, i.e., ActAdd, DirAblate, and Mean-Act, P controllers, they share the same limitations. We informally state our theoretical guarantees that P-control activation steering methods cannot alleviate the steady state error in Proposition below and provide detailed proofs in Appendix B.4.
Proposition 1 (Steady-state error of P-control activation steering) P-control activation steering ensures input-to- stette stability (ISS) for an appropriate range of Kp However, there still exists a steady-state error due the disturbance w(k) to the state of the system. the best case, when w(k) converges to w, under mild condition, the expected error, i.e., the sifference-in-meat r(k)=e(( eventually converges to a steady state e.s x w. Therefore, C., #0 if w +0.
We further provide empirical validation of Proposition I and illustrate the effect of integral and derivative terms (discussed in Section 4.2) in Fig. 3. To achieve this, we apply Sequential P-control activation steering (P Steering) on randomly initialized model with 150 layers deep, and pretrained Qwen2.5-33 Insttrcct.... We plot the scalar signal (e(0) which measures how much of the mean emror at layer & remains aligned with the initial mean error (see Appendix B.6 for further explanation). nonzero plateau of this quantity indicates persistent component of the initial error (i.c., a steady-state error), whereas convergence to values near zero means that this component has been eliminated.
In this light, the behaviour in Figure 3 matches the theoretical discussion: the P curve (blue) decays but clearly settles on a nonzero plateau, confirming that P-control is ISS but admits a steady-state error. The PI curve (red) crosses zero and, after the trasient, remains close to it, meaning that the integral action eliminates the steady-state offsets in < e(O),e(k) >; the large negative dip before convergence is the classical overshoot of PI control, which we discuss in the "Limitations of PT control"
5

---

Published as a conference paper at ICLR 2026
in Section 4.2.1 1 of our main text and analysis in Appendix B.6. The PID curve (green) also settles near zero but with substantially reduced overshoot, showing that derivative action damps the PI transient while preserving its steady-state advantage. We analyze the overshoot reduction of PID control in Theorem in Section 4.2.2 of our main text and in Appendix B.7.2.
3.2
PROPORTIONAL-INTERR
(PID) STEERING
3.2.1 OVERVIEW
To overcome the steady-state error inherent in P-control activation steering, we extend the method by adding integral (I) and derivative (D) terms to the steering vectors. PID Steering thus (i) reacts immediately to errors via the term for greater responsiveness, (ii) removes steady offsets with the I term, ensuring convergence to the desired set point, and (iii) anticipates error trends through the D term, improving stability and reducing overshoot. Together, these properties yield the following advantages:
Generalization. PID Steering extends P-control methods like ActAdd, DirAblate, and Mean-AcT by adding integral and derivative components.
Methodological Agnosticism. Our PID framework can be applied across different activation steering techniques, including ActAdd, DirAblate, and Mean-AcT.
Stability, We theoretical prove and empirical demonstrate that PID Steering reduces steady-state error and overshoot in P-controllers, improving existing steering methods.
Interpretability. Derived from classical feedback control (Minorsky, 1922), the framework inherits the simplicity and interpretability that underpin the wide use of PID controllers.
3.2.2 COMPUTING THE STEERING DIRECTION USING A PID FEEDBACK CONTROLLER
Following Section 3. 1. consider the state-feedback PID controller in Eqn. 10, replacing the state tracking error e(t) with the difference-in-mmanss vector r()). With the PID controller governed by the system dynamics (t)
(x((), (e((t),((t)))) (t)) xtt), we obtain
(tt) (P (x((t (K), / r(r)dr+ Ka dr(t) dt ).t) -t))
(14)
Eqn. 14 defines the continuous time model PID Steering, whose steering vector is given by: dr(t)
r(T)dr+ dt
(15)
In order to obtain the discrete-time formulation of PID Steering, we first discretize the sytem dynamics (t(() = / (Pateer in Section 3. and attain
a1) using Fuler method (Euler, 1768; Hairer et al., 1993), same as
x (k) f()) (x (k=k)==kk))))))) ( -k))))))
(16)
Next, we discretize (t) given in Eqn. 15 to obtain u(k) using Lemma below.
Lemma (Discretizing PID steering vector) Consider the continuous PID steering vector defined in Eqn. 15. The discrete-time PID steering vector is given by:
k-1
a(k)=Krr(( + K, rr()+
Ka(p(k)-rrkk---- -a))
(17)
1=0
Proof of Lemma 1 is in Appendix B.I. With Eqn. 16 and Lemma 1, we now define PID Steering.
Definition (PID Steering) Given large language model whose layers are {f()) ( and
steering function Psteer, PID Steering constructs the steering vectors as follows:
k-1
u (k)=KKr((h K, > -a()))((r())-(((s
(18)
==0
where for non-sequential mapping,
r(k)=E
(xc (2 - DyDitoon/// [x),,q))
and sequential mapping,
=(k) (k) (psteer (z(k- (k 1), ( -1))); r(k) =E
q.p Dynan (Z (k. -B Ditnoa (2(kg)))

---

Published as a conference paper at ICLR 2026
4 THEORETICAL ANALYSIS OF PID STEERING
This section provides theoretical evidence for our claims: (i) adding integral action (PI) reduces stcady-state error that remains under pure P-control (Proposition 3); and (ii) adding a derivative term (PID) preserves bias removal while mitigating oscillations/overshhoo (Theorem and 2). We denote K = I K; = K,I, , and Ka== K.I. Detailed proofs are provided in Appendix B.
4.1 DYNAMICS OF THE AVERAGE ERROR ACROSS LAYERS
To formalize the problem, we consider N pairs of prompts/input tokens from two contrastive datasets, e.g. harmful and harmless, {(g,t .q, ) with corresponding activations (k:) TRd layer k. A steering input (k) perturbs the undesired branch:
x. (k+1) 1! (s, (1) (b)))(())).
(19)
Let e(k) :- N Ex (ax az, (k)), the error dynamics of activation steering is then given by Proposition below.
Proposition 2 (Error dynamics of activation steering) The error dynamics @(k) in activation steering of the form:
e(k+1)= -(())(()))())(((((k))))--)))))-) (((()
(20)
where (()) is the mean local Jacobian of !!(h) at a. (k)) and the disturbance term wrk) collects heterogeneity, See Appendix B.3 for detailed proof and explanations of the terms.
Our control objective drive E(k) zero with input-to-state stability (ISS) for disturbed discrete system Eqn. 20 (Jiang et al., 1999; Edwards et al., 2000).
4.2 STABILITY OF THE ERROR DYNAMICS: ROLES AND CAVEATS OF PI AND PID CONTROL
In the following stability analysis, we consider the orthogonal decomposition for the disturbance w (o(k)=wl) =x)) (k)++ (k), where w (k) E ImA(k) and w + (k) E (ImA(kk))
4.2.1
CONTROL
The following proposition provides a theoretical guarantee of PI Steering's steady-state error reduction.
Proposition 3 (Stabilizing the PI loop reduces steady-state error) Let M2(k) = A(A((I Kp), Capp||AA(A) [|((])] & M < oo and sup, M| S <. I then the Pl closed-loop control is ISS. Furthermore, the integral part exactly cancels the matched disturbance component will The remaining error is due only to the unmatched component w which cannot be compensated. Full proof and term explanations provided in Appendix B.5
and denote | h. Assume sup,
Limitations of P1 control. Overshoot is common under Pl: the closed loop oscillates about the setpoint before settling (Aström & Hägglund, 1995a, Ch. 3, $3.3, pp. 68-69), and large overshoot arise with a high integral gain K; In our steering setting, we explain this by scalarizing the dynamics along a reference direction. The scalarized integral state accumulates past error, pushing the trajectory beyond the setpoint; when the scalarized error changes sign, the integral discharges and the error subsequently approaches zero. See Fig. for an illustration and Appendix B.6 for the formal derivation.
4.2.2 PID CONTROL
The derivative action counteracts Pl-induced oscillations near the setpoint by responding to decreases in the scalarized error, while preserving the integral term't 's bias-removal role, as shown in Theorems and 2. For detailed proofs and explanations, see Appendix B.7.
Theorem (Stabilizing the PID loop preserves bias removal) Let M2(k) = A(k) Kp), and denote || K, =: h, [Ka|== Assume sup ||k))l S M < oo and sup; M, s < If + Mh < 1 (stable PI loop), then there exists E E>0 O such that the PID closed-loop control is ISS. Therefore, the integral part PID design still cancels the matched disturbance component wil.
Theorem (PID reduces the first-overshoot amplitude) Let the first overshoot occur at index ko with amplitude An (definition in Eqn. 54). Then, the first-overshoot amplitude under PID Steering, APID, satisfies A < API, where APl denotes the corresponding amplitude under P1 Steering.
To support the theory, present empirical evidence in Fig. 3. PI and PID controllers clearly improve over P-only control: PI removes steady-state error but causes large overshoot, while adding the derivative term mitigates overshoot and enables faster, cleaner convergence zero.
7

---

Published as a conference paper at ICLR 2026
Table I: Toxicity mitigation results for Gemma-2D and Llama-SB, averaged over 10 runs. Lower is better for toxicity and perplexity; higher is better for MMLU. Bold best, underline second-bess -est within each model."
Seq.
CLS Tox, (%)
0-sbot Tox, (8 (%)
(V)
PPL Wikipedia
PPL Mistral-7B 1 MMLU
Original
4.17 so
13.42a) as
14.17ar
13.98
6.68
53.1
AcADD
13.43
14.17
11.69
1.67
13.00.us.
1.20go so
5.35gn
5,88 ss
14.80g to
6.85gm
51.70ga
Gemman-228
AURA
2.12
9.04
9.72
14.18
7.0420m4
53. 63.00...0
ITI-C
0.74
5.961om
6,102o.a
11.90
7.Hiow
52 22.6.
Mcan-AcT
1.12
5.20
5,802o
14 13
6.8 1.
51. 74.774
Lincar-AeT
0.9G
1.92
11.75
7.21iose
51. 11.631.a.
Mcan-AcT
V
3.23
3.70pm
14.9220.s
6.97
51 1828 6o
Lincar-ATT
1.00
1.641oo
11.98..aa
7.13
51. 11.17 1o.0
PID-AcT (Ours)
3.90ga
3 4.20a na
15.22
7.02gm
51 31.30gn co
Original
5.80
15:00
15.8
5.68
65.30
Ac
15. 15.73ama
16.48a
9.71
6.8
65.50.2
CAA
1822
6.70
7.40z
9.40.o.as
5.50
64.30
Liuma3-88
AURA
8.12
8.80gn
9.52gn
6. 6.05ee
65.50gn
ITI-C
1.00g
6.53gn os
7.19gn os
9.48
6.1 6.17am 14
64.70go
Mean-AcT
1.78 1o20
6.56
9.36 lo.2e
61.35 1o.0
Linear-AcT
1,87
6.50gg in
7.30gm
9.35 17
5.502g os
64.50go as
Mean-AcT
1.21
5.09
1.73
61. 1o.c0
Linear-AcT
/
18
6.47
7.12
9.48
5.46
64.49go
PID-ACT (Ours)
/
0.7220.40
2.3620.91
4 2o.0
9.56
6. 6.0810.
64. 64.50zo
CONTROLLING THE STEERING EFFECT
In this section, we demonstrate the applicability and effectiveness of HID-Stering by using it as a drop-in replacement for the steering vector computation step across multiple steering frameworks.
5.1 TOXICITY MITIGATION
We evaluate the effectiveness of PID Steering for toxic language mitigation in comparison to sequential steering methods, specifically Linear-AcT and Mean-AcT (Rodriguez et al., 2025), by closely following their experimental setup. We apply PID-Steering into Mean-AcT and call it PID-AcT. Results from other baselines, namely ActADD (Turner et al., 2023), CAA (Rimsky et al., 2024), AURA (Suau et al., 2024), ITI-C (Li at.,, 2023), are also reported.
Experimental Setup. Our evaluation is conducted on Gemma2-2B (Gemma Team et al., 2024) and Llama3-8B (Dubey et 2024). using 1,000 randomly sampled prompts from the Praltooxicityroomptt dataset (Gehman et al,, 2020). Toxicity is quantified with a ROBERTA -ased classifier (Logacheva et al., 2022), following the methodology of Suau al. (2024). We also assess toxicity in a zero-shot setting by employing Llama3-88 ----8B--Ireee as an LLM-as-a-judee (Zheng et al., 2023). Additionaly, in Tab. 1, we report QVQ (%) metric measured by QVQ-72B Preview (Qwen Team, 2024) as LLM a--------uuuge
To measure general utility of the intervened models, we report: (i) perplexity (PPL) on a fixed set of 20k Wikipedia sentences, (ii) PPI. of model-generated outputs evaluated with Mistral-7B (Jiang et al., 2023), and (iii) 5-ohot MMLU (Hendrycks et al., 2021) accuracy.
Results. PID-AcT achieves the strongest toxicity reduction with minimal utility loss across both models. As shown in Tab. 1. it lowers toxicity by up to 8.2x on Gemma2-2B and 8.1x on Llama3-8B under both classifier and LLM-judge evaluations. PID-AcT outperforms Mean-AcT and Linear-Ao AcT within the sequential family (Seq.. v) and surpasses strong activation-editing baselines (ActADD, AURA, ITI-C, CAA). Utility remains stable: perplexity is comparable to other baselines; MMLU accuracy aligns with Ac based methods and slightly lower than non-AcT approaches, likely reflecting properties of the ACT framework rather than our method, Overall, AcT-style methods yield stronger toxicity mitigation, with PID-AcT consistently ranking highest.
5.2 JAILBREAKING LARGE LANGUAGE MODELS
We evaluate our method on ActAdd within the Angular Steering framework (Vu & Nguyen, 2025) on the jailbreaking task, which seeks to override model's refusal behavior and elicit harmful outputs.
Experimental Setup. Following (Vu & Nguyen, 2025), replace DIM with our method and baselines RePE (Zou et al., 2023a) and ITT (Li et al., 2023). Refusal directions are built from 80% of ADVBENCI (Zou al., 2023b) and 512 harmless ALPACA (Taori et al., 2023) samples, with the
8

---

Published as a conference paper at ICLR 2026
(a) Cyberpunk concept.
(b) Steampunk concept.
Figure 4: Qualitative results of activation steering in FLUX-Schnell across style concepts with the prompt "Lady bent over with red polka dot umbrella inside a brick building."
remaining 20% for evaluation. General LM ability is tested on TINYBENCHMARKS (Maia Polo et al., 2024). We evaluate across Gemma2, LLaMA3, and Qwen2.5 models (3B-14B).
Table Comparison of Original, DIM, RePE, and PID across models on ASR and general benchmarks. Bold best, underline = second-boo beot within each model (ASR column). Refer to Tab. 3 for results on all tested models.
Method
ASK
tinyArct
GinGGSM8k striet?
IinTIruthQAT MayYMLUt
tinyHellaSwagt
tiny Win-iieGrr
Original
62.29
17.61
68.03
56.43
73.18
70.65
Qwen2.5-3B Instrict
DIM
74.03
61.95
14.80
66.11
54.95
72.40
69.85
ITI
70.19
61.28
15.57
06.62
54.75
72.71
70.12
RePE
68.41
61.05
14.60
65.70
51.30
72.03
69. 10
PID (ours)
76.07
61.20
18.01
67.29
54.10
72.59
69.72
Qween25. 1-1 Instruct
Original
73.96
90.12
74.60
64.50
82.70
73.77
DIM
90.38
72.74
87.01
74.30
63.01
81.94
72.53
ITI
33.65
73.15
89.27
74.55
64.03
82.24
73.31
RePE
25.12
72:10
86.20
T3.90
63.20
81.52
72.60
(ours)
88.96
63.60
82.60
Original
65.33
63.21
62.02
54.39
82.51
Ilaama3. 1-80 Instruct
60:57
00.96
54.17
81.73
79.80
54.33
82.01
RePE
70.42
60.00
53.70
81.35
PID (ours)
94.85
62:30
61.99
61.54
54.24
81.87
64.93
Gemma2 2-9B Instruct
Original
69.31
83.19
76.60
55.07
82.31
72.34
DIM
77.88
68.21
80.14
72.29
51.86
81.45
71.51
ITI
35.57
68.32
81:47
75.33
53.13
81.70
71.82
RePE
28.64
67.50
79.20
71.10
51.10
81.20
71.15
PID (ours)
79.50
67.91
79.24
74.89
52.49
81.39
71.42
Gemma2-277 Instruct
Original
73.45
86.91
76.11
61.36
83,24
75.47
DIM
74.03
72.13
84.70
74.59
59.49
81.79
74.21
ITI
37.36
72.84
86.38
75.51
60.85
82.83
75.11
RePE
24.19
71.03
84.00
73.90
58.79
80.82
73.44
PID
79.80
72.93
86.60
75.67
60.74
82.95
75.06
Results. PID Steering consistently outperforms DIM and scales robustly across models and metrics (see Tab. 2). On Qwen2.5-14D and LLaMA3.1-88. PID achieves the largest ASR reductions 92.7 % and 94.9 % exceeding DIM by 1.5-2 points, while maintaining almost the same performance, with marginal cost, on Benchanchsarks. Smaller models also see consistent gains: +2.0 ASR on Qwen2.5-3B and +1.3 on I.LaMA..2-3B In contrast, I'TI and RePE fail to scale, collapsing on larger models with ASR values of 33.7 and 25.4, respectively, on Qwen2.5- 1- A full version Tab. which also studies Qwen2.5-7B and Llama3.2-30 is provided in Appendix C.2.
5.3 IMAGE GENERATION STYLES CONTROL
We study activation steering in diffusion models using FLUX. FT.UX.1.Schnee J.S.helll''s denoising transformer (Labs, 2024), built on T5-XXI encoders (Raffel al., 2020) and requiring just 4 diffusion steps.
Experimental Setup. Following (Rodriguezeet 2025), we intervene on all normalization layers after most residual blocks FLUX. Style/cconceppexpprsssiiii measured by a CLIP zero-shot classifier with two labels (A picture of (style/concept] vs. A picture of something iineehhinng,. and content preservation by CLIPScore (Hessel el., 2021). Training uses 2,048 COCO Captions (Chen et al., 2015) prompts augmented with wyberpunk/steamppe modifiers from LLaMA-8B-Imm InsItuutttt (source unmodified p. target modified q). Evaluation samples 512 validation prompts to generate images across intervention strengths.

---

Published as a conference paper at ICLR 2026
0-shot (+)
0.90
0.27
O-sher
0.83
0.25
0.80
0.26
0.70
0.75
0.26
0.70
0.00
0.20 0.40
0.60
0.00
1.00
0.00 0.20 0.40 0.60 0.00 1.00
0.00
0.20
0.40
0.60
0.00
1.00
0.00
0.20
0.40
0.60
0.00 1.00
(a) Cyberpunk concept
(b) Steampunk concept
Figure 5: 0-shot and CL.IScoore results for "cyherpunk" and "stcampunk" concept.
lbmaguard3
Pw0.0
lbmaquardS K P=1.0
Iamaguorca
K.P115
CLS Tox
Pw0.0
CLS Tox.
K.P.1.0
CLS Tox
K P=1.5
73:00
77.5
0.13
76.03
0.13
77.37
10.33
0.13
0.75
3.00
0.13
0.09
0.00
0.13
0.09
0.59
0.
24.00
76.43
7757
33
77577
31.57
18.33
0.0
77.51
77.57
37.57
0.3
0.28
0.00
3.0
0.1
0.67
0.66
0.66
0.1
0.00
0.08
0.67
25.65
16.60
72.53
2661
16.53
16
0.03
06.11
19.01
0.66
3.65
6,0
0.1
0.00
0.6
0.4
0.0
12.38
3.0
97.50
66.92
1757
67.55
73.72
11.50
0.0
2.00
0.0
0.00
0.83
0.00
0.55
0.0
0.1
0.0
0.1
0.01
0.1
0.0
0.1
0.01
Imquugee
P.0.0
P-1.0
Imjudge
K P-1.5
0 shot Tbx
K. P=0.0
O-shot Tbx
P.1.0
0 Shsht Tox
P-1.5
20.50
01.22
0.13
82
30.82
00.30
0.13
02.38
02.07
30.28
4.28
4.72
0.13
4.23
4.10
4.00
0.13
4.99
4.03
0.7
51.01
50.30
57.50
3.
00.50
65.31
63:35
0.1
00.00
62.02
64.35
0.3
4.30
4.3
1.61
0.1
4.00
4.45
4.0
0.1
4.55
4.40
4.46
92.42
4.12
1:41
6.66
0.0
56
3.0
10:34
30
60:93
0.5
4.42
0.0
4.39
0.0
0.01
0.1
0.0
0.01
0.01
0.1
0.0
0.1
0.01
K.0
K_D
K.D
K.D
K_D
(a) -emm-em--mmane 2-9b-it on Jailbreaking task.
(b) Gemma---2222 2-22B on Toxicity mitigation task.
Figure 6: Ablation Study on ( KK..K, K2) parameters. We sweep through different values of K, Ka and report (a) Jamaguard3 (ASR in Tab. and Lmjjudge (evaluated using QVQ-72B-Preeieww metrics for Gemma-2-9h-it, and (b) CLSS Tox. and O-shot Tox. metrics for Gemma2-2B.
Results. In Fig. 4, increasing the intervention strength from O I produces a smooth and coherent shift in style-e.g. neon tones for cyberpunk, metallic textures for steampunk-we
preserving core content. Moderate strengths yield strong yet f'ittful stylization, and even at high strengths, semantic alignment remains largely intact. Quantitatively (Figs. 5), style intensity rises monotonically, with PID-AcT outperforming Mean-AcT at mid strengths (0.4-0.3 CLIPScore shows the expected trade-off: both methods decline steadily, with PID-AcT only slightly lower and by a small margin.
5.4 ABLATION STUDY: CONTROLLER GENERALIZATION ACROSS ARCHITECTURES
Figs. 6a and 6b reveal a remarkably consistent PID response surface across different models and tasks. In the -e-m- ITT jailbreak setting (Fig. 6a), increasing K, substantially reduces harmfulness under both A.L.MAguurrh and M.L.e, with the strongest gains concentrated in K; E [0.05, 15.0.11 0.1). However, large K; introduces instability when K, is small, visible degraded pockets at high integral gain; incorporating a small derivative term (K& E [0.01,0.05] reliably suppresses these oscillations and restores smooth, monotone improvements. A nearly identical pattern appears the Gemma-2-21 toxicity setting (Fig. 6b), despite differences in scale and objective: toxicity decreases sharply as K; increases from 0 to 0.05 before saturating, while K₃ stabilizes high-K, regimes particularly when K₁ K,=0. 0. The zero-shot toxicity metric follows the same geometry, with K; driving consistent gains and Ka preventing degradation at strong integral levels. Overall, across architectures, sizes, and evaluation metrics, K, primarily improves bias removal, Ka damps instability, and K₂ modulates amplitude without altering the underlying structure. This setss-s consistency supports our claim that PID behavior stems from architecture-agnosii error dynamics and generalizes without retraining.
6 CONCLUDING REMARKS
We introduced PID Steering, a centrol-theoretic approach activation steering that models layer-wise representations as a dynamical system. This framework unifies prior methods, offers robustness guarantees, and leverages PID dynamics for computing steering vectors. Across language and diffusion models, PID Steering achieves stronger and more stable performance than existing approaches in toxicity mitigation, jailbreak prevention, and style control, while preserving model utility. Our results highlight control theory as principled foundation for developing reliable and generalizable steering methods. A limitation of our work is the use of "stability-fire first, mm--ainn-tt aa-n--t---iie tt-a-time" analytical strategy to find controller gains: it clarifies the role of each component but may miss optimal choices and can overlook broader feasible regions. To address this, numerical methods, for example, LMI- based computations, be employed. We leave these for future work.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This research project is supported by the National Research Foundation Singapore under the AI Singapore Programme (AISG Award No: AISG2-TCC2023-012-5500
This research project is supported by the Ministry of Education, Singapore, under the Academic Research Fund Tier 1 1-8002033-0000001 This research / project is also supported by the NUS Presidential Young Professorship Award (1-0009807-01-000 and the NUS Artificial Intelligence Institute-Seed Funding (A-8003062-00-00)
(FY2023) (A-8002040-004
NYP acknowledges support from the Application Driven Mathematics Program funded and organized by the Vingroup Innovation Fund and VinBigData.
We would like to thank FPT for their generous support in providing discounted GPU access on FPT AI Factory, which enabled the experiments in this work.
Ethics Statement. Given the nature of the work, we do not foresee any negative societal and ethical impacts of our work. However, risks remain: by tightening activation-level control, PID Steering may inadvertently case the generation of nuanced harmful content (e.g., persuasive misinformation or biased narratives). Although it does not expand the baseline risk profile of LLMs, robust safeguards, transparency, accountability, and ongoing ethical review are required for responsible use.
Reproducibility Statement. Source codes for our experiments are provided in the supplementary materials of the paper. The details of our experimental settings and computational infrastructure are given Section 5 and the Appendix. All datasets that we used the paper are published, and they are easy to access in the Internet.
LLM Usage Declaration. We use large language models (L.LMs) for grammar checking and correction.
REFERENCES
Andy Arditi, Oscar Balcells Obeso, Aaquib Syed, Daniel Paleka, Nina Rimsky, Wes Gurnee, and Neel Nanda. Refusal in language models is mediated by a single direction. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL
Karl Johan Aström and Tore Hägglund. PID Controllers: Theory, Design, and Tuning. ISA - The Instrumentation, Systems and Automation Society, 1995a. ISBN 1-55617-516-7.
Karl Johan Aström and Tore Hägglund. PID Controllers: Theory, Design, and Tuning. ISA The Instrumentation, Systems and Automation Society, 1995b. ISBN 1-55617-516-7.
Karl Johan Aström and Richard Murray. Feedback systems: an introduction for scientists and engineers. Princeton university press, 2021.
Reza Bayat, Ali Kalahruua,, Mohammad Pezeshki, Sarath Chandar, and Pascal Vincent. Steering Large Language Model Activations in Sparse Spaces, February 2025.
Leonard Bereska and Efstratios Gavves. Mechanistic Interpretability for Al Safety A Review, April 2024.
Rakesh Borase, DK Maghade, SY Sondkar, and SN Pawar. A review pid control, tuning methods and applications. International Journal of Dynamics and Control, 9(2):818 8-8877, 2021.
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023. URL htt.://ooosmaaaicffferrrreeeeeee
hunps:/transforme--circs
Xinlei Chen, Hao Fang, Tsung- Yi Lin. Ramakrishna Vedantam, Saurabh Gupta, Piotr Dollar, and C. Lawrence Zitnick. Microsoft coco captions: Data collection and evaluation server, 2015. URL https://arii..rr/alb///15......
11

---

Published as a conference paper at ICLR 2026
Emily Cheng and Carmen Amo Alonso. Linearly controlled language generation with performative guarantees. arXiv preprint arXiv::405.15455 2024. URL httpss//aaatv.orr/bb//0000.....
Abhimanyu Dubey, Abhinav Jauhri, Abhishek Pandey, Akash Kadian, Ahmad Al-Dahle, Andrew Letman, Ankit Mathur, et al. The llama 3 herd of models. arXiv preprint 27X1::2407.217888, 2024.
URL httpp::/aaaii..r//aa///200.....
Heather Edwards, YD Lin, and Yuan Wang. On input-to-stade to-ttttettte stability for time varying non- linear systems. volume 4, pp. 3501 3506 vol.4, 02 2000. ISBN 0-7803-6638-7. doi:
10.109//DDC22000992224
Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfeld-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christo- pher Olah. Toy models of superposition. Transformer Circuits Thread, 2022. URL "https:/transformmr- circuits. model/index.htm)
L. Euler. Institutionum calculi integralis. Number v. I in Institutionum calculi integralis. imp. Acad. imp. Saènt., 1768. URL
Zoran Gajic and Muhammad Tahir Javed Qureshi. Lyapunov matrix equation in system stability and control. Courier Corporation, 2008
Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. Real ityPrompts: Evaluating neural toxic degeneration in language models. In Trevor Cohn, Yulan He. and Yang Liu (eds.), Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 3356-3369, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v1/2022 120.6666//112000..nnnnng-g-- -ottiingseem/p.301.... URL httpp:://ccaathooooo.o///// 2wt..200i000.0--eei.e. findings-emll.300/...
Atticus Geiger, Zhengxuan Wu, Christopher Potts, Thomas Icard, and Noah Goodman. Finding alignments between interpretable causal variables and distributed neural representations. In Causal Learning and Reasoning, pp. 160-187. PMLR, 2024.
Google Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. Gemma 2: Improving open language models a practical size. arXiv preprint arXiv:2408. 001 18, 2024.
Emst Hairer, Gerhard Wanner, and Syvert P Nørsett. Solving ordinary differential equations I: Nonstiff problems. Springer, 1993.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. Proceedings of the International Conference Learning Representations (ICIR)) 2021.
Jack Hessel, Ari Holtzman, Maxwell Forbes. Ronan Le Bras, and Yejin Choi. CLIPScore: A reference-free evaluation metric for image captioning. In Marie-Francine Moens, Xuanjing Huang. Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 7514-7528, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: lemnlp- main.595/.
10.18653/v1/2021.emmll
URL Utt..ttts//acclttooog..ooo/
Roger A Horn and Charles R Johnson. Matrix analysis. Cambridge university press, 2012.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. In International conference muchine learning, pp. 2790-2799, PMLR, 2019.
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample. Lucile Saulnier, Lélio Re- nard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang. Timothée Lacroix, and William El Sayed. Mistral 7b, 2023. URL https://aroooorggaa//2100..000
12

---

Published as a conference paper at ICLR 2026
Zhong-Ping Jiang, Eduardo Sontag, and Yuan Wang. Input-to-state stability for discrete-time nonlinear systems. IFAC Proceedings Volumes, 32(2) 1999. ISSN 1474-6670. doi: (6)661))66688-3 URL httpss//wwww.cccccccre...o////// Trttcee/pi$$44744677777556600000 1 14th IFAC World Congress 1999, Beijing, Chia, July.
Kai Konen, Sophie Jentzsch, Diaoulé Diallo, Peer Schütt, Oliver Bensch, Roxanne El Baff, Dominik Opitz, and Tobias Hecking, Style Vectors for Steering Generative Large Language Models. In Yvette Graham and Matthew Purver (eds.), Findings of the Association for Computational Linguisties: EACL 2024, pp. 782-802 St. Julian's, s. Malta, March 2024. Association for Computational Linguistics.
Lingkai Kong, Haorui Wang. Wenhao Mu, Yuanqi Du, Yuchen Zhuang, Yifei Zhou, Yue Song, Rongzhi Zhang, Kai Wang, and Chao Zhang. Aligning large language models with representation editing: A control perspective. arXiv preprint arXiv:2406. 05954, 2024. URL httpss///aaxxi.or//////4666 245559555..
Suhas Kotha, Jacob Mitchell Springer, and Aditi Raghunathan, Understanding catastrophic forgetting in language models via implicit inference. In The Twelfth International Conference on Learning Representations, 2024. URL http:////ppaaawwewww////////////
Black Forest Labs. Flux. http::/giihub.commbiic-fffressss
2024.
Bruce W. Lee, Inkit Padhi, Karthikeyan Natesan Ramamurthy, Erik Michling, Pierre Dognin, Manish Nagireddy, and Amit Dhurandhar. Programming refusal with conditional activation steering. In The Thirteenth International Conference Learning Representations, 2025. URL 10sm.
Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Inference-time intervention: eliciting truthful answers from language model. In Proceedings of the 37th International Conference on Neural Information Processing Systems, 2023.
Shanchuan Lin, Anran Wang, and Xiao Yang. Sdxl-lightning: Progressive adversarial diffusion distillation. arXiv preprint 2rX1v:2402.139922 2024. URL https://arxiv.org/abs/442201
Varvara Logacheva, Daryna Dementieva, Sergey Ustyantsev, Daniil Moskovskiy, David Dale, Irina Krotova, Nikita Semenov, and Alexander Panchenko. ParaDetox: Detoxification with parallel data. In Proceedings of the Annual Meeting the Association for Computational Linguistics (Volume 1: Long Pupers), pp. 6804-68 18, Dublin, Ireland, May 2022. Association for Computational Linguisties. URL http:://cclantoologgo////.....ed
Yifan Luo, Yiming Tang, Chengfeng Shen, Zhennan Zhou, and Bin Dong. Prompt engineer- ing through the lens of optimal control. arXiv preprint arXiv:2310.14201, 2023. URL
https//aaxiv.or//ab//311.0.
Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. An empirical study of catastrophic forgetting large language models during continual fine-tuning. IEEE Transactions on Audio, Speech and Language Processing, 2025.
Felipe Maia Polo, Lucas Weber, Leshem Choshen, Yuckai Sun, Gongjun Xu, and Mikhail Yurochkin. tinybenchmarks: evaluating Ilms with fewer examples. arXiv preprint axXiv:2402.14992 2024.
Samuel Marks and Max Tegmark. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. In First Conference on Language Modeling, 2024.
URL http:://openreviee.nnffffoo
HijdaajyyHjjskk
Samuel Marks, Can Rager, Eric Michaud, Yonatan Belinkov, David Bau, and Aaron Mueller. Sparse feature circuits: Discovering and editing interpretable causal graphs in language mod- els. In The Thirteenth International Conference on Leurning Representutions, 2025. URL http:///ppeeeevvvv.ee///mmmnnadd
Nicolas Minorsky. Directional stability of automatically steered bodies. Journal of the American Society for Naval Engineers, 34(2)2200-309, 1922.
13

---

Published as a conference paper at ICLR 2026
Tam Minh Nguyen, Cesar A Uribe, Tan Minh Nguyen, and Richard Baraniuk. PIDformer: Transformer meets control theory. In Forty-first International Conference on Machine Learning. 2024. URL
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 27730-27744, 2022.
Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models. In Forty-first International Conference on Machine Learning. 2024. URL
https://penneviiew.eefoooo http://wwwweefooumiiiiioooo DittummmddiGGGGGLLLw/pp
Xiangyu Qi, Ashwinee Panda, Kaifeng Lyu, Xiao Ma, Subhrajit Roy, Ahmad Beirami, Prateek Mittal, and Peter Henderson. Safety alignment should be made more than just a few tokens deep. In The Thirteenth International Conference Learning Representations, 2025. URL
Alibaba Qwen Team. Qvq: To see the world with wisdom, December 2024.
URL
https://wwenlmm.iiubbbb
72b-preview.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with unified text-to-text transformer. J. Mach. Learn. Res., (1), January 2020. ISSN 1532-4435.
Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong. Evan Hubinger, and Alexander Turner. Steering Lun-Wei llama 2 via contrastive activation addition. In Lun- Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Long Papers), pp. 15504-15522, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10. 118653/1/2024..cc/ lon..al..-......... 828. URL https://scanthoolgyoo////// acl- long.828/. 828/.
Pau Rodriguez, Arno Blaas, Michal Klein, Luca Zappella, Nicholas Apostoloff, marco cuturi, and Xavier Suau. Controlling language and diffusion models by transporting activations. In The Thirteenth International Conference on Learning Representations, 2025. URL
Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. Quantifying language models' sensitivity to spurious features in prompt design or: How learned to start worrying about prompt formatting. arXiv preprint arXiv:2310. 1324, 2023.
Stefano Soatto, Paulo Tabuada, Pratik Chaudhari, and Tian Yu Liu. Taming ai bots: Controllability of neural states in large language models. arXiv preprint arXiv:2305.1844 18449, 2023. URL
https///arxiv.or//bb//Z22 18445.
Xavier Suau, Pieter Delobelle, Katherine Metcalf, Armand Joulin, Nicholas Apostoloff, Luca Zappella, and Pau Rodríguez. Whispering experts: neural interventions for toxicity mitigation in language models. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org, org, 2024.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-folloving llama model. 2023.
Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam
Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas Turne, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan. Scaling monosemanticity: Extracting interpretable features from claude sonnet. Transformer Circuits monosemanicityindex..... Thread, 2024. URL https://transformer circuits, ont.
Rachel SY Teo, Laziz Abdullaev, and Tan Minh Nguyen. The blessing and curse of dimensionality in safety alignment. In Second Conference on Language Modeling, 2025.
14

---

Published as a conference paper at ICLR 2026
Alexander Turner, Sam Ringer, Rohin Shah, Andrew Critch, Victoria Krakovna, and Evan Hubinger. Activation addition: Steering language models without optimization. arXiv preprint DrXiv:2308.10025 2023. U httpss//aaniiw.rr/aa//223.0.. 102.4
Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan Vazquez, Ulisse Mini, and Monte MacDiarmid. Steering language models with activation engineering. arXiv preprint arXiv:2308. 10248 10248, 2024. URL URL.inss//aard...rrbb2220.... 10248.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Antonio Visioli. Practical PID control. Springer, 2006.
Dimitri von Rütte. Sotiris Anagnostidis, Gregor Bachmann, and Thomas Hofmann. A language model's is guide through latent space. Forty-first International Conference on Machine Learning. 2024. URL
Hieu M. Vu and Tan Minh Nguyen. Angular steering: Behavior control via rotation in activation space. Advances Neural Information Processing Systems, 2025.
Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, An- drew M. Dai. and Quoc V Le. Finetuned language models are zero-shot learners. International Con- ference on Learning Representations, 2022. URL http:////ooooooooooooo/oooo//oe/
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXhv:2412.15116 2024.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging LLM-as-a-judge with MT-bench and chatbot arena. In Thirty-sevntth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023. URL
Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann- Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun. Zifan Wang. Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song. Matt Fredrikson, J. Zico Kolter, and Dan Hendrycks. Representation engineering: A top-down approach to ai transparency. arXiv preprint 23Xiv:23 10.01405, 2023a. URL https///arxivvoor//as///22 10..//21100000000 405..
Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredríkson. Universal and transferable adversarial attacks aligned language models. arXiv preprint a3X5::2307.15043, 15044
2023b. URL
15

---

Published as a conference paper at ICLR 2026
Supplement to " "Activation Steering with a Feedback Controller"
Table of Contents
A Related Works
16
Theoretical Proofs
17
B.1 Discretized PID controller
17
B.2
Background on Input-to-state Stability & Notations
18
B.3
Dynamics of the Average Error Across Layers
18
B.4
Proportional (P) Control
19
B.5
Proportional-Integral (PI) Control
20
B.6
Oveshoot Mcchanism
23
B.7
PID Control
25
................................
B.7.1
Stability of PID closed loop
25
B.7.2
Overshooting under PID law of control
28
B.8
Beyond Locally Linearized Activation Dynamics
B.8.1
Nonlinear remainder in the averaged error dynamics
30
B.8.2
Magnitude of (k)) under P/PI/PID steering
C Additional Experimental Results
32
C.1 Qualitative Examples of Concept Steering
32
C.2 Jailbreaking Large Language Models
32
C.3 Empirical evidence for the stability interval
32
A RELATED WORKS
Recent works increasingly frame large language models (LLMs) dynamical systems, where gen- eration a trajectory in latent space. This view shifts activation steering from heuristic nudging to principled control: rather than biasing outputs without guarantees, controllers enforce constraints tra- jectories with formal assurances (Cheng & Amo Alonso, 2024). In our PID-steering framework, this dis- tinetion is key: treat the model plant with hidden states evolving under controlled interventions.
Controllability of LLMs. Soatto et al. (2023) model decoder-only LLMs as discrete-time stochastic systems and prove that, under idealized assumptions (Euclidean embeddings, well-trained semanties), they are controllable in the space of meanings. This establishes that any semantic state is theoretically reachable, but probabilities may be negligible practice, and emergent behaviors like chain-of-thought are not captured. Their results highlight both opportunities and risks: controllability opens adversarial attack surfaces but also suggests defensive controllers.
Prompting open-loop control. Luo al. (2023) recast multi-round prompt engineering as optimal control problem, where each prompt is a control input maximizing task reward. This provides unifying formalism for prompt strategies, akin to open-loop control. Yet, the framework remains conceptual: metrics in discrete language space are poorly defined, and no guarantees of stability or convergence are provided.
Closed-loop activation control. Cheng & Amo Alonso (2024) propose Linear Semantic Control (LiSeCo), which projects activations into safe subspaces at each decoding step via a closed-form controller. This yields lightweight, guaranteed control of simple attributes (e.g., toxicity, sentiment).
16

---

Published as a conference paper at ICLR 2026
However, the linearity assumption only approximates LLM embeddings, hwwpoidiggggggggtgegeeeoo are local rather than global, and Long-horizon stability remains unaddressed.
Dynamic representation editing. Kong al. (2024) introduce RE-CONTROL, which learns value function on hidden states and applies gradient- interventions at test time. This dynamic approach generalizes steering into Bellman-optimal control problem, balancing alignment with fluency. Still, accuracy of the learned value function is critical, test-time optimization adds overhead, and local interventions may not guarantee global alignment.
Together, these works move activation steering from heuristics to control theory. Soatto et al. (2023) prove fundamental controllability (but under strong assumptions), Luo et al. unify prompt strategies as open-loop control (without guarantees), Cheng & Amo Alonso (2024) derive closed-fore activation control (limited linear approximations), and Kong al. (2024) extend to dynamic optimal control (with overhead and approximation risks)
B THEORETICAL PROOFS
B.1 DISCRETIZED PID CONTROLLER
Implementing continuous-time controller on digital hardware, such as PID, requires discretizing its derivative and integral terms (Aström & Hägglund, 1995a, p.95)
Lemma (Discretizing PID steering vector) Consider the continuous PID steering vector defined in Eqn. 15. The discrete-time PID steering vector is given by:
u( (kk)=Kr((h K. Σ
(17)
Proof. We follow the discretization procedure for PID controllers in (Aström & Hägglund, 1995a, Sec. 3.6, Ch. 3). For simplicity, the sampling period is normalized to 1
Proportional term in Eqn. 15.
P(t) (,)()).
The discrete-time form is obtained by substituting sampled variables for their continuous counterparts:
(pr))),
(21)
Integral term in Eqn. 15.
I(t)=K, [ r(T)dr
dl KKr(t))
dt
Using forward Euler with 1=1,
Hence
[(++1)=[(4++<<<)) (Kr
which is equivalent
--1
A-1
I(k)=1(00+ K, Σ r(f) =( rr()
(22)
j=0
j=0
since =0.
Derivative term in Eqn. 15.
D(t)=Ka Ka dr(1) dt
Approximating the derivative by the backward Euler difference with = gives
D(k) -
(23)
Combining equation 21, equation 22, and equation 23 yields
= (ik)=Krr(K + K. Σ
(r(r()) - 1))-
1=0
17

---

Published as a conference paper at ICLR 2026
B.2 BACKGROUND ON INPUT-TO-STATE STABILITY & NOTATIONS
Background on Input-to-state Stability (ISS) In our proofs, the input-to-state stability (ISS) of system can be established either through the definition of an ISS system in (Jiang et al., 1999, Def. 2.1) and (Edwards et al., 2000, Def. 2.1). or via the use of an ISS-Lyapunov function, as outlined in (Jiang et al., 1999, Def. 2.2, Prop. 2.3) and (Edwards et al., 2000, Theorem I). We also rely the definition of a Lyapunov function and the difference Lyapunov equation for linear discrete-time time homogeneous dynamical systems in (Gajic & Qureshi, 2008, Ch. 1, p. 8). The existence of a solution the Lyapunov equation, together with its bound, is stated in (Gajic & Qureshi, 2008, Ch. 4, 110).
For reference, we briefly note that input- souut-tt--tttee stability (ISS) extends the classical notion of Lyapumov by explicitly accounting for external inputs: the state remains bounded and eventually converges whenever the input bounded. A Lyapunov function provides energy-like certificate for stability, while the associated Lyapunov equation offers a constructive method for obtaining such functions in linear settings. These notions are central for analyzing stability and will be used throughout our proofs.
Conventions and assumptions (used throughout). Let ||-I| denote the Euclidean norm on R2; for matrix M gaxd we also write ||M]] for the operator norm induced by the Euclidean norm, i.c. ||M]| == ||P||=/|=1|| Mxx] (the spectral norm) (Horn & Johnson, 2012, pp. 343-346). We assume (i) sup, Ak)]l oo; (ii) w(k) is bounded (for a signal
set || x= sup,, ||w (x)]l) (iii) the controller gains are static and time-invariant scalar multiples of the identity, i.e., K, = KTI, K;: K,I, and K, We use the standard meaning of the classes K. and K. as in (Jiang al., 1999).
B.3 DYNAMICS OF THE AVER AGE ERROR ACROSS LAYERS
To formalize the problem setup, we consider N pairs of contrastive prompt/input tokens ((q," .9, )) where q,ºt carries the desired property and q, represents the opposite. For discrete time (layer) k; let x (k) E Rdd denote the corresponding activation vectors. The layer-to-layerr evolution
az (k(k+) = 1:a² i==
(24)
with f/ :R + Rd differentiable on the operating region. A steering input ulk) is applied on the undesired branch:
x (k+ f.(h) (x, uk)).
(25)
Defining a (k) = M 1 x. (k)) we track the per-pair and average errors as
e, (b: :0 xt (k) x, (k:), e(k):= (k) x (k))
(26)
Furthermore, we define (k) as the Jacobian of g!(() at क. (k)):
N
A (zt (k)),
A(t) = N Σ A.K)),
(27)
The dynamic of the average error ē(k) is then given by Proposition 2.
Proposition 2 (Error dynamics of activation steering) The error dynamics E(k) in activation steering of the form:
e(k+1)= A(k)e(k))) A(k))(k)++w(h
(20)
where A(k) is the mean local Jacobian of f/(() x (k) and the disturbance term w(k) collects heterogeneity. See Appendix B.3 for detailed proof and explanations the
the terms.
Proof. The evolution of the average error e(k) through layers can be described as follows:
N
1
(k+ (k+1)=
f! (2 (k)) (x) (k) (k)+
(28)
N
Linearizing (.M) around .r. (k), we obtain
5(x)) (ao (k) (,;k)))))))))))))))))))))))))))) ~ (x) (k)) + ,000 (xt
(29)
where J ea denotes the Jacobian of l.(
18

---

Published as a conference paper at ICLR 2026
Setting 6,(k)= (k) (k) yields
(/4)) ( ( 5 (k) (x; + (((())c((())aaa))
Insert this into Eqn.28 we obtain
N
e(k+1)=
N Σ -(()))()))(())u(())))
(30)
Recall that
N
e(k))=((k)++ + ((()) then
Σ
(k)
N
i=1
and
N
A0((1= = A(+)+ (A)()),, then
(o)
-0) =0.
N
Therefore,
1 (()():(4)AA())((1
E(k+
N
(())ē A(())((k) + Σ (o) (1))))A) (k)
(31)
N
N
A (k)
é (k)
(k) (k)
1=1
We then obtain the final state-space model for the dynamics of E(1)
((++1)=A(k)e(t>) ((k))
(32)
where
N
w(k)
(k),
N
which acts time-dependen exogenous disturbance to the model B.4 PROPORTIONAL (P) CONTROL Consider proportional control with
akk)=Kpp K,ē(k), (K,=K,=0)
(33)
The dynamics Eqn. 20 then become
e(k)= (+))))((()((
(34)
where (
With a suitable choice of Kp, the system can be made input-to-state stable (ISS); that is, there exist KCC-function 3 and /C-function 7 such that, for all disturbance with bounded sup norm and all initial states e(0),
|[(A)]] < ((I)()))) + >(1wwlloo) ke ZzZo
(35)
see (Jiang et al., 1999, Def. 2.1).
In particular, the error decays from the initial condition and remains bounded under bounded disturbances.
Proposition 1 (Steady-stade error of P-control activation steering) P-control activation steering ensures input-to-ste tt-tt- stability (ISS) for an appropriate range of Kp. However, there still exists a steady-state error due the disturbance w(k) to the state of the system. In the best case, when w(k) converges to w, under a mild condition, the expected error, i.e., the difference inneenne-i-mmas,,, r(k) = =((k) eventually converges to steady state e.s w. Therefore, C., #0 if w +00
19

---

Published as a conference paper at ICLR 2026
Proof. Assume supk ||A(K)] I M < ce, K, K pl with p 0. Since
M₂ - ( Kp) (1) -p) by aub-multiplicativeprppos property of matrix norm we have
(36)
P (1- (1-,,1+ M/ we have 1<.
Expanding recursively,
k-1 +
e(k) =
2
Hence,
1-9x || 5q -1 1-
1
(0)
|| bo.
1-9
(37)
Since < 1, we can set (s,k) = gh which is a KCCffunction (decaying to zero as k + oc), and which is KC-function, satisfying Eqn. 35. Therefore, the system is ISS.
However, there exists a steady-state error due to the disturbance w(k). In the best case, when A(k) converges to A and (k) converges w, the error e(k) eventually converges to steady state given by
2. = (I A(1 (11pp)) -w. w..
Therefore, 3, #0
Remark 1 (Convergence rate versus Kp.) From Ineq. 37, smaller q yields faster convergence. Because
d(p)=Mi1-2) M((1-)) Pe (1-1-10 (1-111
M 11), M(p-1), PE [1,1+ M)
have -- 00fopp00f0-- and (p) M for p> 1. Therefore the contraction factor q(p) minimized
p-=1
=
and increases as moves away from within the admissible interval.
B.5 PROPORTIONAL-INTEGRR (PI) CONTROL
To reduce the stcady-state error, the proportional controller extended with an integral action, resulting proportional-integee integral (PI) control law:
u u(k) { + (,s()),
s(++1)=s((++e({)
(K,=0),
(38)
The dynamics Eqn. 20 then become
A())
(39)
We use the following orthogonal decomposition for (k))
w(k)= [ (k) (w) (k))
where wi (k)) e Im)(() and w (k) (ImA(k))
The impact of will wol((k) (k) on the error can be eliminated by PI control, discussed below. On the other hand, P-only control is not able to do so, because keeping ē(k) = 0 requires ukk) 0, leaving no component in (ak)) that can compensate for will (k).
Since will (k) E A ((k), (k), it can be expressed as
well || (k) = A(k) (k).
8 (k) (k)=K, A(k)) (k)
Let (k)))==())<>> s (k) and = 8 x* (k)) Therefore, 8(k+1) (k) d(k)
(40)
20

---

Published as a conference paper at ICLR 2026
Insert s(k)= = (k))+ (()) and w (k) (k) (()==(())K])
+ (kk) into Equ. 39,
e(k+1) -()))))))((()()(
-)) 41
(k)
(41)
(k)
We introduce the lifted state (PI(k) - [e(k)]
with its dynamic derived from Eqn. 40-41 as follow
[(k))
(pr(k+1))= Mc(t)(P((k)) - (A))())
(42)
where
M.(k) Mp( -G( -(k)=A(k)) and
with -
(w)
wp1(k)
-d())
Proposition 3 (Stabilizing the PI loop reduces steady-state error) Let M2 = A(K) KD). and denote K l: Assume sup; A(k)] M < oc and supş (p)]l then the PI closed-loop control is ISS. Furthermore, the integral part exactly cancels the matched disturbance component will, The remaining error is due only to the unmatched component w-,, which cannot be compensated. Full proof and term explanations provided in Appendix B.5
+ <1,
Proof. Using the sub-multiplicatiiitty of the induced matrix norm and the triangle inequality, and noting that [k)] S. [|(+)]] (+) K S Mh, we obtain
[()) || (A:]
[|]|[())))++++/185)000++1++01110 (+:]l,
(43)
(44)
Introduce
|e(k)]]
H == [ Mh
| (k)]l
2 (k) =
v(k) =
|((()]]]
[k)())
Then Eqn. 43- 44 give the comparison system
=kk+1) S +u(k)
(45)
Expanding Eqn. 45 recursively yields
=(k) < 11 11 2(0)+ Σ 1²11--b(1))
(46)
1=0
Consider the characteristic equation of H:
M - X2 (q+
Since 4- Mh. < 1, the maximal root A satisfies X <1, hence the spectral radius p (T) <1.
Letr:=p(H)) < be the spectral radius of H. By the Gelfand formula for induced (operator) norms,
lim
|| H 11/kp//==
(Horn Johnson, 2012, p. 349).
100000
Fix any e (r,1). Then, by the definition of the limit, there exists N E N such that
| <p p
forall k> - 1H S 0
Vk.N.
Define the constant
C max 1, max |H || 1bb bb.
OSKSN
Then:
21

---

Published as a conference paper at ICLR 2026
If we have IroskgN, we have |H* < maxosksN 11 || 11* hence || || SCp.
|p* C, hence s=pp.
Therefore. Applying Eqn. 47 to Eqn. 46 gives
| S Cp
for
(47)
R-1
||=())]]
|| [1|=00]]
Σ
H (1)
<c//|=<0)]+
<cD*|==00)|+
(48)
where ||=||>::= == sup
By construction,
[(k) [k)r(())
=1=()))))
[(k)
Combining this identity with Eqn. 48, the ISS estimate follows with
C'
B(s,k):=Cp-* K.,
>(s)= 1-p E
which proves that the closed loop Eqn. 42 ISS.
The integral part exactly cancels the matched disturbance component will The remaining error is due only to the unmatched component w-,, which cannot be compensated, and to the variation rate d(k) when A (l)) and w(k) change over time. In the best scenario, if a steady state exists. i.c. A(k) A and (k) + w with w ImA, then w = 0, d= 0, and thus ē(k) +0
Remark 2 (Convergence rate versus K2) From proposition 3, the convergence rate of (pr depends on the smaller p. the faster the convergence. We also adopt the convention (as in the proof) that p (r,1). Equivalently, we examined r() p(11) and proved that with h (1-q)² this quantity is minimized.
4M
Proof Consider the characteristic polynomial of H: 2 29- (q (1) =0.
Its discriminant is
A - A( Mh) (q 1)² 4.h.
If
(1-g)² then 4M
r(h)= 9+++ A(h)
and r(h) decreases as increases.
If A()) <0 (i.e., 4M
(1-9)²
< 1-g then
r(h) = -
and r(h) decreases decreases.
Hence r(t) can achieve its best (smallest) value at h (1 - g)
(49)
4M
for which the error converges zero the fastest.
Nevertheless, as we discuss in the next section, in some situations such a large value of may become practical obstacle for Pf control.
22

---

Published as a conference paper at ICLR 2026
B.6 OVESHOOT MECHANISM
Phenomenon. A common issue in standard PI settings is overshooting: the closed loop oscillates around the setpoint before settling (see Aström & Hägglund (1995a, 3, $3.3). In our terms, the integral part accumulates past error and can push the output beyond the setpoint; subsequent sign changes of the
gradually "discharge" the integral, producing decaying oscillation. The big overshoot undesirable when we prefer a more stable response near zero. Below we analyze the same mechanism for our PI steering setting.
Citing an observation from Vu & Nguyen (2025): in the absence of steering, the cosine similarity between error vectors at different layers is consistently positive, i.e.,
cos cos/(e())
> for all layers
so the layerwise errors share (approximately) the same direction. Consequently, with (*+) serving as the trajectory setpoints and { } the system output, an overshoot event occurs when the instantaneous error reverses its initial orientation, namely when
(e()), <0.
We now introduce the definitions used below.
Scalarization along a direction.
Let
e(0)
U
and project onto o:
e((k)==u su(k):=vT
|0)
From the PI loop dynamics Eqn. 42 we obtain the scalar PI Plpair
c. + ((++1))a=k)).(f(f (k) + e. (k: (k)).
b(1))s,(k))) w. (A:),
(50)
(51)
with a(k) v.
T -, (k)u),, b(k) T A(k) = v G(k),, and projected disturbances w. (k) == v w- (A:). (k) = o d(k). Empirically (and consistently with the angular-steering observation in our setup), have A(k) 2 0 for all k; together with the gain K; -/l with h22 O, this implies 6(k) 2 O.
Also, the assumption sups Mp and M -MMu A(k)] yeilds
0g S Mh,
(52)
Since the system Eqn. 42 is ISS, so is the system Eqn. 50-Eqn. 51. other words, both (x) and (k) decay. Recall that
h-1 ())
(k)
Hence, (k) can only decrease when e (k) < O, which is precisely the moment when overshoot occurs. These overshooting and decaying phenomena are observed in empirical simulation, see Fig. 7. Below, we define the overshoot in our setting.
Definition (Overshoot and its amplitude) We say an overshoot occurs from time kg to ki + m if (k) < 04k == ka + +L....,, + m and e,, (k) > 0for k = k2 1,kg +m. Its amplitude is defined as (k)
Aa
max
(i)).
(53)
kty SISK, -
In standard PID settings illustrated in Äström & Hägglund (1995a, Ch. 3, $3.3)), it is observed that the overshoot amplitude decays over time. This decay also consistent with the ISS property of the closed loop: as both e, (t) and (t) are driven down, subsequent oscillations tend to diminish in magnitude. In our simulation (see Fig. 7), the first overshoot appear to be representative. Hence, while we are not yet able provide formal proof, the empirical evidence and ISS intuition justify the first overshoot which is typically the dominant one and serves as a representative indicator of oscillatory behavior.
This assumption is for Proposition 4. Suppose that att) 2 O; equivalently, p (1 A 1], which is the result proposition This assumption is expected to entail no loss of generality relative to the [a(1) S < assumption.
23

---

Published as a conference paper at ICLR 2026
1.60
1.40
0.08
1.20
ze(0), e(k)
1.00
<e(O), e(k)>
0.06
0.80
0.60
0.04
0.40
0.20
0.02
0.00
0.00
-0220
-20
-15
-10
-5
0
5
0.1
0.15
0.2
0.25
<e(0), s(k)>
<e(0), s(k)>
(a) Pl
(b) PID
Figure 7: Scalar errors across time step of randomly initialized model after applying Pl and PID controller. Colors from blue to red denote the time (layer) dimension.
Proposition (Agressive gain leads a large first overshoot) Let kn the first sign-chunge time, i.e.,
( 2 vj=0.1.....kg -,
(Ko) <0,
and let k be the first time the trajectory returns the nonnegative side,
(k1) 20,
c.(1) <0 vi= ku, 1,....I -1
As in E3,.53, the first overshoot amplitude is
Au Kosish (e) (fmax))
(54)
Assumesup; A(K)] < Do and Therefore,
M, [4) Denote K, =:h. and given q Mh <
Mh
Mh (kn 1)
Mh(ko-10
+1
Ao S Mh
(0)
|d]
2 |oo0
1-q (1-q)3
1-9
1-q/
1-9
(55)
Proof. Before the first frossing (j /
we have s.()) hence from Eqn. 50
e. =((+))==ajje(()-((({((())
- (()) (i) s |
so by induction (oddcciooe_())s///ec())++++ |//loo JummingEqqm
ka
h--2
Σ (1) S (0) i=0
1-q + (ko - 11) ho 1|/loo-
Sk 1 1) Σ i=0
e (t)
(56)
1-9
Since a(k) 2 and (ko - 1) 2 0, at the crossing step.
leo(ko)] (ko) < b(ko (ku, - 1) + |/2/|
(0) + (ko 1|]| | 00 ko W
< Mh
+ +1|20|bo0
1-4
1-9
(57)
Mh
ho -1
1-q e. (0) Mh 1)]l (Mh
+1) 1|/lo
1-g
Assume that (k) is small enough s.t during the overshoot time, s, nonincreasing (since e < a.e. on k - 1)), so (i) < (ko - 1) for i [ko.k1 - 1]. Using Eqn. 50 again and unrolling m steps from kn.
(ko s (eo + Σ (Mhss 1)+ (|10ll)))))))))))
k=0
(58)
$ (e,() + Mhs (ko 1)+ 1|0110
1-9
24

---

Published as a conference paper at ICLR 2026
Taking the maximum m (0.1.....k) ko} and substituting Ineq. 56 and Ineq. 57 into Ineq. 58 yields
1 1-9 (1- e0 (
Mh
Mh 1 |Id|oo + Mh(ko 1)
Ao S Mh
[1- (ko- 1)+
w
1-4
(59)
Consequently, the right-hand side of Incq.59 is monotone increasing in (via the factor Mh) and increases q decreases (through the factors r In particular, more aggressive PI leads larger first-overshoot amplitude.
Remark 3 ("Fast-PI" specialiatiio))) With the tuning used in our analysis in remark 2, h= (1-g)²
4M
(the value that mininizes the comparison-systen rate), so Mh (1-4)² Plugging into equation 59 gives
Ao - 1-9 In particular, in the disturbance-free case (||0| 100110 = =||||| ||d|lx< =0) we obtain
1-9 (1 (Ko - 1) |||| +
ko
+
w
1lo0
1-9
1-9 + (0),
Ao
4
so stronger proportional action (smaller q) comes with a fistovrrss envelope, even though the closed-loop settles faster.
B.7 PID CONTROL
B.7.1
STABILITY OF PID CLOSED LOOP
We consider the PID update
u(k) = K ((k) K;s(k) Ka(((()-e((k-1))) 1));
and define the auxiliary matrices
Mp(k): := A(+) Kp),
G(k))= A(k))=A(())K)
together with the error increment
(e())==(())
Ac(-1)=0
Using the plant relation, we obtain
Ae(k+1) (1] G(+))(() (K) + w w=((k))
(60)
Introduce the lifted state from the auxiliary PI state in Equ. 42
e(k)
(FID(K)==
3(k)
Ae(k)
Then the closed-loop evolution reads
(M(k) + (k)
(61)
where
M2(k) 1
-G(k) - -H())
w (k:
Mk(k): = (, -G()) -H(
(P)D(k) =
d(k)
w (k:)
Theorem (Stabilizing the PID loop preserves bias removal) Let M,(k) A(k) K,), and denote K, =: h. |[alll =: Assume sup, suppa|| ||(())] < M and sup; M, S q + Mh < 1 (stable PI loop). then there exists E >> such that the PID closed- loop control is ISS.
Therefore, the integral part in PID design still cancels the matched disturbance component wil.
25

---

Published as a conference paper at ICLR 2026
Proof. We establish the ISS for system 61 using the method of ISS-Lyapunov function, see (Jiang et al., 1999, Def. 2.2, Prop. 2.3). It then suffices to construct a candidate Lyy-unnp function VPI((k) satisfying that there exist class Koc functions a 02,003 and a class I. function a such that an (l((rTD(()) & VrID(ÉPI (h:) - 0z (X()P(D))))))
(62)
and
- aa -V) (()))
(63)
Step 1: Candidtate (k) and Pl-closed loop baseline
Define
(k) == (EL((P((A),A)) + ++)))
r>0,
(64)
where ()r((pr,k)= 2p (P) with P(k)= > and there exists some μUL > such that
(,k P(k+1)M - P(k) f -PPi,,
Ak.
(65)
Regarding the existence of such VpI. recall the homogeneous PI-loop ČPI(k + (4)
with (k) (p( -G(A))
I being asymptotically stable. Suppose there is = (At) 0
bounded so that the pair (M,(k), / Q(+)) is observable for all k, hence the difference Lyapunov equation
M, (4) (L)+(k))) (k) =
admits a unique positive definite solution P(k) PT for all k, and uniform bound P| >= sup; <o (see (Gajic & Qureshi, 2008, Ch. 4. 110))
Step 2: Condition (i) as in Eqn. 62
We write VIID as a quadratic form
P(k) 01 0 r1
VPID (k) (k))(k) (K)
P. (.()) =
(66)
Clearly (k( = ( and P. (k)) 0 because P(k) > and rl O: hence P. (kk)) symmetric positive definite for k.
By the spectral theorem, there exists orthogonal matrix (. and diagonal A. (k) = diag(A: (k).... An, (k)) with positive entries such that (k) = U. (K)KA,, Moreover, because P. (k)) is block diagonal, its eigenvalues are precisely the union of the eigenvalues of ((k) and the repeated eigenvalue r. Using the uniform bounds already established for P(k) (there exists > O with Amin (P))) 2 and Amax (P(k)) <
P| == sups [(k]]] < oo), we obtain the
k-independent bounds
< 80. For every vector 2 and every symmetric posittii definite M, Amin (M) M Amax (M] Applying this to M = (k) and ==<PID in Eqn. 66 yields
Amin (.))) 2 :ii 0,
Amaa (k)) Tx =
>. (t) > VPID(k) & X. (K]))DD(()))
Therefore, choosing the class-NC.s functions
an (s):= az(s)==X_s²
we obtain the desired bound
@n (1)))i)(()1)) s (k)n(k) s szz (1)))
(67)
which establishes condition (i) in 62.
Step 3: Condition (ii) in Eqn. 63.
From Eqn. 64,
AVPI (k) = AVri ((e)(()) PID
+r +r((Ae(==D)F--IA((t))
(68)
PI part under PID update
26

---

Published as a conference paper at ICLR 2026
Bounding the PI part under the PID update.
Under rule (Ka= =0),
M.(k):= = Mp(k) -G(R)
(pr(k+1)) +
then
A VPr ((r())(())) =(ri(N) (M((A))(MM(())
(B) GrI(Ks)
+2(r1())TMM(()) M((K) P(R+)))r)))))))))))())) (or(())) ((st)) P(k+1)eerr
By Eqn. 65, inequality and Young's 's inequality, ||00
A AVri (Gor(k)) PT (um
<
[
|
P
|P]
||xūP((+)]1
61
for any 61 >0 0.
Under PID rule (Ka
(pr(k+1))|pm = (((k))p((()) 5(k) + wpr(k))
with the "perturbation"
6(k) (K)))e(())
Hence,
(Vr(((r((()) =
(vn(((r( pT) Vrr((rr(A))
PID
(() + +(GG(+())) +5())) P8(k)
(69)
Bounding each term in Eqn. 69
Ver(G S (/o
Applying Young's inequality for inner product, there exists e> 0
1(mr((+1))pp)
{((())=((((((+++))|| (()) P8(k)
(70)
< ||] 12
(71)
Since (k 1)
+(((p((++1)]p
[pp
IP {
:
= 2s ||| |
( IPI M
#C2l|xppr(k:)11
where C2 P||
6(k) ||CC:)/|||P||0000P2PPPPPPPPAAAA/
Therefore,
(VP((((((H))
PID
+2+|| /ttp:////.j.....aa/////////////a
(72)
= =- -2 P M || (4A (
+Ca))(pp(())11?
27

---

Published as a conference paper at ICLR 2026
where C3 C1+
Bounding the increment term.
From Eqn. 60 and applying the inequality (=+v+2)*=3(0 +
MM) (1 ||
| A: 1)11
M, (k:)
urdD (A)]l
and
= 3r (11 (P)
(73)
- +1
Combination
Combining Eqn. 68, Ineq. 72, and Ineq. 73,
A AVPI( (k) Pir 2 (M ((PP(
(r(1-3322 M2 -|
where C= C3+r. Define
S(r,e) 2 2:
3r (1 - + Mh),
(74)
((rs:))==((133 2 (2 (1)
(75)
ISS of the PID loop follows and
Feasible choices.
We are free to choose any E > and r>> such that S(r,s) > O. One convenient selection is
pPi
-(r,))=&uupn
r=
8
8||
M,
8 (k) 1 Mh)
With e,r fixed above, pick E> 0 small enough to satisfy T(r,s,)) namely
P 1))+
+ M2
Under these choices, A (PID(K) (k) as a3 (k) -4 Ae 122 ( for some a3,04,,3
0, which satisfies condition (ii) as in Eqn. 63 and proves ISS of the PID closed loop.
B.7.2
OVERSHOOTING UNDER PID LAW CONTROL
Developing from Sec. B.6, we introduce scalar PID recursion along v:
e. ((k)) Ae (h w (k),
(4)) (k)), (76)
where
a(k): = M, (k)u, b(k)==v T G(k)u, (e, = Ae(k), w (k)=u² - w (k))
c(k):=v* Hk)),
By construction a(k) < < b(k) < Mh, c(k) < Mt with M = supk A(+))] h:= |K,]] and t:= Kall.
We now impose an additional requirement on the derivative gain K₂ so that, without the effect of noise, the PID update secures the monotonic decrease of e, (l:) before the first negative peak of scalar error (k).
Remark 4 (Pre-overshoot monotonic decrease of scalar errors) Assume the setting of Proposi- tion 4 and further suppose the scalar error trujectory before the first largest overshoot under PID law is smooth in the sense that there exists R 1 such that
ea(k-1) < R
for all K x= 1,2 1,
(77)
(k)
28

---

By the same reasoning, as Published as a conference paper at ICLR 2026
where Au sisko (e)(1) =le (fmax)) from Eqn.54. Assume that w and =0. If, in addition, the derivative gain satisfies
1 1-9 (R-1)M
Kall S
(78)
then under PID law
e, (k(k+)) S (k)
for all k= 0, maa
Proof. Before ko, we have (k) 0, sk( 2 (since 8, accumulates ey and s, -). For
ku <kSimaz (.) proved in remark
Hence
e, c((t11)--achaa000-00000------110
S a(()) (())) e, (k: < (R- e Cx()),
where the last inequality is exactly Eqn. 78.
Note for In practice, one may estimate a conservative from PI-law traces and use small safety factor
Adding Disturbance:
With bounded disturbances, the scalar update reads
e, S [R+(R-1)cmax (x( () + l (A))
so the same one-step monotonicity conclusion holds whenever
I (A:) S (1 (R-+(R (k)) for all prefisst- steps.
this smallness condition on disturbances fails at some step, one-step monotonicity may be lost, but the ISS bounds proved earlier still guarantee geometric decay up to disturbance radius.
Remark 5 (before the first negative peak, the integral state positive) Assume the setting of Remark, 4. Hence,
(k) >
for all lo pmax 1.
Proof. We argue by contradiction. Suppose there exists the first r Max 1] such that (. S Then (T (1) and since the first negative lobe, (T) < 0. Compute the one-step change ey: e
=(1 -(--o())lee(7))-
>0
because ((T) 5g (T) <0 and (T) 00. Hence e
long as both e. (k) < and s. (k))) < hold, we have
e, (k++1 - e, (1-9) (1)
Umin
where bmin inf, b(k) > Mcanwhile (k+1) sk) ek. < sy (k) on that interval, so (kk) is non non-increasing; equivalenty s, (A: is non-decreasing. If e, stayed negative forever, then En (T(kk) would eventually be arbitrarily large, forcing e,, to cross 0 in finite time. This contradicts the choice of max the first negative peak. Therefore such cannot exist and 8, (k) for all k: kmax 1.
xc, s.(k) would grow without bound and the increments e, 11++1)ee.((
Theorem (PID reduces the first-overshoot amplitude) the first overshoot occur index ko with amplitude Ag (definition in Eqn. 54). Then, the first-overshot amplitude under PID Steering, APID, satisfies A) APIDD < API, where API denotes the corresponding amplitude under PI Steering.
Proof. Under the PI law we have
AP
a(imaa 1) ( 1)+ 1)80 (fmaz (iinaz-)) w (fmax 1),
29

---

Published as a conference paper at ICLR 2026
while under the PID law
A
((maa 1)e (Imaz - +b(bbb(((p (immar 11) (Imaz (0pnax-11 +cllmar (Imaz-) w.
(79)
(Iman 11), 1),
(80)
Due to the monotone decrease before this first largest overshooting condition stated the previous part and the fact that c(k) > O, have c(ko 1) (ho 1) <0. Therefore,
Ap PID <
b(imar ()maa - 1) a(imas e) (imax - 1) w (Imaz- -) = A8" (81)
Remark is neccessary because monotonic decrease of et( before the first peak is both a key technical property for proving Theorem 2 and desirable feature of PID control itself. Indeed, noted by Aström & Hägglund (1995a, p.70), poorly tuned derivative gains may produce non-men non-monotonicity, in which case reducing only the first overshoot does not translate into improved overall behavior.
B.8 BEYOND LOCALLY LINEARIZED ACTIVATION DYNAMICS
In the main text, analysed the averaged error dynamics under P/PI/PID steering by applying first-order Taylor expansion of the activation map around the desired "plus" trajectory z (k), leading linear timevnarying (LTVV) model with additive disturbance term. In this appendix, make explicit how the higher-order nonlinear terms enter the dynamics and discuss their impact on the closed-loop behaviour. We also justify why the perturbations
8, (k) = e (k) (A:)
remain controlled in norm under the proposed controllers, even under strong steering.
B.8.1 NONLINEAR REMAINDER IN THE AVERAGED ERROR DYNAMICS
Recall that the minus activations after steering are given by
I; (k) + t (k) = (k) (k)),
(k ( (k)))
where =k)) (k)- .; (k) denotes the layer-wise wise error for pair and u (kk) is the steering control shared across pairs. We consider the exact Taylor expansion of the activation map ((k) around the desired activation (k:):
f Mb) (a (k) + (k)) 11
+J f( JJ
(4)))
(82)
where J J (x) denotes Jacobian of y((²) at x and the (-)) term collects the higher-order Taylor remainder.
Averaging over i and expressing everything terms of the error variables leads the following averaged error dynamics:
(83)
where
ε(k)
(k),, ((())=e((k)-(((
(84)
N 1 N
A(k)
Σ N
A,(k)= =
Σ
J p0 (2+(()) (1)).
(85)
N
1=1
A,kk) -
(86)
The disturbance term wkk) now has two contributions:
1 N N
N (.) NONENNN -(()
w (k)
(87)
The first part is the "heterogeneity" term that already appears in the linearized analysis (capturing pair-to-pain variation in Jacobians and errors), while the second part arises purely from the higher-order Taylor remainder in Eqn. 82.
30

---

Published as a conference paper at ICLR 2026
In other words, when we do not truncate the Taylor series after the linear term, the closed- -lop dynamics still take the form
but (k) is augmented by the additional nonlinear contribution dें (())))))(())tu(()))))))))
From the perspective of input-to-state stability, this is a benign modification: all ISS results in the main text only require that the disturbance w (A) is uniformly bounded. The higher-order terms in Eqn. 87 simply enter w (k), and as long as (I))+((()))))))) remains bounded and the activation maps f/ have bounded higher derivatives along the trajectories of interest, these terms are also uniformly bounded across layers. In that case, all ISS statements and Lyapunov-based bounds derived in the main text continue to hold; the only effect is that the (conservative) upper bound on ||ē(k)]] scales with the enlarged disturbance norm ||
B.8.2 MAGNITUDE OF (k) UNDER P/PI/PID STEERING
A natural concern is whether |6, (A)ll -Ile, (A) (k)] may become large under strong steering, potentially making the higher-order terms Eqn. 87 significant. We show here that, under our gain conditions, (k) primarily measures residual mismatch that remains controlled in norm.
By definition.
(k(k) = (1:6)=-;;000 ( (x (1) x, (1:) (k:).
(88)
so that
z; (k: (k)))(()==) (A) (k).
(89)
Thus, (k) measures the residual mismatch between the desired activation 2; (k) and the steered activation 27 (Rc) + ukk)) at the same layer. When the controller functions as intended, the steered
trajectory x (k) + (A)) stays close to x (k), and hence [|,] remains small in norm. We make this more precise by examining (k))) for the three controller classes.
Recall that we decompose the per- per-pair error into its mean and heterogeneous components,
N
e())==(()++
e(k)
(k),,
ek) E(k)).
(i) P control.
For P control we have
u(k) K ēkk))
and therefore
(k(k) = (k) (k))) = -ē(k) (ikk)+ K
(90)
= =(Kp -(Kp-))e(1) (k). (k)
(91)
Our P-gain stability condition requires
1
M K <1+ M
so K2 p is necessarily small in norm. This cancels most of the contribution of ē(k) in 8,(k): in particular, the component of (k)) along e(k) is scaled by (K,- -55-11 1). not by Kp. As a result, [k)]]]
dominated by the heterogeneity term (k)] rather than by the overall stecring magnitude |[(()]]
(ii) PI control. For Pl control the steering law is
k-1 s(k) (()==[e)) 1=0
=kk)= (k))
(kk)) c,(k)+u(kk)
(92)
= =-((k) - ((a)
(93)
(94)
All contributions to (k) remain linear é(-) and its history through skk). The gain conditions derived in the main text ensure closed-loop stability and imply that
e(k), s(k)
remain uniformly bounded. Consequently, [|u(k)] remains in a comparable bounded range, and so does [k)]]] the PT controller does not cause 5, (k) blow up, even under strong steering.
31

---

Published as a conference paper at ICLR 2026
(iii) PID control.
For PID control we add derivative-like term,
(K) Ka)e(k), -e(k)==(k)) -
which yields
6,(k) (.(k)+
(95)
= = e ((()))-((((())((((((-(--(---()))
(96)
(97)
Again, all terms are linear in e(.), s(-), and ( Under the PID gain conditions, the closed-loop
system is stable and the signals
ekk) s(k),
Ae(k)
remain bounded in norm, implying bounded (| and consequently bounded [[)(())]
In summary, even when the raw error || ee) may become large under strong steering, the controller designed to cancel the dominant component of e(k) in u(k) and to keep the integral and derivative terms bounded. The residual mismatch 6, (k) is primarily governed by heterogencity and bounded controller memory, rather than by the overall steering magnitude. As long as || (k)] stays bounded, the higher-order nonlinear contributions in Eqn. 87 remain bounded as well, so the ISS guarantees derived from the locally linearized model continue to apply, with the disturbance norm [ capturing both heterogeneity and curvature effects.
C ADDITIONAL EXPERIMENTAL RESULTS
C.1 QUALITATIVE EXAMPLES OF CONCEPT STEERING
Fig. 8 and show that varying the intervention strength E [0.1] produces a smooth and controllable progression of stylistic traits in the generated images. At low strengths (ar 0.2), subtle cues emerge, such as faint neon accents for the cyberpunk style or mild metallic shading for steampunk, while the overall image remains close to the original prompt. At moderate strengths (a se 0.5), stylistic features become more salient: cyberpunk generations exhibit vivid neon lighting and futuristic cityscapes, whereas steampunk outputs show prominent brass textures, gears, and industrial motifs. Importantly, in this regime, the central semantic content of the prompt (i.e., objects, entities, and spatial composition) is preserved with high fidelity. At high intervention strengths (a 2 0.8), stylistic traits dominate the visual appearance, often saturating the scene with strong color palettes or dense textures, yet semantic alignment the original prompt remains largely intact, indicating that the steering primarily affects style without eroding core content.
C.2 JAILBREAKING LARGE LANGUAGE MODELS
Tab. 3 reports comprehensive comparison of attack success rate (ASR) and general benchmark performance across multiple instruction-tuned models under different defense methods. Overall, PID consistently achieves the highest ASR among defenses, while maintaining comparable performance on downstream benchmarks.
We further compare our PID steering with recent steering-vector generation methods on a better safety- aligned variant of Gemma- -BBaa-99---- 9-II, namely Gemma2-99 -a-aa-------naaa Instrnct- Seett-WWth-Dppppee Deeper-Safety-AAle et al., 2025). Deeper safety alignment here trains the model to sustain refusal behavior beyond the first few tokens: instead of only shaping the opener, it conditions on partially harmful or misleading prefixes and optimizes the model to "recover" back to safe behavior later in the sequence. Practically, this extends safety pressure across positions refusals remain stable under mild coercion, prefilling, or decoding variance, while seeking to preserve general-task utility. Under this strong safety-aligne regime, PID-based steering still achieves non-trivial attack success rate and outperforms other state-of-tie -fete---- activation steering methods considered in our study.
(Qi
C.3 EMPIRICAL EVIDENCE FOR THE STABILITY INTERVAL
Proposition 4 and Remark 3 (Appendix B.6) show that the PI gains that maximise the asymptotic conver- gence rate of the linearised error dynamics (the "fastest convergence" choice of K, and K,) also induce large overshoot in the correlation trajectory (k))) Empirically, this manifests as poorer steering performance: aggressive integral action speeds up convergence but increases oscillation and overshoot, which is consistent with classical PI tuning principles (see, e.g., (Aström & Hägglund, 1995a)).
Theorem 2 further shows that the derivative term Ka provides damping: for fixed Kp and K,, increasing K2 reduces the overshoot of the error trajectory. Although we do not establish formal
32

---

Published as a conference paper at ICLR 2026
02
0.5
n22
0.4
0.6
0.8 1.0
Figure 8: concept cyberpunk.
optimality theorem, is natural expect that the theoretically fastest PI gains, when combined with a suitably chosen derivative term, can outperform more conservative K; values that lie strictly inside the stability interval (under the same K, = Intuitively, because transformer depth is finite and we do not know at which layer the error will effectively settle, it is desirable to drive the error down quickly while preventing excessive overshoot, with Ka acting as the compensating damping term.
33

---

Published as a conference paper at ICLR 2026
ARILON
02
82
0.6
0.8
oco
D.
Figure 9: Concept steampunk
Our empirical results corroborate this interpretation. The stability intervals for K₁ K, (-0.23, 0.23) for Gemma-2-9B-it and (-0.1 13.155,,0 O. 03155)) for Gemma-2-2B. In Figs. 6a and 6b, we sweep across multiple K; values these ranges. For Gemma-2-922 theoretical convergence rate is 0.056. When K₂ increases from 0.0 to 0.01, the Llamaguard3 score rises from 76.61 78.53. This pattern suggests that with K2 00 the aggressive integral term produces noticeable overshoot that harms performance, whereas adding small derivative term introduces sufficient damping to recover, and some cases improve, steering performance.
the K; that yields the fastest
34

---

Published as a conference paper at ICLR 2026
Table comparison of Original, DIM, ITI, RePE, and PID across models on ASR and general benchmarks on all tested models. Bold = best, underline = second-best within each model (ASR column).
Method
ASR
AinyArct
LinyGSM8k strict?
MinyMMLU
tiny TruthQA1
tinyHellaaSwag!
tiny WinGGradde
Original
62.29
17.64
68.03
56-43
73.18
70.65
Qwen2. 5-3
Instrvct
DIM
24.03
14.80
66.11
51.95
ITI
70.19
61.28
15.57
66.62
54.75
72.71
70.12
RePE
68.44
61.05
14.60
66.70
54.30
72.03
09.40
PID
76.07
61,20
16.01
67.29
54.10
72.59
69.72
Original
68.36
81.68
72.57
56.41
78.87
75.19
Qwen2 5-B
Instruct
DIM
96.15
65.15
80.81
71.19
55.25 22
74:12
ITI
84.61
65.76
79.48
71.23
55.63
78.36
74.75
RePE
80.32
65.10I
78.90
70.60
55.00
77.73
74.15
PID
96.46
66.61
80.78
71.22
55.52
78.28
74.58
Qwen2. 4-14B
Original
73.96
90.12
74.60
64.50
82.70
73.77
DIM
90.38
72.74
87.01
74.30
63.01
81.94
72.93
Instruct
ITI
33.65
73.15
89.27
74.55
64.03
82.240
73.31
RePE
25.42
72:40
86.20
73.90
63.20
81.52
72.60
PID
92.65
72.13
88.96
74.52
63.60
82.60
73.04
Liama3 3.m33,2--3 2-3B
Original
55.86
59.10
63.48
50.19
75.91
58.63
DIM
$8.46
54.24
58.63
61.68
49.78
75.10
57.94
Instruct
ITI
76,92
53.67
57.77
61.85
19.95
75.22
58.16
RePF:
70.15
53.40
57.00
61.10
49.50
74.75
57.53
PID
89.76
53.93
57.26
62.01
50.19
75.07
57.88
Llamn3. LB Instruct
Original
65.33
63.
62.02
51.39
82.51
65.56
DIM
93.26
62.01
60.57
60.96
54.17
81.73
64.81
ITI
79.80
61.26
61.85
61.37
54.33
82.01
65.21
RePE
70.42
61.40
60.00
60.20
53.70
81.35
64.45
PID
94.85
62.30
61.99
61.54
51.24
81.87
61.93
Original
69.31
83.19
76.60
55.07
82.31
72.34
Genma2-99 Instruct
DIM
77.88
68.21
80.14
72.29
51.96
81.45
71.51
ITI
35.57
68.32
81.47
75.33
53.13
81.70
71.82
RcPE
28.61
67.50
79. 79.20
71.10
51.10
81.20
71.15
PID
79.50
67.91
79.24
71.89
52. 19
81.59
71.12
Original
73.45
86.91
76.11
61.36
83.24
75.47
Gemman-227 Instruct
DIM
74.03
72.13
81.70
74.59
59.49
81.79
74.21
ITI
37.36
72.84
86.38
75.51
60..85
82.83
75.11
RePE
24.19
71.03
84.00
73.90
58.70 79
80.62
73.44
PID
79.80
72.53
86.60
75.67
60.74
82.95
75.06
Gemmnn2-9B-- 2------- Despr-Alige Aign
Original
69.07
82.93
76.18
54.82
82.27
72.11
DIM
11.91
67.58
79.76
72.12
51.83
81.33
71.17
ITI
23.12
68.19
81.41
75.22
53.05
81.58
71.79
RePE
11.23
67.37
78.96
70.92
51.06
81.08
71.09
PID
34.75
67.53
79.11
74.77
52.44
81.46
71.33
35


Reviews:
[{"review_id": "iLAht6SRVJ", "reviewer": "Reviewer_NQp6", "summary": "This paper presents a novel perspective on activation steering for large language models and diffusion models by introducing a control-theoretic framework for managing model behavior. The paper provides theoretical analysis using input-to-state stability ISS and demonstrates empirical gains on toxicity reduction, jailbreak resistance, and style control tasks across several models.", "questions": "See above", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. It is novel to connect the control theory and activation steering in LLMs.\n2. The paper provides a solid mathematical formalization of steering as a dynamic control process. \n3. Experiments are comprehensive, covering both text (toxicity, jailbreak) and vision (style transfer) domains.", "weaknesses": "1. Lack of baselines: The experiments compare primarily against the sequential steering vector ****baseline and, to a lesser extent, static activation addition. This is an insufficiently broad comparison. Many other steering methods, such as CAA and ITI, could serve as competitive baselines. \n2. The paper evaluates toxicity using LLaMA-3-8B both as the generator and as the evaluator, which introduces a self-evaluation bias: the same model family that produces text also judges its toxicity. Such evaluations are not independent and can underreport toxicity. A separate toxicity classifier such as GPT-4 or at least cross-model correlation analysis should be used for reliable measurement.\n3. The paper does not report ablations on the $K_P, K_I, K_D$, though the performance of control systems is typically sensitive to them. Similarly, computational overhead, such as inference latency and memory increase is unreported, which is important for large models.\n4. The theoretical analysis assumes locally linearized activation dynamics (Eq. 29–32). While this yields tractable ISS proofs, the approximation error may be significant in deep nonlinear transformers, especially under strong steering perturbations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZNHQrzU05a", "reviewer": "Reviewer_hCNg", "summary": "This paper introduces a control-theoretic framework for activation steering in LLMs, named PID Steering. The motivation here is that traditional activation steering methods, such as Activation Addition, Directional Ablation, and Mean Activation Transport,modify internal model activations to control behaviors like toxicity or refusal, but they lack formal guarantees and often rely on empirical heuristics.\n\nBased on this the authors introduce PID for alignment, removing residual bias and mitigating overshoot when aligning.\n\nResults demonstrate how PID control improves convergence and stability, reducing steady-state errors. Empirical experiments across multiple models (Gemma2, LLaMA3, Qwen2.5) and tasks (toxicity mitigation, jailbreak prevention, and image style control) show that PID Steering outperforms prior activation steering methods, achieving more consistent and interpretable control without harming overall model performance.", "questions": "Please see the weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper proposes an interesting methods of control-theoretic interpretation of activation steering.\n\n- The proposed PID Steering introduces components that correct long-term drift and prevent oscillations making it robust.\n\n- The approach is tested across diverse tasks (toxicity, jailbreaks, style transfer) and modalities (text and images), showing robustness and generality.", "weaknesses": "- Selecting appropriate PID gains (Kp, Ki, Kd) is nontrivial, and may result in suboptimal solutions.\n\n- While results are broadly positive, the paper lacks detailed ablation studies on scenarios where PID might underperform or destabilize.\n\n- Claims that PID Steering is lightweight are mentioned but not benchmarked in terms of latency or inference-time overhead.\n\n- It is unclear to me how controller parameters generalize across model architectures or domains without retraining.\n\n- The much smaller models such as Gemma is also used in this paper. Although, larger models (24B or 70B) are not tested, I was wondering if the authors could already make a size comparison between the models and present the observed differences?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MygxNrVV59", "reviewer": "Reviewer_A2rW", "summary": "This paper aims to address the lack of theoretical guarantees, presence of steady-state errors, and overshoot issues in existing activation steering methods. It analogizes the inter-layer activation propagation process in LLMs to a dynamical system, framing popular methods as instances of a Proportional (P) controller. Building on this, it introduces the Proportional-Integral-Derivative (PID) controller from control systems theory and proposes the PID Steering framework.", "questions": "1. Do the trial-and-error costs associated with manual PID parameter tuning vary across different task scenarios? Could parameters be highly sensitive and difficult/costly to tune in certain task types?\n\n2. If testers used harmful prompts for evaluation, would PID Steering suffer performance loss?\n\n3. Why the activation steering vector is added to every layer?  If added to only part optimal selected layer(s), the method offer advantages over identifying and intervening?\n\n4.  Unclear Visual Representation in Figures: Figure 3, as key evidence supporting the core argument, lacks clarity in visualization and exposition. One theoretical cornerstone is that the integral term eliminates steady-state error. However, the PI curve in the figure ultimately displays saturation and large overshoot, obscuring this advantage and making it difficult for readers to appreciate the effectiveness of this step. Adding a subplot specifically comparing P and PI could highlight this advantage.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. This work provides a unified control-theoretic perspective, categorizin existing methods (e.g. ActAdd, DirAblate, Mean-AcT) as specific types of P controllers, thereby logically leading to the integration of the PID controller concept and the proposal of the PID Steering framework.\n\n2. The paper goes beyond conceptual proposal by providing rigorous mathematical derivations, including error dynamic modeling and PID controller discretization.\n\n3. PID Steering is designed as a lightweight, plug-and-play module that can be seamlessly integrated into various existing steering frameworks (e.g., Angular Steering, Mean-AcT) by replacing their original steering vector construction, facilitating widespread adoption.", "weaknesses": "1. Manual tuning of PID parameters: the PID controller parameters $K_p, K_i, K_d $ appear to require manual tuning. Finding optimal settings involves significant trial and error, which is somewhat cumbersome.\n\n2. Lack of analysis on $I, D$ Independent contributions and combinatorial effects. The experimental section fails to demonstrate the independent contributions and combinatorial effects of the $P, I,$ and $D$ modules, since the authors claim their method is a plugin fashion. Lack of the experiment setting to demonstrate the benefits by introducing the $I$ or $D$ to the $P$-class controllers.\n\n3. Incomplete experimental comparison with cited methods: While the introduction and background sections systematically review mainstream activation steering methods, Section 5 does not include comparative tests against all mentioned methods (e.g., ActAdd). This prevents intuitive verification of the PID framework's optimization effects across different underlying $P$ controllers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)