Steering and Rectifying Latent Representation Manifolds in Frozen Multi-modal LLMs for Video Anomaly Detection
Authors: Zhaolin Cai
Fan Li
Huiyu Duan
Lijun He
Guangtao Zhai
Venue: ICLR 2026 Poster (2026)
Keywords: Video Anomaly Detection; Multi-modal LLM
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Video anomaly detection (VAD) aims to identify abnormal events in videos. Traditional VAD methods generally suffer from the high costs of labeled data and full training, thus some recent works have explored leveraging frozen multi-modal large language models (MLLMs) in a tuning-free manner to perform VAD. However, their performance is limited as they directly inherit pre-training biases and cannot adapt internal representations to specific video contexts, leading to difficulties in handling subtle or ambiguous anomalies. To address these limitations, we propose a novel intervention framework, termed SteerVAD, which advances MLLM-based VAD by shifting from passively reading to actively steering and rectifying internal representations. Our approach first leverages the gradient-free representational separability analysis (RSA) to identify top attention heads as latent anomaly experts (LAEs) which are most discriminative for VAD. Then a hierarchical meta-controller (HMC) generates dynamic rectification signals by jointly conditioning on global context and these LAE outputs. The signals execute targeted, anisotropic scaling directly upon the LAE representation manifolds, amplifying anomaly-relevant dimensions while suppressing inherent biases. Extensive experiments on mainstream benchmarks demonstrate our method achieves state-of-the-art performance among tuning-free approaches requiring only 1\% of training data, establishing it as a powerful new direction for video anomaly detection. The code will be released upon the publication.

Published as a conference paper at ICLR 2026
STEERING AND RECTIFYING LATENT REPRESENTA- TION MANIFOLDS IN FROZEN MULTI-MODAL LLMs FOR VIDEO ANOMALY DETECTION
Zhaolin Cai', Fan Li Huiyu Duan Lijun He², Guangtao Zhai
Xiinjang University an Jiao Tong University
3 SShanghai Jiao Tong University 'Shanghai Al Laboratory
ABSTRACT
Video anomaly detection (VAD) aims to identify abnormal events in videos. Tra- ditional VAD methods generally suffer from the high costs of labeled data and full training, thus some recent works have explored leveraging frozen multi-modal large language models (MLLMs) in a tuning-free manner to perform VAD. How- ever, their performance limited as they directly inherit pre-training biases and cannot adapt internal representations to specific video contexts, leading to diffi- culties in handling subtle or ambiguous anomalies. To address these limitations, we propose novel intervention framework, termed Steer VAD, which advances MLLM-based VAD by shifting from passively reading to actively steering and rectifying internal representations. Our approach first leverages the gradient-free representational separability analysis (RSA) to identify top attention heads as la- tent anomaly experts (LAEs) which are most discriminative for VAD. Then hier- archical meta-controller (HMC) generates dynamic rectification signals by jointly conditioning on global context and these LAE outputs. The signals execute tar- geted, anisotropic scaling directly upon the LAE representation manifolds, ampli- fying anomaly-relevan dimensions while suppressing inherent biases. Extensive experiments on mainstream benchmarks demonstrate our method achieves state- of-the-art art performance among tuning-free approaches requiring only 1 %% of train- ing data, establishing it as powerful new direction for video anomaly detection. The code will be released upon the publication.
INTRODUCTION
Video anomaly detection (VAD) aims to identify events that deviate from expected patterns, which plays a critical role in intelligent surveillance (Sultani et al., 2018), industrial quality control (Roth et al., 2022; Liu et al., 2024b), and autonomous systems (Yao et al., 2023; Bogdoll et al., 2022). While traditional paradigms, encompassing supervised (Liu et al., 2018: Landi et al., 2019), weakly supervised (Wu et al., 2024b) and unsupervised approaches (Lv et al., 2021; Liu et al., 2021), have demonstrated notable success but they are constrained by the reliance on large-scale training data. This dependency not only incurs substantial computational and annotation costs but also limits their ability to generalize to unseen scenarios, hindering real-world deployment (Noghre et al., 2025).
The advent of multi-modal large language models (MLLMs) (Zhu et al., 2023; Liu et al., 2023) has opened a new frontier for VAD by leveraging their powerful capabilities. While fine-tuning these models for VAD is effective (Zhang et al., 2024a;b), these methods reintroduce the burden of computational costs and data requirements, undermining the advantages of pre-trained models. Recent studies have shifted towards luning-free paradigms (Shao et al., 2025), which utilize frozen MLLMs detect anomalies by their generated interpreting text features (Zanella et al., 2024; Ye et al., 2025). Despite their effectiveness, these approaches are constrained by their passive nature. They overlook two critical deficiencies deeply rooted within the models. (1) The first is inherent representational bias of MLLMs, since they are pre-trained web-scale corpora, developing feature space optimized for frequent and prototypical concepts. As a result, their representations exhibit reduced sensitivity to subtle and rare patterss typical in anomalous events, leading to missed or biased detection. (2) The second is contextual ambiguity. Since the semantic meaning of a local
"Correpooningg authors.

---

Published as a conference paper at ICLR 2026
(a) Previous Tall-Trannig based Methods
(b) Existing MLLM based Methods
(c) Our SteerVAD Method
Video raames
Compute Resources
Video Frames
Extra * LLM/VLM
Video Frames
LAE Identification
Backbone Network
Foundation Models
Foundation Models
Training
ssisting
Steering
Costly
Labeled
L. Lossy
Prompt Learning
Accurate Unblased
Manifold
Burden
Datasets
Blased
Rectification
Tox This vidro accident,
No. This video does show any almormai mal events ... ?
The video
Anomaly Frames
Description
Anomaly Frames
Description
Figure 1: Comparison of traditional full-training methods, existing tuning-free methods and our proposed SteerVAD. Our method overcomes the issue of costly training resources and inherent bi- ases with minimal data required from pre-trained foundation models compared to previous methods.
action is determined by its global context, passively relying on isolated features can lead the model to produce confounding representations for visually similar but contextually distinct events. These issues are not mere surface-level classification errors, but stem from structural flaws in the internal representations of MLLMs, highlighting the inherent limitations of passive interpretation.
To address these limitations, we advocate a perspective shift from passive feature reading to active geometric intervention. Our approach builds on the vell-established manifold hypothesis (Whiteley et al., 2022), which posits that high-dimensional natural data concentrates on low-dimensionall struc- tures. We extend this hypothesis to the latent geometry of MLLMs. Within their high-dimensional feature space, the representations for a semantic class coalesce into a coherent, low-dimensional structure, which we term representation manifold. From this geometric perspective, the identified deficiencies can be re-contextualized. Inherent representational bias manifests as systemic prox- imity between the manifolds of normal and anomalous events. The feature space optimized for frequent normal patterns fails to allocate sufficient geometric separation for their low-frequency ab- normal counterparts. This proximity is compounded by contextual ambiguity, manifesting as the local geometric entanglement of these manifolds. In these entangled regions, representations of vi- sually similar but semantically distinct events become indistinguishable, making robust separation extremely challenging. This reframing highlights that the core challenge is not merely the classifica- tion of static features, but the dynamic steering and rectification of the underlying latent manifolds.
In this paper, introduce SteerVAD, novel tuning-free framework that implements this geometric intervention by steering latent representation manifolds within a frozen MILM. SteerVAD opera- tionalizes our geometric insight through two synergistic innovations. First, develop gradient- free representational separability analysis (RSA) to identify internal attention heads termed latent anomaly experts (LAEs), whose representations are effective for separating normal and anomalous events, exhibiting strong potential for rectification. This identification allows our intervention to be focused and efficient, avoiding unnecessary manipulation of the entire model. Second, we design lightweight hierarchical meta-controlke (HMC) that learns to execute dynamic, context-dependent geometric transformations. Conditioned on a global understanding of the scene, the HMC executes targeted anisotropic scaling to the features produced by identified LAEs, thereby actively reshaping their corresponding manifolds by amplifying anomaly-relevan dimensions while suppressing biased ones. By calibrating on only 1% of the training set, this mechanism effectively disentangling the representations of normal and anomalous events without any fine-tuning on the pre-trained model.
Our main contributions are follows:
We introduce a novel intervention paradigm for tuning-free video anomaly detection that moves beyond passive feature interpretation active geometric intervention. Our frame- work is the first to operationalize this paradigm by directly steering and reshaping latent representation manifolds within completely frozen MILM.
We introduce representational separability analysis (RSA), a novel gradient-free geometric method to precisely identify latent anomaly experts whose internal feature spaces are most aligned for VAD, ensuring the data-efficiency of our framework.
We design a hierarchical meta-controller (HMC) that generates context-aware signals to perform anisotropic manifold scaling, dynamically disentangling class representations to overcome pre-training biases and contextual ambiguities.
2

---

Published as a conference paper at ICLR 2026
We establish new ofte-of-theeart performance on UCF-Crinn and XD- Violleeee datasets among tuning-free free methods using frozen MLLM, demonstrating that targeted, data- efficient intervention method as a promising way against expensive fine-tuming approaches.
RELATED WORK
Traditional Video Anomaly Detection, Research in video anomaly detection (VAD) has tradi- tionally focused on models trained specifically for the task, categorized by the level of supervision. Supervised methods (Liu et al., 2018: 8; Landi et al., 2019) have shown strong performance by training on extensive datasets with frame-level labels, but the high cost of annotation limits their practical application. To mitigate this, weakly-supervise leverage more accessible video-level labels, often using multiple instance learning (MIL) frame- works. Unsupervised methods (Tur et al., 2023; Zaheer et al., 2022; Thakare et al., 2023) are trained exclusively on normal data. These models learn to identify anomalies as deviations from learned normality model, typically based on reconstruction error (Xu et al., 2017) or future frame prediction (Luo et al., 2022). While effective, these traditional paradigms require heavily training and struggle to generalize to unseen anomalies, motivating a shift towards new paradigms.
approaches (Li et al., 2022a; Wang et al., 2024)
Video Anomaly Detection with LLMS and MLLMs. The advent of large language models (Vaswani et al., 2017; Touvron al., 2023) and multi-modal large language models (MLLMs) (Zhu et al., 2023; Li et al.. 2023; Wu et al., 2024d), has created a new way for VAD. Initial ef- forts involves adapting these models through supervised fine-tuning (Zhang et al., 2023; Yuan al., 2024). While effective, these methods reintroduce the need for large labeled datasets and high com- putational costs. more recent and promising direction is the tuning-free paradigm, which uses MLLMs in zero-shot or few-shot capacity (Shao et al., 2025). These methods involve querying the MLLM with modificd inputs to generate enhanced textual descriptions or decisions about video, which are then parsed to identify anomalies (Zanella et al., 2024; Ye et al., 2025). Although these approaches effectively leverage the MLLMs without expensive training, they treat the representa- tions of the model as static and immutable, which makes them susceptible to pre-training biases and ambiguity, leaving a crucial gap in how to actively utilize the powerful internals of these models.
Internal Analysis of LLMS and MLLMs. Recent research in mechanistic interpretability of LLMs and MLLMs provides the foundation for intervention (Luo & Specia, 2024). Studies have shown that complex behaviors in foundation models can be localized to specific modules, such as individual attention heads or neuron groups (Zheng al., 2025), which act as functional circuits (Zhou et al., 2024; Bi et al., 2024). Building on this insight, the field of model editing has the ability to make targeted modifications model weights to alter specific behaviors (Wang & Veitch, 2025; Zweiger et al., 2025). These techniques have been applied to language tasks or involve static, input- independent interventions (Nguyen al., 2025; Mahmoud et al., 2025). The application of dynamic, context-dependent rectification of internal geometric structures for a complex, spatio-temporal rea- soning task remains an unexplored frontier. Our work is the first to bridge this gap by introducing actively geometric rectifications on the representation manifolds with frozen MLLM for VAD.
METHODOLOGY
3.1 PRELIMINARY: REPRESENTATION MANIFOLDS IN MLLMS
Our theoretical approach is based on the manifold hypothesis (Whiteley et al., 2022; Genovese al., 2010: Boissonnat & Ghosh, 2010), which posits that high- sigh-dimensional data concentrates on or near a low-dimensional manifold. We extend this the latent space of MLLMs, where an attention head : V + Rahea acts feature extractor that maps semantically similar video inputs into geometrically proximal representations, causing the varied manifestations of a class to coalesce into coherent structure. We model these emergent structures as representation manifolds. Consequently, for given head ht all normal event representations form a manifold Mnorm. while anomalous events form manifold Manom: both embedded within the head's 's high-dimensional feature space. A rigorous topological analysis of these structures is detailed in Appendix A.
Visualizations using UMAP, Figure 2 provides an intuitive illustration consistent with this geometric perspective. The features appear to form two distinct yet intertwined manifolds, visually confirm- ing the problems of representational bias and contextual ambiguity. This geometric entanglement
3

---

Published as a conference paper at ICLR 2026
Manifold Structures (Viewportt) Manifold Structures (Viewpor12)
Maniffld Structures (Viewportt)
Manifool Structures (Viewpor12)
(a) Cubic interpolation surface
(b) Triangulated surface
Figure 2: 3D UMAP visualization of representation manifolds of normal (blue) and anomalous (red) events from InternVL, illustrating their geometric structure. Each manifold is rendered from two perspectives using (a) cubic interpolation and (b) triangulation.
makes reliable separation by simple classifier difficult. Our approach moves beyond passively accepting this topology, we aim to learn a minimal, context-dependent geometric transformation Te that actively rectifies these manifolds achieve clear differentiation.
3.2 FRAMEWORK OVERVIEW
As shown in Figure 3, Steer VAD enables geometric intervention within a frozen MLLM for VAD. The framework rests on two principles: identification relevant representational subspaces and rec- tification of them. To accomplish the first, a representational separability analysis (RSA) identifies few internal attention heads that inherently aligned with VAD, which we term latent anomaly experts (LAEs). Then the hierarchical meta-controller (HMC) drives rectification by integrating the global context with LAE outputs to generate steering signals. These signals perform geometric transfor- mations on LAE feature manifolds, amplifying anomaly cues and suppressing biases. The rectified representations are aggregated and scored by anomaly scorer to produce final decision and curves. Detailed component implementation and analysis of SteerVAD are provided in Appendix
Problem Formulation During the inference, given an input video stream V, the stream is first partitioned into sequence of non-overlapping temporal segments, V (S1, S2 ........ .22...,,y'' Sr For each segment S,, our objective is to compute an anomaly probability Pt E [o. 1] in a real-time, single- pass manner. The identification of LAEs and the training of the HMC and scorer are performed in preceding offline calibration phase, utilizing a small, representative labeled dataset Dealih-
3.3 REPRESENTATIONAL SEPARABILITY ANALYSIS
To identify sub-modules within frozen MLLM that are inherently aligned with the video anomaly detection (VAD) task, propose representational separability analysis (RSA) to discover latent anomaly experts (LAFs), defined as specific attention heads whose feature representations exhibit a high degree of geometric separability between normal and anomalous patterns.
Separability Metric. We compute the Inter-to-Intra Scatter Ratio as the RSA score to quantify geometric suitability of each head. This metric measures the ratio of between-class separation to within-class compactness. For head ht. (layer I, index k), the RSA score is computed as:
||paamon
(1.b)
SRSA (1, 2 (1, k))
o2 2umm (L,
where p Hanomhoomm are centroids for anomalous and normal samples respectively. and o2 aomm/huen (1, k) are their corresponding intra-cluster variances. A high SRSA score sigmifies that head maps normal and anomalous inputs to distinct and compact clusters, making them aligned with VAD task.
(I.k)
Feature Extraction and LAE Selection. The LAE selection performed on the calibration set Dealib via firwae pass for each video through the frozen MLLM. We extract and store features from all attention heads to build feature banks, compute the SRSA score and select the top- K heads with the highest scores as LABs. This process efficiently pinpoints the most informative internal subspaces of the MLLM, providing optimal targets for dynamic rectification.
3.4 HIERARCHICAL META-CONTRONN CONTROLLER FOR DYNAMIC MANIFOLD RECTIFICATION
The hierarchical cota-contr (HMC) dynamically orchestrates manifold rectification via a single MILM forward pass. The HMC receives two concurrent information streams extracted from the
4

---

Published as a conference paper at ICLR 2026
Single Forward Multi-Modal LLM *
ext Inputs
II there are any the abucrmal
Mioo
MroT
Miro
Miro
Mrot
Yes vndeo
occural the video that
person holding pointing
deviate from the normal soone?
Final Context Vector
forward, which indicates
potesttil armed conffict
Hidden States
shooling situation
Full Autoregresson Multi-Modal LLM
Representational Separability Analysis Module
Anamoin Expert Heads
Local Gating Module
Monifalds in 1AFs
Adapter
Anixstrapic Scoting
Adapler
Abnormal
Local Signals
Anomaly
Global Scrutiny Gate
O
Anomaly Soor er
Conmmasy
Global Signal
repea
Manipulatian Sipnals
Context
Heerarciicll Meda-Contraller
A Anomaly Distribution
Figure 3: Framework overview of SteerVAD, We first apply the Representational Separability Anal- ysis find top Latent Anomaly Experts inside frozen MIL.M. During the single pass, The global context vector с and LAE features {h:} are extracted. The Hierarchical Meta-Controller ingests these signals, using Global Scrutiny Gate and Local Gating Module generate manipulation sig- nals (Sglobal. {g.}) }). These signals perform Anisotropic Manifold Scaling to rectify LAE features. lightweight Anomaly Scorer receives the rectified features and outputs the final anomaly curve. Detected anomalous frames can be passed to the full MILM produce textual explanation.
the fine-grained feature vectors (h. from the K LAEs, and a global context vector e E Rddeo The context vector e is obtained by extracting the final hidden state corresponding the first generated token, which serves as a holistic semantic summary of the scene by the MLLM. The hierarchical design of the HMC decouples the rectification task into two synergistic levels.
frozen MILM:
Global Scrutiny Gate. as a global signal to quantify the overall possibility of an anomaly. It is implemented as small multi-layer perceptron (MLP) that takes the global context vector as input:
The global scrutiny gate (GSG) computes holistic suspicion score Sglobul
Sglobal (a Lineary ReLU o BN Linearr)(G E [0.
(2)
where Linear: E du devater Linear2 Rddisor x1 BN denotes Bat Bat chtormid and is the
sigmoid function. A value near 0 indicates a benign scene, prompting the HMC to remain quiescent, while value near 1 signals high-suspicion event that requires significant representational steering. This scalar acts as a master switch, determining the overall intensity of the subsequent rectification.
Local Gating Module. While the global gate provides a coarse-level signal, the local gating mod- ule generates a fine- -raiird per-feature control signal for each LAE. It consists of K parallel, lightweight adapter networks. Each adapter is low-rank network that takes the global context c as input and outputs unique, dense steering vector g: E Rdheod
g: taanhhLinnarg (Linearaswn (c)))
where Lineargown E de projects the context vector into a low-rank bottleneck space, and Linearup E Krad projects it back to the feature dimension of the expert head. The rank r is hyperparameter set to a small value to minimize trainable parameters. The hyperbolic tangent tanh activation constrains the output local signals şi to [-1, 1]. enabling the controller to learn not only to amplify feature dimensions in positive values but also to actively suppress them.
Anisotropic Manifold Scaling, The control signals generated by the GSG and LGM converge the core rectification mechanism, anisotropic manifold scaling. Our goal is apply targeted,
5

---

Published as a conference paper at ICLR 2026
context-dependent geometric transformation the LAE feature space. We operationalize this trans- formation through simple yet powerful element-wise operation, ensuring it incurs negligible com- putational overhead. A detailed geometric interpretation of this operation is provided in the Ap- pendix A. For each LAE feature h; the rectified feature h is computed as:
This formulation instantiates the gebinets fransfrrraaation gas a stable, residual modulation. Thf global signal Sylobal governs the intensity of transformation, while the local vector gi provides its critical anisotropy, enabling the targeted stretching and compressing of the manifold along specific semantic axes potentially linked to pre-training biases.
3.5
ANOMALY AGGREGATION AND SCORING
The anomaly scorer processes the rectified features from selected LAEs by estimating frame level anomaly probabilities and then applying temporal smoothing to yield final coherent anomaly curves.
Probabilistic Anomaly Scoring. For each segment S., the rectified feature vectors (hş E Rdve KKK Ji=1 from selected LAES are aggregated via concatenation into single vector Fl0
Concat(h',, h'x) E TR Khue The aggregated vector then input to anomaly scorer, a logis- tic regression classifier that maps the feature to frame-level anomaly probability pr:
(+() linal + b)
(5)
where w and are the learnable weights and bias, and a the sigmoid function. This scorer ensures minimal computational overhead and prevents overfitting on the small calibration dataset, thereby focusing the learning complexity within the hierarchical meta-controller.
Anomaly Curve Generation. To mitigate noise from transient visual fluctuations in the raw anomaly probability sequence {P1, .2...., Pr)). we employ temporal smoothing via ID Gaussian convolution. This process. predicated on the temporal locality of anomalous events, yields stable anomaly curve A, by evaluating each point within its temporal neighborhood. The final score A, is computed as a weighted average over a temporal window:
A: ET G(t G(t-j;o,,
E G(t j=;jj
where G(z; 0g) (opp-2//(222)) is the Gaussian kernel with standard deviation og.. and normal- ization ensures unit weight sum. This filtering suppresses transient noise while preserving sustained anomalies, yielding smoother, more reliable detection signal for practical deployment.
Hoc Explainability. When segments St flagged as anomalous, the corresponding video frames can be concatenated and re-submitted to the frozen auto-regressive MLLM. The model then generates textual description, offering explanations for the anomaly alert and enhancing the trans- parency and trustworthiness of our framework.
3.6 TRAINING OBJECTIVE
The trainable parameters of our framework consist of the HMC including GSG, LGM and anomaly scorer. We train the framework with composite objective that jointly maximizes classification accuracy and enforces inductive bias for anomaly detection. The main component of this objec- tive is the binary cross-entropy (BCE) loss between the predicted anomaly probability pt and the ground-truth y E 1);
LBCE = - log(p:) + +(1 - y) log(1 - -p))
To mitigate false positives by preventing the model from overreacting to benign activity, we intro- duce sparsity-inducin regularization on the global signal Sglobal for normal samples. Let Brom be the set of normal samples in a training batch. The regularization loss is:
(1 (lobbal)
Lrey
B
B
jEBroo
where glaohal is the global signal for the j-th normal sample. This L2 penalty encourages the con- troller remain dormant (Sglobal 0) for normal inputs.
(

---

Published as a conference paper at ICLR 2026
The final objective is weighted combination of these two losses:
Liotal LBCE + >reg
where Are controls the regularization strength. This composite objective trains the HMC to activate its feature-modulating capacity only when necessary. Promoting sparsity the activation of the global gate enhances the robustness of the model and reduces the potential for false positives.
EXPERIMENTS
4.1 EXPERIMENTAL SETTINGS
Datasets. We evaluate our framework on two widely adopted, large-scale benchmarks for VAD.
UCF-Crime (Sultani et al., 2018) contains 1900 untrimmed real-world surveillance videos sist of 13 distinct anomaly types, totaling over 128 hours. The official split contains 1610 videos (810 abnormal and 800 normal) for training and 290 (140 abnormal and 150 normal) for testing.
Violenee (Wu et al., 2020) comprises 4754 untrimmed videos from movies, sports, and online sources. It includes 6 different anomaly categories. The dataset is divided into 3954 training videos and 800 test videos (500 abnormal and 300 normal).
We use small, randomly sampled 1% subset the training sets from each dataset for our LAE discovery and the training of the HMC. The full official test sets are used for evaluation.
Evaluation Metric. We adopt the standard evaluation metrics for each dataset to ensure a fair comparison with prior works. For UCF-Crime, we report the frame-level Area Under the Receiver Operating Characteristic Curve (AUC). This metric provides comprehensive measure of the ability of model to distinguish between normal and anomalous frames across all decision thresholds. For XD- VD-Vielen,e, we report the Average Precision (AP), which evaluates the precision-recall trade-off' and sensitive to the correct classification of positive (anomalous) instances. For both metrics, higher values indicate superior performance.
Implementation Details. SteerVAD is built upon the InternVL3 (Zhu et al., 2025) as the frozen MLLM backbone. For video processing, each video is divided into non-overlapping segments at an interval of 48 frames and uniformly sample F 16 frames from each segment as the input to MLLM. For the offline calibration (LAE discovery and HMC training), we select the top K attention heads the LAEs based on the RSA scores. The HMC and anomaly scorer trained for 1000 epochs using the Adam optimizer with a learning rate of 1 x 10-3 and batch size of 64, the regularization weight Aveg 0.1, and the standard deviation for the Gaussian smoothing kernel = 6. All experiments were conducted on a single NVIDIA RTX A6000 GPU.
4.2 COMPARISON WITH STATE-OF-TIE-ARR METHOODS
As detailed in Table 1, our proposed SteerVAD establishes a new state-of-the-art (SOTA) in the tuning-free vidco anomaly detection approaches, demonstrating consistent superiority on both the UCF-Crime and XD-Violence benchmarks. This strong performance is attributed to active in- tervention mechanism, where a hierarchical meta-controller executes targeted geometric steering within the feature space. Unlike passive methods that rely fixed representations from pre- trained MLLM, our approach dynamically rectifies the feature space to counteract inherent model biases. This process selectively amplifies anomalous signals that would otherwise be obscured.
Furthermore, SteerVAD narrows the performance gap with fully-trained fine-tuned methods while operating with exceptional data efficiency. On UCF-Crime, our model achieves an AUC of 87.15% which is highly competitive with the 99.51% AUC of the Holmes VAD.. Crucially, SteerVAD utiliz- ing a minuscule fraction of the data demanded by its counterparts. This validates that leveraging the global understanding of an MLLM through targeted geometric steering presents compelling and efficient alternative to expensive fine-tuning for adapting MILMS to specialized downstream tasks.
4.3 VISUALIZATION OF LATENT SPACE RECTIFICATION
To provide a qualitative assessment of our manifold rectification strategy, we employ t-SNE to visu- alize the latent feature distributions from the XD- Violence and UCF-Crime datasets in Figure 4, The

---

Published as a conference paper at ICLR 2026
Table Comparison with existing methods on the UCF-Crime and XD-Violence datasets.
UCF
XD
Mode
Methods
Backbone
Params
AUC (%)
AP (%)
Wu et al. (Wu et al., 2020)
I3D
82.44
73.20
MIST (Feng et al., 2021)
13D
82.30
RTFM (Tian al., 2021)
13D
84.30
77.81
S3R (Wu al., 2022)
13D
85.99
80,26
Weakly Supervised
MSL (Li et al., 2022b)
13D
85,30
78.28
UR-DMU (Zhou et al., 2023)
I3D
86.97
81.66
MFGN (Chen et al., 2023)
I3D
86.98
79.19
Wu et al. (Wu et al., 2024a)
VIT
86.40
66.53
CLIP-TSA (Joo al., 2023)
87.58
82,19
Yang et al. (Yang et al., 2024)
VIT
87.79
83.68
VadCLIP (Wu et al., 2024c)
VIT
88.02
84.51
TUR et al. (Tur et al., 2023)
Resnet
66.85
Self Supervised
BODS (Wang & Cherian, 2019)
I3D
68.26
GODS (Wang & Cherian, 2019)
I3D
70.46
GCL (Zaheer et al., 2022)
ResNext
71.04
Unsupervised
DYANNET (Thakare al., 2023)
I3D
84.50
Holmes- (Zhang al., 2024b)
ViT+LLM
~B
88.96
87.68
Fine- Tuned
Holmes-VAn VAD (Zhang al., 2024a)
VIT+LLM
~BB
89.51
90.67
Full-Training Free
Zero-Shot CLIP (Radford al., 2021)
VIT
0..4B
53.16
17.83
7S ImageBind (Girdhar al., 2023)
VIT
1.22B
55.78
25.36
LLAVA-1.5 ..5 (L.iu et al., 2024a)
ViT+LLM
~BB
72.84
50.26
Tuning-Free FFree
LAVAD (Zanella al., 2024)
BLIP-2+LLM
14B
80.28
62.01
Multimodal VAD
EventVAD (Shao et al., 2025)
ViT+LLM
7B
82.03
64.04
Few- Pw--Parameter Learning
VERA (Ye al., 2025)
VIT+LLM VIT+LLM
8B
86.55
70.54
HiProbe VAD (Cai et al., 2025)
8B
86.72
82.15
SteerVAD (Ours)
ViT+LLM
-B
87.15
83.02
Original
Rectified
Original
Rectified
Componente
60
Noreal
componontt2
Homea Absura
I-SNE Companent
Atneraal
I-SME Component
25
-25
20
-30
20
-10
10
-20
20 Component
40
50
Component
Component
Component
Figure 4: Visualization of manifold rectification via t-SNE on aggregated features from the LAAS for the XD- and UCF-Crime datasets.
original features extracted by the frozen MILM exhibit a high degree of overlap, where the represen- tations of normal (blue) and anomalous (red) samples are largely indistinguishable This observation is consistent across both datasets, highlights the representational bias that hinders downstream sep- arability. In contrast, after the application of our hierarchical meta-controller, the rectified feature space is fundamentally restructured. The same samples now form two distinct, compact clusters with large inter-class margin. This enhancement separability qualitatively demonstrates that our method effectively remaps the latent manifold to mitigate the inherent bias, yielding representations more conducive to classification. Further visualizations are available in Appendix G.
4.4 ABLATION STUDIES
Effectiveness of the Hierarchical Meta-Controller. We validate the design our hierarchical meta-controller (HMC) through the ablation studies presented in Table 2 (Part A). The strategy that trains a linear classifier on frozen raw LAE features, achieves only 81.33% AUC, confirming the need for feature rectification. We then explore context-agnostic modulation strategies. A static steer-
8

---

Published as a conference paper at ICLR 2026
Table 2: Ablation studies on the UCF-Crime dataset, analyzing the impact of key components and
expert selection strategies.
Configuration / Strategy
GSG
LGM
Steering
AUC (%)
Part A: HMC Component Ablation
Full Model
Anisotropic
87.15
w/o Global Gate
X
Anisotropic
85.94
-
w/ Additive Steering
Additive
85.02
w/o LGM (Static)
X
Static Scaling
84.21
Lineear Probing
NA
81.33
Part B: Expert Selection Strategy Ablation
RSA (Ours)
87.15
Mid-to-Last Layer
84.68
Random Selection
69.57
First-to-Mic Layer
63.16
Table 3: Stability analysis of LAE selection and downstream performance across 10 independent runs with different random seeds on the 1% calibration subset.
Seed ID
7270
860
5390
5191
5734
6265
466
4426
5578
8322
Statisties
Selected LAES
Identical (L18114, L231124, L211121, L22117)
100% Match
87.05
87.12
87.18
87.14
87.09 87.21
87.16
87.10
87.13
87.19
87.15 + 0.04
ing variant, which applies a single, learnable channel- scaling vector to LAE features, improves performance by +2.88% AUC. Employing a more expressive additive steering mechanism, which instead modulates features with learnable additive bias vector, further boosts the AUC to 85.02%. However, these limited gains underscore the inherent constraints of unntext-unaware modulation. Relying solcly the local gating module (LGM) to perform its context-dependent anisotropic scal- ing, without the global scrutiny gate (GSG), results in a 1.21% % AUC degradation from our full model. This highlights the GSG's critical function high-level gate that determines if modulation is necessary. thereby preventing the LGM from altering features benign frames indiscriminately. This result demonstrates that the synergistic interplay between the global gating mechanism and the local, fine-grained modulator is crucial for optimal performance.
Effectiveness of Representational Separability Analysis. We validate our RSA against several heuristic selection strategies, with results shown Part B from Table 2. Choosing heads at random first-to-mid layers yield poor performance due to their failure to capture task-relevant semantics. Positional heuristics selecting heads from mid-to-last layers provide better results. The late-layer heads perform better due their encoding of higher-level semantic features, but the selection lacks precision. Our RSA method, which directly optimizes for geometric separability, significantly out- performs all baselines. This performance advantage stems from RSA's ability to measure the manifold disentanglement of each head rather than relying on qualitative architectural assumptions. By filtering for subspaces where normal and anomalous patterns naturally diverge, RSA ensures that the subsequent steering mechanism operates the most geometrically compliant representa- tions. The selected experts represent stable, task-specific functional circuits intrinsic to the frozen backbone, distinguishing them from transient or noisy features often selected by heuristic methods. Calibration Stability and Robustness. To empirically verify that these identified circuits rely on intrinsic model properties rather than artifacts of the limited 1% calibration data, we conducted rigorous stability analysis. As detailed in Table 3, we performed 10 independent calibration runs using different random seeds. The results demonstrate exceptional robustness: the RSA algorithm identified the strictly identical set of Latent Anomaly Experts (specifically heads L18H4, 1.33244, L21H21, and L22H7) across all 10 runs, resulting in negligible AUC fluctuation (a As 0.04%). Furthermore, comparative analysis reveals that increasing data coverage from 1% to 100% yields marginal gains, confirming that the geometric signatures of these experts are sufficiently distinct to be robustly captured even with minimal supervision. This definitive consistency proves that our expert selection mechanism is invariant to specific data splits, establishing a solid and reproducible foundation for the subsequent geometric rectification.

---

Published as a conference paper at ICLR 2026
Data Efficiency and Geometric Saturation. To rigorously quantify the calibration efficiency, we systematically investigate the performance trajectory as the calibration set size scales from the minimal 1% subset (N = 16 videos) up to the full 100% training set. As detailed in Table 4, the model exhibits a distinct phenomenon of asymptotic saturation. While increasing the data coverage from 1% 100% causes the computational cost escalate linearly from less than minute approximately 49 minutes, the resulting performance gain is statistically negligible, yielding only +0.27% improvement AUC. This sharp plateau indicates that the HMC does not rely on massive data density to memorize patterns; instead, it rapidly converges to an optimal steering policy by capturing the intrinsic functional eircuits of the backbone. This confirms that the geometric signature of anomalies is low-rank property that can be robustly estimated with minimal supervision. Visual evidence Appendix further corroborates this stability, showing that the spatial distribution of identified experts remains invariant across all data scales.
Table 4:
Data efficiency analysis. The rapid performance saturation indicates that the geometric steering policy converges to optimal solution with minimal supervision (1%).
Data Ratio
AUC (%)
Improvement
Training Time
% (Ours)
87.15
1 min
10%
87.29
+0.14
~ 5 min
25%
87.35
+0.20
~ 13 min
50%
87.39
+0.24
~ 25 min
100%
87.42
+0.27
~ 49 min
Table 5: Hyperparameter analysis of selected LAEs (K) and input frames (F) on UCF-Crime.
Number of Experts (K)
Number of Frames (F)
Value
AUC (%)
Value
AUC (%)
85.91
82.54
4
87.15
8
85.03
8
87.09
16
87.15
16
86.88
24
87.23
Hyperparameter Analysis We evaluate two critical hyperparameters in SteerVAD: the number of selected latent anomaly experts (K) and sampled frames (F) from each segment. As detailed in Table the performance exhibits clear parabolic relationship with K; the AUC peaks 87.15% when K = 4 by striking optimal balance between incorporating diverse, anomaly-centric features and mitigating the noise introduced by less discriminative experts higher values. The choice of F reveals classic pattern of diminishing returns. Extending the input frames to 16 yields substantial performance gain by integrating richer contextual information, this improvement saturates thereafter offering negligible further gains that do not justify the considerable increase in computational over- head. Therefore, we adopt K 4 and compelling trade-off between model efficacy and computational efficiency.
16 as our final configuration, representing the most
CONCLUSION
In this paper, we introduce a novel intervention framework for tuning video anomaly detection, moving from passive fcature interpretation to active geometric rectification within frozen MLLMs. Our SteerVAD operationalizes this by using representational separability analysis (RSA) to pin- point task-aligned latent anomaly experts and employing hierarchical meta-controller (HMC) to dynamically rectify their representation manifolds through context-aware anisotropic scaling. This active steering mechanism effectively overcomes pre-training biases and resolves contextual ambi- guities, leading to new state- of-te-ff-the-r performance on the UCF-Crime and XD- Violence bench- marks among tuning-fir free methods. Our work demonstrates that targeted. dynamic intervention is a powerful and data-efficient alternative to costly fine-tuning, paving the way for more adaptable applications of frozen foundation models for video anomaly detection.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
This work was supported in part by the National Natural Science Foundation of China under Grant 62225112, 62595732, 62401 365, 62271312, 62132006, U24A20220, in part by the Sichuan Science and Technology Program under Grant 2026YFHZ02055, and in part by the XJTU Research Fund for Al Science, and in part by the China Postdoctoral Science Foundation under Grants BX20250411, 1,
2025M773473.
ETHICS STATEMENT
Our work strictly adheres the ICLR Code of Ethics. The research is conducted exclusively on pub- licly available, standard academic benchmarks (UCF-Crime and XD-Violence) which are widely used for evaluating video anomaly detection algorithms. We acknowledge the dual-use nature of video anomaly detection technology. However, primary motivation is to advance the field for societal benefit, such as enhancing public safety through intelligent surveillance and improving in- dustrial quality control. We believe that our framework represents step towards more responsible AI by promoting transparency and intervention accountability. Unlike opaque fine-tuning methods that alter millions of parameters, tent anomaly experts (LAEs). This architectural choice makes the model's 's decision-making process more scrutable. Furthermore, our post-hoc explanation capability is merely descriptive feature; it provides a user-facing check on the system's 's outputs, allowing human operators to verify alerts against generated textual rationales. By focusing on rectifying pre-training biases at a geometric level, our work also contributes to mitigating one of the core ethical challenges in deploying large models. We advocate for the responsible development and deployment of this technology, ensuring its application aligns with ethical principles and contributes positively to human well- well-being.
approach isolates intervention to small, identifiable set of la-
REPRODUCIBILITY STATEMENT
We are fully committed to ensuring the complete reproducibility of our research. To this end, provide an exhaustive description of our methodology in Section 3, supplemented by a rigorous theoretical analysis in Appendix A and detailed implementation specifics, including architectural layouts and tensor transformations, in Appendix B. All crucial implementation details, including the specific frozen MILM backbone used (internVLL3) dataset preprocessing steps, and complete list of hyperparameters, are clearly documented in Section 4 and Appendix C. Our experiments are built on standard public benchmarks and common libraries (PyTorch, Hugging Face Transformers) to ensure broad accessibility. Furthermore, to significantly enhance the verifiability of our results, we will provide comprehensive ISON file samples of our experimental run outputs in the supplementary materials. We also pledge to release our source code upon the publication of this paper, which we believe will allow the community fully replicate our findings and extend our work.
REFERENCES
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang. Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. arXiv preprint ar arXiv:2502. 13923, February 2025. doi: 10448550/rXxv.25222.3333
Jing Bi, Junjia Guo. Yunlong Tang, Lianggong Bruce Wen, Zhang Liu, and Chenliang Xu. Unveil- ing visual perception in language models: an attention head analysis approach. arXiv preprint arXiv:2412. 181. December 2024. doi: 10.48550/axxi.24422..88
Daniel Bogdoll, Maximilian Nitsche, and J. Marius Zollner. Anomaly detection in autonomous driving: A survey. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 4487-4498 IEEE, June 2022. doi: 10.1109/
CVPRW56634..202200006
11

---

Published as a conference paper at ICLR 2026
Jean-Daniel Boissonnat and Arijit Ghosh. Manifold reconstruction using tangential delaunay com- plexes. In Proceedings of the Twenty-Sixth Annual Symposium on Computational Geometry, SoCG '10 pp. 324-333, New York, NY, USA, 2010. Association for Computing Machinery. ISBN 9781450300162. doi: 10.1145/1810999. 1013. URL https: /ooi .rgg 1811013.
Zhaolin Cai, Fan Li, Ziwei Zheng, and Yanjun Qin. Hiprobe-vad: Video anomaly detection via hidden states probing in tuning-free multimodal lims. arXiv preprint arXiv:2507. /73944 2025.
Yingxian Chen, Zhengzhe Liu, Baoheng Zhang, Wilton Fok, Xiaojuan Qi, and Yik-Chung Wu. Mgfn: Magnitude-contrastvve glance-and-focu network for suakly-supervised video anomaly detection. In Proceedings of the AAAI conference on artificial intelligence, pp. 387-395, 2023.
Jia-Chang Feng, Fa-Ting Hong, and Wei-Shi Zheng. Mist: Multiple instance self-training frame- work for video anomaly detection. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pp. 14004-14013, IEEE, June 2021. doi: 10.1109/CVPR44447.2200 .0.399.
Christopher R. Genovese, Marco Perone-Pacifice Isabella Verdinelli, and Larry A. Wasser- man. Minimax manifold estimation. ArXiv, abs/1007.0549, 2010. URL https://aag semanticschon wtwpilaarrrrrrrreeeee Serg/CorpusIDD 8833773.
Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Imagebind one embedding space to bind them all. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15180-15190. IEEE, June 2023, doi: 10. 10.1009/VVP552229.22222 2023.221799....145..
Hyekang Kevin Joo, Khoa Vo, Kashu Yamazaki, and Ngan Le. Clip-tsa: Clip-assisted tem- poral self-attention for weakly-supervised video anomaly detection. In Proceedings of the IEEE International Conference on Image Processing (ICIP), pp. 3230-3234. IEEE, 2023, doi: 10.1 10.1109/ICIP4933922222 200.:355..2222.1.1 0e..22222...2228.
Federico Landi, Cees G. M. Snoek, and Rita Cucchiara. Anomaly locality in video surveillance. arXiv preprint arXiv: 10364, January 2019, doi: 125550/arXiw.1911103333
Bo Li, Yuanhan Zhang. Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. -lavaionevision: easy visual task transfer. arXiv preprint arXiv:24088. 03326, October 2024. doi: 10-44550/axXx.2440..33666
Guoqiu Li, Guanxiong Cai, Xingyu Zeng, and Rui Zhao. Scale-aware spatio-temporal relation learning for video anomaly detection. ECCV 2022, volume 13664, pp. 333-350, Springer Nature Switzerland, 2022a. doi: 10.1007/978- 1-1197788--0311977777 20.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning. pp. 19730-19742 PMLR, 2023.
Shuo Li, Fang Liu, and Licheng Jiao. Self-training multi-sequence learning with transformer for weakly supervised video anomaly detection. In Proceedings of the AAAI Conference Artificial Intelligence, pp. 1395-14900 1403, 2022b.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34916, 2023.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 26286-26296. IEEE, June 2024a. doi: 10.1 0D09/CVPR52233.022.0077
Jiaqi Liu, Guoyang Xie, Jinbao Wang, Shangnian Li, Chengjie Wang, Feng Zheng, and Yaochu Jin. Deep industrial image anomaly detection: A survey. Machine Intelligence Research, 21(1): 104-135, January 2024b. ISSN 2731-5398. doi: 10.10077/s11633-023
1459-z. URL http:
//dx.dot wot.iiorg/.............
633 0203-023---2 1459-2..
12

---

Published as a conference paper at ICLR 2026
Wen Liu, Weixin Luo, Dongze Lian, and Shenghua Gao. Future frame prediction for anomaly detection a new baseline. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6536-65455 IEEE, June 2018. doi: LLL109/CVPRR2218800066
Zhian Liu, Yongwei Nie, Chengjiang Long, Qing Zhang, and Guiqing Li. A hybrid video anomaly detection framework via memory- aegoor-augmeeted flow reconstruction and flow-guided frame predic- tion. In Proceedings of the IEEE/CVI International Conference Computer Vision (ICCV), pp- 13568-13577. IEEE, October 2021. doi: 1A110///CV489922220.0000
Haoyan Luo and Lucia Specia. From understanding to utilization: survey on explainability for large language models. arXiv preprint arXiv:2401. 12874, 2024.
Weixin Luo, Wen Liu, Dongze Lian, and Shenghua Gao. Future frame prediction network for video anomaly detection. IEEE Transactions on Puttern Analysis and Machine Intelligence, 11(11): 7505-7520, 2022. doi: 10.1 109/TPAMI.2021.... 200..22222..111990. .399999..
Hui Lv, Chen Chen, Zhen Cui, Chunyan Xu, Yong Li, and Jian Yang. Learning normal dynamics in videos with meta prototype network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15420-15429. IEEE, June 2021. doi: 10.1109/ CVPR46437.2021.0115
Omar Mahmoud, Buddhika Laknath Semage, Thommen George Karimpanal, and Santu Rana. Im- proving multilingual language models by aligning representations through steering. arXiv preprint arXiv:2505. 125844, August 2025. doi: 10.48550/arXiv.55000 12584.
Duy Nguyen, Archiki Prasad, Elias Stengel-Eskin, and Mohit Bansal. Multi-attribute steering of language models via targeted intervention. arXiv preprint arXiv:2502.12446, July 2025. doi: 10. 10.48550/rxxiv.552222244
Ghazal Alinezhad Noghre, Armin Danesh Pazho, and Hamed Tabkhi. A survey on video anomaly detection via deep learning: Human, vehicle, and environment. arXiv preprint arXiv:2508. 14203, 2025.
Qwen, :, An Yang. Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang. Jiaxi Yang. Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2025. URL /tttpp:///rxiv...
abs / 2412 2...2... 15115.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and llya Sutskever. Learning transferable visual models from natural language supervision. arXiv preprint arXiv:2103.00020, February 2021. doi: 10.48550/r2Xiv21103.00
Karsten Roth, Latha Pemula, Joaquin Zepeda, Bernhard Scholkopf, Thomas Brox, and Peter Gehler. Towards total recall in industrial anomaly detection. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 14298-14308. IEEE, June 2022. doi: 10.1109/CVPS52688.... 222.2266882222.11992
Yihua Shao, Haojin He, Sijie Li, Siyu Chen, Xinwei Long, Fanhu Zeng. Yuxuan Fan, Muyang Zhang, Ziyang Yan, Ao Ma. Xiaochen Wang, Hao Tang, Yan Wang, and Shuyan Li. Eventvad: training-free event-aware video anomaly detection. arXiv preprint arXiv:2504.13092 July 2025. doi: 00.448550/axxiv2244.13000
Waqas Sultani, Chen Chen, and Mubarak Shah. Real-world anomaly detection in surveillance videos. In Proceedings of the IFFF/CVF Conference on Computer Vision and Pattern Recog- nition, pp. 1479-6488 IEEE, June 2018. doi: 1.111099/VP..2211..00 8.00078..
Kamalakar Vijay Thakare, Yash Raghuwanshi, Debi Prosad Dogra, Heeseung Choi, and Ig-Jae Kim. Dyannet: A scene dynamicity guided self-trained video anomaly detection network. In Proceed- ings of the JEEE/CVF Winter Conference on Applicattons of Computer Vision (WACV), pp. 5530- 5539. IEEE, January 2023. doi: 10.1099/ACVV56688.2000
20w.VVV56668.22...00002 2023 20230000
13

---

Published as a conference paper at ICLR 2026
Yu Tian, Guansong Pang, Yuanhong Chen, Rajvinder Singh, Johan W. Verjans, and Gustavo Carneir.. Weakly-supervised video anomaly detection with robust temporal feature magnitude learning. In Proceedings the IEEE/CVF International Conference Computer Vision (ICCV), pp. 4955-4966 IEHE, Octoher 2021. doi: 10. 109//CCV48922.2000 .00493.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. 19972 2023.
Anil Osman Tur, Nicola Dall' Asen, Cigdem Beyan, and Elisa Ricci. Unsupervised video anomaly detection with diffusion models conditioned on compact motion representations. In International Conference on Image Analysis and Processing. pp. 49-62. Springer, 2023.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. arXiv preprint T.Xiv:1706.0362.. August 2017. doi:
Haibo Wang, Chenghang Lai, Yixuan Sun, and Weifeng Ge. Weakly supervised gaussian contrastive grounding with large multimodal models for video question answering. In Proceedings of the ACM International Conference on Multimedia, pp. 5289-5298. ACM, October 2024. doi: 10.
114573664647.36808266
Jue Wang and Anoop Cherian. Gods: Generalized one-class discriminative subspaces for anomaly detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 8200-8210. IEBE, October 2019, doi: 10. (09/CCCV2001900088.
Zihao Wang and Victor Veitch. Does editing provide evidence for localization? arXiv preprint arXiv:2502. 2025.
Nick Whiteley, Annie Gray, and Patrick Gubin-Delanchy, Statistical exploration of the manifold hypothesis. arXiv preprint arXiv:2208. 11665, 2022.
Jhih-Ciang Wu, He-Yen Hsieh, Ding-Jie Chen, Chiou-Shann Fuh, and Tyng-Luh Liu. Self- supervised sparse representation for video anomaly detection. In ECCV 2022, volume 13673, pp. 729-745. Springer Nature Switzerland, 2022. doi:
Peng Wu, Jing Liu, Yujia Shi, Yujia Sun, Fangtao Shao, Zhaoyang Wu, and Zhiwei Yang. Not only look, but also listen: learning multimodal violence detection under weak supervision. In ECCV 2020, volume 12375, pp. 322-339. Springer International Publishing, 2020. doi: 10. 10.1007/ 978-3-030-58577-8.2
Peng Wu, Xuerong Zhou, Guansong Pang, Yujia Sun, Jing Liu, Peng Wang, and Yanning Zhang. Open-vocabulary video anomaly detection. arXiv preprint 1xXv:23311007042 2024a.
Peng Wu, Xuerong Zhou, Guansong Pang, Zhiwei Yang. Qingsen Yan, Peng Wang. and Yan- ning Zhang. Weakly supervised video anomaly detection and localization with spatio-temporal prompts. In Proceedings of the ACM International Conference on Multimedia, pp. 9301-9310. ACM, October 2024b. doi: 10.1 1i1115//6646477.3688 1442.
Peng Wu, Xuerong Zhou, Guansong Pang, Lingru Zhou, Qingsen Yan, Peng Wang, and Yanning Zhang. Vadelip: Adapting linguae models for weakly supervised video anomaly detec- tion. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 6074 04422082 2024c.
Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, Zhenda Xie, Yu Wu, Kai Hu, Jiawei Wang, Yaofeng Sun, Yukun Li, Yishi Piao, Kang Guan, Aixin Liu, Xin Xie. Yuxiang You, Kai Dong, Xingkai Yu, Haowei Zhang, Liang Zhao, Yisong Wang, and Chong Ruan. Deepseek-v12: mixture- of-experts languae models for advanced multimodal understanding. arXiv preprint arXiv:2412. 10302, 2024d.
Dan Xu, Yan Yan, Elisa Ricci, and Nicu Sebe. Detecting anomalous events in videos by learning deep representations of appearance and motion. Computer Vision and Image Understanding, 156: 117-127, March 2017. doi: 1...016/..viu.201110.0000
14

---

Published as a conference paper at ICLR 2026
Zhiwei Yang, Jing Liu, and Peng Wu. Text prompt with normality guidance for weakly supervised video anomaly detection. arXiv preprint uxXiv2244408833/ 2024.
Yu Yao, Xizi Wang, Mingze Xu, Zelin Pu, Yuchen Wang, Ella Atkins, and David J. Crandall. Dota: Unsupervised detection of traffic anomaly in driving videos. IEEE Transactions on Pattern Anal- ysis and Machine Intelligence, 45(1):444-459, 2023. doi: 10.1 1W9/TPAMM.002..3550700
Muchao Ye, Weiyang Liu, and Pan He. Vera: Explainable video anomaly detection via verbalized learning of -anguugg models. In Proceedings of the Computer Vision and Pattern Recog- nition Conference (CVPR), pp. 8679-8688 June 2025.
Tongtong Yuan, Xuange Zhang, Kun Liu, Bo Liu, Chen Chen, Jian Jin, and Zhenzhen Jiao. Towards surveillance Andeo-and-language understanding: New dataset, baselines, and challenges. In Pro- ceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp- 22052-22061. IEEE, June 2024. doi: 10. (UU9//CVPR2233..224..2222
M. Zaigham Zaheer, Arif Mahmood, M. Haris Khan, Mattia Segu, Fisher Yu, and Seung-lk Ik Lec. Generative cooperative learning for unsupervised video anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14724-14734. IEEE, June 2022. doi: 2.1109/CCPP55228.222222000
Luca Zanella, Willi Menapace, Massimiliano Mancini, Yiming Wang. and Elisa Ricci, Harnessing large language models for maaiiing-free video anomaly detection. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 18527-18536. IEEE, June
2024. doi: 10. 11.10//CVV55273.2224001111
Hang Zhang, Xin Li, and Lidong Bing. Video-Ilama: An instruction-tuned audio-visual language model for video understanding. arXiv preprint arXi::2306.0288 02858, 2023.
Huaxin Zhang, Xiaohao Xu, Xiang Wang. Jialong Zuo, Chuchu Han, Xiaonan Huang, Changxin Gao, Yuchuan Wang, and Nong Sang. Holmes-vad: towards unbiased and explainable video anomaly detection via multi-modal Ilm. arXiv preprint arXiv:2406. 12235, June 2024a.
Huaxin Zhang, Xiaohao Xu, Xiang Wang, Jialong Zuo, Xiaonan Huang, Changxin Gao, Shanjun Zhang, Li Yu, and Nong Sang. Holmes-vau: towards long-term video anomaly understanding at any granularity. arXiv preprint 4rXiv:2412.06171, December 2024b.
Ziwei Zheng, Junyao Zhao, Yang. Lijun He, and Fan Li. Spot risks before speaking! unraveling safety attention heads in large vision-language models. arXiv preprint arXiv:2501.00002) 2025.
Hang Zhou, Junqing Yu, and Wei Yang. Dual memory units with uncertainty regulation for weakly supervised video anomaly detection. arXiv preprint arXiv:2302.05160, 2023.
Zhenhong Zhou, Haiyang Yu, Xinghua Zhang, Rongwu Xu, Fei Huang, Kun Wang, Yang Liu, Junfeng Fang, and Yongbin Li. On the role of attention heads in large language model safety. arXiv preprini arXiv:2410. 2tln:2410.33000 13708, 2024.
Deyao Zhu, Jun Chen, Xiaogian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: en- hancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304. 10592, October 2023.
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504. /0479, 2025.
Adam Zweiger, Jyothish Pari, Han Guo, Ekin Akyürek, Yoon Kim, and Pulkit Agrawal. Self- adapting language models. arXiv preprint 250.::9500.0094433 2025.
15

---

Published as a conference paper at ICLR 2026
A THEORETICAL ANALYSIS OF THE REPRESENTATION MANIFOLD
This section formalizes the theoretical underpinnings of our geometric steering methodology. We argue that the emergence of structured, low-dimensional representational geometries is natural consequence of applying continuous deep networks to high-dimensional, semantically coherent data. Our argument proceeds by first establishing the topological properties of the input data space, then analyzing how these properties are preserved under the MLLM's ' feature mapping.
A.1 TOPOLOGICAL PROPERTIES OF THE INPUT DATA SPACE
We begin by characterizing the topological structure of the data space C IR from which video segments are drawn. A semantic class C (e.g.. "normal" or "anomaly") corresponds to a subset of this space, denoted Ve C V.
Compactness. We model the data subset Ve for any given class C as compact set. This model grounded two properties. First, video data is inherently bounded; pixel intensities are confined finite range (e.g.. [0, 255)) and video dimensions are fixed, making Ve bounded subset of RN, Second, while the semantic boundary between classes can be ambiguous in practice, we posit that the underlying generative process for class occupies topologically closed region in the data space. This idealized assumption is a necessary abstraction for analytical tractability, allowing us model the core data distribution of N that is both closed and bounded is compact. This compactness provides a stable topological foundation, ensuring that continuous maps such the attention heads in an MILM exhibit well- behaved properties when applied to the data.
well-defined set. Under the Keine-Bore' theorem, subset
Piecewise Senbe-Connettenesss. The data structure for complex semantic categories is inherently piecewise, not monolithic. A high-level concept such "anomaly" encompasses diverse family of distinct scenarios; for instance, "arson" in a building and a "vehicle collision" on street are both anomalies, yet continuous transformation path of semantically valid videos exists in the high-dimensional data space to connect them.
This intrinsic heterogeneity dictates that the data space for class C is most accurately modeled as union of path-connected subsets. Formally, we express this structure as:
Mc
Vc = U (()
(10)
1=1
where each component (L is a path-connected and compact set, representing a semantically co- herent subcluster (e.g., "theft" and "vandalism" as components of "anomaly")). Crucially, these components are not necessarily disjoint; a single event, such as breaking a car window to steal an item, can naturally belong multiple subclusters. This potential for overlap in the input space is critical feature, as it implies that the resulting representation manifolds in the latent space may also intersect, creating complex, entangled geometry that is difficult to separate.
This topological structure has a direct implication for the learned representation space. As the feature extractor h is continuous map, it preserves path-conneetednnss. Consequently, the image of Ve under h, which we term the representation manifold Me = h(Vc), is also a collection of path- connected manifold components:
Mc Mc = U h
Me U t=1
M
(11)
Here we formally define the representation manifold Me as this complete, set-thcoretic union. Cru- cially, the MI.M''s pre-training vast semantic data induces critical property in this structure: the manifold components (M(') are not scattered arbitrarily across the latent space but are mapped into bounded and largely contiguous region. This geometric coherence ensures that the global proper- ties of the parent manifold Mc specifically its centroid (center of mass) are not just well-defined, but serve geometrically meaningful representation of the class's central tendency. Consequently,
16

---

Published as a conference paper at ICLR 2026
RSA metric is not naive simplification but robust measure of the separation between the centers of these high-level structures.
This leads to arate two single, cleanly-defined manifolds, but to distinguish between two families of potentially overlapping and entangled manifold components (M } and (Manim))
precise geometric formulation of the VAD challenge: the task is not to sep-
A.2 FEATURE EXTRACTORS AS CONTINUOUS MAPS
An MLLM feature extractor, such as an attention head, is modeled as a function h V + R.dh This function is a composition of finite number of layers h - fL f1-10.... f1 each consisting of linear transformations and continuous, non-linear activation functions (e.g., GELU, Softmax). Since the composition of continuous functions is itself continuous, the entire feature extractor constitutes a continuous map from the input space V to the representation space RR
A fundamental theorem in topology states that the continuous image of path-connected set is path- connected. Given our model of the input data space where each component V(( is path path-connected and the map is continuous, it follows that the image of each component, MM6 h(v(()) ) must
also be path-connected subset of Tho This formalizes the observation that feature represen- tations semantically coherent subclusters coalesce into distinct geometric structures within the representation space, collectively forming the full representation manifold Mc.
A.3 LOCAL EUCLIDEAN STRUCTURE AND INTRINSIC DIMENSIONALITY
For M to qualify as a differentiable manifold, must also be locally homeomorphic to a Euclidean space. This property emerges from the smooth mapping of local degrees of freedom in the input data to the representation space.
Under the central tenet of the manifold hypothesis, we assume that local variations within small neighborhood of a video E C (e.g., an object's 's position, orientation) can be parameterized by low-dimensional vector u U C Rdx, where U is an open set and dmM < N is the number of local generative factors. This defines video in this neighborhood as a local parameterization v(u).
The feature extractor acts on this parameterization, defining local coordinate chart Φ U + Rhd
via the composition:
Φ(u) h(v(u))
(12)
Since h is composed of differentiable (or piecewise differentiable) functions, the map is also differentiable (or piecewise differentiable). This map embeds a patch from the low-dimensional parameter space IRoM surface within the high-dimensional representation space R
The local geometry of this embedded surface is characterized by the Jacobian of the chart:
Oo Zu
(u)
XdM
(13)
The d column vectors of the Jacobian span the tangent space TμM at a point <(u). We operate under the reasonable assumption that for non-degenerate natural variations, this map an immersion, meaning the Jacobian has full column rank, i.e., rank(J,) dm. This non-collapsing condition ensures that the local dimensionality is preserved. The existence of such differentiable atlas (a collection of charts covering M) establishes M as differentiable manifold.
A.4 IMPLICATIONS AND RATIONALE FOR GEOMETRIC STEERING
Geometric Problem Formulation. The preceding analysis provides a rigorous geometric frame- work for understanding the VAD task and justifying intervention strategy. From this perspective, the VAD challenge arises when the representation manifolds for normal (Maomm) and anomalous (Manom) data are proximal or intersect in Ad,, Such proximity, where infpe.am.qq. Ma............. ||p all2 xe 0, is often due to shared local features (e.g., "a person running" is common to both jogging and fleeing). This entanglement makes reliable separation by static classifier difficult.
17

---

Published as a conference paper at ICLR 2026
Intervention Controlled Duality of Transformation. As introduced in the methodology, hierarchical meta-controller (HMC) learns a context- context-dependent transformation Te R Rdod + R. via anisotropic manifold scaling. This transformation exhibits powerful duality:
Diffeomorphic Remodeling, When all scaling factors are non-zero, To acts as a local dif- feomorphism, smoothly remodeling the geometric structure by anisotropically stretching and compressing the manifold along discriminative axes while preserving local topology. This alters the local metric tensor, effectively increasing the geometric distance between manifolds without tearing their topology.
The true expressive power of our method is unleashed when the framework drives one or more scaling factors to zero. This induces a singularity, rendering the transforma- tion locally non-invertible and fundamentally changing its nature from remodeling to structural simplification. Geometrically, this singular transformation acts as a projection, mapping the fea- ture space Rd onto a lower-dimensional effective subspace spanned by the dimensions with non-zero scaling. Information within the projection's 's null space, the dimensions corresponding zero scaling, is actively discarded. This mechanism serves as form of potent, context- aware feature selection, allowing the model to completely nullify dimensions associated with pre-training biases or trsk-irrelevant variance. This projection effectively creates what can be conceptualized as a stable anchor space; representations are anchored to this simpler subspace, and their final structure becomes robust perturbations in the suppressed dimensions, whose outputs are mapped to stable fixed point (c.g., the origin) within the projected subspace.
Singular Projection.
This capability justifies choice of anisotropic manifold scaling. It is not merely heuristic but highly expressive yet efficient transformation capable of learning both smooth, diffeomorphisms and radical, context-aware projections for dimensional reduction. This provides minimal yet powerful mechanism to achieve the desired manifold separation.
Theoretical Objective and Practical Realization. The ultimate objective of our adaptive trans- formation is to maximize the geometric separation between transformed manifolds, thereby resolv- ing structural ambiguity in their latent spaces. This objective can be formally expressed as maximiz- ing a metric like the Hausdorff distance, dis (Taaarr)) Te (Maamm)), While direct optimization of such a metric is often intractable, our practical training objective, binary cross-entropy (BCE) loss serves as an effective and stable surrogate. By penalizing misclassifications, the BCE loss implicitly compels the HMC to learn transformation Te that renders the manifolds maximally separable to linear classifier. This aligns the tractable optimization process with our fundamental geometric goal, training expressive transformation to achieve well-defined structural objective.
This concludes the theoretical derivation, which demonstrates that representation manifolds are a predictable consequence of applying continuous deep networks to structured data and provides rigorous foundation for our geometric steering approach.
A.5 TOPOLOGICAL GENERALIZATION TO OPEN-SET SCENARIOS
A fundamental consideration in applying the manifold hypothesis to real-world surveillance is the unbounded, open-set set nature of video data. While the input space of raw pixels V is theoretically infinite, posit that the latent representation space induced by pre-trained MLLM imposes structured topology that facilitates generalization to unseen events. Here, we extend our theoretical framework to formally account for open-set robustness.
Open-Set as Modular Component Expansion. Recall from Eq. 11 that the anomaly manifold Manom modeled not a monolithic shape, but as union of semantically coherent path- connected components: Manom = U, M. Under open-set conditions, an unseen anomaly type does not violate this manifold structure; rather, it manifests as the emergence of novel topolog- ical component, M(( Critically, due to the generalized discriminative capacity of the foundation model, this new component ( is mapped to a region topologically distinct from the normality manifold Mnorm. Our framework relies on the principle of geometric invariance, while the intrinsic coordinates of (k) are unknown during calibration, its geometric divergence from the normality baseline remains consistent, detectable property.
18

---

Published as a conference paper at ICLR 2026
RSA Geometric Consistency Filter. The Representational Separability Analysis (RSA) oper- ationalizes this principle by functioning as a geometric filter. Rather than searching for attention heads that classify specific known anomalies, RSA identifies functional subspaces where the topo- logical separation between the support of MMorr and any divergent signal is naturally maximized. Subspaces exhibiting chaotic mapping where open-set samples might collapse into the normality manifold are assigned low separability scores and are strictly excluded from the intervention scope.
Entropy-Drivee Steering Policy. Consequently, the intervention policy learned by the hierarchical meta-controller (HMC) is not one of closed-set classification, but of context-relative divergence correction. Conditioned on the global context c, the HMC identifies directions in the tangent space T.MM that exhibit high deviation from the expected context coherence (analogous to high semantic entropy). This formulation explains the model's 's ability to handle unseen data, novel anomalies act as high-entropy perturbations that trigger the pre-learned steering mechanism to project them away from Mnorm: regardless of their specific semantic content.
B DETAILS OF STEERVAD
This section provides detailed description of the implementation of our proposed SteerVAD frame- work, complementing the methodology presented in Section 3. We elaborate on the precise mathe- matical formulations, the architectural specifications of our trainable modules, and the algorithmic procedures for latent anomaly expert (LAE) discovery, model training, and real-time inference.
B.1 PREPARATION OF STEERVAD
B. REPRESENTATIONAL SEPARABILITY ANALYSIS (RSA)
The representational separability analysis (RSA) systematic, gradient-free procedure designed to identify select set of latent anomaly experts (LAEs). The objective is to pinpoint attention heads whose internal feature spaces are most naturally aligned with the VAD task, meaning they maximize criterion of class separability. This section provides the detailed mathematical formulation and algorithmic implementation, complementing the conceptual overview Section 3. The complete procedure is formally outlined in Algorithm 1.
At its core, RSA leverages a geometric separability criterion inspired by the Fisher Discriminant Ratio evaluate and rank the representational efficacy of attention heads. This metric is deliber- ately chosen for its computational tractability, as it operates solely on first and second order statistics (mean and variance), and circumvents the need to train auxiliary classifiers per head, which would incur prohibitive computational and optimization overhead. More fundamentally, its mathematical objective of maximizing between-class seatter while minimizing within-clan class scatter directly aligns with our architectural goal: to steer the manifold and rectify them for unbiased detection. An at- tention head that scores highly under this criterion induces feature geometry in which intra-class compactness and inter-class margin are simultaneously optimized, rendering the representation ide- ally suited for subsequent classification by simple and parameter-efficient linear scorer.
For each attention head hy, (at layer I, index k), its RSA score is computed as the ratio of the between-class scatter to the total within-class scatter:
SB(1,
SRSA (l,A(l, k) Sw (1, k) +
(14)
where e is a small constant for numerical stability.
The numerator, Sy (1, k),, represents the between-class scatter. It is defined as the squared Euclidean distance between the centroids of the anomalous and normal feature distributions, rewarding heads that map the classes to distant locations in the embedding space. It calculated as:
Sp(,k)
(t.A)
1 pond
(15)
The denominator, Sw (1, k), represents the total within-class scatter. It quantifies the compactness of the clusters by summing the squared Euclidean distances of each feature vector to its respective class centroid. A lower value indicates more tightly formed, less diffuse clusters. It is defined as:
19

---

Published as a conference paper at ICLR 2026
Sw (L, =
Σ (LD
IX
(l.)) anom
112
x
(LA)) bormi 1
(16)
where D (1.A)
norm/anom denotes the set of feature vectors {x} produced by head (4, for all normal and anomalous samples the calibration set, where each x E RRdhodd The term represents the centroid (mean vector) of the corresponding feature set.
Algorithm Representationa Separability Analysis (RSA) for LAE Discovery
1: Input: Frozen MLLM M, calibration dataset D Dealib-
2: Parameter: Number of experts to select K.
3: Output: A set of K Latent Anomaly Experts (LAEs).
4:
> Phase 1: Prepare a baass-bllanced calibration set 5: Construct the calibration Dealib to balanced by randomly undersampling the majority class.
6:
b Phase 2: Feature Extraction
7: Initialize empty feature banks for all heads (1, k)):
8: Bltch) <- (
9: for each video segment S, with label in Dealib do
10: Perform a single forward pass S. through M.
11: For each attention head (1, k). extract its output feature vector h hi,k"
12: if y/ is normal then
Add h
(t) hi.k to (I.A)
14: else
Add hi to Blick)
15:
16: end
17:
end for
18:
Phase 3: Score Calculation
19: Initialize an empty list of scores S +- [-
for each attention head k)) do
21: Compute centroids: fmorm mean (1(,k)
Manom mean (((A))
(I.k) (1,k)
22:
Compute between-class scatter (1, k)
Manon Wnorm 1-
23:
Compute within-class scatters:
24:
Sw.wum = -negg(a g(t.)) |h
Mnomm
(1.b)
25:
Swanno = -Ehe || Mano
(1.k)
26:
Compute RSA score SRSA (1, k)) Sb(1 (l),,,)/((S)) (S),,.)) + Sw.anom c).
27:
Add (1, k, (R, k)) to S.
28:
end for
29:
b Phase 4: LAE Selection
30: Sort S in descending order based on SRSA-
31: return the top K heads from the sorted list S.
Equivalence to the Variance-Based Metric. To ensure robust and unbiased estimation, we com- pute the RSA score on class-balanced calibration set, constructed by randomly undersampling the majority class to match the size of the minority class. This procedural detail is critical as it establishes a direct mathematical link between the scatter-based formula used for implementation (Eq. 14) and the more intuitive variance- variance-based metric presented in Methods (Eq. 1).
Specifically, the variance of cluster is defined o² = + Σx x 44 Consequently, the within- class seatter for a class Selass becomes:
o², On a balanced dataset, the total within-class seatter
Sw (1, k) N o² amom (1. k) N 2 m (I, k) N (c (1, k) 2 k))
(17)
20

---

Published as a conference paper at ICLR 2026
Substituting this into our RSA score gives:
Maaom
(1,K) Anoem
12
SRSA k)
(18)
N (o anom (1, 1c) T
(1, +
Since our goal is to rank the attention heads, maximizing this score is equivalent to maximizing the metric from Eq. I, as N is a positive constant that does not affect the ranking order. This balancing act prevents the statistics of a majority class from dominating the score and justifies our use of the more computationally direct seatter formulation.
This principled, quantitative analysis allows us to survey all attention heads within the frozen MLLM and rigorously select the top-K performers. These selected heads, designated as latent anomaly experts, are not just randomly chosen sub-modules; they are empirically validated as the most infor- mative and geometrically aligned subspaces for the VAD task, making them the optimal targets for subsequent manifold rectification.
B.1.2 HIERARCHICAL
(HMC) AND ANOMALY SCORER
This section provides detailed implementations of trainable components in SteerVAD. We move be- yond the conceptual overview in Section to detail the precise tensor transformations, architectural choices, and design rationales essential for reproducibility. The dimensions specified here corre- spond to our experiments using the InternVL model, where the hidden dimension is dmoddl = 3584 and the attention head dimension is dhead 128.
Global Context Vector Rationale The efficacy of our context-aware rectification hinges upon the extraction of the global context vector, с Rooss As introduced in Section 3.4, this vector is derived from the final hidden state of the frozen MLLM at the first generated token. This choice motivated by the auto-regressive nature of the underlying language model. The model generates the initial token by constructing a holistic semantic summary through comprehensive encoding of the entire input sequence, yielding a hidden state that encapsulates global contextual understanding. Further from structural standpoint, the causal attention mechanism ensures that this representation purely conditioned on the inputs, free from the influence of any generated text. This yields an unbiased and information-rin summary of the scene's 's semanties, providing ideal input for the HMC while remaining computationally efficient as a natural byproduct of the single forward pass.
Global Serutiny Gate (GSG) The global scrutiny gate is used distill the high-dimensional global context vector into a single suspicion score Sglobul- It processes batch of context vectors of shape [B dmodel] to produce batch of scores of shape [B,1]
Table 6: Architecture of the global scrutiny gate (GSG). The input batch size is denoted by
Layer
Input Shape
Output Shape
Activation Notes
Input e
B, 3584
Global context vector
Linear-1
[B, 3584
[B, 128
Bormhd
B, 128
B, 128
Stabilizes training
ReLU
BB 128
[B, 128
Non-linearity
Linear-2
[B, 128]
[B, []
Output logit
Sigmoid
[B, 1]
[B,
Produces 5global
Architecture. The module is implemented as a lightweight MLP, whose architecture is detailed in Table 6. The inclusion of Bat chNorml d layer is a deliberate choice stabilize the training dynamics on the small calibration set. It normalizes the feature activations from the first linear layer, preventing the model from becoming reliant on specific dimensions of the context vector whose statistics might be inconsistent across small batches.
Local Gating Module (LGM) The local gating module is designed for parameter-efficient context-dependent feature modulation. It consists of K parallel low-rank adapters, where K is the number of selected LAEs. Each adapter takes the same global context vector e and generates a unique steering vector g: for its corresponding LAF.
21

---

Published as a conference paper at ICLR 2026
Table 7: Architecture of single low-rank adapter within the LGM.
Layer
Input Shape
Output Shape
Activation / Notes
Input e
B, 3584
Shared across K adapters
Linear-down (r 4)
B, 3584
[B, 4]
bias-False
Linear-up
B, 4]
[B 128
bias=False
Tanh
[B 128)
[B, 128)
Produces gi E [-1,
Architecture. Each of the K adapters is identical in structure, as detailed in Table 7. The design choice set bias=Falss in both linear layers is intentional. It ensures that the adapter's s operation is purely modulatory (scaling existing features) rather than additive (shifting features), preserving the original manifold's position while only altering geometry. This aligns with our core objec- tive of rectifying, not replacing the learned representations of MILM.. Numerical computations are performed using bfloat16 precision, consistent with the native format of the InternVL backbone.
Algorithm Training Procedure for HMC and Anomaly Scorer
1: Input: A batch of pre-extracted LAE features, context vectors, and labels {((h;) Ct y1))
2: Parameters: HMC parameters θимс. Scorer parameters 8Scorer: regularization weight Areg-
3: Initialize batch loss Lhatch 0.
4: for each sample ([h:), c, y) in the batch do
5:
Generate Rectification Signals
6: Compute global suspicion score Sglobal GSG OHMC).
7:
Compute local steering vectors {gi i=1
K
LGM(c; @HMC).
8:
PPrforr Anisotropic Manifold Scaling gi) for i = 1...., K.
9:
Compute rectified features: h h + Sglahal
Score and Compute Loss
11:
Concatenate features:
Fiima Concat(h(,:.... h'x))
12:
Compute logit 2 = AnomalyScorrer(Fim (F'maal (ssover).
13:
Compute classification loss LBCE BCEWithoogisLLss((::
14:
b Compute Regularization Loss for Normal Samples
15:
Initialize Laeg O. if y is normal then
16:
18: end Lreg = (8giohal)2
Accumulate total loss: Coatch Coarch + LBCE + >ueg
20:
end for
Perform backpropagation on Lrucch to compute gradients V Oumee Laach
22: Update parameters 0μмс and fscoerr using an optimizer (e.g., Adam).
Anisotropic Manifold Scaling The core rectification mechanism, anisotropic scaling, is an element-wise tensor operation that integrates the outputs of the GSG and LGM to modulate the raw LAE features. The data flow for batch of inputs during inference is follows:
1. Inputs: The process begins with two tensors: the raw LAE features H E BxxXxdxa and the global context vectors с R Bxxx dentell
2. Signal Generation: The context e is passed through the GSG to produce the global scores Sglobal Bx1, and concurrently through the K adapters of the LGM produce the stacked
local steering vectors E R B dhuas
3. Rectification: The global score Sglabal is unsqueezed to shape of [B, 1] to enable broadcast- ing across the K experts and thead dimensions. The final rectified features H' are computed via the element-wise operation:
H' ④ (1 unsqueeze(sgama)) G)
(19)
where 3 denotes the Hadamard product. This mechanism provides dynamic, fine-grained con- trol over the feature geometry. For instance, high global score (Sglatal & 1) combined with positive steering signals in G amplifies the norm of corresponding feature vectors, while nega- tive signals attenuate them, thus actively steering the representation on its manifold.
22

---

Published as a conference paper at ICLR 2026
Algorithm 3 Real-time Inference Pipeline of SteerVAD
1: Input: Untrimmed video stream V: Pre-trained HMC and Anomaly Scorer, LAE configuration.
2: Parameters: Frames per segment L; Sampled frames per segment F; Gaussian kernel std. dev.
Ggi Anomaly threshold Tanomaly-
3: Output: Smoothed anomaly curve ; Set of textual explanations & (optional).
4:
Phase 1: Frame-level Probability Scoring +S1,...,TTT) each of length
5: Partition V into T non-overlapping segments
6: Initialize empty list of probabilities P <- [].
7: for each segment S, from = 1 to T do
8: Uniformly sample F' frames from S..
9: Perform a single forward pass through the frozen MLLM to extract raw LAE features (hitr and context Ct.
10:
With gradients disabled:
(t) global GSG(c:),, {gite LGM(c:)..
h h, (1 + "global"
gi) for i , final Concat()
(e)
F Fl0
13:
4 AnomalyScorer( (F )
14:
15: P: = (=t), where a is the Sigmoid function.
Append pr P.
17:
end for
Phase Anomaly Curve Generation
19: Let N Nreems be the total number of frames in V. Initialize frame-level scores Anw E R frames
with zeros.
20: for f. to 7 do
Let [start, end) be the frame indices for segment St.
22:
Araw[start: end] Pt.
Assign segment probability to corresponding frames
23: end for
24: Compute smoothed curve gmooth GaussianConvolutioni CatpenoooouuitooDDA....... ((Arii 0g)-
25:
Phase 3: Post-Hoc Explanation (Optional)
26: Initialize an empty set of explanations & <- 0.
27: for 1 to T do
28:
Compute representative score for segment St, e.g., ā. = mean ancannAsmoohh [start end)).
if' a Tanomaly then
30:
Let V be the original frames of segment S..
Construct prompt: Q..
32:
Generate explanation: E, MLL.M.eenerat((Q.. V/). Add (1, E2) E.
33:
34:
end if
35:
end for
36:
return Asmonth: E.
Anomaly Scorer The design of anomaly scorer prioritizes robustness over complexity. Its primary role is map the rectified feature space to final anomaly probability.
Architecture and Rationale. The scorer first reshapes the batch of rectified features H' E X into a flattened feature matrix fimal E BB dhea), For our configuration with K 4, this results in an input shape of B, 512). This matrix is then processed by single linear layer (nn. Linear (512, 1)) to produce batch of logits z E R Bx1
This minimalist design, equivalent to logistic regression classifier, is deliberate methodological choice that acts as a form regularization. By constraining the classifier's capacity, we ensure that performance gains attributable to the quality of the rectified representations produced by the HMC, rather than the power of the final classifier. compels the HMC to learn a feature space that is linearly separable possible.
23

---

Published as a conference paper at ICLR 2026
Training Procedure The HMC and anomaly scorer trained end-to-end using the composite loss function from Equation 9. The complete training process detailed in Algorithm 2.
B.2 REAL-TIME INFERENCE OF STEERVAD
The inference pipeline of SteerVAD is designed for efficiency, enabling the processing of long, untrimmed videos in single pass and culminating in a final, interpretable output. The entire process, formally detailed in Algorithm 3, integrates segment-wise scoring, temporal aggregation for curve generation, and optional mechanism for generating textual explanations for detected anomalies.
The procedure begins by partitioning the input video into a sequence of non-overlapping temporal segments. Each segment processed independently through single forward pass of the frozen MILMM to extract raw LAE features and a global context vector. The pre-trained HMC and Scorer modules then compute a segment-level anomaly probability. This raw sequence of probabilities, while informative, can exhibit high-frequency noise due to transient visual fluctuations.
To address this and produce temporally coherent result, we introduce an aggregation step. The segment-level scores are first mapped to a frame-level representation. Subsequently, we apply a temporal smoothing filter, as implemented in our visualization code via standard 1D Gaussian convolution. This operation, which efficiently realizes the weighted averaging formula in Equa- tion smooths the raw scores to produce the final anomaly curve, effectively suppressing spurious spikes while preserving sustained anomalous events.
Finally, enhance the framework's utility and trustworthiness, an optional post-hoc explanation module can be activated. For any segment where the smoothed anomaly score surpasses prede- fined threshold, the corresponding video frames are re-submitted to the frozen MLLM's generative interface. By prompting the MLLM describe the events in these frames, obtain natural language explanation for the detected anomaly, bridging the gap between a numerical score and a lumannblddrstanddbbee reason.
C FURTHER EXPERIMENT DETAILS
This section provides additional details regarding our experimental setup to ensure full reproducíbil- ity. We outline the specific implementation environment, the prompts used to query the MLLM, and a comprehensive summary of all hyperparameters used in our framework.
C.1 IMPLEMENTATION DETAILS
Model and Architecture. Our experiments are conducted using the Intern VL3-8B (Zhu et al., 2025) model as the frozen MLLM backbone. This model is based on the Intern 30000/V V2.5 vision encoder and a Dwen2.5-7B-ban
language model (Qwen et al., 2025). The language model comprises 28 transformer layers, with each layer containing 28 attention heads. This results in total of Niotal 28 28 784 attention heads available for the representational separability analysis (RSA). The hidden dimension of each attention head is dhead 128.
Hardware and Software. All experiments were performed on a single server running Ubuntu 22.04.4 LTS. The hardware configuration features dual AMD EPYC 75F3 CPUs (128 logical cores), 755 GB system RAM, and a single NVIDIA RTX A6000 GPU with 48 GB of VRAM. Our implementation is built upon PyTorch (v2.3.0) and the Hugging Face Transformers library (v4.51.3), utilizing CUDA 12.1 for GPU acceleration.
C.2 PROMPT DETAILS
During the development of our framework, we explored several prompts to elicit the most effective global context vector from the MLLM. Below are the main variants we considered. For all experi- ments reported in this paper, we utilized the structured system prompt (Prompt 3), which we found yield the most consistent and task-aligned representations for anomaly detection.
24

---

Published as a conference paper at ICLR 2026
Prompt (General Query).
"If there are any abnormal events that occurred in the video that deviate from the normal scene, please identify them and describe what happened."
Prompt (Structured Response Query).
"Please carefully watch the video and identify if any abnormal events occur. Your task is to first provide clear and concise summary of the video content, then state whether any mentioned abnormal events are present. yes, specify which events occurred and provide detailed description of what happened, including the timing and context. no, explain why you believe the video is normal. Please structure your response follows:
Yes or No answer first with anomaly detection.
Video Summary: A brief summary of the video content.
Details: If Yes, describe the abnormal events in detail. If No, explain your reasoning.
Take your time to analyze the video thoroughly and ensure your assessment accurate. Take deep breath and work on this problem step-by-step."
Prompt (Final System Prompt).
System Identity
You are professional Al assistant specialized in video analysis, focusing on ab- normal behavior detection in real-time surveillance.
Task Context
Analyse if there are abnormal events happened in video frames.
Core Objectives
1. Analyze the following video frames.
2. Detect high-risk abnormal events which should not happen in this video.
3. Perform spatiotemporal analysis across consecutive frames.
Output Requirements
Implement two-level response system with Yes or No first, then subsequent para- graphs must include: Detected event type(s), Detailed description of the behavior, and Risk level assessment (low/mediun/high).
C.3 HYPERPARAMETER CONHIGURATION
Table 8 provides a comprehensive summary of all hyperparameters used throughout our framework, including those for LAE discovery, HMC training, and real-time inference.
D FURTHER EXPERIMENTS
D.1 PERFORMANCE EVALUATION
To provide a comprehensive understanding of the practicality of our framework, this section details the computational performance of SteerVAD, We evaluate the model size, training resource require- ments, and the inference costs associated with the trainable components, demonstrating that our active intervention approach is not only effective but also remarkably efficient.
Model Size and Complexity. The primary advantage our tuning-free free framework is the minimal size of the trainable modules. The hierarchical meta-controller (HMC) and the anomaly scorer operate on top of a frozen, multi-billion parameter MILM, yet their own footprint negligible. Table summarizes the architectural and size statistics of these components, based the final configuration used in our experiments (K = experts). The total number of trainable parameters is approximately 0.52 million, which occupies about 1 MB of disk space. This is several orders
25

---

Published as a conference paper at ICLR 2026
Table 8: Consolidated list of hyperparameters for the SteerVAD framework.
Hyperparameter
Value
Description
Data Processing Model Architecture
48 frames
The duration of each non-overlapping video segment.
F
16 frames
The number of frames sampled from each segment input to the MLLM.
K
The number of top-scoring latent anomaly experts (LAEs) selected by RSA.
HMC Training
Epochs
1000
The total number of training epochs for the HMC and anomaly scorer.
Batch Size
64
The number of samples per batch during training.
Learning Rate
x
10
The initial learning rate
Adam optimizer.
Aneg
0.1
The weight for the sparsity regularization loss (Lrug)
dnidden
128
The hidden dimension of the MLP in the Global Scrutiny Gate (GSG).
4
The low-rank dimension of the adapters in the Local Gating Module (LGG)..
Inference Analysis
6.0
The standard deviation of the ID Gaussian kernel for temporal smoothing. The score threshold to trigger the optional post-hoc explanation module. small constant for numerical stability the RSA score calculation.
Tanomaly
0.5
10
of magnitude smaller than the frozen backbone, highlighting the parameter-efficient nature of our method. The computational overhead is similarly minimal, requiring just over million FLOPs per segment inference, ensuring that the bottleneck remains the single forward pass through the MLLM.
Table 9: Architectural size and computational complexity of SteerVAD's trainable modules.
Component
Parameters
Size Complexity
Parameter Count & Disk Size
Global Scrutiny Gate
459,265
Local Gating Module
59,392
HMC (Total)
518,657
0.99 MB
Anomaly Scorer
513
<0.01 MB
Total Trainable
519,170
"0.99 MB
Computational Complexity (per inference)
HMC FLOPs
1,037,568
Scorer FL.OPs
1,024 1,038,592
Total Added FLOPs
Table 10: Training cost statistics for calibrating the HMC and Anomaly Scorer.
Metric
Value
Total Training Epochs
1000
Batch Size
64
Learning Rate
0.001
Total Training Time
27.83 seconds (0.46 minutes)
Average Time per Epoch
0.03 seconds
Peak GPU Memory Usage
203.88 MB
Training Throughput
4168.81 samples/second
Training Cost. The calibration of the HMC and anomaly scorer is extremely efficient. The train- ing is performed on a small, representative calibration set randomly sampled 1% videos from the training set from UCF-Crime and Xd- vd-leneee As detailed in Table 10, the entire training pro- cess completes in approximately one minute on a single commodity GPU. The memory footprint is minimal, and the training throughput is exceptionally high. This low-cost calibration phase makes SteerVAD easy to adapt to new domains or datasets without incurring the prohibitive expenses as- sociated with fine-tuning large foundation models.
26

---

Published as a conference paper at ICLR 2026
Table 11: Inference cost as function of the number of sampled frames (F) per segment.
Sampled Frames (F)
Avg. Inference Time (s)
Peak VRAM (GB)
0.43
17.2
8
0.78
18.3
16
1.52
20.4
24
2.29
22.3
Inference Cost Analysis. The primary factor influencing inference cost in our framework is the number of frames, F, sampled per video segment to be processed by the MLLM. Table 11 presents the trade-off between the temporal density of this sampling and the resulting computational demands in terms of latency and GPU memory. As expected, both inference time and peak VRAM usage scale near-linearly with the number of sampled frames. This analysis demonstrates clear and predictable trade-off, allowing practitioners to balance detection performance with their available computational budget. The configuration used in our main experiments (F 16) was chosen to provide a robust temporal context while remaining well within the capacity of common research hardware.
Table 12:
Cross-das dasssedttsst generalization comparison. To ensure fairness, we reproduce HiProbeVAD under identical settings with VL3-8B backbone with same promptl setting in App.C.2.
Source (Calibration)
Target (Evaluation)
Metric
HiProbeVAD (Rep)
SteerVAD (Ours)
Gap
UCF-Crime (1%)
XD-Violence (Unseen)
AP (%)
70.66
71.31
+0.65
XD-Violence (1%)
UCF-Crime (Unseen)
AUC (%)
80.82
81.04
+0.22
Zero-Shot Generalization. We further evaluate the robustness of SteerVAD by testing its zero- shot transfer capability across distinct domains. In this setting, the HMC module is calibrated on source dataset and directly applied completely unseen target dataset. To provide a rigorous benchmark, we compare our method against the state-of-the-an probing baseline. HiProbe-VAL implemented with the same backbone and settings. As presented Table 12, SteerVAD demon- strates consistent superiority over the baseline in both transfer directions, retaining high performance despite significant shifts in anomaly definitions and environmental contexts. This cross cross-dataset sta- bility indicates that the hierarchical meta-controller does not overfit to superficial Nomain-specinf cues within the small calibration set. Instead, effectively learns a transferable geometric rectifica- tion policy, identifying and amplifying high-entropy divergences that remains valid across diverse video distributions.
Table 13:
Cross-mode generalization performance. The framework is applied to different most widely used and advanced frozen MLLM backbones.
Backbone
Params
UCF-Crime (AUC %)
XD-Violence (AP %)
LLaVA-OV (Li al., 2024)
7B
81.52
74.65
Qwen2.5-VI. (Bai et al., 2025)
~B
84.11
79.82
VL3 (Zhu et al., 2025)
8B
87.15
83.02
Cross-Model Generalization. Our active manifold steering strategy is a fundamental principle rather than technique coupled to a specific architecture. We applied the SteerVAD framework to other prominent MLLMs, as presented in Table 13, different models including LLaVA-OV and Qwen2.5VL with our steering mechanism unlocks strong anomaly detection capabilities. On both the UCF-Crime and XD-Vis DD-Violence datasets, these models achieve level of performance that is highly competitive. This outcome powerfully suggests that the existence of latent, geometrically steerable expert subspaces is general property of MLLMs. SteerVAD framework should be un- derstood as a aggostic paradigm for adapting a wide range of frozen foundation models to complex perception tasks, rather than solution tied single model.
27

---

Published as a conference paper at ICLR 2026
E EXTENDED ROBUSTNESS AND GENERALIZATION ANALYSIS
In this section, present comprehensive empirical evaluation of the SteerVAD framework's adaptability. Building on the data efficiency analysis in the main text, we focus on the model's sen- sitivity to class distribution imbalances, the structural stability of the expert selection process under random initialization, and its capability to generalize to strictly unseen open-set scenarios. Further- more, substantiate these quantitative results with extensive visual evidence of the underlying latent geometric structures.
E.1 VISUALIZATION OF GEOMETRIC SATURATION
While the quantitative saturation is discussed in the main paper. provide further intuitive visual- ization of the RSA score distribution across all attention heads (L x H). To demonstrate the univer- sality of our findings, generate these heatmaps for all 14 categories within the under varying calibration set sizes. As illustrated Figure and 6, the heatmaps of anomaly sensi- tivity exhibit remarkable invariance. The high-scoring regions, which represent the latent anomaly experts, emerge distinctly at the 1% scale and maintain their spatial positions and relative inten- sities data scales 100%. This visual evidence, consistent across diverse anomaly semantics, powerfully corroborates that the geometric signature of anomalies is a stable, low-rank property, confirming that massive calibration data is redundant for identifying the core functional circuits.
dataset
Menton
Capley
ABUntot hnstaa Cugay
Darany
Cugary
Buplo
Figure 5: Evolution of anomaly expert activations across different calibration data scales. The consistent distribution of hotspots from I % to 100% visually confirms that the identification of anomaly experts saturates rapidly.
28

---

Published as a conference paper at ICLR 2026
tomrallo Dipation
AmarSin Lowaon
Preting
Figure 6: Evolution of anomaly expert activations across different calibration data scales. The consistent distribution of hotspots from 1 % to 100% visually confirms that the identification of anomaly experts saturates rapidly.
29

---

Published as a conference paper at ICLR 2026
Table 14:
Impact of normal/abnormal ratio in the calibration set on UCF-Crimm performance. The model maintains robust detection capabilities across a wide range of class distributions.
Ratio (N:A)
1:9
2:8
3:7
5:5 (Default)
6:4
7:3
8:2
9:1
AUC (%)
83.67
85.69
85.75
86.34
87.15
87.00
85.70
85.62
83.72
E.2
SENSITIVITY TO CLASS DISTRIBUTION IMBALANCE
The robustness of the hierarchical meta-controller (HMC) against label distribution shifts is criti- cal factor for real-world deployment. We systematically analyzed the sensitivity of our framework varying ratios of normal versus abnormal videos in the calibration discriminative steering vector that effectively projects anomalous features away from the normal- of examples to model the direction of geometric
Theoretically, learn
ity manifold, the HMC requires a contrastive divergence. As detailed in Table 14, empirical results demonstrate significant resilience. While balanced distribution (5:5) yields the peak performance, the method maintains robust AUC scores above 86% across wide effective range (from 4:6 to 6:4). Performance degradation becomes no- table only at extreme imbalances. such regimes, the scarcity of counter-example prevents the HMC from establishing stable normality baseline, leading to a less precise rectification policy,
Crucially, this finding highlights distinct advantage of our few-shot calibration paradigm han- dling the inherent sparsity of real-world anomalies. Unlike fully supervised methods that require massive, balanced datasets for convergence a condition rarely met in practice due to the long-tail nature of abnormal events, our framework requires only minute, representative subset for calibra- tion. Consequently, even in scenarios where anomalies are extremely rare in the wild, the ability to construct small, locally balanced calibration from limited samples allows SteerVAD to mitigate the data sparsity problem effectively. This capability provides a robust foundation for learning in extremely sparse scenarios, where traditional approaches would fail acquire sufficient signal.
E.3 ROBUSTNESS TO SUPERVISION GRANULARITY
To evaluate the robustness of SteerVAD under coarser supervision signals, conducted ablation study using video-level labels. this setting. all segments belonging to an anomalous video are labeled as positive during the calibration phase. As shown in Table 15, the model achieves $7.01 % AUC with coarse video-level labels, which is statistically comparable to the 87.15% AUC obtained with standard segment-level labels. This minimal performance drop (-0.14%) demonstrates that the R-A-based expert selection and HMC modulation are highly robust to label noise, effectively distilling the anomaly signature even from coarse annotations.
Table 15:
Impact of supervision granularity on UCF-Crime performance, SteerVAD maintains high performance even with coarse video-level labels.
Supervision Type
Granularity
AUC (%)
Segment-Level (Default)
Fine
87.15
Video-Level
Coarse
87.01
E.4 STRUCTURAL STABILITY OF EXPERT SELECTION
While the main text confirmed the stability of final AUC performance across random seeds, we further scrutinized the structural consistency the selected feature subspaces. Figure presents the jaccard similarity matrix of the Top-20 ranked attention heads across 10 independent random seeds. The heatmap reveals a high degree of overlap between any pair of random seeds. This indicates that the set of identified functional circuits is not sensitive to specific data samples. The algorithm consistently converges to the same subspace of expert heads, reinforcing that RSA captures stable, intrinsic functional components of the frozen backbone rather than fitting transient data noise.
30

---

Published as a conference paper at ICLR 2026
Jaccard Similarity Between Seeds
1.0
Seed
200
Seed
14005
ser
0.6
Seed
Seed
Seed
0.2
Beod 10
Sweet
5005
2003
Seeds
Figure 7: Jaccard similarity analysis of the Top-20 identified anomaly experts across 10 random seeds. The high similarity scores confirm that the expert selection is structurally robust to sampling variations.
Table 16: Detailed per-class AUC (%) performance on UCF-Crime. The active steering mecha- nism demonstrates consistent robustness across diverse anomaly types, with particularly high per- formance dynamic events.
Class
AUC (%)
Class
AUC (%)
Abuse
68.84
Robbery
84.02
Arrest
82.58
RoadAccident
89.64
Arson
59.02
Shooting
85.74
Assault
95.17
Shoplifting
81.57
Burglary
75.47
Stealing
80.18
Explosion
79.72
Vandalism
88.16
Fighting
83.03
Average
87.15
E.5 DETAILED ANALYSIS OF ANOMALY SEMANTICS AND FUNCTIONAL CIRCUITS
To investigate the operational boundaries and internal stability of our framework, synthesize granular performance analysis across all 13 anomaly classes (Table 16) with visualization of the structural consistency of attention head selection (Figure 8).
Geometric Divergence with Contextual Mimicry. As detailed in Table 16, the framework ex- hibits performance heterogeneity that correlates with the semantic ambiguity of the event. Cate- gories characterized by high-velocity motion and explicit visual conflict, such as Assault (95.17%), Road Accidents (89.644 (99.6%)) and Vandalism 18%),,%), achieve peak AUC scores. These events gen- erate representations with large displacement vectors relative to the normality manifold, which are casily amplified by our steering mechanism.
In contrast, categories such as Abuse (68.84) (88.84%) and Burglary (75,47%) present a distinct challenge rooted in contextual mimiery. Visually, burglary closely resembles normal entry, and abuse often mimics normal physical interaction. These anomalies possess a smaller geometric footprint and high semantic overlap with normal activities. The difficulty here lies not merely in geometric separation, but in resolving the intent behind visually ambiguous actions, which suggests that future improvements for these classes may require incorporating longer-range temporal reasoning into the steering logic.
Latent Structural Invariance. verify that the identified anomaly experts represent stable func- tional circuits rather than artifacts of random initialization, Figure 8 visualizes the selection fre- quency of top attention heads across independent random seeds. The resulting heatmap reveals striking structural invariance. A specific subset of heads is consistently selected with nearly iden- tical across all runs, while the majority of the background heads remain inactive. This consistent convergence pattern demonstrates that RSA robustly identifies intrinsic functional anchors within the frozen MLLM that are uniquely sensitive to anomaly semantics, independent of the stochasticity in data sampling.
31

---

Published as a conference paper at ICLR 2026
THI0 Saate
1225"
Figure 8: Visualization of top attention head selection frequency across 10 independent random seeds. The emergence of consistent high-frequency heads confirms that the model reliably identifies the same intrinsic functional circuits for anomaly detection, demonstrating robustness against ran- dom calibration splits.
32

---

Published as a conference paper at ICLR 2026
E.6 GENERALIZATION TO OPEN- SEN AND UNSEEN ANOMALIES
To empirically validate the topological generalization framework proposed in Appendix A.5, we conducted a stress test on the UBnormal dataset. SteerVAD achieves a compelling 72.21 % AUC on this benchmark without any re-calibration. This result provides strong empirical evidence for semantic entropy hypothesis, even for strictly unseen anomaly types, the anomalous events manifest as high-entropy geometrie perturbations in the latent space, which are effectively rectified by our entropy-driven steering policy.
F MECHANISM DESIGN VALIDATION
This section provides deeper investigation into the architectural design choices of our framework, specifically validating the rationale behind using linear metric for RSA and the architectural con- straints of the HMC.
F.1 VALIDITY OF LINEAR RSA VS. NON-LINEAR METRICS
To justify the use a linear metric (RSA) for analyzing complex manifolds, we compared against two computationally intensive metrics, Silhouette Coefficient and k-Nearest Neighbor (k-NN) Pu- rity. We visualize the trade-off between computational cost and selection consistency in Figure 9.
The top panel highlights the dramatic efficiency gap that RSA completes calibration in milliseconds (0.14s), whereas k-NN incurs computational costs that are orders of magnitude higher (6.87s). De- spite the speed difference, the head stability heatmaps for RSA, Silhouette, and k-NN visually indistinguishable. All three metrics highlight the exact same expert regions. This visual evidence, combined with the strictly identical expert overlap shown in Table 17, conclusively proves that lin- earity is not a limitation but a feature, RSA serves as a lossless, high-speed proxy for identifying the manifold structure.
Table 17: Comparison of RSA with non-linear selection metrics regarding expert overlap and com- putational efficiency. RSA identifies the identical set of experts as expensive non-linear metrics but with significantly higher speed.
Metric
Lincarity
Expert Overlap
Calibration Time
Speedup
RSA (Ours)
Linear
100% (Reference)
0.14
Silhouette
Non-Linear
100% Identical
0.86 s
8x
k-NN Purity
Non-L.inear
100% Identical
6.87 s
1
F.2 ABLATION OF HMC INPUT DESIGN
We explicitly designed the HMC to condition purely on the global context vector c. To verify this de- sign, we implemented a local-aware variant where the input to the LGM adapter is the concatenation of the global context and the local feature ((c, h,)).
As shown in Table 18, the local-aware variant results in a performance decrease of 0.80% AUC. This confirms that including the high-dimensional local feature causes the lightweight HMC to overfit to local noise. By restricting the input to the global context, we enforce top-down modulation mechanism, ensuring that rectification signals are robust and context-driven.
G QUALITATIVE ANALYSIS
The improved geometric organization of our SteerVAD corresponds to refined semantic attention mechanism, as evidenced by the activation heatmaps in Figure 10. Initially, feature activations from the base model are scattered, indicating a failure to prioritize the semantically critical regions and apparent distraction by biased objects within the video frames. Upon rectification by our framework, the focus of Steer VAD is unequivocally sharpened. The result- ing feature activations are highly concentrated and precisely highlight the key anomalous entities,
Visualization of Attention
33

---

Published as a conference paper at ICLR 2026
Average Computation Time per Method (Total over all prompts 5 seeds)
Average Time (seconds, log scale)
10
0.86s
0.14s
RSA
Silhouatte Method
k-NN
Figure 9: Validation of RSA efficiency and consistency. (Top) Average computation time per method, showing RSA's superior speed advantage. (Bottom) Heatmaps of head importance scores generated by RSA. Silhouette, and k-NN. The nearly identical visual patterns confirm that RSA captures the same underlying geometric structure as computationally expensive non-linear metrics.
Table 18:
Design ablation for HMC input signals on UCF-Crime. Conditioning solely on global context prevents overfitting to local feature noise and ensures robust top-down modulation.
Design Variant
Input Signal
AUC (%)
Observation
Loaal-Aware
[c, hg] (Concat)
86.35
Overfits to local noise & Self-Bias
StecrVAD (Ours)
Global Context Only
87.15
Robust Top-Down Modulation
such the central figures in the video. This demonstrates core capability of our hierarchical meta- controller: it dynamically modulates the feature channels to amplify anomaly-relewe signals and attenuate noise, thereby correcting the semantic gaze of the model for improved task performance.
Case Studies on Anomaly Detection. Figure demonstrates the end to end effectiveness and robustness of SteerVAD across normal and abnormal examples from both benchmark datasets. For an abnormal event in UCF-Crime, the anomaly score curve accurately rises at the moment of action, conclusion supported by the generated explanation. Crucially, the model maintains low score in a busy but normal surveillance scene, demonstrating resilience to false positives. This robustness extends the challenging XD-Violence dataset. The model precisely captures abnormal event while correctly discerning high motion sports clip as normal, a key challenge that confounds many methods. This ability distinguish chaotic yet benign activity from genuine threats, coupled with the semantic coherence of the generated explanations, confirms that our framework delivers not only quantitatively superior results also practically reliable and interpretable detections.
34

---

Published as a conference paper at ICLR 2026
Original
Rectified
Figure 10: Semantic impact of feature rectification on an example from XD-Violence.
No derriiion frus clcce cad alouctig nasse pata di spads Thack aora ralls AE Zalabaaa bulutie
Lady
Yoe Thastde spous cocepieee as loats Palonogg cokaaua (AOC51 ucven porvent.
Eaurayy Pubilily
Anamaly Soves
Azondy
300
26
400
26
403
300
X6
Frame Number
Frumt Number
Qualitative results on UCF-Crime dataset.
Ko. Tieiako akon uaditdadd cocki cige The scae Eiiing covorall og paice you crad by facking
Therr depitt plydiaal pentaltteotttt belower heckey gang acailice be Secces zampl met-e ciaavee
apilee
appindl
Anomaly Scove
1.00
He Fas and
Faroos 2001 300.5-18 0026632
latel rap
Anumaly Sarve
Sew SOUROCU 21 label BI ooule
0:75
Frumi Namber
Frime Numbeer
(b) Qualitative results on XD-Violence dataset.
Figure 11: Qualitative results of SteerVAD on UCF-Crime and XD-Vielence datasets. Each panel shows representative video snippet and the anomaly curve with one normal and one abnormal video. The shaded regions denote the ground truth. Further generated descriptions are also provided.
G.1 ADDITIONAL VISUALIZATIONS
To further illustrate the effectiveness and interpretability of our framework, we provide additional qualitative results and attention visualizations.
Figures 12 and 13 present comprehensive selection of qualitative results on the UCF-Crime and XD-Violence datasets, respectively showcasing representative examples from all anomaly categories within each dataset. These examples reinforce the findings from our main strate the consistent ability of SteerVAD to generate precise anomaly scores and coherent textual explanations across wide variety of complex scenes. The inclusion of all categories highlights the model's robustness in correctly identifying diverse anomalies while avoiding false positives in dynamic but normal environments, further validating the practical reliability of our method.
studies and demon-
Furthermore, Figures 14 and 15 offer more visualizations the semantic impact of our manifold rec- tification process. We visualize the model's effective gaze both before and after intervention. The original focus heatmaps frequently reveal the MLLM's inherent biases, showing attention scattered across irrelevant background elements or non-discriminative objects. In stark contrast, the rectified focus heatmaps demonstrate clear and decisive shift. Our HMC successfully steers the model's attention to the core anomalous action, providing compelling visual evidence that our method not only improves geometric separability but also enforces more task- semantic understanding.
35

---

Published as a conference paper at ICLR 2026
H LLM USAGE STATEMENT
In the preparation this manuscript, the large language model (LLM) is solely utilized for the pur- pose of language polishing and grammar correction improve clarity and readability. We confirm that all core research ideas, the conceptual framework, experimental design, data analysis are en- tirely the original work of the authors. The authors take full and sole responsibility for the accuracy, integrity, and all claims made within this work.
36

---

Published as a conference paper at ICLR 2026
Ns. Theside nad right with cutgle volicles inflddig suisced monig bothe spaan dee smuntay beacer
ipperrig co scpreon aagees peron i parkul) wong to th ailooun anbotted dodieeo
Ailk speail ragitano and vddiddl an botavii cocually la vol
189
Allongo x254
Ancualy Store
Auerab Prbdllyy
Anemaly
0
1000
200
202
400
663
200 Frame Number
10:00
1230
Frame Number
Yos. Tacoiiio alnve guist atout uttu publei butk allos Eackands thore a lafigt Losk of suzpert ta our his suigt fir Thi laddeall enso, wicc powes sppitead fae sak
Yeu Thesike slove egoss pooplc whar is be she aea clamudica look. The piepa as cribiegg peaking and frtonk each odoor iedoutas kyskee oudies The setea poitil
AsseeilES 3209 rpp
Nooeelo00 x254. rp
Ancualy Sore
Anomalyy Scone
3.75
Oneer Thws
8.75
Chocoel Trui
2,50
Frame Number
Frane Number
dbrigh Siopptilips und tedvicade Rlpeur to ust: the winid Sustebo: tdlateenae and atilnin liitell the tralent comtist euch rensery wode
buay malltalle Suddlady capeo
The Aical is chacptigg linkk Lag hane duar Des and
aethon tale Borbed eery. 5082 seperct hurrley Remundaiee teant sulvvilly
Barglayy02 x254
Angnally Store
Anomaly Score
M m M
500
50e
2006 Frame Number
2006
X008
20e3
203
9000
19:0 1000
Frame Number
You Aa eaploiios sccun worlggeea enutag buse ligk cad secloo The und is audka vichte affecs poaatd tinger bazadide angtos the steeeesedd
Yos The &null is de vidoo apecs to lavshal plypeii dioccation la be Sismss ye cut metre's
Eipe people Dhaginl iniliatig y plysica
Esprostent mmpt
Fightightt ctpl
Anyeaay Score
3.75
3,500
0,50
Number
Number
vide, penu nalling
Sharty etee ega bopiene scram meer the bid dawe kodl: due aastaaee or aifteriiiigg
spprasctin sorder be peroee scoter meets be tevobrdd in feree traeccien ledtag acciee swyy qpickiy makuainn Ferter orgatom, wir rageek prod moleery nomana
exlottyciio da vtaaky sock obizhiig kig ciraass made ia pas by sndd
opennnn
Haabbery 706 2ast
Angnaly Scoce
Trul
0.30
1.00 200
34h
403
600
0,30
230
4.0
460
100
Frame Number
Frame Number
Tes is the rites, agrap of incivibuuis are indemuttio tho serred recetrr Use parcra in otting whiee owen axchrrtdreen, wiR (erpl peiniig geerinn texirrt De unded person, Ind catige
Yis Two indiv agparra be engaged appyyiod shooottcc with perwe Mandivn zals stng Lighting ev Tho plapic catas of tacitouutee pull cukioc-kakk pastilly aidii
Shape (ffrnggee cppt
Shootingtos 2tp4
Anomale Score
Avernbl
9.35
Goacal
Anumaly
50
11
M1
Frame Number
Frame Number
soudkotiso secosss Th0 peosu or (i0 bosan als Ba viluds parall auaddaag taking
apparn
Yes The ixtivid is swn kiiig fay door wir vidldn drange The belavaroecalaaa
ssaut coeaac tait) riiing ssaguey buzgle any.
eeceecttnted advvi and peor hasts batt perpery and poteria by byrthadem
Vertaiien
Azgnaly Score
Asumally Score
200
-00
600
209
1003
1250
1400
0,30
500
E20
DM0G
Frame Number
Frame Number
Figure 12: Further qualitative results UCF-Crime dataset. Covered all 13 abnormal category and normal examples.
37

---

Published as a conference paper at ICLR 2026
cancntinn aod net bucorrun lighing The anti in hommen sar bere os Ph
Taeskko sutery vaccoeda unfera 1dade hasde bibras wazs dadan vensi barlans and pellenung for malles
1.00
1905
003001
Aunmaly Scote
225
100
300
1000
1308
330 Frintie Numbee
2.00
1040
15.00
0000
26
600
900
1000
Framc Number
one pablas, 1044 to spaaii be oidlactiq parki
You Tie udoo dann tne playts we fos wilee juas aal secls, blcck paio, dlolkaly dioupp
ond takidaa
subilly contrrtaate see thigr, repates zio tisi. Teems borm pappte
agrossllyy
free eppcdia
OOV ripakel Blg
Annmaly Score
Acaec's hunak
Anomaly Soote
Aevredy Pritidiry
1.30
1.5
5.00
26
1090
12:0
1409
250
1003
1229
1500
1159
F Framc Number
Frame Number
You Thessik patauly slsro quckia befaes buss sunciceea skadkag millis setaoaaii pppinee and umale las apprtalle eut sus tuans when mare cretmadd
Yea Them ridioocadoo rationa lecavees grouped people aluli pokatii gou uid chilks grodp autontad pesuaaul suppoloo rolabi sdadiaa posiis Alomstkc
inoke
vicraiiaa prstoctt Vasrcaa LNI maap raps
1.00
Ted baVYgghil
Wi Lind B4-tstoop
2.5
trak
0.75
Groad Truit
Annmaly
1.50
1.5
NA
Frume Number
Frame Number
bassivigg annind citam and peodite tutisaa distrctiin taitt pits There mvdeaa
Yoa Tlaside aloya aovou kalues La calopos alines jooakllbeeee laik anders aceiiaal lxoepolleiin exicetacs whar da advvaball eo eor and bc...
contral
pilaillll lageling c pottcc oouiittass
1.00
1.30
2.79
Thais
Annmaly
1.59
V1
E.00
150
2000
Number
trumss aprag of raen mem, muming d Burptee by dendis
lupasaaedyy undicatioo
Ex
Fonvrrt, Eageati go by lise out Tny't widk a nes for rol. evevet mows
wnttiggn im phepalll etrgghty by tndrating Righ
ontwitne
1.00
City afdil
500.37.26 06.3600 cep
N0000500mnk 10 OLLOE
Wel By
Aunnoly Secre
035
200
401
600
800
1030
150
20
388
300
Frunc Number
Frame Numther
Yee Them yn eeglasoe ners viine venidate the fait, prodacing Larg oiaddel ansok disida Drplotine innorrllllkkeeeeee paese aetiont Pebitii sermendag
expiedion oemiiis froter & centap hhen ot beeeaas linos Tiotepuctictteeee dedteeenndddddeeee
Enger pagente
VICSP2NN
Anonaly Score
2.3
6.0
135
103
400
1200
100
66
Frame Number
Fram: Numberr
Figure 13: Further qualitative results on XD-Violence dataset. Covered all abnormal category and normal examples.
38

---

Published as a conference paper at ICLR 2026
Original
Rectified
Difference
Original
Rectified
Difference
Original
Rectified
Original
Rectified
Difference
Original
Rectified
Difference
Original
Rectified
Difference
Original
Rectified
Difference
Figure 14: Further visualizations of SteerVAD on XD- Vi-lenne datasets. Our steering method effectively rectify the biased ambiguous focus the original videos.
39

---

Published as a conference paper at ICLR 2026
Original
Original
Difference
Original
Rectified
Difference
B F-
Original
Rectified
Difference
ENTRADA
ENTRADA
Original
Rectified
Difference
Original
Rectified
Figure 15: Further visualizations of SteerVAD UCF-Crime datasets. Our steering method effec- tively rectify the biased or ambiguous focus on the original videos.
40


Reviews:
[{"review_id": "38PDrQdXwF", "reviewer": "Reviewer_wJq5", "summary": "This paper aims to address the limitations of using frozen multi-modal large language models (MLLMs) for tuning-free video anomaly detection (VAD), particularly pre-training biases and contextual ambiguity. The authors propose SteerVAD, a novel active intervention framework. The method first employs a gradient-free Representational Separability Analysis (RSA) to identify internal attention heads that are most discriminative for VAD, termed Latent Anomaly Experts (LAEs). Subsequently, a lightweight Hierarchical Meta-controller (HMC) is trained on a minimal data subset (1% of the training data). During inference, the HMC uses a global context vector and LAE features to generate dynamic signals that perform targeted, anisotropic scaling on the LAE's representation manifolds. This intervention actively separates the representations of normal and anomalous events. The method achieves state-of-the-art performance among tuning-free approaches on the UCF-Crime and XD-Violence benchmarks.", "questions": "1.Can the authors justify the use of a simple, centroid-based linear metric (Eq. 1) to identify experts in the complex, non-convex, and entangled manifold structures described in Sec 3.1 and Figure 2? Is there evidence that this simple metric is sufficient and not just a marginally better heuristic than layer selection?\n\n2.Was any stability analysis performed on the LAE selection? If the 1% calibration set is re-sampled, how much do the resulting Top-K LAEs vary? How can we be confident the method's foundation is not fragile and dependent on a lucky 1% split?\n\n3.What is the rationale for designing a \"blind\" HMC that only uses the global context $c$ to generate intervention signals, without seeing the local feature $h_i$ it is supposed to be rectifying? Have the authors compared this against a \"Local-Aware\" HMC (e.g., $g_i = \\text{LGM}(c, h_i)$) to justify this strong design assumption?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.The paper proposes a novel paradigm shifting from \"passive feature reading\" to \"active geometric intervention.\" It elegantly addresses pre-training bias by identifying LAEs via RSA and applying dynamic, context-aware rectification using the HMC. This core hypothesis is strongly supported by t-SNE visualizations (Figure 4), which show the feature space transforming from \"entangled\" to \"linearly separable,\" and by attention heatmaps (Figure 5, 9, 10), which confirm this geometric reshaping leads to a semantic \"re-focusing.\"\n\n2.The method exhibits exceptional efficiency. In terms of data, it requires only 1% of the training data for calibration. For parameters, the trainable HMC and Scorer total only 0.52M (< 1MB). In terms of training cost, the entire calibration process takes less than 30 seconds on a single GPU. This extremely low adaptation cost makes it highly valuable for deployment and scaling in the MLLM era.\n\n3.SteerVAD achieves SOTA performance on both UCF-Crime and XD-Violence. Ablation studies (Table 2) demonstrate the necessity and superiority of both the HMC's dynamic context-aware design and the RSA expert selection strategy. Finally, the method shows excellent performance in cross-domain (Table 10) and cross-model (Table 11) generalization experiments, proving its robustness as a model-agnostic paradigm.", "weaknesses": "1.There is a conceptual contradiction in the paper's theory. The theoretical foundation (Sec 3.1, Appendix A) emphasizes the need to model complex, non-convex, and entangled manifolds, which is confirmed by visualizations in Figure 2. However, the RSA metric (Eq. 1) used to discover these structures is a simple, linear metric based on centroids and variance, which implicitly assumes the manifolds are Gaussian-like and convex. This use of a simple linear metric to solve a complex non-linear problem weakens the convincingness of RSA. RSA may just be a slightly better heuristic rather than a robust metric that truly understands manifold geometry.\n\n2.The method's foundation, the selection of LAEs, lacks a critical stability analysis. The entire LAE discovery process relies on a minimal (1%) calibration dataset. The paper provides no evidence that this selection process is robust. If a different 1% random sample of data were used, would the chosen K=4 \"experts\" be completely different? If the LAE selection is highly sensitive to data sampling (i.e., unstable), the method's foundation is fragile, which would weaken the rationale for all subsequent interventions.\n\n3.The core mechanism, the HMC, suffers from a \"blindness\" problem in its design. According to the design (Sec 3.4), the HMC generates all intervention signals ($s_{global}$ and $g_i$) based solely on the global context vector $c$. This means the HMC is completely \"blind\" when deciding how to \"stretch\" or \"compress\" the local feature $h_i$, as it does not take $h_i$ itself as an input. This design makes a very strong, unverified assumption: that the global context $c$ already contains all information needed to correct $h_i$. The paper does not justify why this intervention is reasonable, nor does it compare it to a more intuitive, \"Local-Aware\" design (e.g., where the intervention signal $g_i$ is determined by both $c$ and $h_i$).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "a2t33j4cmB", "reviewer": "Reviewer_ypVo", "summary": "This paper introduces SteerVAD, a tuning-free framework that performs video anomaly detection by actively steering and rectifying the latent representation manifolds of frozen multi-modal large language models. Instead of fine-tuning or relying solely on prompts, SteerVAD analyzes and adjusts the internal geometry of MLLM representations to reduce bias and ambiguity. It identifies anomaly-sensitive attention heads through a Representational Separability Analysis and dynamically refines them using a Hierarchical Meta-Controller that performs context-aware, anisotropic scaling of latent features.\n\nExperiments on UCF-Crime and XD-Violence show that SteerVAD achieves state-of-the-art tuning-free performance, reaching 87.15% AUC and 83.02% AP with only 1% of labeled data and no backbone updates. The framework offers strong interpretability, computational efficiency, and provides a new geometric perspective for adapting frozen MLLMs to complex video understanding tasks.", "questions": "1. The authors assume that video data form a closed and bounded manifold that allows for well-defined geometric steering. Could you clarify how this assumption is justified in practice? In particular, how does SteerVAD behave if the underlying video distribution is open-set or exhibits non-manifold structures? Would your RSA and HMC modules remain stable under such violations?\n2. The paper claims that the conditional distribution $P(Y|V,Z)$ is invariant between training and testing. This seems questionable given the high intra-class variation in anomaly content and context. Could you provide empirical evidence or theoretical reasoning to support this invariance? Alternatively, have you evaluated SteerVAD under distribution shifts (e.g., unseen anomaly appearances or environmental changes)?\n3. In Table 1, most baselines are training-free, while SteerVAD involves learning a few additional parameters. Could you explicitly define how you distinguish between training-free and tuning-free?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper introduces a fresh and well-motivated idea — geometric steering and rectification of latent representation manifolds in frozen multi-modal LLMs — offering a new paradigm for tuning-free adaptation.\n2. The proposed RSA and HMC modules are conceptually clear, lightweight, and technically sound. They enable dynamic, context-aware control over latent representations without modifying the backbone parameters.\n3. SteerVAD achieves state-of-the-art performance among all tuning-free methods on UCF-Crime and XD-Violence, approaching fine-tuned models while using only 1% labeled data.\n4. The method provides both geometric and semantic interpretability through manifold visualization and text-based explanations, while maintaining low computational and memory cost.\n5. The approach is broadly applicable to other tasks involving frozen MLLMs, demonstrating promising potential beyond video anomaly detection.", "weaknesses": "While the paper presents an innovative and well-designed framework, several conceptual and methodological issues remain unclear.\n1. The assumption that video datasets form closed and bounded manifolds is theoretically strong and may not hold in practice. Real-world video data are inherently open-set and dynamic, so the claimed manifold topology should be regarded as an approximation rather than a strict property.\n2. The paper assumes an invariant conditional distribution $P(Y|V,Z)$ between training and testing, which is questionable in anomaly detection. Even if the anomaly types remain consistent, the visual and contextual variations across instances can cause significant distributional shifts, and the authors do not discuss how SteerVAD handles such variability.\n3. The distinction between training-free and tuning-free paradigms is not clearly defined, leading to potential unfairness in the comparisons. Most baselines in Table 1 are training-free, while SteerVAD involves training a small number of additional parameters. The authors should clarify the conceptual difference between these settings, separate the result categories.\n\nOverall, the paper would benefit from a more precise theoretical justification of its assumptions, a clearer comparison protocol, and a deeper discussion on robustness under distributional variation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wBgqvoKZFl", "reviewer": "Reviewer_TUs8", "summary": "This work addresses video anomaly detection (VAD) through a novel multi-modal large language model (MLLM)-based paradigm. The core contribution lies in mitigating the representational bias of MLLMs caused by the domain gap between pre-training web-scale data and downstream video tasks. Using a frozen MLLM, the authors propose a tuning-free framework combining representational separability analysis (RSA) and hierarchical meta-controller (HMC). RSA identifies anomaly-discriminative representations, which HMC subsequently refines through representation manipulation. The method is evaluated on UCF-Crime and XD-Violence datasets.", "questions": "1. Although the MLLM remains frozen, feature modification via HMC may risk knowledge forgetting. The authors should discuss whether this occurs.\n2. It is required to use frame-level labels to supervise the training? Or if the method can operate with coarser (e.g., video-level) annotations.\n3. The paper omits evaluation on unseen anomalies in the same dataset (i.e., anomaly types absent from the calibration set), instead of cross-dataset evaluation in Table 10.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper is well-structured and clearly articulated.\n2. The tuning-free approach is novel, by leveraging pre-trained representations through selection and scaling.\n3. Ablation studies on RSA and HMC effectively validate the design choices.", "weaknesses": "1. Limited Ablation on Calibration Set: The rectified representations heavily depend on HMC tuning via the calibration set. However, the paper lacks critical ablation studies on: 1) Diversity: Performance impact of varying calibration set compositions (e.g., random subsets, different sizes); 2) Class Balance: Sensitivity to ratios of normal/abnormal videos in the calibration set.\n2. Fair Comparison Issues: Table 1 should include parameter counts for specific backbones (especially Multimodal VAD series) to enable fair evaluation. The observed performance drop when switching from InternVL3 to Qwen2.5VL underscores this need in Table 11.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HqTlmBhq6y", "reviewer": "Reviewer_abCn", "summary": "This paper introduces SteerVAD, a novel tuning-free framework for video anomaly detection (VAD). It proposes an active intervention paradigm that steers the internal representation manifolds within a frozen multi-modal large language model (MLLM). The method first identifies the most task-aligned attention heads, termed Latent Anomaly Experts (LAEs), via a gradient-free Representational Separability Analysis (RSA). It then employs a lightweight Hierarchical Meta-Controller (HMC) to perform context-aware, anisotropic scaling on the LAE features, dynamically rectifying their geometry to enhance separability between normal and anomalous events. The framework is calibrated with only 1% of the training data and achieves state-of-the-art performance among tuning-free methods on standard benchmarks.", "questions": "See weaknesses.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Novel Paradigm: The core idea of actively steering latent manifolds to overcome pre-training biases is highly innovative and represents a significant conceptual shift in the field.\n\n2. Comprehensive Evaluation: The paper provides thorough experiments, including ablation studies, hyperparameter analysis, and cross-dataset/cross-model generalization tests, which robustly support the claims.\n\n3. Compelling Visual Evidence: The t-SNE visualizations offer clear and intuitive proof that the proposed method effectively disentangles the feature manifolds.", "weaknesses": "1. Marginal Practical Gain: While achieving SOTA, the absolute performance improvement over very recent strong baselines (e.g., +0.43% AUC over HiProbeVAD on UCF-Crime) is modest. This small margin raises questions about the practical advantage and cost-effectiveness of the method's added complexity compared to simpler probing-based approaches.\n\n2. Limited and Misleading Interpretability: The claim of enhanced interpretability is not fully substantiated. The provided \"post-hoc explanation\" is essentially standard video captioning by the MLLM and does not elucidate the internal decision process of the HMC or the reasons behind its specific steering actions. The HMC itself remains a black box, offering no insight into why a scene is deemed suspicious or how the specific rectification strategy is chosen.\n\n3. Questionable Efficacy of the RSA Module: A significant methodological concern arises from the performance of the features selected by the proposed RSA. As shown in Table 2, a simple linear classifier on these RSA-selected LAE features (\"Linear Probing\") achieves only 81.33% AUC. This is substantially lower—by over 5% AUC—than the 86.72% achieved by HiProbeVAD (Table 1), which also operates by probing internal MLLM features. This result critically undermines the premise that RSA effectively identifies the most powerful \"anomaly experts\" within the model. It suggests that alternative feature selection or probing strategies might yield a stronger baseline, potentially making the subsequent complex intervention of the HMC less necessary or justified.\n\n4. Unsubstantiated Claims of Robustness: The paper attributes its cross-dataset generalization ability to the HMC. However, the HMC is a complex, context-aware module trained on a very small (1%) calibration set. This introduces a non-trivial risk of overfitting to the specific calibration data, which could paradoxically harm generalization. To credibly claim that the HMC improves robustness, a direct comparison is essential. The authors should provide the cross-dataset generalization results (as in their Table 10) for a strong baseline like HiProbeVAD. Without this comparison, it remains unclear whether the observed generalization is a property of the HMC or is already achievable by a simpler, more robust probing method, which would be a significantly more efficient solution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)