PANDA: Towards Generalist Video Anomaly Detection via Agentic AI Engineer
Authors: Zhiwei Yang
Chen Gao
Mike Zheng Shou
Venue: NeurIPS 2025 poster ()
Keywords: Video Anomaly  Detection
AI Agent
Agentic AI Engineer
Multimodal Large Language Model
Primary area: applications
TL;DR: Generalized Video Anomaly Detection via Agentic AI Engineer

Abstract: Video anomaly detection (VAD) is a critical yet challenging task due to the complex and diverse nature of real-world scenarios. Previous methods typically rely on domain-specific training data and manual adjustments when applying to new scenarios and unseen anomaly types, suffering from high labor costs and limited generalization. Therefore, we aim to achieve generalist VAD, \ie, automatically handle any scene and any anomaly types without training data or human involvement. In this work, we propose PANDA, an agentic AI engineer based on MLLMs. Specifically, we achieve PANDA by comprehensively devising four key capabilities: (1) self-adaptive scene-aware strategy planning, (2) goal-driven heuristic reasoning, (3) tool-augmented self-reflection, and (4) self-improving chain-of-memory. Concretely, we develop a self-adaptive scene-aware RAG mechanism, enabling PANDA to retrieve anomaly-specific knowledge for anomaly detection strategy planning. Next, we introduce a latent anomaly-guided heuristic prompt strategy to enhance reasoning precision. Furthermore, PANDA employs a progressive reflection mechanism alongside a suite of context-aware tools to iteratively refine decision-making in complex scenarios. Finally, a chain-of-memory mechanism enables PANDA to leverage historical experiences for continual performance improvement. Extensive experiments demonstrate that PANDA achieves state-of-the-art performance in multi-scenario, open-set, and complex scenario settings without training and manual involvement, validating its generalizable and robust anomaly detection capability. Code is released at https://github.com/showlab/PANDA.

PANDA: Towards Generalist Video Anomaly
Detection via Agentic AI Engineer
Zhiwei Yang!
Chen Gao
Mike Zheng Shou
Xidian University
Show Lab, National University of Singapore
Abstract
Video anomaly detection (VAD) is a critical yet challenging task due to the com- plex and diverse nature of real-world scenarios. Previous methods typically rely on domain- specific training data and manual adjustments when applying to new scenarios and unseen anomaly types, suffering from high labor costs and limited generalization. Therefore, we aim to achieve generalist VAD, i.e., automatically handle any scene and any anomaly types without training data or human involve- ment. this work, we propose PANDA, an agentic Al engineer based on Ms. Specifically, we achieve PANDA by comprehensively devising four key capabilities: (1) self-adaptive scene-aware strategy planning. (2) goal-driver heuristic reasoning, (3) tool-augmented self-reflection, and (4) self-improving chain-of-memory Con- cretely, we develop self-adaptive scene-aware RAG mechanism, enabling PANDA to retrieve anomaly-specifie knowledge for anomaly detection strategy planning. Next, we introduce a latent anomaly- -uidded heuristic prompt strategy to enhance reasoning precision. Furthermore, PANDA employs progressive reflection mech- anism alongside a suite context-aware tools to iteratively refine decision merisiionmmainng in complex scenarios. Finally, chain-of- memory mechanism enables PANDA leverage historical experiences for continual performance improvement. Exten- sive experiments demonstrate that PANDA achieves state-of-the-ar performance in multi-scenario, open-set, and complex scenario settings without training and manual involvement, validating its generalizable and robust anomaly detection capability. Code is released at https://gthub..om//oooo/bbbb..
Introduction
Video anomaly detection (VAD) [I
2 3 4 aims to identify abnormal or suspicious events in
video streams, playing vital role wide range of real-world applications such as intelligent
surveillance [5]. traffic monitoring (6). autonomous driving [7]. and industrial safety [2].
Existing VAD methods follow specialist-oriented paradigm and require manual participation when deploying for new scenarios and anomalies. Broadly, they can be categorized into: training deeeedn and training- free (Fig. [a)). Specifically, training-depea dependent methods rely newly annotated data to train models for each target scenario. The manual and training costs make such methods lack generalization and versatility. Besides, training-free methods typically employ pre-trained large language models (LLMs) or vision-language models (VLMs) as the backbone, thereby eliminating the need for model training. However, they still depend heavily on manual engineering when deploying for new scenarios and anomalies, such as scenario-specifce preprocessing steps. handcrafted prompt templates, rule curation, and prot-pro.....ng. These static pipelines still lack adaptivity, making them brittle when confronted with uncertainty, long-term temporal dependencies, or complex, dynamic scenarios. Moreover, the hand-crafted nature restricts them from towarding generalist VAD.
+Corresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Any Scenarios
Any Anomaly Categories
Engineering
Solection
Task- arienied
P
Manual Enginsering
Tool-ugmmented sall self-rafiaction
Pra-grocassing
knowledge
Agentic
Engineer
VAD Model
LLM VLM
Sull-improving chate-oh-memory
planning
Training domain scenurios
Non-training domain scenarios
Tank-oreented Manual Engineering Post-prooessing
Goal-driven houristic reasoning
Retraining
derminngdeepedentttno VAD
Training- VAD
Generalist
Manual-free
Training-free
(a) Existing specialized methods
(b) Our generalist PANDA
Figure 1: PANDA vs. Existing specialized methods. When facing arbitrary scenes and anomalies, PANDA can automatically adapt without the need for training refinements and manual adjustments, still achieving superior performance.
To overcome the limitations of existing methods and free ourselves from the burden of domain-specific training and handcrafted pipeline design, our vision is to develop a eeneral-purpose video anomaly detection method that can be self-adaptive to new scenarios and novel anomaly types without requiring any training data or manual pipeline assembly. The recent success of Multimodal Large Language Models (MILMs) in wide range of visual understanding tasks offers a promising foundation for realizing this vision. Therefore, we introduce PANDA, an agentic Al engineer for generalized VAD (Fig. [b)). Drawing inspiration from how human engineers systematically analyze problems, adapt to complex environments, and iteratively improve through tool use and experience, PANDA adaptively perceives the enviroment based on user-defined requirements, formulates detecion plans, perform goal-driven reasoning, invokes external tools to enhance decsion making, and continuously accumulates expeirence in memory for eeff-improvement.
Technically, the proposed PANDA is distinguished by the following aspects. (I) self-adaptive scene- aware strategy planning. Faced with new scene or user-defined anomaly detection requirements, PANDA first conducts environment perception and understanding. then retrieves relevant anomaly rules from an anomaly knowledge database. Based on the environment context information, a scene-adaptive Retrieval Augmented Generation (RAG) mechanism is designed to construct tailored anomaly detection plans. (2) Goal-driven heuristic reasoning. PANDA injects task task-specific prompts guided by latent anomaly cues, which steer the reasoning process toward more accurate and focused decision-making, (3) Tool-augmented self-reflectonn. PANDA iteratively assesses uncertainty and activates suite of curated tools, such object detection, image retrieve, or web search, to acquire additional information and resolve ambiguous decision decision-making. (4) Self-improving chain-of-memory. PANDA integrates historical experiences to justify current reasoning decisions or self-reflection. By progressively accumulating contextual cues across temporal spans, it enhances both the stability and accuracy of its decisions over time. Taken together, PANDA embodies agentic Al engineer that proactively perceives diverse environments, formulates adaptive strategies, performs goal-driven reasoning, and progressively improves through tool-augmented reflection and a chain-of-nn -o-mmorryy mechanism, enabling generalizable video anomaly detection across complex real-world scenarios.
Extensive experiments across multiple challenging benchmarks show that PANDA achieves state- of-the-art performance in multi-scenario, open-set, and complex scenario settings, without training and manual involvement. These results highlight PANDA's strong potential as autonomous and general-purpose purpose solution for real-world VAD.
2 Related Work
has long been a critical research topic in the computer vision field due to its wide range weal-world applications. Existing VAD methods are specialist- oriented and can be broadly categorized into
Video anomaly detection (VAD) 8 9 10
and training-free approaches.

---

Training-dependent VAD. These approaches rely on varying levels of annotated data and typically
fall into three categories: semi-supervised VAD 20 T2
[3,. weakly-supervised VAD 15
1ó
17 18 19. and Instruction-tuned VAD 20
21 22 23 24
For example, Ristea et al. 225 proposed
an efficient anomaly detection model based on a lightweight masked autoencoder. Yang et [4] introduced text prompt-driven pseudo-labeling and self-training framework for weakky-superrvied
VAD. Zhang et al. [23] presented a model combining an anomaly-focused temporal sampler with an instruction-tuned MLLM to detect anomalies. While these training-dependent methods often perform well within the domain of the training data, they typically suffer from sharp performance degradation when deployed in out-of- of-diitributton environments or faced with novel anomaly types. This limits their applicability in the
scenarios where anomalies are diverse, and context-sensitive.
Training-free VAD. Inspired by the recent success of LLMs 126 27 and VLMs 22
29
30
training-free VAD methods 31 [32 have gained increasing attention. These approaches aim to leverage the powerful prior knowledge embedded in foundation models without requiring domain- specific training. For instance, Zanella et al. (31] proposed the first anguage-model-bee
training- free VAD framework, which improves anomaly scoring by aligning cross-moss mross-modal features between LLMs and VLMs while suppressing noisy captions. Yang et al. [32) developed a rule-based anomaly inference framework by prompting LLMs to perform inductive and deductive reasoning over anomaly rules. Despite removing the need for training, these methods often rely on static prompting patterns and require substantial manual engineering (e.g. handcrafted wee/post-processing which limits their adaptivity and robustness in complex, real-world scenarios.
Distinct from both paradigms above, PANDA is an agent-based framework that embodies the characteristics of an agentic AI engineer, which is capable of autonomously performing VAD without training and manual engineering when faced with various real-world scenarios. By incorporating a progressive reflection mechanism and suite of perception-enhancci refine its predictions through self-reflection and tool invocation. This enables PANDA to dynamically handle diverse and challenging scenarios in the real world.
tools. PANDA can adaptively
Method
In this section, we present the core architecture and reasoning process of PANDA, an agentic AI engineer for generalized VAD. PANDA is designed to dynamically perceive diverse environments and perform progressive, tool-enhanced reasoning and self-refinement, as shown Fig. 2 achieves this through four synergistic modules: (1) self-adaptive scene-aware strategy planning, (2) goal-driven heuristic reasoning, (3) tool-augmetted self-reflection, and (4) self-improving chain-of- of-eoor-mmmory
3.1 Self-adaptive Scene-aware Strategy Planning
To achieve VAD in general and unconstrained environments, it is essential to dynamically perceive the current video context and construct targeted detection strategies. Given the deeeeddeeeneence of many real-world anomalies and the variability in visual conditions, PANDA first performs self-adaptive perception of the input video to extract high-level environment contextual information.
Environmental Perception. Given a user-defined detection query Userquery and an input video sequence V f2.. fe....,.... fN3 containing N frames, PANDA uniformly samples M keyframes F (Ji, f2,..., .fM) and constructs perception prompt Prompt pereeption combining F with the UseIquery This prompt is fed to a VLM, which returns structured environmental information:
promptpercqpiion
EnvInfo VLM(F, P.ompt,ropppp.mmm peropppeeccpttionn
{Scene Overview, Potential Anomalies,
Weather Condition, Video Quality)
Here, Scene Overview provides high-level summary of the scene, including location type (e.g., street, shop, parking lot) and observed activities. Potential Anomalies refers to types of sus- picious behaviors that may plausibly occur in the current scene context. Weather Condition captures attributes such as time of day (day/night) and weather (e.g., sunny, rainy). Video Quality summarizes resolution and clarity (e.g., low-resolution, blurred, noisy).
3

---

tyee
Burpley, Pipiadoo, Fighing
Robbary, Shoot Bropliting Stealin, Handalam
Anomaly Knowledge
VLM Resson
Enverrenionaal Informaation Booe
Detoction Strolegy Planning
Ressoning Rasuls
Type
Fulls
Weather Condition
Scona Ovensew
Dayt brigtt light
d
Putential Anomaliss
Stealing
Durglay
Prog rooossnni
Companion stracts clak wh thatt
image
Rakes
Raasor The vidko ahoon Beco indiidduaaa
Status
0.5
rauffoonn
Apploaatio
overal cuality vikoo is kow and Llurry
Obeenv customer insrrct with
suffictert datal cprfidietiy
romES stoorma
Iec pan
CoM
Tookit
Rataction
Irrsion
Raglannin
VIM Filsoring
Image Dencise
Seff- NaltRRetaction
Strstegs
Rolieed Reasoning Resuts
Frig gieneee Fintence
Head for fie acrance
mage denas
Supur reSORAA
New Anomaly Raies
Cullomes sppathdly
Razzor
Sudus
The trowsinn
Alnommal hood fount
image Super-redtutior
Toxls to usen
Detectionn
Now Houriili Prompi
tamss surpiiduue behowirr
Image Super readiction
bataovar the
shoplit arunady.
Object Datection
2011
Long
Image Retrive
CoM
image Dabir
Sull-Reflctioo
Paplaniing Detection Strotegy Update
Ressoring Roults
Rassoring
Reanon Anciyyta
image
Mor informat neaded
catermnn wamara
Ienvadayy loven
witto paying
Ourumerr labing
Scone 0.5
Search
Toota to use Well Barch
Vode Search
Focuiin on wortnn
badk wchoout paylng,
womeen canyinn bug
befiaiio sere...
typical shop tring warl..
Figure Overview of the proposed PANDA. As shown in the figure, upon receiving user- defined query, PANDA first performs environment perception and plans a scene-adaptive detection strategy. PAÑDA then executes the plan with goal-driven heuristic reasoning. When encountering ambiguous cases, PANDA enters a reflection phase, revising its plan and invoking external tools to refine the decision. Throughout the process, PANDA maintains both short-term and long-term Chain-of-Memon (CoM), enabling it to accumulate experience and continually improve over time.
RAG-Based Strategy Planning. With the structured environment context in hand, PANDA pro- ceeds to plan its detection strategy. To avoid hallucinations and improve reliability, this planning is performed via retrieval-augie aureaal-aggeeette augmenntd generation (RAG) (33, driven by multimodal large language model (MLLM). First, based on User PANDA constructs a knowledge base prompt Prompt know and then generates a structured general anomaly knowledge base using the MLLM:
Ka MLLM (User.uuy. Promptxnow)
[Event Type, Anomaly Rules, Application Scenes)
Here, Event Type indicates anomaly categories specified by the user. Anomaly Rules are detection rules associated with each anomaly type. Application Scenes are contextual environments where anomalies are likely to occur.
For each anomaly type, we predefine rule-scene pairs to form the knowledge base. PANDA then uses the perceived EnvInfo as query to retrieve the top-k most relevant anomaly rules:
Rules, = (etrievveTopx(EnvIInt. k.).
Finally, PANDA integrates the Userquayy EnvInfo, and Rules, to construct planning prompt Prompt plan which is passed to the MLLM to generate the detection strategy plan:
Plangtrategy MLLM(Prompttpaa)
(Preprocessing, Potential Potentiaal Anomalies,
(4)
Heuristic Prompt].
Here, Preprocessing specifies optional visual enhancement steps (e.g.. brightness adjustment, de- noising, super-resolution). Potential Anomalies refines the anomaly list based on rule relevance and scene understanding. Heuristic Prompt includes step-by-step reasoning instructions for each potential anomaly, enabling the downstream inference module to perform structured, chain-of-thougle analysis.

---

By integrating self-adaptive environment perception and RAG-enhanced strategy planning, PANDA ensures that subsequent anomaly reasoning is goal-driven and context-aware, significantly improving robustness in open-world settings.
3.2 Goal-Driven Heuristic Reasoning
The reasoning module serves the component of PANDA for analyzing video anomaly events. PANDA supports both offline and online inference modes. In this section, we focus on the offline setting, while the implementation details section will describes the online mode settings.
Under the guidance of detection strategy plan constructed subsection 3.T] PANDA performs goal-driven heuristic reasoning using VLM. Given the Querquary. clip-level video segment Velip = {C1, ........ CT) (cach video clip Ct contains s video frames), and the Planatrategy PANDA first applies the preprocessing tools specified in Plangtrategy to obtain an enhanced video clip:
Velip Preprocessing (Viip) { C,, CT)
Next, PANDA constructs reasoning prompt based on the Plamstrategy'
Promptreasoning (Memoryleat 1-stepa Potential Anomalies, Rules,, Heuristic Prompt, Enhancement and Reflection Info].
The fields Potential Anomalies, Rules,, and Heuristic Prompt are directly inherited from the planning stage. The Enhancement and Reflection Info field incorporates information produced during the self-reflection stage (To be described in subsection (.33), including tool-based refinements and updated anomaly rules and heuristic prompts. To enhance temporal awareness,
PANDA equips short-term memory component Memory I-steps which records the past reasoning steps as textual memory. In addition to textual memory, PANDA also maintains a corresponding visual memory stream Memory 1-steps
visual which stores visual frames aligned with the latest steps, allowing the model access fine- ane-grained visual cues during inference.
Finally, driven by the potential anomaly targets and enriched contextual knowledge, PANDA performs heuristic reasoning with the following formulation:
i-sieps Resultycasnnic Ereasoning VLM Memory visual Promptrewoningg
reasoning
(Status hataaa//bbboramlnssssffcci..... Score E [o. 1], Reason : (+)}.
(6)
Here, Status indicates the result of the VLM judgment: Normal indicates the clip is confidently classified as non con-anomal Abnormal denotes strong evidence of anomaly, and Insuff iciint suggests the current information is inadequate to make definitive judgment. Score is the probability of the existence an abnormal event for the clip corresponding to each status. Reason the reason for the status judgment given by the VI.M. When the result is Insuf ficint, PANDA will trigger the reflection mechanism to gather additional context or observations before re-entering the reasoning loop.
3.3 Tool-Augmented Self-Reflection
In complex scenarios, PANDA may not be able make a clear decision whether video segment is normal or abnormal. In such ambiguous cases, it returns an Insufficient status, which triggers the reflection module. PANDA adopts rogl-augeented self-reflection mechanism enhanced by a specialized set of tools - [tooli, tool,............ tool,, for visual content enhancement and auxiliary analysis, including image deblurring, denoising, brightness enhancement, image retrieval, object detection, and web search, etc. These tools assist in gathering additional evidence to support the piceision-making process.
Experience-Drivee Reflection. Given an Insufficient Reason from the current reasoning step, PANDA first queries its long chain-of-memory (Long COM, will be introduced in 3.4) to retrieve the most similar history reflection cases:
Experience rellectionn RetrieveeTopl(hnsufficcieee Reason, Long CoM).
(7)
5

---

PANDA then constructs reflection prompt using video context information, including the User EnvInfo, Planstrategy Rules, short chain-of -emory -eeeooy (short CoM), Insufficient Reason, and Experience,etee i Promptyefsscion {Userquery EnvInfo, Plangerategy Rules,, short CoM, Insuff fiinnt Reason, exsperieneepee cause of uncertainty and recommend an appropriate reflection plan:
This prompt is fed into the MI.LM to analyze the
Resul Erellcction Eysllttelllction
MILLM
["Insufficient Reason" (+),
"Tools to Use" [[tool1, params) ..... (tromm,
"New Anomaly Rule": (?),
"New Heuristic Prompt": (+).
Here, Insufficient Reason refers to the underlying cause of decision uncertainty, inferred by the MLLM in conjunction with VLM output and contextual information such as environmental cues and anomaly rules. Tools to Use specifies the names of tools used for information enhancement and their corresponding parameters. New Anomaly Rule and New Heuristic Prompt represent the updated anomaly rule and the reformulated heuristic prompt, respectively.
Tool Invocation. PANDA executes the tool functions suggested in the reflection result to enhance both visual and semantic information. The tool invocation process is formulated as:
Resul troo, augmented Tool ToolIInooe tool..,,
= {Text Enhancement Info,
Visual Enhancement Info ē Uc.)-
Here, Text Enhancement Info includes summaries from tool outputs (e.g., detected objects, web search results), while Visual Enhancement Info includes processed video clip ε and retrieved historical keyframe set C₂ =
Refined Reasoning.
PANDA updates the reasoning prompt with the newly acquired textual cues:
Prompt reasoinng
refined
Prompt reasaing U
Text Enhancement Info, New Anomaly Rule,
(10)
New Heuristic Prompt)
and re-reasoning the enhanced video clip input:
Result t Pressoning
reflection
VLM(G, U c.. Memory visual promptressoinng
1-steps refiee
(11)
If the returned status Normal or Abnormal, PANDA resumes reasoning the next timestep. If the status remains Insufficient reflection is re-triggered. To prevent infinite loops, we limit the number of reflection rounds to r. If after rounds the result still Inauf cient, PANDA assigns default anomaly score corresponding to the Insufficient status and skips the current segment and continues next timestep.
3.4 Self-Improving Chain-of-Memory
To enable PANDA to become increasingly "smarter" over time by accumulating experience through the iterative cycle of reasoning, reflection, and refined reasoning, PANDA equips self-improving chain-of-memory (CoM) mechanism as shown in Fig. 3 This mechanism enhances both long term context awareness and consistency decision-mall making across video sequences. The CoM comprises two components: short chain-of -oroo-o--mmoryyy (short CoM) and long chain-of-ma of-inoor-mmmry (long CoM).
Short CoM. In the reasoning stage, short CoM includes both the textual reasoning trace 2-steps Memory 2steps and its visual counterpart Memory visual as described in subsection 3.2 In the re-
1-steps
flection stage, short CoM is represented by the set of past reflection outputs: Resul teffecttoo 2 feflulltelletttion / Result/
history
[Result]

---

Rests
Visurd StaneHaannn
Supred Semm/Barne
Seep 2. Reffertinn Rendir
innufiction
Steperdi Ulowe! Seaee//Setts//ee /Sontt//Stwee/eeee SSore/
Short
Norwel Amnara
Lang CAM
V. Hrsnals Prong!
Berisc
Humrnyy Experincce
Roffeettio
Sop Re-rrossring Fenaks
spdatte
Experivncc
Figure 3: Illustration of Self-Improving Chain-of-Memory.
Long CoM. PANDA also maintains temporally evolving long CoM: Long-CoM
{M1, M2........ Mr), where each memory unit M, time step encapsulates three key outputs: Mc = tr Resultrenicttoon: Result tesuuttrcaae trssoinggssnnn
refined J. This structure ensures that PANDA retains a complete trace of all decision stages-iniss rcasoning reasoning, reflective analysis, and post-reflection decisions. At the start of video, LongCoM is empty by design, and PANDA relies on ShortCoM's local window memory for initial reasoning and reflection. As more clips are processed, LongCoM gradually accumulates traces, supporting memory-comee cootsstttnnssttnttt reasoning and reflection planning. With this self-improving chain-of- ofmmmoy, PANDA leverages accumulated historical experience to inform both reasoning and reflection, leading progressively more stable and accurate anomaly detection over time.
4 Experiments
4.1 Experiment Setup
We evaluate PANDA on four benchmarks: UCF-Crime 17 XD-Veolence 16 UB- normal 34 and CSAD, which represent three distinct settings-mulli--eeeeoo multi-scenario (UCF-Crime and XD-Violence), open-set (UBnormal), and complex scenario (CSAD). UCF-Crime is a large-scale dataset comprising 1,900 long, untrimmed real-world surveillance videos. It covers 13 types of abnormal events such as fighting, abuse, stealing, arson, robbery, and traffic accidents. The training includes 800 normal and 810 abnormal videos, while the test consists of 150 normal and 140 abnormal videos. XD-Violence is another large-scale dataset focused on violence detection. contains 4.754 videos collected from surveillance video, movies, and CCTV sources, encompassing 6 categories of anomaly events. The training and test sets include 3,954 and 800 videos, respectively. UBnormal is a synthetic open-set video anomaly detection dataset with total of 543 videos. defines 7 categories of normal events and 22 types of anomalies. Notably, 12 anomaly categories challenging benchmark for evaluating gen- eralization under open-set conditions. CSAD is a complex-scent anomaly detection benchmark constructed in this work. It consists of 100 videos (50 normal and 50 abnormal), sampled from UCF-Crime, XD-Violence, and UBnormal. CSAD includes videos with challenging conditions such as low resolution, poor illumination, high noise levels, and long-range temporal anomalies. designed to assess model robustness in complex and degraded environments.
Datasets.
in the test set are unseen during training, making
we report the Area Under the Curve (AUC) the frame-level receiver operating characteristic for UCF-Crime, UBnormal, and CSAD. For XD-Violence, we follow the evaluation criterion of average precision (AP) suggested by the work [16] to measure the effectiveness of our method.
Evaluation Metrics. Following the previous methods [16 17
Implementation Details. We adopt Langgraph 35 to build the whole agent framework and all experiments are implemented using PyTorch (36) on the A6000 GPU. We use Qwen2.5VL-77 [28 as the VI.M for perception and reasoning stages, and Gemini 2.0 Flash [27 as the MILM for planning and reflection. During the RAG process, the anomaly knowledge base and environment information

---

Table Comparisons with previous state-of of-the--tthhhe methods on different datasets. "Expl." stands for "Explanation" indicating whether the output results include interpretations of the detected anomalies. Methods categorized as "Semi", "Weak", or "Instru-Tuned" require training data adapt specific scenarios anomaly types.
Multi-Seenatto
Open-Set UB (AUCS))
Camplex Scenaria
Methods
Supervision
Expl.
Manual-Free
Mode
UCP OCF-AAUS))
XD (APS)
CSAD (AUCS)
Specialized
[CV992204]
Offline
[CV9R2034]
Offiine
57.98
HL-Net
RTFM
[CEVSIN])
59.91
VaOCLIP
88
TPWNG [CРЛР2Р3
Weak
Offline
87.79
VERA
[C9882223]]
Weak
Ofline
86.55
70.11
64.52
Holmey VAU
[CР882223]
Instru-Tuncd
Oflline
88.
87.68
56.77
72.47
25 CLIM
Training fice
Oflline
53.16
17.83
46.2
32.45
LLAVA
28
[CУR82:34]
Training free
Offline
72.84
50.26
53.71
47.78
LAVAD
[evR8204
Training free
Offline
80.28
62.01
64.23
57.26
Anomaly Ruler
pocvacc
Traening-free
Offline
71.90
Generalized method
Offline 84.89
70.16
75.78
73.12
PANDA
Training-free
Oaline
82.57
63.57
72.41
71.25
are encoded via the all-MiniLM--L-v2 model [37]. with the knowledge base indexed using FAISS for efficient similarity retrieval. To improve the inference efficiency, the input video is sampled at FPS, and a video clip of s = 5 frames is inferred at each time step. PANDA supports both offline and online reasoning modes. In offline reasoning mode, the perception phase is sampling M 300 frames uniformly for the whole video, while only the initial M 10 frames are sampled in online mode. The number of knowledge entries for each type of anomalous event in the anomaly knowledge base is H = 20. The maximum number of reflection rounds is set to 3. The short CoM length / during the reasoning stage. We retrieve the top k 5 anomaly rules from the anomaly knowledge base for each user query. More implementation details, including prompt templates and tool usage, provided in the supplementary material.
4.2 Comparison with State-of-the-Art Methods
Table compares the performance of PANDA against state-of-the-art specialized VAD methods, including both training-dependent and training-free methods. As shown, PANDA significantly outperforms all existing training-free baselines across all four datasets, even under online settings.
On the UBnormal dataset, which adopts open-set evaluation protocol where test-time anomalies are unseen during training, PANDA surpasses both This highlights PANDA's strong generalization capabilities.
and training- free approaches.
PANDA also exhibits notable advantages on CSAD, the complex cmmple--scene scene benchmark introduced in this work, where traditional methods tend to fail under low-quality or temporally extended anomalies. PANDA's superior performance across diverse datasets and conditions demonstrates its robustness and effectiveness ceneral-purpooe solution for real-world video anomaly detection.
4.3 Analytic Results
Analysis of reflection round r. Table 2a) shows the effect of varying the number of reflection rounds on PANDA's performance. We observe that performance improves gradually when increasing from to Although r 5 yields a slight additional improvement compared to 3, introduces more computational overhead due to repeated tool invocation and reasoning steps. To balance efficiency and effectiveness, we adopt
as the default setting in all experiments.
Analysis of rules number k. Table 2 (b) analyses the influence of the number of retrieved rules k used during RAG-based anomaly strategy planning. When too few rules are retrieved (e.g., k 1), the system lacks diverse contextual cues to support robust reasoning, resulting in performance degradation.

---

Table 2: Key hyperparameter analyses on the dataset. (b) Analysis rules number
(a) Analysis reflection round r.
(c) Analysis of short CoM length i.
Reflection Round
UCF-Crime (AUC%)
Rules
UCF-Crime (AUC%)
Short CoM
UCF-Crime (AUC%)
Number
length
83.83
82.79
82,92
84.89
84.89
84.89
84.91
9
83.92
9
84.03
Table 3: Ablation study the capability of PANDA. Planning refers the self-adaptive scene-aware detection strategy planning, which contains detection planning, adaptive environment perception, and RAG with anomaly knowledge. Rellection denotes the tool-augmented self-reflection mechanism. Memory corresponds to the chain-of-memory module, encompassing both short-term and long-term components.
Key Capabilities of PANDA
UCF-Crime (AUC%)
Planning
Reflection
Memory
Performance
X
X
75,25
X
80.37 (+5.12%)
x
82.63 (+2.26%)
V
84.89 (+2.26%)
Conversely, setting & too high may introduce noisy or irrelevant rules that dilute reasoning quality. Finally, PANDA achieves optimal performance when setting k = 5.
Analysis of short CoM length 1. Table c) analyzes the impact of varying the short CoM length I during the reasoning phase. The best performance achieved when 5. When the memory length is reduced to = 1, performance drops noticeably due to insufficient temporal information, which limits the model's ability to leverage recent reasoning traces. On the other hand, increasing the memory length to 9 also leads to performance degradation, likely because excessive memory introduces historical noise that distracts from the current decision-making process.
Ablation Study. Table presents an ablation study examining the contribution of each core capa- bility in PANDA, including self-adaptive scene-aware strategy planning (Planning), tool-augmented self-reflection (Reflection), and -f-emoy (Memory). The third row serves as the baseline, where PANDA performs direct reasoning solely based on the user-defined query, without planning, reflection, and memory modules. As shown, the performance is relatively poor, with an AUC of 75.25%. Equipping PANDA with the planning capability yields substantial improvement of +5. 15.12% in AUC. This demonstrates the effectiveness of scene perception, rule retrieval via RAG, and context- aware strategy plans in inspiring the potential of PANDA. Adding the reflection module further improves performance by +2.26%, suggesting that the self-reflection mechanism, enhanced by the integration of external tools, expands PANDA's capability resolve challenging and ambiguous cases. Finally, incorporating the memory mechanism results in another +2.26% gain, validating the effectiveness of the thain-of-memory design. This module enables PANDA accumulate experience across time and use it to refine decisions. In summary, each of PANDA's capabilities plays vital role in enabling generalizable and reliable anomaly detection. The synergistic integration of all modules empowers PANDA as highly capable agentic Al engineer for generalized VAD. For the ablation study on the key components corresponding to each capability, please refer to the Subsection C.I the supplementary material.
4.4 Qualitative Results
Figure 4 shows visualized example from the UCF-Crime test set to illustrate PANDA's reasoning- reflection process. The left side of the figure shows the anomaly score curve over time. On the right, we visualize PANDA's internal reasoning and reflection process. When the model encounters

---

10 03
Vidao Time Reasoning Reaults: ("score": 0.0, "status" :: "nnormal", "resson"; "The scana shows a quiet store counter with no visible signs abnonmal activity. There are no people present, and the enwiromennt appears undisturbed.]]
Video Time 02:25 Reasoning Results: ("score": 0.2, "status" : "Insufficient". "reason" "The video shows static view of store counter with no sigrificant actions interactions occurring There is no clear indication of abnormal behavior based on the provided rules and prompts.")
Reflection Results: ("resson": "The primary Reffection
reulficent information is the lack activity the video. suggest vidao quality might also appeens to oventy relant on immediate action, patemnaly missing precunsory behavior that could indicato an Impending anomaly. Also, the surrent do account for pro-orime action analyss.".
static
While discemible anomalias that
low activily. doesn't definitvally rule potantial
"tools to use": [["oool name:: "Image Retrieve": "quary": "Simlar time day, static view store counter, idantity any nntaricss susplelous behavkor bafora anomaly occurs B
"new, anomaly. rule": "Security camera shows still scene in normally operating business during off-peak hours. Roqulras check of entrance to the store for loitering for longer than thve minutes."
"new heuristic prompt". "Analyze the current scene in the context of past irstances with similer condition.. Specifically, look for subtle changes in the environment le.g. object slightly out of place) or any person entering loitering outside the storafront within the last hour, and datermine if these subilla changes are ")
suspicious based
Re-reesoning Results: I"acora": 0.2 "status" "insufficient. "raaso": "The scane appears to be a typical store
interior with no signilicamt activity or changes observed. There are no clear indications of abnormal events such as foroed entry, auspicious behavlor, of physical attrraaionn..1
Rellection Round 2
Reflection Results: ("resson": 'The páiary resson for iraufficiett information is the lack of activity and low video quality The VLM struggling ideiity subtle that might indicate early potential anomalies Ika shapliting, burglary, robbery, aspecially given the grainy low
video historical static nature scere multipe clip., current rules focus elightly different scene spectfic.".
indicates pattern
"tools to use": [|*tool name": "Image aperrResolution: "quary": ""..
"new anomaly. rule": " "Lack of pustomer activity combined with repeated glanoess towards merchandise near the ontrance in conveniance store setting during non peak hours may indicato potent al shoplitting or pre- crime surveiliance.
"new houristic prompt": "Fonus on kdantifyin s. subfll changes in the anvinonment, such as items being moved individuals displaying nervous behavior, even if the video quality low. Pay cluse attention to hands and pocketts for potential signs of shoplltinng Consider if the lack of activity is unusual for the time of day jearty morring, according to historical dabs).
Re-reasoning Results: ("spore": O.7. "status": "abnormal. "reason":
the store quickly
moves bahind
which inconslatant register mices suspicion potential robbery.))
bahavior. This suddan
and
postioning near
Video Time 06:00 Reasoning Results: I'score": 0.0, "statue": "normal", "reason": "The scene shows static view of store counter with no visible movemant or Interaction that would suggast an abnormai evant. The environment appears calm and undeturbed.") ")
Figure 4: Visualization of qualitative results on UCF-Crime. On the left a visualization of the anomaly score curve. The right side shows the specific reasoning and reflection process of PANDA.
uncertainty and cannot confidently determine whether an anomaly is present, it transitions into the reflection phase. PANDA first analyzes the reason behind the insufficient status from the reasoning stage, and then invokes external tools to acquire complementary information to support decision- making. This case highlights PANDA's capacity for progressive self-refinement and dynamic tool invocation, demonstrating its effectiveness in tackling complex, real-world video anomaly detection scenarios. For more visualization samples, please refer to the Supplementary Material.
5
Conclusion
In this work, we presented PANDA, an agentic AI engineer for generalized VAD that eliminates the need for training data or manually crafted pipelines when faced with various real-world scenarios. PANDA integrates four core capabilities: self-adaptive scene-aware strategy planning, goal-driven heuristic reasoning, tool-augmented self-reflection, and the self-improving chain-of-memory. These capabilities work in concert to enable PANDA adaptively detect anomalies across diverse, dynamic, and previously unseen environments. Our extensive experiments across multiple benchmarks, includ- ing multi-scene, open-set, and complex scenarios, validate PANDA's strong generalization ability and robust performance without any training. These findings highlight PANDA's potential as generalist VAD solution for real-world scenes.
Acknowledgement
This research is supported by the National Research Foundation, Singapore under its Al Singapore Programme (AISG Award No: A1S23-RP-2002-030).
10

---

References
[1] Peng Wu, Chengyu Pan, Yuting Yan, Guansong Pang. Peng Wang. and Yanning Zhang. Deep learning for video anomaly detection; A review. arXiv preprint arXiv:2409.05.2 003383. 2024.
[2] Mahmudul Hasan, Jonghyun Choi, Jan Neumann, Amit K Koy-Chowdhury, and Larry S Davis. Learning temporal regularity in vidco sequences. In CVPR, pages 733 7223-74,272 2016.
[3] Yannick Benezeth, P-M Jodoin, Venkatesh Saligrama, and Christophe Rosenberger. Abnormal events detection based on spatio-temporal co-nceurences. In CVPR, pages 2458-2465, 2009.
[4] Yang Cong, Junsong Yuan, and Liu, Sparse reconstruction cost for abnormal event detection. In CVPR. pages 1449934566 2011.
[5] Cewu Lu, Jianping Shi, and Jiaya Jia, Abnormal event detection at 150 fps in matlab. In CVPR, pages 2720-2727, 2013.
[6] Vijay Mahadevan, Weixin Li, Viral Bhalodia, and Nuno Vasconcelos. Anomaly detection in crowded scenes. CVPR, pages 1975-1981, 2010.
[7] Yu Yao, Xizi Wang, Mingze Xu, Zelin Pu, Yuchen Wang. Ella Atkins, and David Crandall. Dota: Unsupervised detection of traffic anomaly in driving videos. IEEE TPAMI, 45(1):444445 2022.
[8] Guansong Pang, Cheng Yan, Chunhua Shen, Anton van den Hengel, and Xiao Bai. delf-trained deep ordinal regression for end-to-end video anomaly detection. In CVPR, pages 12173- 12182, 2020.
Weixin Luo, Wen Liu, and Shenghua Gao. Remembering history with convolutional Istm for anomaly detection. ICME, pages 439-444, 2017.
[10] Wen Liu, Weixin Luo, Dongze Lian, and Shenghua Gao. Future frame prediction for anomaly detection-a new baseline. In CVPR, pages 6536-6545, 2018.
[11] Dong Gong, Linggiao Liu, Vuong Le, Budhaditya Saha. Moussa Reda Mansour, Svetha Venkatesh, and Anton van den Hengel. Memorizing normality detect anomaly: Memory-augmented deep autoeneoder for unsupervised anomaly detection. CVPR, pages 1705-1714, 2019.
[12] Zhiwei Yang, Jing Liu, Zhaoyang Wu, Peng Wu, and Xiaotao Liu. Video event restoration based on keyframes for video anomaly detection. In CVPR, pages 14592-14601, 2023.
[13] Zhiwei Yang, Peng Wu, Jing Liu, and Xiaotao Liu. Dynamic local aggregation network with adaptive clusterer for anomaly detection. In ECCV, pages 404-42 2022.
[14] Zhiwei Yang, Jing Liu, and Peng Wu. Text prompt with normality guidance for weakly supervised video anomaly detection. In CVPR, pages 18899-18908, 2024.
[15] Peng Wu, Xuerong Zhou, Guansong Pang, Lingru Zhou, Qingsen Yan, Peng Wang, and Yanning Zhang Vadelip: Adapting vision-language models for weakly supervised video anomaly detection. In AAAI, volume 38, pages 6074-6082, 2024.
[16] Peng Wu, Jing Liu. Yujia Shi, Yujia Sun. Fangtao Shao, Zhaoyang Wu, and Zhiwei Yang. Not only look. but also listen: Learning multimodal violence detection under weak supervision. In ECCV, pages 322-339, 2020.
[17] Wagas Sultani, Chen Chen, and Mubarak Shah. Real-world anomaly detection in surveillance videos. In CVPR, pages 6479-6488, 2018.
[18] Peng Wu, Xuerong Zhou, Guansong Pang. Zhiwei Yang. Qingsen Yan, Peng Wang, and Yanning Zhang Weakly supervised video anomaly detection and localization with pages 9301-9310 2024.
prompts. In ACMMM,
[19] Heng Lian, Zhiwei Yang, Zhaoyang Wu, Bo Lin, Tianqiang Huang, and Qiaoru Miao. Vlial: Vision- language instance awareness learning via vlms for weakly supervised video anomaly detection. In RoboSoft. pages 23-30, 2025.
[20] Hang Du, Sicheng Zhang, Binzhu Xie, Guoshun Nan, Jiayang Zhang, Junrui Xu, Hangyu Liu, Sicong Leng. Jiangming Liu, Hehe Fan, et al. Uncovering what why and how: A comprehensive benchmark for causation understanding of video anomaly. In CVPR, pages 18793 18803, 2024.
11

---

[21] Jiaqi Tang, Hao Lu, Ruizheng Wu, Xiaogang Xu, Ke Ma, Cheng Fang. Bin Guo, Jiangbo Lu, Qifeng Chen, and Yingcong Chen. Hawk: Learning understand open-world video anomalies. NeurIPS. 37:139751-139785, 2024.
[22] Huaxin Zhang, Xiaohao Xu, Xiang Wang. Jialong Zuo, Chuchu Han, Xiaonan Huang, Changxin Gao, Yuehuan Wang, and Nong Sang. Holmes-vad: Towards unbiased and explainable video anomaly detection via multi-modal Ilm. arXiv preprint arXiv:2406. 12235, 2024.
[23] Huaxin Zhang, Xiaohao Xu, Xiang Wang, Jialong Zuo, Xiaonan Huang. Changxin Gao, Shanjun Zhang, Li Yu, and Nong Sang. Holmes-vau: Towards long-term video anomaly understanding at any granularity. arXiv preprint arXiv:2412.06177 2024.
[24] Zhiwei Yang. Chen Gao, Jing Liu, Peng Wu, Guansong Pang. and Mike Zheng Shou. Assistpda: An online video surveillance assistant for video anomaly prediction, detection, and analysis. arXiv preprint arXiv:2.503. /9944, 2025.
[25] Nicolae-C Ristea, Florinet-Alin Croitoru, Radu Tudor lonescu, Marius Popescu, Fahad Shahbaz Khan, Mubarak Shah, al. Self-distilled masked auto-encoders are efficient vidco anomaly detectors. In CVPR, pages 15984-15995, 2024.
[26] Josh Achiam, Steven Adler. Sandhini Agarwal, Lama Ahmad, lige Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXi::3303.08772 2023.
[27] Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katic Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312. I 1805, 2023.
[28] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, Sibo Song, Kai Dang, Peng Wang. Shijie Wang, Jun Tang, et al. Qwen2. 5-v1 technical report. arXiv preprint arXiv:2502.13923 2025.
[29] OpenGVLab Team. Internv12: Better than the best-expanding performance boundaries of open-source multimodal models with the progressive scaling strategy. 2024.
[30] Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. arXiv preprint arXiv:2311.10122 2023.
[31] Luca Zanella, Willi Menapace, Massimiliano Mancini, Yiming Wang, and Elisa Ricci. Harnessing large language models for training-free video anomaly detection. In CVPR, pages 18527-18536, 2024.
[32] Yuchen Yang. Kwonjoon Loe, Behzad Dariush, Yinzhi Cao, and Shao-Yur Yuan Lo. Follow the rules: reasoning for vidco anomaly detection with large language models. In ECCV, pages 504-322. Springer. 2024.
[33] Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni. Vladimir Karpukhin. Naman Goyal, Heinrich Küttler, Mike Lewis. Wen-tau Yih, Tim Rocktäschel, et al. Retrieval augmeeted generation for knowledge- intensive nlp tasks. NeurlPS, 33:9459-9499 9444,,4744 2020.
[34] Andra Acsintoae, Andrei Florescu, Mariana-Iuliana Georgescu, Tudor Mare, Paul Sumedrea, Radu Tudor Ionescu, Fahad Shahbaz Khan, and Mubarak Shah. Ubnormal: New benchmark for supervised open-set video anomaly detection. In CVPR, pages 20143-2011 203-20013,.00 2022.
[35] LANGCHII.. Langgraph: Multi-agent framework for llms. 2025. URL hupp://github. com/tangchain- ai/langrraph. Accessed. pages 01-24, 2025.
[36] Sagar Imambi, Kolla Bhanu Prakash, and GR Kanagachidambaresan. Pytorch. Programming with TensorFlow: solution for edge computing applications, pages 87-104, 2021.
[37] Nils Reimers and Tryna Gurevych. monoligual sentence emheddings multilingual using knowledge distillation. In EMNIPP, 2020.
[38] Ayush K Rai, Tarun Krishna, Feiyan Hu, Alexandru Drimbarean, Kevin McGuinness, Alan F Smeaton, and Nocl E O' connor. Video anomaly detection via spatio-temporal pscudo-anomaly generation: A unificd approach. In CVPR, pages 3887-3899, 2024.
[39] Yu Tian, Guansong Pang, Yuanhong Chen, Rajvinder Singh, Johan W Verjans, and Gustavo Carneiro. Weakly-supervisee video anomaly detection with robust temporal feature magnitude learning. In ICCV, pages 4975 4986, 2021.
12

---

[40] Hang Zhou. Junqing Yu, and Wei Yang. Dual memory units with uncertainty regulation for weakly supervised video anomaly detection. In AAAI, volume 37, pages 3769-3777, 2023.
[41] Muchao Ye, Weiyang Liu, and Pan He. Vera: Explainable video anomaly detection via verbalized learning of Vision-language models. In CVPR, pages 8679 8688, June 2025.
[42] Tianheng Cheng, Lin Song, Yixiao Ge, Wenyu Liu, Xinggang Wang, and Ying Shan. Yolo-world: Real-time open-vocabulary object detection. In CVPR, pages 16901-16911, 1, 2024.
[43] Xintao Wang. Lianghin Xie, Chao Dong, and Ying Shan. Real-esrgan: Training real-world blind super- resolution with pure synthetic data. In ICCV, pages 1905-1914, 2021.
[44] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In ICML, pages 8748 8763, 2021.
[45] Tavily. Tavily search api. https: //wp://wwww.. http://wwwtaa.l....../. tavily.com com
13

---

Technical Appendices and Supplementary Material
A Overview of Technical Appendices and Supplementary Material
This technical appendices and supplementary material provide additional information not included in the main paper. Specifically:
Section
B
offers further clarification on methodological and experimental details.
Section presents additional ablation studies and parameter analysis experiments.
Section D
details the toolset employed by PANDA.
Section
shows the prompts used at each stage of the PANDA framework.
Section
provides additional visualizations of qualitative results across various datasets.
Section
G discusses the current limitations, broader impacts, and future work directions.
B
Additional Methodological and Experimental Details
B.1
Visualization of the PANDA Detailed Pipeline
Figure 5 presents visualized case study on test example from the UCF-Crime dataset, illustrating the detailed execution process of PANDA across its core components: anomaly knowledge base construction. self adlaaideetie scene-aware strategy planning, goal-driven heuristic reasoning, pool-augmented self-reffection, and the self-improving chain-of- of-memory mechanism.
B.2 Clarification on Evaluation Modes and SOTA Comparison
In Table of the main paper, we distinguish between offline and online settings based whether future information is utilized when reasoning over a given frame or clip. future information accessed, the method is considered offline; otherwise, it falls under the online setting. For offline evaluation, we follow the SOTA methods AED-MAE, LAVAD, and AnomalyRuler, which are compared in Table 1 of the main paper, and apply temporal smoothing (mean filtering, window size=10) 10) on the final anomaly scores.
Among the compared methods, Holmes- VAU is fine-tuned VLM-based approach that leverages detailed anomaly annotations via instruction tuning. It was originally evaluated only on UCF-Crime and XD- Violence. Its results on UBnormal and CSAD are reproduced by us, without any re- re-training, using its publicly released model. As seen in Table of the main paper, PANDA significantly outperforms Holmes VAU on UBnormal, an open-set dataset that includes unseen anomaly types. While Holmes- VAU slightly surpasses PANDA in online mode on CSAD, this is primarily due CSAD including large number of videos derived from UCF-Crime and XD- Violence- the original training sets Holmes VAU. Notably, PANDA still achieves superior performance under the offline setting. underscoring its strong generalization capability. These results demonstrate the limitations of relying solely on fine-tuned VLMs when facing domain shift and complex real-world conditions. Additionally, report new evaluation results for three prominent training free baselines: ZS-CLIP VL- and LAVAD, on UBnormal and CSAD. PANDA consistently and substantially outperforms all of them across both datasets. Together, these experimental results reinforce the strength of PANDA as a generalist, fully automated VAD agent, capable of adapting its reasoning to scene-specific conditions without supervision or hand-crafted engineering.
Additional Experiments
C.1
Ablation of Key Component
Table A the results of an ablation study evaluating the contribution of PANDA's six core components. The third row in the table corresponds to baseline that directly queries the VLM using only the user-defined anomaly description without leveraging any PANDA modules. This baseline yields notably poor performance, confirming that naive prompting alone insufficient. As more modules detection strategy planning. adaptive scene perception, RAG with anomaly knowledge, self-refiection, short CoM, long CoM the performance steadily improves. When all six components are combined, PANDA achieves its best overall performance. These results demonstrate that each individual module contributes positively to the final performance and validates the effectiveness of whole PANDA framework design.
incrementally added-namely
14

---

Table 4: Ablation results of PANDA component.
Key Components of PANDA RAG with Anomaly Knowledge Self: ReflecRllttton
Dataset
Delection Strutegy Planning
Belf-AAapotion Scene Scene-Aware
Short CaM
Long CoM
UCF-Crime(AUC%)
75.25
78.92
80.57
83.94
84.89
Table 5: Additional Experiments. (b)
Impact of different
inpur clip length
Analysis speed.
Different MLLMs
UCT Crine
Input Clip Length
UCF-Crime (AUC%)
Datasets
Averuge speed of inference
(AUC%)
(Number of frames)
Name
(FPS)
Qwen2 5-728
84.03
84.25
UCF-Crime
0.82
DeepseekV3
84.72
84.56
XD-Violence
0.86
Gemini Flash
84.89
83.15
CSAD
0.53
C.2 Impact Different MILMS
Table cablprajcompares the performance of PANDA when integrated with different multi-modal large language models (MLLMs), GPT-4o and Gemini 2 Flash represent proprictary models, while DeepScek-V. V3 and Qwen2.5-728 open-source alternatives. As shown, GPT-4o achieves the highest performance. However, adopt Gemini Flash our main pipeline due to its strong trade-off between performance and cost-effectiveness.
Notably, although Qwen2.5-72B yields the lowest performance among the compared models, it remains significantly superior to prior training-free baselines. Given its open-source nature and ease of local deployment, it serves as practical and scalable option for essource-consrrane scenarios.
C.3 Effect of Input Clip Length
Table 5b analyzes how varying number of frames in each input video clip affects PANDA's performance. As the input length increases from to frames, detection accuracy steadily improves. suggesting that short- range temporal cues are beneficial to the reasoning process. However, when the clip length is extended to frames, performance noticcably drops. We hypothesize that this is due to the binary labeling strategy used during evaluation- may contain both normal and abnormal frames, this scoring scheme introduces noise, leading to performance degradation.
clip is anomalous, frames of the clip scored anomalous. For longer clips that
C.4 Analysis of Inference Speed
Table reports the average inference speed of PANDA across different datasets. As observed, PANDA achieves similar inference times on UCF-Crime. XD- Violence, and UBnormal. However, a noticeable slowdown observed on the CSAD dataset. This is primarily because CSAD contains videos with complex conditions and scenarios, leading PANDA to more frreuently enter the reflection stage. Since the reflection stage involves invoking additional tools, it introduces greater computational overhead. Despite this, the overall average inference speed of PANDA remains acceptable for non-time-sensitive applications, demonstrating its practical fcasibility in real-world deployments where latency is not critical constraint.
D Toolset Details
PANDA is equipped with modular and extensible set of tools designed to enhance video content analysis, mitigate visual degradation. and provide external contextual information. These tools are automatically selected and invoked during the self-reflection stage based on the detection context. Below, we provide detailed summary of each tool integrated into the PANDA framework.
Object Detection. PANDA employs the YOLOWorld [42] model pretrained on a wide set of open-world con- cepts. It supports fine-graines category-specifie detection including actions like "person hitting another", "person setting something fire", and "person stealing". This enables robust anomaly-rclated seene understanding through bounding-box localization and category labels.
15

---

Image Denoising. PANDA uses OpenCV's fast non-local means filter for image denoising. It reduces color and spatial noise in frames using adaptive filtering. helping enhance clarity in low-light noisy environments.
Image Deblurring. PANDA applies unsharp masking with Gaussian blur subtraction sharpen edge details for motion blur or out-of-focus issues. This lightweight enhancement improves perceptual clarity without need for retraining.
Image Brightness Enhancement. PANDA uses OpenCV's CLAHE (Contrast Limited Adaptive Histogram Equalization) on the L-channel in LAB color space. This ensures localized brightness normalization for dimly ltt or overexposed frames.
Image
PANDA integrates the Real-ESRGAN + model for resolution enhancement.
improves detail preservation and restores textures in low-resolution videos using deep RRDBNet-based super-resolution pipeline.
Image Retrieval. PANDA uses CLIP-based visual-textual retrieval to match current queries (e.g.. "robbery incident") with previously seen keyframes. Cosine similarity between CLIP embeddings is used for scoring relevance.
Web Search. PANDA leverages the Tavily Search API 4S for querying web content related to unknown uncertain anomalies. Search results are parsed into structured summaries that can referenced the reasoning process.
Image Zooming. PANDA leverages a bicubic interpolation -bapddd zooming tool to magnify regions that require enhanced spatial detail using a specified zoom factor. This is useful when detecting small-scale interactions distant activities.
All tools are dynamically invoked during the reflection stage via the MLLM generated reflection plan. Each tool outputs enhanced frame and structured summaries that are used augment the reasoning prompt for follow-up reasoning steps.
E
System Prompts
In this section, we present the detailed prompts used by PANDA its core stages. Bigurr@illustaatte the prompt used during anomaly knowledge base construction. Figure ( shows the self-adaptive environmental perception prompt. pigureBspresentt the prompt for anomaly detection strategy planning. Figure geuuonydmmoosttttess the goal-driven heuristic reasoning prompt. Figure 10 displays the prompt used during the tool-augmented self-reflection phase.
Additional Visualization Results
Figures
11 and
12 further show the qualitative results of the samples on the XD- Violenee and UBnormal test
sets.
G Discussions
G.1
Limitations
PANDA currently integrates curated of commonly used tools for enhancement and reasoning. While these tools suffice most general scenarios, expanding the toolkit to accommodate domain specifie modalities (c.g.. thermal imaging) could broaden PANDA's applicability.
G.2 Broader Impacts
PANDA advances the paradigm of automated video anomaly detection by integrating vision- languu-la models unified agent framework. This work has potential to improve
decision time tool augmentation the robustness and interpretability of security monitoring systems, enabling adaptive deployment across diverse environments without training. Moreover, We also recognize the importance of ethical considerations: systems like PANDA must be deployed responsibly, with attention to privacy protection, fairness, and minimizing unintended surveillance harms.
G.3 Future Work
Enhancing Real-Time Adaptability. To make PANDA more suitable for practical deployments, future work could focus reducing inference latency and optimizing tool invocation paths.
16

---

Improving, Spatial Reasoning and Localization. Currently, PANDA focuses on frame-level or clip-level anomaly identification. Incorporating spatial anomaly localization, such identifying the precise region object involved in an abuormal event, could significantly expand its utility in surveillance systems.
17

---

Asxeab Bas Freeptt
Plese Idly ne lesct the Siliw syps of soraa cocpe tts Arros,
Ansealy Kavvinige Bos Canirention
Asuuaby Baawridgy Bam
Talee ID: SHD-MRE
"Foet Type" "iliig
Fightay Read Aucdbas Relbwry Sheorn Shipling Secang Vanditon
persiall ail lig
Ga Buck
Appliatl
"Applicatle Soanot "Cheling Seporaato. Parmaay"
Scioaligthe Scatoo aran, Stratop Planiig
Natuore
Prongt
rocled with predan The shes aapaas well-is vin auis of cartted peth
"Semo Dvceries"
"ragnccoitg" "truac Demeico brage Doblur, "Saelitii
Rillbay".
cnethar pasally expleyas, satted the cenrrrmen aggins type rea lekeg".
"ikii "helppocanee inseriie ppoduars Dios centrre lees check if siree haves the store withert the luving vamead"
'Haarak Proeper"
wincs comsatedt lioo
furns theme red ciken weather
co scew antady kad
"Sediig" "O intwait heneee
and lawe:
carara moctiiig will los cluiity une poirasss The lmage ia Eurry and aavigntne inpirrr daerta
diecall aidna hans ave aabro es odser cacceuos vidieii partistad Pay topie
atrectng vem "Rellary duydiiiigg en bibly cearedies incena aou..
"Peustial
Prouge
"lakkey "Duse sallen sagrostin belanter chccaall the cabler edsect celkees angges, lives deean ronry Awale
Reaveiaz
Goall Biince Hearitk Heneakag
Vidao Town Recuniigg Rouite l'uwe "sersaa" "eawee "Ther villle
ep rollory fiis
scrrc apparo sobi bptca Idaal centerace vith carenco zowing anlaappta
Votuo Time Mtctt Rts etaeettg Heeute: t'oeer" ILC. "saaur" "aouttites" "roaee be vodee teee soow pencral veow soner with people we awound hot o deepi villery.
qail
Honeer Experience
"tauffiiier Brass"" fooee dhen pracral af sooe wills an ductii of ani absooaal
"Is Bockee Rosti: "rowe: 1 "he V strgphng the the lon
oid pouuedd doylin saiding bus hely
Frongt
BAG
"air aweadly rake "Tepone iarge duity dhnigh upes ouck redio daady and you koad io at, objict rncrctenn and bay bag mge and enavenent beaarg on domntrnn cases titinn. coectitigg daliee
"aor berritk preepe "Alko supoeeekuee attnnceees fica on cach atal non chotloet comnicr commcranti sachhechh Fudy laegaigs
poylig
Relactiss Reaals P'ouurd "Th VIM reoon bufffien relorriia heoa videe sipaices hik wiee nenoeeee ther Wah ralieeeeee the fedler socerie experimees the antoen..
Boco ibaud impon by the quilytu ta andoe
ksag: sppt roxiadao
"own assenaly rate" Tadaace image cartty trrugh eetto exceadiarlyy analiss buge wilbet
The looks
aich fecesing slverfying "how, Jiee bite, peuaeeee emeee" "ffer sagm revkiing the reages mylyye valeo with
Nares
"Saige Roukkaic"
keen Puy hose stemnio wheceer the coseeer place mouih prdketss bupe wiheet diaa popeeas lnseraction with cashier Cousider cortoral's
sove poisrilly deaanal onseraiie clisaly momall")
Xctaed Rissniig
Re Rerastriing Beutic: I'sxss" a3, "woaus "lasatf "ruase" "Tho casea spparee le syscal daus inacies us digaiiiuss anle oharee Th puse pelknn
indiatians ofcloeeral encis rack an doglitiig duiling or other laoad ancull
"Butbetke Rovel" "rmuus" "he VLM rppning toutteer inkermmen the subliite Let somectttd alll
dence
VS aiik rega endeur thiphiingg wace ovetennd.
Pronge
BAG
actoee tus while wecnings ngo rmpocuoe
"geay"
Expeltise
"aor pate Dove
placig tage poclct
belurias conaaiion c20 shor paec tu ideadl perertal doyl tag dact
Speciiially.
ditat an back thy cn will egyry bwaw, plaving tles in bollego paikut will aud paying.
Raflstion Rarad
witeo quilyy which-oe otle colle wheee that enght shepl oi or madin torn mape samal doing durodds while leading arcad taepicnaly. is pesazial bull.
Rufletive Rorass ("ocassr" "The VLM spprrtng teafficion indernation
Teal Unie
ViMd own the
raper the "cweeur" of ereiion for
perice
Lagiaga for capliiigg
not, Jaank &u you we'k
body
placing
wot cweck dis
The Laroks
"1. Siga Desettl Buaykiker out Officns
"pery" "hddy langaigg cers stting") "I] paruuns the sker oe white re terree weth serve revvenent io cenceiennt atoneti. prastalty
"aee sneraal "Aaaguu indi sdhall lagg
Ted Nara: "Wib Suuci" Deoane "aats cacine skeptckng)
Dac of the koy ta karatiing abapi to lanca nowciiitaagee dafllelllon hugia
eiccumeee wettisee dieeenne the shutiiedd an serhtigggn
"aen, procpe" as iidie focuinng aubita body langrage
kens, and incendiaer or bast add aclion are mecchadd ins Courider the cainics oftee ferter awayy they wits emphoyessu. aoviding i uneud Daess
noar
Refbed Rascning trvepp
abwonal ottardi dl by aascanad "-
sweding whih coopiiing The and idadd bob larguas edidiay but evoldane eye oenuct nervees can aniits, digs vith boaisss proppa doggitinn??
("eaeh "Maas"" "dowerer"" "wowe" "The
Vilas Thas 0005 Ra Reasing Recaiic Plcce" on "mas"" "eemal, "aurr"" The scae dhees sade viee af www.coa conaratte knsrccion word sagget ensratecnt meverna upprrs ain uxbarree
Figure 5: Visualization of the PANDA detailed pipeline. We show more detailed pipeline for PANDA by visualizing the results of a test sample on UCF-Crime.
18

---

Anomaly Knowledge Base Construction Prompt
are an expert in designing detection for video anomaly detection, user's specilied Lypes abnormal events, your is generale 20 comprehenaive and diverse detection rules for each event zype.
User Requirement:
(user query)
Each rule must inclade the following four fields:
1. Rule 10: unique identifier for the rule (e.g., "F16-x001"),
2. Event Type:
The corresponding abnormal event calegory (e.g., "FighLing")
3. Rule Description: concise and clear description of the behavioral pattern this event.
that
Applicable Scenes: List real-worid scenarios where this may be applicable (e.g., "Street, Shopping mall, School playground"). rules ahould
wide range of realistic situations for each type.
Flease output zesponse the Iollewing structured JSON fermat: [[
"Rule ID": "FIS-X901",
"Event Type": "Fighting",
"Sule lescription"; "Fulling hair or grabbing clothes during struqqle", "Applicable Scenes"; "Shopping mall, Playground, Street corner"
Figure 6: System prompt for anomaly knowledge base construction.
Self-Adaption Environmental Perception Prompt
are expert in anomaly perception. task to perform an initial understanding and analysis of the provided video frames based the user'a spacified remuiremenss.
User Requirement: (user_query)
Please respond by completing the following four aspects:
1. Scene Overview: Describe the environment shown the video (e.g., shopping mal1, office, street, survei' corridor: and briefly summariza the main aclivilies or events
Neather condition: Describe the visual lighting conditions (e.g.,
saytimm//igh:ttme,, sunny/overeast, bright/dim).
3. Video Quality: Comment on the overall quality the video (e.g., clear, blurry, noisy, low frame rate, low light.etc.).
4. Potential momalless: Based on the visual content, what types of abnormal events are likely to occur in this video (e.g., Fighting, Stealing).
Please return your output strictly in the following JSON formal:
"Scene Overview"; "...",
"Neather Condition": "...", "video Quality": ",..", "Potential Anomalies": "..."
Figure System prompt for self-adaptive environmental perception.
19

---

Anomaly Detection Strategy Planning Prompt
You are a strategic expert in video anomaly detection, specializing plenning effective detection strateqies based or user-defined requirements, video enviranment information, and sceno-speci anomaly rules. Your role is to generate optimal plan that quides the analysia module acourately detecting anomalies in the curzent video User Requirement: {user_query:
Video Enviromment Information:
1. Scene Overview: (env info.get ("Scene overview", "Unknown"))
2. Weather Condition:
("Weather Condition", "Unknowm"))
Video Quality: (env_ seav_info.gel ("Video Quality", "Uaknowa") Pozential Anomalies: (env info. ("Poteentiall Anomalies", "Unknowm"))
Anomaly ly Detectior Rules:
janomaly_rutee rules;
Based on the user's requi rement, the irenammnary video envi ronment information, and the provided anomaly rules, please design a atrategy tailored for this video scenario, Your response must include the follewing three components: 1. Preprocesaing Recommendations and Pipeline
Suggest sequance preprccess's steps (e.g., Image Reblu-ring, Brightness Enhancement, Image Deneising) that help improve video quality and aupport better ancmaly detection, especially if the video is of poor quality.
2. Pocential Anomaly Types
Based on the preliminary video environment information and the given scene- related anomaly rules, further infer and 1ist the most possible types of anomalies this scenario.
3. Heuristic Prompts for Using the anomaly rules guidance, craft Beeln-of-thoughee style heuriatic prompos for each potential anomaly type. These prompts are intended to assist Visual Language Model in performing accurate anomaly judgments.
Please return your output strictly in the following JSON format:
"Preprocessing"; "SLepl -> Step2 ...", "Pozential Anomalies"; "Fighting, Stealing, "Heuristic Prompts":
"Fighting": "Heuristic prompt with reasoning stepa...", "Stealing"; "Heuristic prompt with reasoning steps...",
Example Outgut:
"Praprocassing":
"Image Brightness Krhancement "Posentisl Anomalies": "Fighting, Stealing",
Deroising",
"Heuristic Prompts": {
"Pighting": "Observe the number of people, their movements, and interactions. If two individuals are zepeatedly making aggressive centact, consider i: potenzial fight.",
"Stealing": "Identify solitary individuals interacting with objecta, especially if they conceal or "eava quickly without paying."
Figure 8: System prompt for anomaly detection strategy planning.
20

---

Goal-Driven Heuristic Reasoning Prompt
You are a highly skilled expert video anomaly detection, specializiag icentirying abnormal evidence.
through temporal and spatial analyais viaual
Given sequence of video frames, requirements, potential anomalies, anumaly detection rulea, heurietic prompts, and entancement/reflecttae information, your task is to asaess the likellhood of abnormal eventa in the curzent video clip.
You must output:
soft anomaly score between +00.0 (clearly normal) and ++10 (clearly ebnormal)
status label from: **"norma'AA, "*"abnormal"", ""insufficient"".
reason justifying your decision.
User Requirement: (uaer_query]
Historical Detection Info:
(history_restt petpslt__prestttttttteeeeepppeee history result, cessll__prupppelteeeeeeeeeeeeepe prumpt.strip() "No relisble historical detection information available.",
Current Video clip Index: Clip [index)
Potential Anomal ies:
belanning_info.ee ("sotential, anomalies', ())
Anemaly Detection Rules: (anomalyy (oomaly_rules]
Heuristic Prompt: (planaing info.get ('heuristic_prompeee (1)
Enhancement and Reflection Information: eeermattod_anhanncmmmn prompp)
Your analyais should follow three spepa:
1. Deacribe the main visible actions and interactions between people objects in the scane.
2. Assess how strongly these actions match any known abnormal event patterns using the provided rules and prompts, The provided anomaly rules may not be compreheneive, so you also apply your own expert zeasoning.
Rased on your assessment, assign score label, expla'n your reasoning elearly.
the [Enhancement and Reflection Information] section provides additional information, should refer it emphatically.
Sceriny Guidelines:
score close to **1.** indicates clear and confident abnorma.. behavior.
score close to **0.0** indicates clearly normal behavior.
score near *40.5*A means uncertain, ambiquous behavior or mixed signals.
Examples valid reasons "insufficient";
"The entire scene is too blurry dark, making it difficuit to distinguish any actions."
"A11 persans are sither occluded or out of frame." "
"Only partial Limbs are visible and motion cues are unclear."
Please strictly output your reaponse the following JSON format:
"score": float, //anomaly score 10.0, 1.0)
"status"; "normal httpmmmmmminmuiiiienn...... "reason": "A detailed explanation of your reasoning..."
Figure System prompt for goal-driven heuristic reasoning.
21

---

Toel-Augmented Self-Reflection Prompt
You are a reflection assistant within video anomaly detection aystem.
The current VLM anslysis module has returned "insufficient statu" for determining whether en abnormal event occurred in the given video clip. Your task to critically analyze the situation based on the provided context and recommend solutiens.
Here is contextua" information:
User rement:: {user_query]
Video Rnvmentnen Information:
**Senn Overview: (env info.get ("Scene Overviex", "Unknowa"))
("weather Condition", "Unknowr:)) **Video Quelity: jenv ann__iio.gettteeoeo ("Videu Quality", "Caknowa")
**eathher Condition:
Anomaly Detection Rules:
aanomaly [ammalyrules] r_rules)
Fotential Anomalies: (planning iafo.get ('Potential Anomalies', (1)
Historical Detection Reaults:
[historical, results]
Current VIM Output Reason: [zeason)
Information Echancement tools Already Used: (Lools_ade sasss_alrdd___eee used]
History Experience:
(memory_comm context)
Based on this information, your taaks are:
1. Analyze and determine the primary reasons for the insufficient in.formation.
2. Recoemend which touls the available options
used enhance
infozmation
better anomaly detection. (teal descriptionn text)
3. For any selected toal that requires 'query' input (e.g., image retsieve, web bab_seerch), generate an appropriate query based on the context; otherwise leave the 'quezy' field empty.
4. Propose a new representative anomaly datection rule derived from the current situation to better support future VEM anslysis.
5. Propose an additional heuristic prompt based on the context and your analysis to better qui the toward an accurate judgment.
Please output your response the following structured format:
"reason": "o.r analysis or the information is insufticient...", to "tools_to_uu _o_uus"
"tools
"taol_name"s name": "Ona of the most critical tools.", "quezy": "generated quezy if needed, otherwiss leave empty"
"tool
}
"new anomaly rule"; "...a representative anomaly rule derived from your analysis and context...",
"new heuristic promp=": ". add.additional guidance help the VLM make more accurate judgment..."
Impostant Notes:
When calling tools, make sure you don't duplicate any of the information enhancement tools that have already applied, use only one of the most critical
time.
If all available informacion. enhancement tools been exhauated, you should directly suggest in the 'now Feuristie prompt' how to guide the VTM analysis medule to make the most reasonable judgment based incomplete evidence. If the current context provides enough information to make clear judgment, plesse directly guide the VLM anslysis module in the 'now www.isstiepprmeeeeeeeeeeeeeeeeep to conclude whather the event is abnormal or normal.
Figure 10: System prompt for togl-augeetted self-reflection.
22

---

Valeo Time texd Reasening Roeatt:: Phcoed" d.I. "straa" "ooemaal" "raaeet "The video showe ancher oppeting live stull. These vuibli agga of stecmal evens rach Fightng a2ooting, mi, state car accidons, or cptieaes the tunes prosseed)) ?
Beasoning
invoraal liee fighting raporteg
prstart bu trame prodid snoggh viraa decull to or opicdes The cread appears to gathire, hut quecttc achoo or beborses that would indicue these snstraker not chormmblee
absue, accidees
Broed
Roflection Reeates reflaction ("easee" "The VLM oaturr indicatie that shile the sideo casws a peotos, in lacke the stoal deail aecessary
centimn the proseree qpertic abeoenal meme Thes sagarra the temal VI.0 anatpeu conlde ideetrry actione hatuntarr indicatov analee:
npbrag,
desper
didddihhd willn
bece provided rakes..
"taak, une [Phddd_ sare "itteet Detrotioa "paar"" "I]
"een, aroomnllyy ralee "looared oct siekoce aggrosioos on tiii a protes crowd, sads sigunling slaving, or playiea olarcatioon betwees
caaing danalaano Chece for and charsi purvenanta ralicatiee of ofiiting flestgg from poicntad explaaes Als, jay clese
inkondiinn betwoan proseteo and catautiko ceurriss if abese or dasko are secaring")
Readls
62 "aaae" Tesellicinn "romee" "The
fedemosi
crowd people bet rospecifi wrkin beksiers indeaiing fighing shooingg lke, ahese, cideet or expliiion observed The cwol aobano be pabecs sesettily and there to vartte evicace of staerral ceets:
floeed
Refection Beadss ("rawer peinary rasus inwelicket infermaies
iubiliy didoem-pealii
withs
crecsifes
decdees,
Inoom pprien Jacutaa VLM lacks detulad autos isdiidad and actitttes cantin
abnarmul cvants
The veject detction basi was cloady usd ragaering that bask objet mosggitinn sakt provid crough naight The Vi.id can only desemmns tha bers conee can fauzz oat ane potoria figtng dasamva acteass". spery"
"Iaaag rake Sacder, doodod digenaal looaliee
preses liniiing
Saleer") "
"sen
agrinie befactor (daoiing parking objoc thoseegg withe
"scw Jauietic promut" "Amndoe ee the counds surrazzna and isteattion clacky. Fecas os idcatiliia siane of sagrcmies vnch sa pualings
aasv iecinida being lasaded. Look eleacs being throwes sian propecty Jassgg say indeativon of centlis natheriics.
croversar
rapily, sapacaling otlec luralá ubeeaa" "I
depcts bensdnbal auch
Receaseing Roulss Cseard' ool Saate" "Stnerasa"" "ummer" "The cppodaa be craaae in with vense indiniduab wariing
(aaderaa bokdng Sap. There was acae sggaay converssi a di socoo whidi i tyskal of ckst
Video 53:30 Roonsning Rendin Phicee" U.A. "aara"" "shnormaal" "Mauee" "hee cwnet appoare to be exgaged in graser. aome people
dags
Figure 11: Visualization of qualitative results for a sample on the XD-Vinlence test set.
Vidao Toma E0x15 Reaseeing Roulte ('scse" B.I "aata"" "neemaa, "rasce" "The manding amond. This ane no vith rign aboomnal bekostoreach aypoylaiing wallnng drasc acciderss or menning The pedeerion.sppe to be stalesay or reasing donly, which Jos nat swads the otkata for of Tstal ahaorne eveiss
cain night with fir pedturtioe
Vilor Tine d0:10 Reaumita Roulo: ("score" 6.2. "sttaa "tmosliiciaa"" "reaarr' "The vidce flanss skon peloozana mooing edes moowing but da low luitisns tade ditiiaal dacce pecilekkaaee doos decooloiiats ustsen er dhe aucoaaaaiig conimennar. Taav cu daar acioaion aatasmal beawser houd on prwnieed mike and pronees
Refiectio Hland
Refeectiee Readiv: ("osssss "hh pireary maces for lsu tsurffciee inforvattr we low lighting maderer vndro puslisy and the zadecriawk sciaes benn diffical docmn. This nakes il land to Jeamine if the poloszan cobitiirs atcomnal betanier such walling drank straaking. prcoeecc
the sightine aclin latle complinic licuaayaa.. *"),
anoeaaty cute" "Pedariss reabeis uneralle vrhlaance shill dippery earbics, inticttn pourile imsoicavvn or mediccal leaa."
aspacisily under advare conditonn exch ae low
"een, bourtutt pronge" Toors on steriifrng any indicaios emmn mevenent, each exaping, eurahiigg or uxider charges in direction, appcure they
denggaaed
crnikor
lighting"
Reoremoning Rosalss Tworeh 02, "Sate"" Tesafficeent
The videe foxmes polctiaa condition make k diffict to discn specife asion or hdlevios tha inciatt aeoomaal ovent. There so cler indiatioes ajyywalking. eellegdrani drank codders or offer moneii besel on the villle riseradoe
Refiedicn Brend
Reflection Recalte Phonen" "The piinay mears fo trafficirr irbernttos are fiegy contities and praevistly reniirire which siruty apelled
suacffc
tinghrwer
"odds ind" Teu Lueee" "lang Domiiind "saay"."" "I]
"uex seeesely rukk" Rolaed vaallity dooteee duotecopoootiin adirlt, taking it diffiod to ideeif nomel belovicr or detett ped
anarales in tightinn ana womter contitunc".
prompe". nakee emnlc
art perticaly maste nught Proentit detecing destaticna walking coniteitz the anvrrerane of the scone")
charpe
or colluaans,
Becrossiing ("ocre" "sarss" "sheorseat" "rossos" of cecor belasoo which but does the dlarredd bi
oppers to wolking onnteally.
desoritine of W Breek: The vinilily concitions wids
Vidai Roenoning Roals: ("ooe"" 62, "aata" "abaarred"" leace"" "A polosiias consing sbout the devignated JAY-X001 jeywalking mosoelk sealy, indiaiing prresial ie riak
opears unoving
Figure 12: Visualization of qualitative results for sample on the UBnormal test set.
23

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction of the paper describe in detail the contribution and scope of the research.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We describe the limitations of the method in the supplementary material.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]

---

The methodology section 3 of the paper describes in detail the implementation process and details of the entire methodological framework.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The experimental part of the paper details the experimental environment platform on which the method is executed, well as the specific parameters of each module the method, which fully ensures the reproducibility the experimental results.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
25

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The dataset used is publicly available, and will share codes and database publicly once the paper is accepted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://niis.cc
Soblic//uuides/Coeeeubbiiss wtw.oeeeSuubiissiooooooooooogg for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ wttps/cccppblcccuui.eeeeeeaccaea
Stteee///ooSSSeeissSioica www.ssssiicggcoooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The implementation details section 4.1 of the paper details the experimental implementation details and its parameter settings.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Our paper performs deterministic evaluations on the fixed dataset. In the case of fixed random seeds, our experiments perform deterministic results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
26

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Details of the execution of the experiment and the platform which it was run are described in the experimental setup of the main paper and in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: The research conducted in the paper conforms, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Broader impacts described in detail in the supplementary material.
Guidelines:
27

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negatíve societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper does not contain such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
The creators or original owners of assets used in the paper are properly credited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
28

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
batasets.papersstt
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
No new assets introduced in the paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not include crowdsourcing experiments and research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not contain experiments IRB required.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
29

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: Our method uses VLM and MLLM for inference, and the type of LLM used described in the experimental setup subsection 4.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy hbtp:://heerpss.ccc////eeeeeeeep for what should or should not be described.
30


Reviews:
[{"review_id": "sZy3lWDWN2", "reviewer": "Reviewer_B3X7", "summary": "This paper proposes PANDA, a new paradigm for video anomaly detection (VAD) targeting general scenarios and open anomaly types. PANDA leverages multimodal large models (MLLMs) and simulates a detective-like “perceive-reason-reflect-learn” decision process. Through adaptive scene perception, goal-driven reasoning, tool-augmented reflection, and self-improving memory chains, PANDA achieves anomaly detection that requires no training or manual adjustments, adapting to different scenes and anomaly types. Extensive experiments on multiple datasets (UCF-Crime, XD-Violence, UBnormal, CSAD) covering diverse, open, and complex scenarios show that PANDA achieves SOTA-level performance without any training.", "questions": "Please respond to each item of the weaknesses mentioned above.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. The paper introduces a “detective agent” paradigm, incorporating RAG (retrieval-augmented generation), chain-of-thought self-reflection, tool-chain support, and memory chain mechanisms into VAD, achieving anomaly detection without training or manual prompt engineering.\n2. The approach breaks free from the heavy dependence of traditional VAD methods on specific scenes, event types, rules, or models.\n\nWeaknesses & Questions:\n\n1. The extensive use of RAG, repeated reasoning, and tool-chain calls in PANDA incurs significant inference latency and computational cost—preliminary estimates suggest FPS is even less than 0.1. The authors should provide detailed inference timings and discuss this issue, as high computational cost could significantly limit practical use.\n2. The novelty is limited; the core idea is to replace the prompts in existing MLLM-based VAD methods (such as SUVAD, LAVAD, VERA, etc.) with RAG and chain-of-thought techniques.\n3. During the lengthy reasoning process, model hallucinations may be amplified at each step. The authors should conduct case studies and thoroughly explain whether severe hallucinations occur and how much they affect result correctness.\n4. Technical details are lacking; the authors should supplement detailed implementation of RAG and tool invocations.", "was_revised": true, "final_justification": "After reading the responses from the authors, I believe that PANDA demonstrates a certain degree of innovation and practical significance in the direction of “training-free, manual-free, and generalizable VAD for open scenarios.” Although there are still some shortcomings, such as considerable engineering complexity and limited depth in ablation studies of certain modules, the overall contribution and experimental support are now at an acceptable level. Therefore, I decide to raise my score.\n\nI hope the authors will further improve the implementation details and failure case analysis in the final version, making this work even more valuable to the community.", "rebuttal": "**Thank you for your thoughtful and detailed feedback. We appreciate your recognition of PANDA’s “detective agent” paradigm and its advantages of operating without training or manual prompt engineering. Below, we address each of your concerns point by point:**\n\n---\n\n**Q1: Inference latency and FPS concerns.**\n\nWe respectfully clarify that PANDA’s actual speed is **significantly higher than the initial estimate (< 0.1 FPS)**, and is broadly sufficient to meet the practical requirements under 1 FPS sampling settings, which aligns with our experimental setup.\n\nAs shown in the supplementary material (Sec. C.3, Tab. 1c), the measured average inference speed across datasets is:\n\n|Dataset|Inference Speed (FPS)\n|:-|:-:\n|UCF-Crime|0.82\n|XD-Violence|0.86\n|UBnormal|0.79\n|CSAD|0.53\n\nAmong them, **CSAD is a purposefully extreme benchmark composed entirely of visually degraded scenes**, and thus its speed reflects a worst-case condition. For the other three datasets — UCF-Crime, XD-Violence, and UBnormal — which represent real-world surveillance distributions and open-set scenarios, **PANDA achieves an average speed of 0.823 FPS**, which **closely aligns with the 1 FPS processing rate** and supports timely anomaly detection in practical deployments (1 FPS sampling settings).\n\nImportantly, **PANDA is a training-free and manual-free VAD framework that supports both online and offline inference**, while existing SOTA training-free LLM/VLM-based methods (e.g., LAVAD, AnomalyRuler) rely on **multi-stage processing involving manual prompt design, external visual parsing, or result aggregation, making them unsuitable for online operation**.\n\nTo illustrate this comparison, we measured the average inference FPS on the UCF-Crime dataset with existing SOTA training-free LLM/VLM-based VAD baselines, all evaluated on the same hardware setup using a single A6000 GPU. As shown in the table below, PANDA outperforms previous methods **by 6~10 in speed**, while also supporting **training-free, manual-free, and fully autonomous online detection**.\n\n| Method| Inference Speed (FPS)\n|:-|:-:\n|LAVAD (CVPR 2024)|0.08\n|AnomalyRuler (ECCV 2024)|0.13\n|**PANDA**|**0.82**\n\n*Note: For LAVAD and AnomalyRuler, the inference process involves **multiple sequential stages**, where each stage must be completed before the next can begin. For example, LAVAD first generates captions for all video frames, then performs caption cleaning and summarization, followed by initial scoring and anomaly score refinement based on the captions. Therefore, in our evaluation, we report the **average FPS across all inference stages** to provide a fair and comprehensive comparison.*\n\nWe will include these clarifications and the comparison table in the revised version.\n\n---\n\n**Q2: The novelty is limited; the core idea is to replace the prompts in existing MLLM-based VAD methods (such as SUVAD, LAVAD, VERA, etc.) with RAG and chain-of-thought techniques.**\n\nWe respectfully clarify that **PANDA is not a simple replacement of prompts with RAG and CoT techniques**, nor an incremental extension of existing VAD methods. Instead, it is a **fundamentally new agentic AI framework**, built from scratch using LangGraph, to enable **generalist VAD** — meaning it can automatically adapt to new scenes and novel anomaly types in a training-free and manual-free manner. In contrast, existing methods typically require **retraining or manual intervention** when faced with such open-world scenarios.\n\nSpecifically, PANDA introduces a **paradigm-level shift** from training or manual dependence to **training-free, manual-free, and generalist** paradigm. It is built as a detective-like agent and is empowered with the following **cooperating abilities**:\n\n- **Self-adaptive scene perception** with environment-aware rule retrieval;\n- **Goal-driven heuristic reasoning** that aligns anomaly rules with real-time vision-language observations;\n- **Tool-augmented self-reflection**, allowing the PANDA to detect uncertainty and improve its own perception pipeline dynamically;\n- **Chain-of-memory experience alignment**, enabling context tracking across sliding windows and continue learning from history experience;\n- **Online processing ability**, where the system makes decisions sequentially without accessing future frames — whereas existing methods, such as **SUVAD, LAVAD, and VERA**, require access to the entire video for global context and can only operate in an offline manner.\n\nIn summary, these prior works like SUVAD, LAVAD, and VERA rely on handcrafted or fine-tuned prompts and equire offline access to global video information, they **lack four critical capacities**:  \n① **self-adaptive scene perception**, ② **self-reflection under degradation**, ③ **long-term memory across temporal sequences**, and ④ **online processing** — all of which are essential for **generalist VAD in complex, real-world settings**.\n\nWe believe PANDA is the first agentic VAD framework that **not only performs well across open-set and degraded scenes**, but also does so **fully autonomously**, without training or handcrafted tuning.\n\n---\n\n**Q3: Model hallucinations concern.**\n\nIndeed, hallucination remains one of the most recognized challenges when deploying large models (LLMs/VLMs), especially in lengthy reasoning process. In PANDA, we fully acknowledge this concern and have explicitly designed several key components to mitigate hallucination propagation throughout the pipeline. We elaborate below:\n\n**1.Scene-Aware Planning with RAG Mitigation**\n\nDuring the strategy planning phase, PANDA integrates scene perception with RAG. Instead of relying on free-form hallucinated outputs, the system uses explicit environmental cues (e.g., scene type, weather condition, potential anomalies) to retrieve relevant anomaly rules from a curated knowledge base. This constraint dramatically reduces the likelihood of hallucinated, scene-irrelevant planning responses.\n\n**2.Goal-Driven VLM Reasoning with Multi-State Output**\n\nDuring Reasoning, PANDA explicitly passes potential anomaly types, inferred from scene-aware planning, into the reasoning prompt — **thus narrowing the decision space**.\nMoreover, instead of forcing binary outputs (“normal” vs. “anomaly”), we introduce an **\"Insufficient\" reasoning state**, allowing the model to abstain from uncertain predictions rather than hallucinate a confident but incorrect answer.\n\n**3.Tool-Augmented Reflection to Ground Ambiguities**\n\nIf a clip is deemed “insufficient” by the VLM, PANDA triggers tool-augmented self-reflection, which includes reliable enhancement tools (e.g., image deblurring, brightness enhancement, object detection) that provide objective, grounded information. These tools act as external signal correctors, significantly reducing the reliance on speculative internal reasoning.\n\n**4.Chain-of-Memory for Decision Consistency**\n\nPANDA maintains both ShortCoM and LongCoM memory. These modules serve as explicit external memories, enabling the system to reference prior decisions and reflection rationales — reducing the chance of making inconsistent judgments across similar cases, a common hallucination symptom.\n\nOverall, these mechanisms above enable PANDA to effectively mitigate model hallucinations, as further supported by the component ablation study in Table 3 of the main paper. Nonetheless, hallucination remains a stubborn challenge for large models. Despite our efforts to alleviate it through multiple strategies, we still observed occasional hallucination cases during the final experiments. We will add hallucination case studies in the revised version to illustrate remaining failure cases and reflect on potential future improvements. **Notably, despite these challenges, PANDA — as a training-free and manual-free generalist model — still significantly outperforms existing methods across diverse scenario datasets, as shown in Table 1 (main paper), indicating its overall reliability and effectiveness.**\n\n---\n\n**Q4: Missing implementation details — RAG and tool invocation.**\n\nThank you for pointing this out. We appreciate the opportunity to clarify the technical implementation behind PANDA's core modules, and we will add further elaboration in the revised version.\n\n**1. RAG Details**\n\nPANDA encodes both the pre-built anomaly rule base and environment information using the all-MiniLM-L6-v2 sentence embedding model.\n\n- The rule base is indexed using FAISS for fast similarity retrieval.\n\n- During the scene-aware planning phase, the environment information is embedded and used as a query to retrieve the top-k most relevant rules, which are then fed into the MLLM for reasoning.\n\n**2. Tool Invocation Details**\n\nPANDA integrates a modular toolset, where all tool functions are pre-defined with their input/output formats and functional descriptions. These descriptions are made available to the MLLM during the reflection stage — as illustrated in **Supp. Fig. 6 ({tool_description_text})**.\n\nDuring reflection, the MLLM considers the VLM’s output, historical context, and available tool descriptions to determine whether additional information is needed. It then outputs a structured field tools_to_use, specifying:\n\n- The tool name to invoke\n\n- Optional parameters (e.g., query strings)\n\nEach tool name directly maps to a callable function in PANDA’s toolset. Finally, PANDA parses this tools_to_use field and executes the corresponding tool function with the provided parameters."}, {"review_id": "5J3PuIo5Gf", "reviewer": "Reviewer_AEpx", "summary": "This work aims to develop a generalisable video anomaly detection method which should handle any scene and any anomaly types without training data or human involvement.  Self-adaption and scene-aware strategy planning,  goal-driven heuristic reasoning,  tool-augmented self-reflection, and  self-improving chain-of-memory are the four main components of the proposed method.  Extensive experiments demonstrate that the proposed method (dubbed as PANDA) achieves state-of-the-art performance in multi-scenario, open-set, and complex scenario settings.", "questions": "Please respond to the weaknesses mentioned above.", "limitations": "Failure cases are not discussed in the paper.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\nThis work proposes PANDA, training-free VAD that can automatically handle any scene and any anomaly types without training data or human involvement.  PANDA is evaluated on four VAD benchmarks representing three distinct setting multi-scenario, open-set, and complex scenario settings.\n\nWeekness:\n1. In the implementation details, it is mentioned that “the maximum number of reflection rounds r is set to 3.” The paper suggests that PANDA is able to assign a normal or anomalous score to all previously insufficient samples within these rounds. However, intuitively, there may be hard cases where even after three reflection rounds, PANDA might still fail to confidently assign a label. Could the authors clarify how such unresolved or ambiguous cases are handled, if any remain after the maximum number of reflection rounds?\n2. All the methods compared in Table 1 perform anomaly detection in an offline setting. As stated in the supplementary material (L19–L20), \"If future information is accessed, the method is considered offline; otherwise, it falls under the online setting.\" However, PANDA online setting still outperforms offline, training-free methods. This seems counterintuitive. Could you clarify why PANDA achieves such strong performance even in the online setting?\n3. An analysis of how frequently the reflection mechanism is invoked i.e., the number of clips requiring reflection rounds would be valuable. Additionally, it would be insightful to report how many clips transition from \"insufficient\" to either \"normal\" or \"anomalous\" after each reflection round. Finally, please clarify how many clips, if any, remain in the \"insufficient\" category even after the maximum of three reflection rounds.\n4. Figure 2 lacks clarity and does not accurately reflect the proposed method. According to the methodology, when the VLM reasoning status is marked as “insufficient,” the MLLM reflection module is invoked. However, this is not clearly depicted in the figure. Instead, the arrow from the VLM reasoning block appears to lead directly into another instance of VLM reasoning, which is misleading. To improve clarity, the figure should explicitly illustrate the transition from VLM reasoning to MLLM reflection in the case of an “insufficient” outcome. Additionally, incorporating step numbers into the diagram would help convey the sequential flow of the pipeline more effectively. As it stands, the figure may confuse readers rather than aiding in their understanding of the method.\n5. Since LongCoM evolves over time, it initially lacks any prior information about insufficient cases. How does the Experience-Driven Reflection module handle such scenarios where no historical context is available?\n6. L241-242 is ambiguous “In offline reasoning mode, the perception phase is sampling M = 300 frames uniformly for the whole video, while only the initial M = 10 frames are sampled in offline mode.” please clarify this statement.\n7. PANDA incorporates several tools to enhance video content analysis. Could you elaborate on how these specific tools were selected, and what criteria guided the choice of each component?\n8. Is the anomaly knowledge subjective to each dataset?\n9. In Supplementary Figure 1, which visualizes the detailed PANDA pipeline, the Short-CoM component appears to be missing. For completeness and clarity, it would be helpful to include Short-CoM in the diagram as well.\n10. Supp. Figure 5, where does the “{formatted_enhancement_prompt} come from?\n11. How the proposed method compares to unsupervised VAD such as: \"Generative cooperative learning for unsupervised video anomaly detection\" in CVPR 2023?", "was_revised": true, "final_justification": "Q2: The accuracy values shown in Table 1 on CSAD dataset is not convincing. The proposed method has a gap of 15.86% AUC over the current best training-less method LAVAD. Note that CSAD dataset is sampled from the existing datasets UCF, XD, UB, and the average performance improvement on these datasets is less than 8%. Authors were not able to justify this gap in the rebuttal.  Authors have not disclosed the details of the CSAD dataset so this reviewer could not confirm which type of videos were included. In Q12 authors accept that their method does not work well in low resolution and the case has long-term dependencies. This CSAD dataset is all about low resolution and long-term dependencies.  Therefore the performance of SOTA as well as the proposed method on CSAD is not convincing. A detail of the video numbers included in CSAD would be helpful. \n\nQ4,Q9,Q5,Q10: The figures and the text need improvements and authors have promised to incorporate these improvements. Such major changes are often beyond the scope of a conference paper because reviewers could not enforce the authors to incorporate the promised changes.   \n\nQ8: Response to Q8 is not convincing. It looks the algorithm requires the dataset dependent anomaly class names which is against the claims of a generic algorithm which can be applied to any field or domain without any modifications.  Candidate rule set is constructed based on the user’s goals which varies from dataset to dataset. Therefore, for a test video, the possible anomaly class names and the user's goals should also be available. Without this information, the proposed algorithm would not be able to predict anomaly in a given test video. \n\nQ 11: Comparison with unsupervised anomaly detection methods would require the proposed algorithm to predict anomalies in a given video without knowing the possible anomaly class names and any other user input. Authors need to present results of their method under unsupervised settings if applicable. The authors' response to this comment is not convincing.  \n\nAdditionally, the capability of the proposed method for handling longterm dependencies is not convincing. A breakdown of the performance on the UCF Crime over different anomaly categories would be helpful. Also slow speed is a problem because most abnormal actions are fast and would take only very few seconds. Identifying anomalies by looking at 2 or 3 frames far apart would be quite difficult.     \n\nFor the time being, the rating of the paper is not changed.", "rebuttal": "**We thank the reviewer for their insightful comments and thoughtful questions. We are pleased that you found PANDA’s design, scope, and generality valuable. Your excellent rating on the quality of our work is truly encouraging and deeply appreciated. Below we respond point-by-point to the concerns raised.**\n\n---\n**Q1. How does PANDA handle unresolved “insufficient” cases after the maximum 3 reflection rounds?**\n\nAs stated in the main paper (L196–197):\n> *“If after r rounds the result is still Insufficient, PANDA skips the current segment and continues to the next timestep.”*\n\nThus, if a segment remains “Insufficient” after the maximum r = 3 reflection rounds, PANDA assigns a default anomaly score corresponding to the \"Insufficient\" status. This strategy is a deliberate design choice to balance computational efficiency with performance stability.\n\nAs shown in Tab. 2(a) of the main paper, increasing r from 3 to 5 brings minor gains, resolving some hard cases but significantly increasing inference time, making r = 3 a reasonable trade-off in practice.\n\n---\n**Q2. Why PANDA outperforms other offline methods in online mode?**\n\nWe agree that a method typically performs better in the offline setting due to access to future information. As shown in Tab. 1 (main paper), PANDA’s offline mode consistently outperforms its online counterpart. However, due to fundamental design differences, online and offline methods are often not directly comparable — a well-designed online system may outperform rigid offline ones.\n\nOur agentic AI framework, PANDA, is deliberately designed to function robustly in the online setting, and notably, it outperforms existing training-free offline methods, which we highlight as one of our key contributions.\nBelow, we further explain PANDA’s advantage by analyzing the capabilities enabled by its core components.\n- **Self-adaptive scene perception**: PANDA uses RAG to retrieve context-aware anomaly rules based on current scene information (e.g., scene overview, potential anomalies), aligning reasoning goals with environment.\n- **Goal-driven heuristic reasoning**: PANDA performs CoT-style planning guided by retrieved rules, enhancing grounding between task objectives and VLM predictions.\n- **Tool-augmented self-reflection**: Upon insufficient reasoning confidence, PANDA autonomously invokes enhancement tools (e.g., object detection, image deblurring) to improve clarity before re-reasoning.\n- **Chain-of-memory**: PANDA leverages memory mechanisms (ShortCoM and LongCoM) to accumulate prior context, enabling temporally consistent decisions in streaming settings.\n\nThese mechanisms enable PANDA to make robust anomaly detection — even without access to future frames — much like a human expert acting with present information and past experience. **In contrast, SOTA methods like LAVAD and AnomalyRuler rely on static prompts and lack adaptation, reflection, or memory, limiting their reasoning robustness even in offline mode.**\n\n---\n**Q3. Statistics about reflection mechanism.**\n\nThank you for this detailed suggestion. We analyzed PANDA’s reflection behavior and summarize key statistics below (full results will be added to supp. material).\n\n**1. Frequency of reflection**\n\n|Dataset|Total Clips|Clips Triggering Reflection|Trigger Rate (%)  \n|:-|:-:|:-:|:-:\n|UCF-Crime|7416|858| 11.6%\n|XD-Violence|19382|2956|15.3%\n|UBnormal|619|82|13.2%\n|CSAD|1202|429|35.7%\n|Avg.|—|—|19.0% \n\nAs shown in the table above, the average reflection trigger rate across all four datasets is 19.0%. **Notably**, the CSAD dataset exhibits a trigger rate nearly 2–3 times higher than the other three. This aligns with our expectations, as CSAD is composed entirely of complex scenes, where PANDA is more likely to encounter uncertainty during reasoning and therefore invokes reflection more frequently.\n\n**2. Transition outcomes**\n\nAmong clips initially classified as “Insufficient”, their transitions after each round are as follows (UCF-Crime dataset):\n\n|Reflection Round |Clips Resolved to Normal or Abnormal|Clips Remaining Insufficient|Transition Rate (%) \n|:-|:-:|:-:|:-:\n|After Round 1|334|524 (858→334)|38.9%\n|After Round 2|599|259 (858→599)|69.8%\n|After Round 3|752|106 (858→752)|87.6%\n\nThis shows that **over 87.6%** of uncertain clips are resolved within 3 rounds. We also observed diminishing returns after the second round, which supports our default choice of r = 3 as a trade-off between effectiveness and computational overhead.\n\n---\n**Q4. Fig. 2 lacks clarity in reflection flow.**\n\nThank you for this constructive feedback. To improve clarity, we will revise Fig. 2 in the following ways:\n- **Explicitly depict the “Insufficient” reasoning path**, with a clearly labeled arrow from the VLM block to the MLLM reflection module;\n- **Visually separate normal reasoning and reflection flows** via distinct line styles;\n- **Number key steps (1–6)** to clearly present PANDA’s sequential reasoning process: perception → planning → initial reasoning → reflection trigger → tool-enhanced retry → memory update.\n\n---\n**Q5. How does LongCoM behave at cold-start (no history)?**\n\nThank you for the insightful question. At the start of a video, LongCoM is empty by design, and PANDA relies on ShortCoM’s local window memory for initial reasoning and reflection. As more clips are processed, LongCoM gradually accumulates traces, supporting memory-consistent reasoning and reflection planning. This progressive warm-up is intentional to ensure robustness under cold-start conditions. We will clarify this in Sec. 3.4.\n\n---\n**Q6. L241–242 ambiguity.**\n\nThank you for catching this. We will revise this sentence in the final version. The correct statement should be:\n> “In offline reasoning mode, the perception phase is sampling M = 300 frames uniformly for the whole video, while only the initial M = 10 frames are sampled in **online mode**.”\n\n---\n**Q7. Tool selection.**\n\nAs detailed in Sec. 3.3 of main paper and Supp. Fig. 1, PANDA selects tools based on two signals:\n- **VLM reasoning feedback**: When the VLM returns **“Insufficient”**, it provides a natural language rationale (e.g., “too blurry”, “too dark”), which helps identify the root cause.\n- **Memory-guided experience**: PANDA retrieves similar past “Insufficient” cases and their successful tool usage from LongCoM, informing effective tool reuse.\n\nThese cues are combined into a reflection prompt, along with all **tool descriptions**. The MLLM then selects the most appropriate tools, which are invoked accordingly. We will clarify this more explicitly in the final version.\n\n---\n**Q8. Is the anomaly knowledge subjective to each dataset?**\n\nWe would like to clarify that PANDA’s anomaly knowledge base is **not subjective to specific datasets**, but is dynamically built based on:\n- User-defined anomaly categories \n- Scene context perceived at test time.\n\nGiven anomaly is context-dependent, PANDA embraces this by adapting the rule base per instance:\n- It first constructs a candidate rule set based on the user’s goals;\n- Then, during the scene perception stage, it retrieves context-relevant rules using environmental cues (e.g., scene overview, potential anomalies);\n- During reflection, it further refines or expands rules based on reasoning ambiguities or failures.\n\nThis design ensures the knowledge base is context-grounded, adaptive, and generalizable, rather than statically tailored to any dataset — enabling open world anomaly detection.\n\n---\n**Q9. Missing ShortCoM in Supp. Fig. 1.**\n\nThank you for pointing this out. While ShortCoM is implicitly integrated into the construction of both the **Reasoning Input and Prompt** and the **Reflection Prompt**, we agree it is not explicitly shown in Supp. Fig. 1. We will revise the Supp. Fig. 1 to clearly depict ShortCoM.\n\n---\n**Q10. Supp. Fig. 5, where does the “{formatted_enhancement_prompt} come from?**\n\nThank you for raising this point. `{formatted_enhancement_prompt}` refers to the **enhanced prompt generated after the reflection stage**, as described in **Lines 148-152 and 188–193 of the main paper**. It integrates three components: {Text Enhancement Info, New Anomaly Rule, New Heuristic Prompt}, where **Text Enhancement Info** summarizes the outputs of invoked tools (e.g., object detection, web search). We will clarify this in the caption of Supp. Fig. 5.\n\n---\n**Q11. PANDA VS \"Generative cooperative learning for unsupervised VAD (GCL)\" (CVPR 2023)**\n\nThank you for highlighting this reference. PANDA differs from GCL in both its goal and capabilities. While GCL focuses on cooperative training and learning from unlabeled videos, PANDA is designed to generalize to unseen scenes and anomaly types without any training, which GCL cannot handle due to its training dependency. Specifically:\n- PANDA adopts an agentic paradigm with scene perception, rule-based retrieval (RAG), heuristic reasoning, and tool-augmented reflection, enabling zero-shot and training-free VAD.\n- In contrast, GCL requires fixed model structures and offline training on large unlabeled data, limiting adaptability when deployed in open-world, complex environments.\n\nQuantitatively, **on the shared evaluation benchmark UCF-Crime, PANDA achieves an AUC of 84.89%, significantly outperforming GCL’s reported 71.04%**, while training-free and zero-data requiring. We will add this comparison to the revised version.\n\n---\n**Q12. Failure case analysis.**\n\nThank you for the helpful suggestion. We will add a failure case visualization in Supp. Sec. C.4.\n\nThe case from *UCF-Crime* (`Shoplifting033_x264.mp4`) involves a man discreetly steals a watch from a store table. Due to low resolution and subtle visual cues, PANDA fails to detect the anomaly — even after 3 reflection rounds, the clip remains “Insufficient” status and is skipped.\n\nThis case shows that PANDA may struggle with fine-grained, low-visibility anomalies. We will explore stronger visual enhancement tools and finer-grained reasoning to address such cases in future work."}, {"review_id": "8cHgPHqJTG", "reviewer": "Reviewer_cvqx", "summary": "This paper presents a solution to make multimodal large language models (MLLMs) behave like a detective with 4 modules: (1) a RAG to make them have strategy planning ability, (2) a heuristic prompt strategy to make them have goal-driven heuristic reasoning, (3) a progressive reflection mechanism to make them have self-reflection, and (4) a chain-of-memory mechanism for self-improving ability. The paper tests the methods on representative datasets including UCF-Crime, XD-Violence, UBnormal, and CSAD.", "questions": "1. Why we must use those modules to model the desired ability? Are they optimal? What are the other solutions?\n2. Are these modules really necessary? VERA has an AUC~86% in UCF-Crime and just need good prompts for MLLMs. \n3. What's the additional computation overhead will be brought by these modules?", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The paper is well-motivated to make AIs behave like detectives in video anomaly detection. The paper includes additional modules to equip the model with the desired ability, and the selection of them is reasonable.\n2. The paper tests the proposed method in various datasets and presents a comprehensive result for study.\n\nWeaknesses:\nThe main concern for this paper is that it doesn't illustrate the necessity of why we must have those modules, and in fact these modules actually could be redundant. \n1. First, the paper doesn't have concrete reasons to explain why we must use those techniques to model the four abilities in detectives. Whether they are optimal design is not discussed. The current solution is more like a combination of hot techniques without illustrating why they are essential in design.\n2. Second, these modules may not be necessary because an important baseline is missing. This paper is categorized into training-free methods but it doesn't compare with another method not fine-tuning LLMs published in CVPR'25. (VERA: Explainable Video Anomaly Detection via Verbalized Learning of Vision-Language Models). The proposed method uses many additional modules to achieve VAD, however, based on the performance on XD-Violence and UCF-Crime, it doesn't outperform VERA which simply uses good prompts obtained from verbalized learning between LLMs. These results could indicate that actually all the introduced modules in this paper is not necessary and a good prompt could outperform them. The paper needs a more thoughtful discussion on models not using fine-tuning with more works such as VERA.\n3. Third, the additional computation overhead for including these modules are not discussed in this paper. This is an obvious trade-off for including those modules and need detailed discussion.", "was_revised": true, "final_justification": "I'd like to raise the score here because the rebuttal results include the discussion of the comparison with recent methods like VERA. This discussion will be useful for illustrating the strengths and weaknesses of the paper. I raise my rating because of this and hope the authors will include this discussion in the future version.", "rebuttal": "**Thank you for your thoughtful and critical review. We appreciate your recognition of the motivation and comprehensiveness of our proposed PANDA. We address your concerns in detail below.**\n\n---\n\n**Q1: Why we must use those modules to model the desired ability? Are they optimal? What are the other solutions?**\n\n**1. Why those modules are necessary.**\n\nThank you for the insightful question. We would like to clarify that the four modules in PANDA — namely: (1) self-adaptive scene-aware strategy planning, (2) goal-driven heuristic reasoning, (3) tool-augmented self-reflection, and (4) a self-improving chain-of-memory — are not arbitrary choices, but are **deliberately designed to achieve generalist VAD that is fully training-free and manual-free even when facing novel scenes or anomaly types — a capability not supported by existing VAD methods.**\nNext, we elaborate on the necessity of those modules below.\n\n**(1) Functional Necessity**\n\nEach module equips PANDA with a key cognitive capacity essential for generalist VAD:\n\n- **Self-adaptive scene-aware strategy planning (via RAG)**: Real-world anomalies are scene-dependent, and thus require context-specific rules. The RAG module lets PANDA dynamically retrieve only relevant rules from a anomaly knowledge base based on perceived context. Static prompts or fixed rules (used in prior work like VERA or AnomalyRuler) cannot generalize.\n\n- **Goal-driven heuristic reasoning**: Existing methods that rely on manually constructed or tuned fixed prompt templates for MLLM reasoning are brittle and poorly scalable. PANDA learns to synthesize them adaptively from environment + user goals + rules.\n\n- **Tool-augmented self-reflection**: When facing ambiguous or degraded input, PANDA “knows what it doesn’t know” and selectively invokes tools (e.g., object detection, web search). This reflects a meta-cognitive ability essential to generalization.\n\n- **Chain-of-Memory**: Many anomalies involve temporal cues (e.g., shoplifting, riot). PANDA maintains both short-term reasoning traces and long-term historical experiences, allowing it to refine decisions across clips — which neither VERA nor other baselines support.\n\n**(2) Empirical Justification**\n- As shown in Table 2 (main paper), PANDA outperforms existing SOTA methods.\n- Table 3 (main paper) demonstrates that the performance of PANDA improves consistently and substantially as each module is incrementally added. \n- **As addressed in Q2**, even when compared to VERA — a recently proposed method trained with coarse-grained prior labels — PANDA still achieves superior results on the XD-Violence and CSAD datasets.\n\nThese empirical evidence indicates that these modules are not redundant but instead contribute orthogonally to generalization and robustness in VAD. \n\n**2. Are these modules optimal? What are the other solutions?**\n\nThank you for the insightful question. While we do not claim these modules represent the “optimal” design, they provide a minimally sufficient and extensible structure to achieve training-free, manual-free, and generalist VAD across diverse benchmarks. \n\nWe acknowledge that these modules proposed in PANDA represent just one plausible design path toward generalist VAD. Another feasible solution is to design reinforcement learning (RL)-based agents — for example, using RL to train a policy that decides how to analyze videos (e.g., which tools to invoke or how to interpret anomaly signals). However, since real-world anomalies are rare and highly context-dependent, this poses significant challenges for reward function design and generalization to complex, dynamic environments.\n\nIn summary, while other solutions exist, we emphasize that PANDA offers a **training-free**, **manual-free**, and **generalist** design that balances flexibility and practicality. We believe it provides a compelling trade-off between generalization, interpretability, and ease of deployment in real-world VAD scenarios.\n\n---\n\n**Q2: The paper lacks comparison with VERA (CVPR 2025), which uses learned prompts to achieve strong results without fine-tuning. Does this mean the proposed modules are unnecessary?**\n\nThank you for raising this important point. We fully acknowledge that VERA is a strong, recent method in the MLLM-based VAD paradigm. However, we respectfully clarify several key distinctions and provide both theoretical and empirical evidence supporting the necessity of PANDA’s modular design:\n\n**1. PANDA is Fully Training-Free and Manual-Free — VERA Requires Dataset-Specific Training and Human Intervention**\n\nA core distinction is that PANDA is a generalist that is **fully training-free and manual-free**, requiring *no training phase, no data annotation, no optimization of prompt parameters, and no human intervention*. The entire pipeline is **globally automated** — from rule generation to detection — making PANDA highly suitable for deployment in novel environments with **no prior data or supervision**.\n\nIn contrast, VERA is a **specialist model** that requires:\n- Coarsely labeled training videos (weakly-supervised);\n- An optimizer-learner loop to iteratively tune prompt questions;\n- Manual inspection for convergence and selection of prompt questions.\n\nThis makes PANDA significantly more lightweight and **deployment-ready in unseen, zero-data environments**, while VERA remains more aligned with weakly-supervised prompt learning and dataset-specific adaptation.\n\n**2. Evaluation Metric Consistency — Our AUC and AP on XD-Violence is Higher than VERA**\n\nVERA reports **AUC** on the XD-Violence dataset, while our main paper follows the original XD-Violence paper [HL-Net, Wu et al., ECCV 2020] in reporting **AP**, which is more robust for class-imbalanced datasets as highlighted by XD-Violence authors.\n\nTo ensure a fair comparison, we additionally report PANDA’s AUC on the XD-Violence dataset. Furthermore, using the official VERA codebase — including the released video features, initial scores, and AP computation scripts — we reproduce VERA’s AP on XD-Violence (**which was not reported in their paper**). As shown in the comparison table, **PANDA outperforms VERA on both AUC and AP metrics.** This demonstrates that PANDA’s modular design does not sacrifice accuracy despite being fully training-free and manual-free, whereas VERA requires training on coarsely labeled video data.\n\n|Methods|Supervision|XD-Violence (AUC%)|XD-Violence (AP%)|CSAD (AUC%)\n|:-|:-|:-:|:-:|:-:|\n|VERA|Weakly-supervised|88.26|70.11|64.52\n|PANDA|Training-free|89.35 (+1.09)|70.16 (+0.05)|73.12 (+8.6)\n\n\n**3. PANDA Outperforms VERA in Complex and Degraded Scenarios (e.g., CSAD)**\n\nWe further evaluate VERA using its released codebase on our proposed CSAD benchmark — a complex-scene anomaly detection set characterized by poor lighting, low resolution, and long-range temporal anomalies.\n\nAs shown in the table above, VERA achieves only 64.52% AUC on CSAD, whereas PANDA significantly outperforms it with 73.12%. This highlights that **prompt-based adaptation alone is insufficient** for handling real-world degradation, where PANDA’s self-adaptive scene-aware, tool-augmented self-reflection, and chain-of-memory modules play a critical role in restoring visual clarity and improving reasoning confidence.\n\nIn summary, while we commend VERA as an elegant prompt-learning method, we emphasize that:\n- PANDA is **completely training-free, manual-free, and generalizable** — capable of handling arbitrary new scenes and anomaly types, whereas VERA requires **coarse-grained annotations and training** on specific datasets;\n- PANDA surpasses VERA on complex and unseen scenes without dataset-specific tuning;\n- PANDA achieves superior accuracy on the XD-Violence under a fair comparison on the same evaluation metric;\n- PANDA offers **fully training-free, manual-free, and globally automated solution**, which is essential for real-world generalist anomaly detection systems.\n\nFinally, we thank the reviewer again for highlighting this important comparison. We will include detailed discussion on VERA in the revised version of our paper.\n\n---\n\n**Q3: What's the additional computation overhead will be brought by these modules?**\n\nThank you for pointing out this important concern. We agree that a discussion on computation cost is necessary, and now provide additional clarification and quantitative analysis.\n\n**1. Inference Speed Across Datasets**\n\nWe have reported the inference speed of PANDA on each dataset in the supp. material (Section C.3, Table 1c):\n\n|Dataset|Average Inference Speed (FPS)\n|:-|:-:\n|UCF-Crime|0.82\n|XD-Violence|0.86\n|UBnormal|0.79\n|CSAD|0.53\n\nAmong them, **CSAD is a purposefully extreme benchmark composed entirely of visually degraded scenes**, and thus its speed reflects a worst-case condition. For the other three datasets — UCF-Crime, XD-Violence, and UBnormal — which represent real-world surveillance distributions and open-set scenarios, **PANDA achieves an average speed of 0.823 FPS**, which **closely aligns with the 1 FPS processing rate** and supports timely anomaly detection in practical deployments (1 FPS sampling settings).\n\n**2. Trade-off Justification**\n\nWe acknowledge that PANDA introduces additional reasoning and reflection steps compared to fixed-prompt baselines. However, this is a **conscious trade-off**:\n\n- PANDA trades marginal compute for **robustness under uncertainty**, **scene adaptivity**, and **open-set generalization**;\n- Unlike training-dependent systems, PANDA avoids all training costs;\n- Unlike VERA, PANDA does not require pre-optimized prompts or dataset-specific training to adapt.\n\nWe believe that this modest overhead is justified by PANDA’s real-world usability and improved generalization in complex and unseen scenarios. We also plan to reduce PANDA’s computational load and further improve its inference speed in future work by incorporating multithreaded tool invocation and adopting lighter LLM/VLM backbones."}, {"review_id": "4Q8scqHMsR", "reviewer": "Reviewer_wQDY", "summary": "The paper proposes a training free model for VAD called PANDA. An LLM acts as the core reasoner. A four stage pipeline has been introduced to first pull scene-specific rule snippets from text, then a language-vision model evaluates videos with those rules. After this, there is a reflection module to use external tools such as super resolution if the confidence is low and then re running the evaluation. Finally a memory system is implemented to guide the later decisions.", "questions": "Please refer to the weaknesses part, but here is the main questions:\n\n1- The first question I have is regarding the real-time performance of the model. Please refer to the first weakness.\n\n2- I would appreciate it if authors can clarify on the unseen anomalies and also the effect of varying the number of rules. Evaluate PANDA on anomaly categories absent from the knowledge base to reveal generalisation behaviour is very helpful.\n\n3- Please estimate annotation time per anomaly category or provide an automated rule-generation procedure, and discuss how the knowledge base scales to hundreds of categories. Clear evidence that the rule set can be created with minimal manual effort, or that it can be learned automatically would help the decision.\n\n4- Please polish the presentation of the paper and fix spelling issues etc.", "limitations": "The societal-impact section acknowledges surveillance use but does not analyse privacy risks or potential misuse. A brief discussion would help.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n\n- The framework design, combining planning, reasoning, utilization of external tools and memory for generalist VAD is compelling idea, which is extending from LLM agents to the video domain.\n\n- There are comprehensive empirical study with four datasets and ablations on design choices.\n\n- The paper is very clear and easy to follow.\n\nWeakness:\n\n- My main concern is the real-time performance of the model. In VAD we inherently rely on real-time systems. Catching an anomaly in a reasonable time frame is an important aspect of VAD. The authors report average inference speeds of 0.53–0.86 FPS across four datasets. In PANDA the raw video stream is first down-sampled to one frame per second and inference is carried out on 5-frame clips drawn from that 1 FPS stream. The authors (and your supplemental table) report effective throughputs of 0.53 – 0.86 analysed frames per second. To my understanding for example at 0.86 FPS the agent processes each sampled frame in about 1.16s, so the backlog grows slowly. In practice the display or decision stream will drift behind real time by roughly one second per minute of video. At the lower end, 0.53 FPS approximately doubles that drift. For time-critical response scenarios the agent would require further optimization. If the authors could clarify this or provide comparison with other models mentioned in the paper it would strengthen the papers promise.\n\n- Another concern is that the model still depends on a manually curated anomaly-rule knowledge base. Authors need to clarify on the effectiveness and the cost of building and maintaining this base, and its effect on generalisation to unseen anomaly categories. I understand that the domain seems to be anomalous \"actions\" and not dealing with unseen or novel anomalies, thus I need clarification with this point.\n\n- There are some minor typos such as \"planing\" or \"Kowleage\" in table 3. It would be great if authors can go over the paper for a final clean up.\n\n- One last minor concern is that, the model relies mostly on already developed models and the contribution is limited to the design of the system.", "was_revised": true, "final_justification": "While most of my concerns were properly addressed by the authors. Considering other reviewers comments, I still would like to keep my initial assessment.", "rebuttal": "**We sincerely thank the reviewer for the positive assessment and thoughtful comments. We are glad you found the PANDA framework compelling and clearly presented. Below, we address each concern in detail.**\n\n---\n\n**Q1: Real-time performance concern.**\n\nWe thank the reviewer for this detailed analysis and fully agree that real-time inference is vital for practical VAD systems. We provide a detailed clarification on the real-time performance below.\n\n1. **Measured throughput and drift analysis**  \n   As shown in **Supp. Tab. 1c**, PANDA achieves the following average inference speeds across different datasets:\n   \n   |Dataset|Average Inference Speed (FPS)\n   |:-|:-:\n   |UCF-Crime|0.82\n   |XD-Violence|0.86\n   |UBnormal|0.79\n   |CSAD|0.53\n\n   Notably, **CSAD is an extreme stress-test benchmark**, composed entirely of visually degraded videos. This is not reflective of typical real-world anomaly distributions, but is used to evaluate robustness under worst-case conditions.  \n   Across the other three realistic and diverse datasets, PANDA achieves an **average speed of 0.823 FPS**, meaning each sampled frame is processed in approximately **1.22 seconds**, resulting in a mild temporal drift under the 1 FPS sampling setting — as the reviewer insightfully observed.\n\n   In practice, this level of delay is often acceptable for real-world anomaly detection, as abnormal events typically unfold over several seconds. PANDA is often able to raise an anomaly alert during the early stages of most anomalies, which satisfies the latency requirements of many surveillance and safety-critical scenarios. Moreover, we plan to further reduce PANDA’s computational cost by **introducing multi-threaded tool pipelines and adopting lighter-weight LLM/VLM backbones** in future work.\n\n\n2. **Comparison with existing training-free LLM/VLMs-based methods**  \n\n   Real-time VAD using LLM/VLMs is inherently challenging due to their reasoning complexity. Existing SOTA training-free MLLM-based methods like LAVAD and AnomalyRuler involve **manual intervention** (e.g., pre-processing prompts or post-filtering predictions) and **require offline access to global video information**, making them unsuitable for online deployment**. PANDA, to our knowledge, is the **first system to support online, training-free VAD in an agentic manner**. \n   \n   To illustrate this comparison, we measured the average inference FPS on the UCF-Crime dataset with existing SOTA training-free LLM/VLM-based VAD baselines, all evaluated on the same hardware setup using a single A6000 GPU. As shown in the table below, PANDA outperforms previous methods **by 6~10 in speed**, while also supporting **training-free, manual-free, and fully autonomous online detection**.\n\n   |Method| Avg. FPS|Online\n   |:-|:-:|:-:\n   |LAVAD (CVPR 2024)|0.08|✘\n   |AnomalyRuler (ECCV 2024)|0.13|✘ \n   |**PANDA**| **0.82**|✔\n\n   *Note: For LAVAD and AnomalyRuler, the inference process involves **multiple sequential stages**, where each stage must be completed before the next can begin. For example, LAVAD first generates captions for all video frames, then performs caption cleaning and summarization, followed by initial scoring and anomaly score refinement based on the captions. Therefore, in our evaluation, we report the **average FPS across all inference stages** to provide a fair and comprehensive comparison.*\n\nThis discussion and table will be included in the revision.\n\n---\n\n**Q2: Concerns regarding reliance on an anomaly-rule knowledge base, generalization to unseen anomalies, and rule number sensitivity**\n\nWe sincerely thank the reviewer for these insightful questions. Below, we respond to each aspect in detail:\n\n**1. On the construction and cost of the anomaly-rule knowledge base**\n\nWe would like to clarify that **PANDA does not require any manual annotation or dataset-specific rule design**. As shown in *Supp. Fig. 1 (top)*, the anomaly knowledge base is **automatically constructed at runtime** based on the user’s specified anomaly detection needs.\n\n- This process is driven by MLLMs (e.g., Gemini 2.0 Flash), using dynamically composed prompts based on the user requirement;\n- For example, generating 20 rules for each of 13 anomaly categories on the UCF-Crime dataset took approximately **95 seconds** in total — demonstrating low overhead and strong scalability, even when extended to hundreds of categories.\n\nImportantly, PANDA does not rely rigidly on the initial rule base. It performs:\n\n- **Scene-aware knowledge retrieval** in the perception stage, selecting only context-relevant rules;\n- **Dynamic rule augmentation** during reflection, where new rules may be added or revised based on ambiguous cases and context.\n\nThis ensures that PANDA’s reasoning process remains adaptive, even when the initial knowledge base lacks coverage for the current anomaly type.\n\n**2. On generalization to unseen anomalies**\n\nSince PANDA is fully training-free, it starts with no pre-learned notion of known anomaly categories. The only guidance comes from **user requirement** at runtime.\n\nBecause anomaly is inherently context-dependent — the same action (e.g., “running”) may be normal in a park but abnormal in a hospital. PANDA reflects this principle and building the anomaly rule base on demand, according to the specific detection goal and scene context.\n\nMoreover, PANDA’s adaptive mechanisms — self-adaptive scene perception, tool-augmented self-reflection, and chain-of-memory — allow it to handle anomalies not explicitly defined in the initial rule base.\n\nTo further verify this, we conducted an **open-set evaluation on the UBnormal dataset**:\n- The anomaly rule base was constructed **only using categories from the training set**;\n- The test set included **completely disjoint anomaly types**;\n- As shown in the table below, PANDA’s performance only dropped ~1.5% AP, and still outperformed all SOTA methods — demonstrating robust generalization to unseen anomalies.\n\n|Method|Setting| AP (%)\n|:-|:-|:-:\n|ZS CLIP (ICML 2021)| Training-free|46.20\n|LLAVA-1.5 (CVPR 2024)| Training-free|53.71\n|LAVAD (CVPR 2024)| Training-free|64.23\n|AnomalyRuler (ECCV 2024)|Training-free|71.79\n|PANDA|Training-free (w/o test anomaly rules)| **74.23**\n|PANDA|Training-free (w test anomaly rules)| 75.78\n\n\n**3. On rule number effect**\n\nWe have analyzed this factor in Tab. 2(b) of the main paper. When too few rules are retrieved (e.g., k = 1), the system lacks diverse contextual cues to support robust reasoning, resulting in performance degradation. Conversely, setting k too high (e.g., k = 9) may introduce noisy or irrelevant rules that dilute reasoning quality. Finally, PANDA achieves optimal performance when setting k = 5.\n\n   |Rules Number k|UCF-Crime (AUC%)\n   |:-:|:-:\n   |1|82.79\n   |5|**84.89** \n   |9|83.92\n\n---\n\n**Q3: Clarification on contribution scope beyond model reuse**\n\nThank you for this remark. While it is true that PANDA leverages existing MLLMs and VLMs, we respectfully argue that its contribution lies in the **novel agentic AI paradigm and system-level architecture** designed to unlock **training-free, manual-free, and generalist video anomaly detection** — a capability not achieved by prior work.\n\n**1. Paradigm-level innovation**\n\nPANDA is not a naive wrapper over existing models. It introduces a deliberately designed four-stage reasoning stack —  \n*Perceive → Reason → Reflect → Memory(Learn)* — mimicking a human detective’s workflow. Each module equips PANDA with essential abilities:  \n- **Self-adaption scene-aware strategy planning** → addresses open-scene adaptation;\n- **Goal-driven heuristic reasoning** → supports goal-driven, explainable, and reliable decisions;\n- **Tool-augmented self-reflection** → enhances robustness under visual degradation;\n- **Chain-of-memory** → enables temporal abstraction and learning from prior experience.\n\nThis transforms MLLMs from static responders into situationally aware, reflective agents that operate sequentially over video data — enabling a new class of VAD agents.\n\n**2. First to unify training-free + online VAD**\n\nPANDA is, to our knowledge, the **first training-free MLLM-based VAD system that supports online detection**, addressing a critical practical need. In contrast:\n\n- **ZS CLIP**, **LLAVA-1.5**, **LAVAD**, **AnomalyRuler** — all require offline processing and manually tuned or learned prompts;\n- None support self-reflection, dynamic rule retrieval, or temporal memory;\n- None operate without scene-specific annotations or tuning.\n\n**3. System-level design *as* contribution**\n\nPANDA unifies RAG, tools, reasoning, and memory into a **dynamic and autonomous agent**, shifting from static pipelines to **reasoning-driven learning**. This design is non-trivial and central to PANDA’s contribution.\n\nWe will further emphasize these points in the revised Introduction and Section 3.\n\n---\n\n**Q4: Typos.**\n\nWe sincerely thank the reviewer for pointing out the typos. We will correct all typos such as \"planing\" → \"planning\" and \"Kowleage\" → \"Knowledge\". We will also thoroughly revise and proofread the entire paper and supplementary materials to ensure a polished and professional presentation in the final version.\n\n---\n\n**Q5: Privacy and societal impact.**\n\nThank you for the reminder. While PANDA does not collect personal identifiers, we acknowledge potential risks if used inappropriately (e.g., unauthorized surveillance, biased rules).\n\nWe will update the societal impact section to discuss:\n- The importance of transparent and auditable rule design;\n- Ethical deployment boundaries;\n- Bias mitigation through automated rule auditing."}]

Author rebuttal:

Meta-review: The reviewers and the AC concur that the paper demonstrates sufficient novelty, provides detailed implementation, and presents strong results.  This paper went through extensive discussion. 
AC recommends acceptance and request authors to incorporate all reviewers queries in the final version.
Decision: Accept (poster)