Title: A Unified Reasoning Framework for Holistic Zero-Shot Video Anomaly Analysis
Venue: NeurIPS 2025
Accepted: yes
Keywords: Video Understanding

Vision-Language Model

Test-time Reasoning

Anomaly Detection
Abstract: Most video-anomaly research stops at frame-wise detection, offering little insight into why an event is abnormal, typically outputting only frame-wise anomaly scores without spatial or semantic context. Recent video anomaly localization and video anomaly understanding methods improve explainability but remain data-dependent and task-specific. We propose a unified reasoning framework that bridges the gap between temporal detection, spatial localization, and textual explanation. Our approach is built upon a chained test-time reasoning process that sequentially connects these tasks, enabling holistic zero-shot anomaly analysis without any additional training. Specifically, our approach leverages intra-task reasoning to refine temporal detections and inter-task chaining for spatial and semantic understanding, yielding improved interpretability and generalization in a fully zero-shot manner. Without any additional data or gradients, our method achieves state-of-the-art zero-shot performance across multiple video anomaly detection, localization, and explanation benchmarks. The results demonstrate that careful prompt design with task-wise chaining can unlock the reasoning power of foundation models, enabling practical, interpretable video anomaly analysis in a fully zero-shot manner. Project Page: https://rathgrith.github.io/Unified_Frame_VAA/.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses video anomaly detection and analysis with a training-free framework. The framework consists of two main components: Intra-Task Reasoning (IntraTR) and Inter-Task Chaining (InterTC).  In IntraTR, the authors use pre-trained VAD models and Video LLMs to generate a list of possible anomalies as refined prompts to LLMs for reasoning. In InterTC, the authors leverage VLMs to generate bounding boxes for anomalies and draw them on the frames, followed by another VLMs with refined prompts for analysis. The authors have conducted experiments on the UCF-Crime, XD-Violence and UBNormal dataset.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n- The proposed method is straightforward and likely effective.\n- Drawing bounding boxes on frames as clues for VLMs is interesting.\n- The experimental results are promising.\n\n\n\n**Weaknesses:**\n- **Concerns about Proposed Method**\n    - The novelty of IntraTR is minor because it simply takes captions from VLMs and refined prompts (anomaly tags) as inputs for LLMs to perform VAD on ambiguous clips. In addition, the design can work if the VLM is powerful enough to find anomalies in video clips and the VAD baseline is good enough to localize ambiguous segments. \n- **Concerns about Performance Improvement**\n    - Given that the authors adopt better VLM on captioning/tag generation (e.g., LAVAD uses BLIP-2, an image-based model, and this paper uses VideoLLaMA3-7B, a video-based model), the performance gain could come from the VLM rather than the design of the IntraTR. For example, the third row in Table 3(a) shows that using frozen VLM and LLM can achieve comparable performance (80.38%) as LAVAD (80.28%) on UCF, showing that the proposed method benefits from the strong VLM/LLM. \n    - Table 3(a) does not clearly show how effective adding anomaly tags is. \n- **Concerns about Experiments**\n    - The component name in Table 3(a) and the paragraph (line261-271) do not match the corresponding name or appear in the method section (e.g., I could not find “soft-margin” and the authors use “Score-based reasoning gate” instead of “Score-gated Reasoning”).   In addition, the design of the simplest baseline (first row in Table 3(a)) is unclear but the performance is surprisingly good. Finally, the difference between LLM-Scoring and Prior-Reasoning is unclear because they both ask LLM to provide the scores and I did not see a reasoning prompt such as \"explain what happens in the scene\" in $p_{VAD}$ in line 531. The unclear table and descriptions make the reviewer hard to understand how effective each component is. \n    - The authors claim that the proposed method is computation efficiency (line 169) but do not provide detailed analysis. I would like to see the computation complexity comparison between this work and other baselines. \n   - In general, the refined prompt with generated tags is why VAD performance improves. Recognizing the anomaly tags for the video clips provides LLMs useful information for anomaly detection and understanding. It would be good if the authors can provide qualitative results of anomaly tags or refined prompts. Showing those results can strengthen the technical reliability of the proposed method. \n- **Potential Anomaly Clips**\n    - The authors set the window size to a fixed value for each video (line 237). Because the framework highly relies on the selection of anomaly clips, I wonder if some clips may not be fully captured with a fixed window size. In addition, as IntraTR only updates the prompt to refine anomaly scores, the temporal boundaries of the anomaly clips remain the same throughout testing, limiting the improvement of VAD.\n\nQUESTIONS:\n**Important questions and concerns for the authors to address. Responses will strongly influence the final rating**\n- Elaborate more on the novelty of IntraTR. Explain what the results will be if VLMs cannot capture correct suspicious behaviors (tags) or VAD baseline wrongly localizes the ambiguous segments. Providing experiments can be helpful but is not required.\n- Address the concerns for Table 3 mentioned in the weakness section.\n- Provide computational analysis to prove the claim in line 169.\n\n\n\n**Optional questions and concerns for the authors to address. Good responses can increase the final rating.**\n- Provide qualitative results for anomaly tags or refined prompts $p^*_{VAD}$\n- Analyze what clips may not be fully captured with a fixed window size (Potential Anomaly Clips in the weakness section). Discuss possible solutions that can change the anomaly boundaries to find better anomaly segments. \n\nPlease find the relevant detail for some questions in the weakness section.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-23"}

comments: SUMMARY:
This paper proposes a unified, training-free reasoning framework for video anomaly analysis, aiming to integrate three key tasks: Video Anomaly Detection (VAD), spatial localization (VAL), and semantic understanding (VAU), thereby enabling comprehensive zero-shot anomaly analysis. Unlike previous methods that focus solely on temporal detection or a single sub-task, this framework employs a chained task reasoning mechanism that sequentially infers "when, where, and why" anomalies occur during inference—without any additional training or fine-tuning.

STRENGTHS AND WEAKNESSES:
Strengths：
1. This paper is well-motivated, as it tackles the anomaly reasoning task under a training-free paradigm.
2. This paper is easy to read. 

Weaknesses：
1. The authors claim to address the sub-task of spatial understanding; however, I could not find any quantitative evaluation of spatial performance in the main paper. Moreover, in the qualitative visualizations provided in the appendix, it is unclear where the ground-truth annotations for the spatial data come from.
2. The authors apply Gaussian smoothing as a post-processing step, but I have two concerns. In addition to the standard deviation $\sigma$, Gaussian smoothing also involves a window size parameter, which is not specified in the paper. Furthermore, the authors do not conduct a sensitivity analysis regarding the choice of window size.
3. How do the authors obtain anomaly scores for every segment of the video if they only sample 180 frames?
4. According to the methodology, the authors use both VLMs and LLMs, but the framework diagram does not clearly indicate which specific model is used at each stage.

QUESTIONS:
See in the weaknesses section.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 2
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-30

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a Unified Reasoning Framework for Holistic Zero‑Shot Video Anomaly Analysis, which leverages frozen vision–language models (VLMs) and large language models (LLMs) to perform, without any additional training, three core anomaly tasks: Temporal Video Anomaly Detection (VAD) via an Intra‑Task Reasoning (IntraTR) pipeline that computes frame‑wise anomaly scores, extracts contextual “anomaly tags”.Spatial Video Anomaly Localization (VAL) by chaining the VAD priors (tag list and refined scores) into a frozen VLM localization head, yielding bounding boxes for anomalous regions;Textual Video Anomaly Understanding (VAU)\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- This work presents a novel zero-shot approach for anomaly detection\n- Method validation across multiple anomaly detection formsFramework demonstrates functionality on various modes of anomaly detection like VAU,VAD and VAL\n-Testing conducted on three benchmarks (CCTV, dash-cam/movie violence, synthetic scenes) with performance metrics compared against existing methods. Results show 91.3% AUC on XD-Violence versus 85.4% for baseline approaches.\n-Systematic ablation analysis - This work examines individual component contributions including LLM scoring, tag priors, gated refinement, and fixed versus adaptive gating margins.\n\n**Weaknesses**\n- Absence of zero-shot CoT baseline comparisons No empirical evaluation against alternative zero-shot chain-of-thought approaches.\n- Dataset contamination UCF-Crime, XD-Violence, and UBnormal datasets may overlap with model pretraining data. Evaluation lacks recent benchmarks or challenging scenarios that would validate generalization claims. Authors could refer to more recent datasets like MSAD[1]\n- Insufficient justification for modular architecture : Separation of vision (VLM) and reasoning (LLM) components lacks empirical support. No comparison with end-to-end multimodal models (GPT-4V, VideoLLaMA) to assess performance trade-offs or computational efficiency.\n[1] Advancing Video Anomaly Detection: A Concise Review and a New Dataset.Liyun Zhu, Lei Wang, Arjun Raj, Tom Gedeon, Chen Chen\n\nQUESTIONS:\n**Suggestions**- Overall, the quality of the work can be improved by comparing the proposed zero-shot technique with other prompting methods, especially Chain-of-Thought (CoT) and recently released reasoning-based models. In addition, the issue of data contamination also needs to be addressed to fully validate the zero-shot performance of the proposed method.\n- **Zero‑Shot CoT Baselines**  Addition of chain‑of‑thought (CoT) baseline would help.  \n\n- **Data Contamination** :- To avoid data contamination issues authors could provide results using LLM/VLM  released before a dataset.\n \n- **Ablate Monolithic Multimodal LLMs**   Evaluate an end‑to‑end GPT‑4V or VideoLLaMA pipeline without discrete VLM/LLM modules.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a unified reasoning framework for video anomaly analysis. By leveraging intra-task reasoning and inter-task chaining, the approach integrates temporal detection, spatial localization, and semantic explanation into a unified framework, improving  both interpretability and generalization. The framework achieves state-of-the-art zero-shot performance on multiple benchmarks, demonstrating the potential of foundation models, when guided by well-designed prompts, to deliver practical and interpretable video anomaly analysis.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths：\n1. The paper proposes a unified framework that seamlessly integrates multiple tasks in video anomaly analysis, including detection, localization, and explanation.\n2. The approach achieves state-of-the-art performance in a zero-shot setting, without requiring any additional training or fine-tuning, which highlights its strong generalization capability.\n\nWeaknesses：\n1. Since anomaly events can be either very brief or span extended durations, how sensitive is the method’s performance to the choice of hyperparameter l? A discussion or analysis on this would be helpful.\n2. In Table 2, the adaptive m setting does not outperform the fixed m version. Could the authors provide more insight into why adaptive m yields inferior results in this case?\n3. The proposed test-time reasoning involves multiple sequential steps and the use of multiple vision-language models (VLMs). What is the overall inference speed?\n4. The equations in the paper should be numbered for easier reference and clarity.\n\nQUESTIONS:\n1. How sensitive is the method to the choice of hyperparameter l, given the varying duration of anomaly events? A brief analysis would be helpful.\n\n2. In Table 2, why does adaptive m underperform compared to fixed m? Please explain the possible reasons.\n\n3. What is the inference speed of the full test-time pipeline involving multiple VLMs? Any runtime or resource analysis would be useful.\n\nLIMITATIONS:\nThe paper does not explicitly discuss the limitations or potential societal impacts of the proposed method. The authors are encouraged to:\n\nDiscuss the computational cost and inference latency, especially given the use of multiple VLMs at test time.\n\nAddress possible failure cases, such as ambiguity in anomaly definition or limitations in zero-shot generalization to unseen domains.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-05"}