Title: Generalizing Single-Frame Supervision to Event-Level Understanding for Video Anomaly Detection
Venue: NeurIPS 2025
Accepted: yes
Keywords: Video Anomaly Detection

Inexact Supervision

Single Frame Supervision
Abstract: Video Anomaly Detection (VAD) aims to identify abnormal frames from discrete events within video sequences. Existing VAD methods suffer from heavy annotation burdens in fully-supervised paradigm, insensitivity to subtle anomalies in semi-supervised paradigm, and vulnerability to noise in weakly-supervised paradigm. To address these limitations, we propose a novel paradigm: Single-Frame supervised VAD (SF-VAD), which uses a single annotated abnormal frame per abnormal video. SF-VAD ensures annotation efficiency while offering precise anomaly reference, facilitating robust anomaly modeling, and enhancing the detection of subtle anomalies in complex visual contexts. To validate its effectiveness, we construct three SF-VAD benchmarks by manually re-annotating the ShanghaiTech, UCF-Crime, and XD-Violence datasets in a practical procedure. Further, we devise Frame-guided Progressive Learning (FPL), to generalize sparse frame supervision to event-level anomaly understanding. FPL first leverages evidential learning to estimate anomaly relevance guided by annotated frames. Then it extends anomaly supervision by mining discrete abnormal events based on anomaly relevance and feature similarity. Meanwhile, FPL decouples normal patterns by isolating distinct normal frames outside abnormal events, reducing false alarms.  Extensive experiments show SF-VAD achieves state-of-the-art detection results while offering a favorable trade-off between performance and annotation cost.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Single-Frame Supervised Video Anomaly Detection (SF-VAD), using one annotated abnormal frame per video to balance annotation efficiency and detection precision. The Frame-guided Progressive Learning (FPL) framework estimates anomaly relevance via evidential learning and generalizes supervision to event-level by mining abnormal intervals and decoupling normal patterns. Three SF-VAD benchmarks are constructed, and experiments show superior AUC and lower false alarm rates compared to prior methods. However, limitations in multi-event robustness and computational analysis weaken its broader impact.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1.Novel Supervision Paradigm: SF-VAD reduces annotation cost significantly while providing fine-grained anomaly guidance, addressing a critical challenge in VAD.\n\n2.Effective Two-Stage Framework: FPL’s progressive learning from sparse frames to events demonstrates strong generalizability, as shown in ablation studies (Table 3).\n\n3.Dataset Contribution: The manually re-annotated SF-VAD benchmarks offer practical single-frame supervision datasets for future research.\n\nWeaknesses:\n\n1.Limited Multi-Event Analysis: The paper lacks quantitative results on videos with complex, overlapping abnormal events (e.g., UCF-Crime’s “Riot” class), leaving uncertainty about FPL’s scalability.\n\n2.Computational Overhead Omission: No comparison of inference time or model size with lightweight baselines (e.g., Conv-AE) is provided, hindering assessment of practical deployment feasibility.\n\n3.Incomplete Baseline Comparison: Missing direct comparisons with recent semi-supervised methods (e.g., MULDE, CVPR 2024) limits clarity on SF-VAD’s positioning in the VAD landscape.\n\n4.Theoretical Underpinnings: The evidential learning formulation (Sec. 3.3) lacks intuitive justification for its choice over standard probabilistic models, making it harder for readers to contextualize.\n\nQUESTIONS:\n1.lti-Event Robustness Test: Provide AUC scores for video clips with ≥3 disjoint abnormal events in UCF-Crime to validate FPL’s ability to handle temporal complexity.\n\n2.Computational Efficiency Metrics: Report inference time per video and model parameters compared to baselines (e.g., PEL4VAD, VadCLIP) to clarify trade-offs between performance and cost.\n\n3.Semi-Supervised Baseline Addition: Include results from recent semi-supervised methods (e.g., MULDE, SWAD) to explicitly contrast SF-VAD’s reliance on minimal abnormal labels.\n\n4.Evidential Learning Justification: Add a brief comparison with standard cross-entropy loss in Stage I to explain why evidential learning better captures anomaly uncertainty.\n\nLIMITATIONS:\nThe authors mention limitations in the supplemental material (e.g., reliance on feature similarity), but they do not adequately discuss how SF-VAD performs when the annotated frame is ambiguous or non-representative of the full anomaly. This oversight raises concerns about the method’s robustness in noisy real-world scenarios.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new paradigm for video anomaly detection under single-frame supervision (SF-VAD), where only one anomalous frame per anomalous video needs to be manually labeled. The authors design a Frame-guided Progressive Learning (FPL) framework to generalize sparse single-frame supervision to event-level anomaly detection. Extensive experiments on multiple mainstream datasets demonstrate the method’s excellent trade-off between detection performance and labeling efficiency.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The two-stage FPL framework combines evidence learning and feature similarity, expanding sparse labels to full anomaly event intervals.\n2. The paper provides detailed performance comparisons and ablation studies, validating the method’s effectiveness.\n\nWeaknesses & Questions:\n\n1. The “single-frame annotation” setting actually contradicts the fundamental reality in VAD that anomalies are hard to annotate or acquire. In this setting, model learning shifts from “finding classification boundaries between normal and anomalous samples in many clips” to “finding the top-K segments most similar to a labeled anomalous frame within a single video”. This essentially transforms the anomaly detection problem into a retrieval task under strong supervision, weakening its generalization and induction challenge.\n2. The choice of anomalous frame has a huge impact on performance. Annotators may not always select the most representative frames; labeling near event boundaries or ambiguous frames could hinder the model from learning typical anomaly patterns. The authors should detail the annotation process and provide case studies showing how the method handles boundary or unclear frames, and analyze the model’s robustness.\n3. Some videos (e.g., in XD-Violence) may contain multiple types or stages of anomalies. Relying on a single frame is insufficient to cover the full semantic and temporal diversity of anomaly events. The authors should explain how their method handles multiple anomaly cases in detail.\n4. I have concerns about the generalization ability of the proposed method. Compared to semi-supervised VAD, weakly-supervised VAD sacrifices labeling cost for greater generalization, yet this method’s training heavily relies on known anomalous frame features. If a novel anomaly type appears, or if the anomaly differs significantly from the labeled frame, detection performance may drop sharply—generalization is questionable.\n5. The framework is based on the classic two-stage MIL for weakly-supervised VAD, but the model structure is overly complex and engineering-oriented, with limited originality.\n\nQUESTIONS:\nPlease respond to each item of the weaknesses mentioned above.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

comments: SUMMARY:
The authors introduce a new annotation regime for video-anomaly detection, Single-Frame supervised VAD, in which each abnormal video is annotated with only one abnormal frame.  They (i) re-label three popular benchmarks—ShanghaiTech, UCF-Crime and XD-Violence—to create SF-VAD testbeds, and propose Frame-guided Progressive Learning.

STRENGTHS AND WEAKNESSES:
This paper formalise a new anomaly detection benchmark: Single-Frame supervised VAD. Which allevaite siginificant amount of annotation works but achieve good performances. 

Weakness: 

The mining seem to relies on several thresholds. Could the authors provide some insight or ablations to how to choose the best thresholds. 

The performance of this Single-Frame-level supervision achieve higher performances than the fully supervised approaches. This seems counter intuitive. Could the authors leverage on this and discuss the potential reasons? 

One of the argument for anomaly detection is that the normal events are more controlled and common in real world. Hence, annotating the normal video/frames is a lot easier than abnormals. Collecting the annotation for single frame anomalies are also time-consuming from my perspective, you need to locate which frame from a raw video contain anomalies, which some times even harder than the video annotations in the WVAD setups. This could be alleviate if the model trained on certain type of single frame anomalies could generalize to unseen anomalous events.

QUESTIONS:
If you train on ShanghaiTech SF-VAD but test on UCF-Crime without fine-tuning, how does performance change?  This relates to my previous question on the single-frame anomaly generalization. 

Does FPL still help when using a lightweight CNN or ViT backbone?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces an innovative Single-Frame Supervised Video Anomaly Detection (SF-VAD) paradigm that significantly enhances annotation efficiency by utilizing just one annotated abnormal frame per abnormal video, substantially reducing the annotation workload compared to traditional fully-supervised methods. To generalize this sparse supervision for comprehensive understanding of entire abnormal events, the authors propose the Frame-Guided Progressive Learning (FPL) framework, which consists of two stages: first estimating the relevance of each frame to the annotated abnormal frame, and then extending this supervision to multiple abnormal events while decoupling normal patterns to minimize false alarms. The method is validated on three newly constructed SF-VAD benchmark datasets, demonstrating superior performance in detecting both obvious and subtle anomalies with reduced false alarm rates.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nPotential to Address Practical Issues: The SF-VAD paradigm significantly enhances annotation efficiency while maintaining detection performance, which holds great significance in real-world applications where annotation resources are limited.\nRigorous Methodology: The paper proposes an innovative SF-VAD paradigm and effectively generalizes sparse frame-level supervision to event-level anomaly understanding through the FPL framework. The theoretical derivations and experimental validations are relatively rigorous, ensuring the reliability of the method.\nWeaknesses:\nInadequate Discussion of Limitations: Although the paper highlights the advantages of the method, it fails to fully discuss its potential limitations or restrictions in applicable scenarios. For instance, the SF-VAD method may not perform well for certain specific types of anomalies.\nCode Requires Organization and Standardization: While the paper promises to make the data and code publicly available, it is advisable to organize and standardize the code to ensure that other researchers can easily reproduce and utilize it. For example, following best practices of open-source projects, such as providing a clear README file, detailed comments, and documentation. Additionally, currently, there is no introduction to the dataset configuration in the paper, requiring reviewers to configure it themselves.\n\nQUESTIONS:\nClarification on Dataset Configuration:\nDescription: The paper mentions constructing three SF-VAD benchmark datasets but does not provide detailed information on how these datasets were configured, including data sources, annotation process, and data splits.\nSuggestion: Please provide a comprehensive description of the dataset configuration in the supplementary material or an appendix. Include details such as the criteria for selecting videos, the annotation process (e.g., how annotators were instructed to label the first abnormal frame), and any preprocessing steps applied to the data.\nImpact on Evaluation: If this information is provided, it will significantly enhance the reproducibility of your work and allow other researchers to build upon your datasets. This could increase my evaluation score if the dataset configuration is well-documented and transparent.\nDiscussion on Method Limitations:\nDescription: While the paper highlights the advantages of the SF-VAD paradigm, it does not fully discuss its limitations or potential failure cases.\nSuggestion: Please include a section in the paper that explicitly discusses the limitations of the SF-VAD method. For example, mention scenarios where the method might struggle (e.g., videos with very low-salience anomalies or those with high noise levels) and provide insights into why these limitations exist.\nImpact on Evaluation: A thorough discussion of limitations will demonstrate a mature understanding of the method's scope and applicability. This could positively influence my evaluation score, especially if you also propose potential solutions or future work directions to address these limitations.\nCode Organization and Documentation:\nDescription: The paper promises to release the data and code but does not mention the current state of the codebase or any efforts made to ensure its reproducibility.\nSuggestion: Please organize and document the code following best practices for open-source projects. This includes providing a clear README file with installation instructions, usage examples, and a description of the code structure. Additionally, include comments within the code to explain key components and algorithms.\nImpact on Evaluation: Well-organized and documented code will greatly facilitate the reproducibility of your work. If the code is released in a user-friendly manner, it could increase my evaluation score by demonstrating your commitment to open science and reproducibility.\nAblation Studies on Different Anomaly Types:\nDescription: The ablation studies section provides valuable insights into the effectiveness of different components of the proposed method. However, it does not specifically analyze the performance on different types of anomalies (e.g., subtle anomalies vs. obvious anomalies).\nSuggestion: Please extend the ablation studies to include an analysis of the method's performance on different types of anomalies. This could involve categorizing anomalies based on their salience or other characteristics and evaluating how well the SF-VAD method performs on each category.\nImpact on Evaluation: A detailed analysis of performance on different anomaly types will provide a more nuanced understanding of the method's strengths and weaknesses. This could positively influence my evaluation score if the method demonstrates robustness across a diverse range of anomalies.\nComparison with State-of-the-Art Methods on Additional Datasets:\nDescription: The paper compares the SF-VAD method with state-of-the-art methods on three datasets. However, it would be beneficial to see how the method performs on additional datasets to further validate its generalizability.\nSuggestion: Please consider evaluating the SF-VAD method on additional publicly available video anomaly detection datasets. Compare its performance with existing methods on these datasets and discuss any differences in performance.\nImpact on Evaluation: Demonstrating the method's generalizability across multiple datasets will strengthen its credibility and applicability. If the SF-VAD method consistently outperforms existing methods on additional datasets, it could significantly increase my evaluation score.\n\nLIMITATIONS:\nPotential Negative Societal Impacts:\nDescription: The paper does not explicitly discuss the potential negative societal impacts that may arise from deploying the SF-VAD method in real-world systems. This includes concerns related to privacy, surveillance, and potential biases in anomaly detection.\nSuggestion: Add a dedicated section to discuss the ethical considerations and potential negative societal impacts of the SF-VAD method. Address issues such as privacy concerns (e.g., detecting behaviors that may be considered private or sensitive), surveillance implications (e.g., misuse in authoritarian regimes), and potential biases (e.g., racial, gender, or socioeconomic biases manifesting in anomaly detection). Provide guidelines or recommendations for mitigating these impacts.\nRobustness Against Adversarial Attacks:\nDescription: The paper does not discuss the robustness of the SF-VAD method against adversarial attacks, which is an important factor to consider when deploying AI systems in security-critical applications.\nSuggestion: Include an analysis of the method's robustness against adversarial attacks. This may involve testing the method on videos subjected to adversarial perturbations and discussing potential defense strategies or methods to enhance robustness.\nCross-Domain Generalization Capability:\nDescription: The paper primarily focuses on specific datasets and does not extensively discuss the generalization capability of the SF-VAD method across different domains or video types (e.g., surveillance footage, social media videos, medical videos).\nSuggestion: Expand the evaluation to include diverse datasets from different domains. Discuss the challenges faced by the method in performing well across various video types and the potential adaptive adjustments that may be required.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-05"}