Title: CLUE-VAD: Structured Semantic Clues for Understanding Explainable Events in Video Anomaly Detection
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: Video Anomaly Detection

Abstract: Weakly Supervised Video Anomaly Detection (WSVAD) aims to identify rare and abnormal events in long untrimmed videos using only video-level labels. While recent approaches have leveraged multimodal learning and pretrained language models, they often treat scenes holistically, failing to provide fine-grained or interpretable insights into the source of anomalies. In this paper, we introduce CLUE-VAD, a novel framework that explicitly decomposes each video segment into three semantically grounded components—Action, Environment, and Object—termed as Textual CLUEs. This structured decomposition enables the model to disentangle overlapping contextual cues and reason about anomalies in a human-aligned and interpretable manner. Our approach comprises three key modules: (i) the Witness Module, which automatically generates dense, clue-specific captions and CLUE-based features using a large-scale video-language model; (ii) the Detective Module, which employs a learnable clue-aware fusion mechanism to dynamically quantify the importance of each semantic clue for anomaly prediction; and (iii) the Reporter Module, which provides fine-grained explanations by attributing anomaly scores to specific keywords and clues. We also construct the CLUE-VAD Benchmark, an enriched evaluation resource with structured segment-level captions for existing WSVAD datasets. Experiments on UCF-Crime and XD-Violence demonstrate that CLUE-VAD achieves strong performance in text-only settings while offering transparent and context-aware anomaly reasoning. Our framework bridges the gap between machine prediction and human interpretation, making it a practical and trustworthy solution for real-world surveillance.