GenAI4ELab/papercli-papers huggingface dataset:
Graph-Jigsaw Conditioned Diffusion Model for Skeleton-Based Video Anomaly Detection
Authors: Ali Karami; Thi Kieu Khanh Ho; Narges Armanfard
Venue: WACV 2025 (main)

Abstract: Skeleton-based video anomaly detection (SVAD) is a crucial task in computer vision. Accurately identifying abnormal patterns or events enables operators to promptly detect suspicious activities thereby enhancing safety. Achieving this demands a comprehensive understanding of human motions both at body and region levels while also accounting for the wide variations of performing a single action. However existing studies fail to simultaneously address these crucial properties. This paper introduces a novel practical and lightweight framework namely Graph-Jigsaw Conditioned Diffusion Model for Skeleton-based Video Anomaly Detection (GiCiSAD) to overcome the challenges associated with SVAD. GiCiSAD consists of three novel modules: the Graph Attention-based Forecasting module to capture the spatio-temporal dependencies inherent in the data the Graph-level Jigsaw Puzzle Maker module to distinguish subtle region-level discrepancies between normal and abnormal motions and the Graph-based Conditional Diffusion model to generate a wide spectrum of human motions. Extensive experiments on four widely used skeleton-based video datasets show that GiCiSAD outperforms existing methods with significantly fewer training parameters establishing it as the new state-of-the-art.
Federated Source-Free Domain Adaptation for Classification: Weighted Cluster Aggregation for Unlabeled Data
Authors: Junki Mori; Kosuke Kihara; Taiki Miyagawa; Akinori F. Ebihara; Isamu Teranishi; Hisashi Kashima
Venue: WACV 2025 (main)

Abstract: Federated learning (FL) commonly assumes that the server or some clients have labeled data which is often impractical due to annotation costs and privacy concerns. Addressing this problem we focus on a source-free domain adaptation task where (1) the server holds a pre-trained model on labeled source domain data (2) clients possess only unlabeled data from various target domains and (3) the server and clients cannot access the source data in the adaptation phase. This task is known as Federated source-Free Domain Adaptation (FFREEDA). Specifically we focus on classification tasks while the previous work solely studies semantic segmentation. Our contribution is the novel Federated learning with Weighted Cluster Aggregation (FedWCA) method designed to mitigate both domain shifts and privacy concerns with only unlabeled data. Fed-WCA comprises three phases: private and parameter-free clustering of clients to obtain domain-specific global models on the server weighted aggregation of the global models for the clustered clients and local domain adaptation with pseudo-labeling. Experimental results show that Fed-WCA surpasses several existing methods and baselines in FFREEDA establishing its effectiveness and practicality.
Active Learning for Vision Language Models
Authors: Bardia Safaei; Vishal M. Patel
Venue: WACV 2025 (main)

Abstract: Pre-trained vision-language models (VLMs) like CLIP have demonstrated impressive zero-shot performance on a wide range of downstream computer vision tasks. However there still exists a considerable performance gap between these models and a supervised deep model trained on a downstream dataset. To bridge this gap we propose a novel active learning (AL) framework that enhances the zero-shot classification performance of VLMs by selecting only a few informative samples from the unlabeled data for annotation during training. To achieve this our approach first calibrates the predicted entropy of VLMs and then utilizes a combination of self-uncertainty and neighbor-aware uncertainty to calculate a reliable uncertainty measure for active sample selection. Our extensive experiments show that the proposed approach outperforms existing AL approaches on several image classification datasets and significantly enhances the zero-shot performance of VLMs.
Metric Compatible Training for Online Backfilling in Large-Scale Retrieval
Authors: Seonguk Seo; Mustafa Gokhan Uzunbas; Bohyung Han; Sara Cao; Ser-Nam Lim
Venue: WACV 2025 (main)

Abstract: Backfilling is the process of re-extracting all gallery embeddings from upgraded models in image retrieval systems. It inevitably spends a prohibitively large amount of computational cost and even entails the downtime of the service. Although backward-compatible learning sidesteps this challenge by tackling query-side representations this leads to suboptimal solutions in principle because gallery embeddings cannot benefit from model upgrades. We address this dilemma by introducing an online backfilling algorithm which enables us to achieve a progressive performance improvement during the backfilling process without sacrificing the full performance of the new model after the completion of backfilling. To this end we first show that a simple distance rank merge is a reasonable option for online backfilling. Then we incorporate a reverse transformation module for more effective and efficient merging which is further enhanced by adopting metric-compatible contrastive learning. These two components help to make the distances of old and new models compatible resulting in desirable merge results during backfilling with no extra computational overhead. Extensive experiments show the benefit of our framework on four standard benchmarks in various settings.
GaitCloud: Leveraging Spatial-Temporal Information for LiDAR-Base Gait Recognition with A True-3D Gait Representation
Authors: Shaoxiong Zhang; Hiromitsu Awano; Takashi Sato
Venue: WACV 2025 (main)

Abstract: Gait recognition using point clouds captured by LiDAR (Light Detection And Ranging) sensors offers better adaptability to variations in walking conditions compared to camera-based methods due to the precise spatial information captured. However existing methods typically project the point clouds into a sequence of 2D depth images extended along the time dimension and adopt gait recognition networks optimized for camera-based approaches. This planar projection compromises the integrity of the 3D coordinates (length width and depth) and results in severe silhouette deformations with varied observation viewpoints similar to the camera-based methods. To better utilize the spatial information in gait point clouds we propose a true 3D gait representation using eff icient point cloud voxelization termed GaitCloud. Additionally we explore the unique nature of LiDAR-captured point clouds and present two improved modules adapted to our method called Layer Encoder (LE) and Horizontal Convolutional Pooling (HCP). Evaluation results using the open-access gait dataset SUSTech1K show that our method outperforms the state-of-the-art achieving recognition accuracies of 93.1% and 89.2% in cross-view and variance experiments respectively. These results demonstrate that 3D gait representation based on point cloud voxelization more effectively utilizes spatial information than depth images offering new possibilities for high-performance LiDAR-based gait recognition. The source code is available at https://github.com/seagrgz/GaitCloud-master.git.
Dequantization and Color Transfer with Diffusion Models
Authors: Vaibhav Vavilala; Faaris Shaik; David Forsyth
Venue: WACV 2025 (main)

Abstract: We demonstrate an image dequantizing diffusion model that enables novel edits on natural images. We propose operating on quantized images because they offer easy abstraction for patch-based edits and palette transfer. In particular, we show that color palettes can make the output of the diffusion model easier to control and interpret. We first establish that existing image restoration methods are not sufficient, such as JPEG noise reduction models. We then demonstrate that our model can generate natural images that respect the color palette the user asked for. For palette transfer, we propose a method based on weighted bipartite matching. We then show that our model generates plausible images even after extreme palette transfers, respecting user query. Our method can optionally condition on the source texture in part or all of the image. In doing so, we overcome a common problem in existing image colorization methods that are unable to produce colors with a different luminance than the input. We evaluate several possibilities for texture conditioning and their trade-offs, including luminance, image gradients, and thresholded gradients, the latter of which performed best in maintaining texture and color control simultaneously. Our method can be usefully extended to another practical edit: recoloring patches of an image while respecting the source texture. Our procedure is supported by several qualitative and quantitative evaluations.
Partial Filter-Sharing: Improved Parameter-Sharing Method for Single Image Super-Resolution Networks
Authors: Karam Park; Nam Ik Cho
Venue: WACV 2025 (main)

Abstract: Numerous deep learning techniques have been developed for Single Image Super-Resolution (SISR) leading to significant performance improvements. However these techniques have also resulted in a substantial increase in parameter size. As a result there is a growing interest in reducing network complexity for more practical usage while still maintaining high SR quality. One such method is parameter-sharing which includes recursive recurrent and multi-scale learning approaches. However sharing identical kernels across layers or up-scaling tasks can reduce the network's representational capacity. To address this we propose Partial filter-Sharing (PS) a new parameter-sharing method that preserves the network's representational power more effectively than previous approaches. Instead of sharing a single filter PS shares segments of filters called partial filters across layers. This approach enables parameter-sharing layers to use diverse filters for each layer or task striking a balance between parameter efficiency and the network's representational ability without imposing excessive computational or parameter overhead. Furthermore the PS framework provides precise control over the network's performance and complexity by adjusting the quantity of partial filters. Extensive experiments demonstrate that our PS framework outperforms traditional parameter-sharing super-resolution (SR) methods without incurring excessive additional parameters or computational cost.
Identify Backdoored Model in Federated Learning via Individual Unlearning
Authors: Jiahao Xu; Zikai Zhang; Rui Hu
Venue: WACV 2025 (main)

Abstract: Backdoor attacks present a significant threat to the robustness of Federated Learning (FL) due to their stealth and effectiveness. They maintain both the main task of the FL system and the backdoor task simultaneously causing malicious models to appear statistically similar to benign ones which enables them to evade detection by existing defense methods. We find that malicious parameters in backdoored models are inactive on the main task resulting in a significantly large empirical loss during the machine unlearning process on clean inputs. Inspired by this we propose MASA a method that utilizes individual unlearning on local models to identify malicious models in FL. To improve the performance of MASA in challenging non-independent and identically distributed (non-IID) settings we design pre-unlearning model fusion that integrates local models with knowledge learned from other datasets to mitigate the divergence in their unlearning behaviors caused by the non-IID data distributions of clients. Additionally we propose a new anomaly detection metric with minimal hyperparameters to filter out malicious models efficiently. Extensive experiments on IID and non-IID datasets across six different attacks validate the effectiveness of MASA. To the best of our knowledge this is the first work to leverage machine unlearning to identify malicious models in FL. Code is available at https://github.com/JiiahaoXU/MASA.
Multispectral Object Detection Enhanced by Cross-Modal Information Complementary and Cosine Similarity Channel Resampling Modules
Authors: Junbo Jang; Chanyeong Park; Heegwang Kim; Jiyoon Lee; Joonki Paik
Venue: WACV 2025 (main)

Abstract: Images obtained from different modalities can effectively enhance the accuracy and reliability of the detection model by complementing specialized information from visible (RGB) and infrared (IR) images. However integrating information from multiple modalities faces the following challenges: 1) distinct characteristics of RGB and IR images lead to the problem of modality imbalance 2) fusing multimodal information can greatly affect the detection accuracy as some of the unique information provided by each modality is lost during the integration process and 3) RGB and IR images are fused while preserving the noise of each modality. To address these issues we propose a novel multispectral object detection network which contains two main components; 1) Cross-modal Information Complementary (CIC) module and 2) Cosine Similarity Channel Resampling (CSCR) module. The proposed method addresses the modality imbalance problem and efficiently fuses RGB and IR images in the feature level. Extensive experimental results on LLVIP FLIR M3FD VEDAI and KAIST benchmark datasets verify the effectiveness and generalization performance of the proposed multispectral object detection network compared with other state-of-the-art methods.
Robust Long-Range Perception Against Sensor Misalignment in Autonomous Vehicles
Authors: Zi-Xiang Xia; Sudeep Fadadu; Yi Shi; Louis Foucard
Venue: WACV 2025 (main)

Abstract: Advances in machine learning algorithms for sensor fusion have significantly improved the detection and prediction of other road users thereby enhancing safety. However even a small angular displacement in the sensor's placement can cause significant degradation in output especially at long range. In this paper we demonstrate a simple yet generic and efficient multi-task learning approach that not only detects misalignment between different sensor modalities but is also robust against them for long-range perception. Along with the amount of misalignment our method also predicts calibrated uncertainty which can be useful for filtering and fusing predicted misalignment values over time. In addition we show that the predicted misalignment parameters can be used for self-correcting input sensor data further improving the perception performance under sensor misalignment.
Anomaly Detection for People with Visual Impairments using an Egocentric 360-Degree Camera
Authors: Inpyo Song; Sanghyeon Lee; Minjun Joo; Jangwon Lee
Venue: WACV 2025 (main)

Abstract: Recent advancements in computer vision have led to a renewed interest in developing assistive technologies for individuals with visual impairments. Although extensive research has been conducted in the field of computer vision-based assistive technologies most of the focus has been on understanding contexts in images rather than addressing their physical safety and security concerns. To address this challenge we propose the first step towards detecting anomalous situations for visually impaired people by observing their entire surroundings using an egocentric 360-degree camera. We first introduce a novel egocentric 360-degree video dataset called VIEW360 (Visually Impaired Equipped with Wearable 360-degree camera) which contains abnormal activities that visually impaired individuals may encounter such as shoulder surfing and pickpocketing. Furthermore we propose a new architecture called the FDPN (Frame and Direction Prediction Network) which facilitates frame-level prediction of abnormal events and identifying of their directions. Finally we evaluate our approach on our VIEW360 dataset and the publicly available UCF-Crime and Shanghaitech datasets demonstrating state-of-the-art performance.
TRNeRF: Restoring Blurry Rolling Shutter and Noisy Thermal Images with Neural Radiance Fields
Authors: Spencer Carmichael; Manohar Bhat; Mani Ramanagopal; Austin Buchan; Ram Vasudevan; Katherine A. Skinner
Venue: WACV 2025 (main)

Abstract: Thermal cameras offer unique detection capabilities in building inspections search and rescue operations and autonomous vehicle perception. Of the different types of thermal cameras uncooled microbolometers are often chosen due to their relative affordability small size and low power consumption. However microbolometers suffer from motion blur rolling shutter distortions and fixed pattern noise which limit the conditions of their use. Nearly all prior methods for microbolometer image restoration account for only one of these degradations and current techniques addressing microbolometer blur and rolling shutter are limited. This paper presents TRNeRF a thermal image restoration method that jointly addresses all three degradations by incorporating the microbolometer image formation model with Neural Radiance Fields (NeRFs). To evaluate TRNeRF this paper introduces a new real-world dataset that is uniquely designed to support two novel quantitative evaluation strategies for thermal image restoration. Experiments demonstrate that TRNeRF is able to recover sharp global shutter and clear thermal images even under extremely aggressive camera motion that causes existing methods to fail. The code and dataset are available at: https://umautobots.github.io/trnerf.
Unsupervised Domain Adaptive Visual Question Answering in the Era of Multi-Modal Large Language Models
Authors: Weixi Weng; Rui Zhang; Xiaojun Meng; Jieming Zhu; Qun Liu; Chun Yuan
Venue: WACV 2025 (main)

Abstract: Unsupervised domain adaptation (UDA) for visual question answering (VQA) has attracted research interest. However with Multi-modal Large Language Models (MLLMs) showing great performance on VQA datasets UDA for VQA based on MLLMs remains unexplored. To fill this gap we propose the first systematic approach to Unsupervised Domain Adaptation VQA based on MLLMs (UDAM). First we introduce semantic context feature alignment and domain query feature alignment which utilize a single token embedding for each modality to capture contextual domain information from unimodal inputs and conduct coarse-grained feature alignment on it thus alleviating domain shifts in the unimodal feature space. Second we propose the novel semantics-guided query feature alignment which differentiates important domain-specific queries from learnable query outputs and conducts fine-grained feature alignment controlled by a semantics-guided weight map to reduce domain shifts in the cross-modal feature space. Third we devise a pair-wise domain-aware prompt strategy which aids UDA by prompting MLLMs to discern the commonality of tasks and the distinctiveness of domains in multi-modal inputs. Extensive experiments demonstrate UDAM's effectiveness in adapting MLLMs to unlabeled new domains.
PULSE: Physiological Understanding with Liquid Signal Extraction
Authors: Shahzad Ahmad; Sania Bano; Sachin Verma; Yogesh Singh Rawat; Sukalpa Chanda; Santosh Kumar Vipparthi; Subrahmanyam Murala
Venue: WACV 2025 (main)

Abstract: The non-contact estimation of vital signs particularly heart rate from video data is a promising method for remote health monitoring. 3D convolutional layers are widely used for this task due to their ability to capture both spatial and temporal features. However traditional 3D convolutions while effective in many cases lack the capacity to adjust dynamically to the temporal variability inherent in physiological signals such as remote photoplethysmography (rPPG) which are characterized by subtle frequency changes over time. To address this we propose PULSE (Physiological Understanding with Liquid Signal Extraction) a framework that employs Liquid Time-Constant (LTC) models with 3D convolutional layers to enhance temporal sensitivity and improve the extraction of these fine-grained rPPG signals. In PULSE traditional 3D-conv layers are deployed for initial feature extraction while LTC-based 3D-conv layers dynamically adapt and guide the temporal processing allowing the model to better track and interpret the subtle variations in heart rate signals under different conditions such as motion artifacts and lighting changes. We evaluated the effectiveness of PULSE in an unsupervised training setting demonstrating that our solution performs well even in the absence of labeled datasets a common challenge in rPPG signal extraction. Experimental evaluations on three public datasets confirm that PULSE achieves comparable or superior results to existing methods proving its robustness and efficacy for real-world non-contact health monitoring applications.
On Which Data Distribution (Synthetic or Real) We Should Rely for Soft Biometric Classification
Authors: Manju R. A; Atul Kumar; Akshay Agarwal
Venue: WACV 2025 (main)

Abstract: Identification of gender is critical not only for human-computer interaction but also for scrutinizing the search space in which an identity needs to be determined. Traditionally "real" facial images are employed for gender identification by computer vision algorithms. Due to the tremendous rise of privacy and advancement in generative networks synthetic face images are heavily developed and can be used for several face-related studies including gender classification. However their effectiveness compared to real images is still unexplored for gender classification. In response this study explores the effectiveness of gender classification networks trained on real and synthetic face images offering novel insights into the effectiveness of these two data distributions. For that we implemented several state-of-the-art gender classification architectures covering convolutional neural networks (CNNs) and vision transformers (ViT). Our research builds on the rigorous evaluation of 8 Deep Neural Networks (DNNs) across 4 diverse datasets and 6 types of image corruptions. To make the research interpretable we have also used several explainable mechanisms including Grad-CAM and t-SNE visualizations. In brief the impact of the proposed research is multifold: (i) understand the effectiveness of real vs. synthetic data distributions in network training and (ii) whether the synthetic models reflect the true physical world distribution to ensure that the models trained on them are resilient against image perturbations.
SimuScope: Realistic Endoscopic Synthetic Dataset Generation through Surgical Simulation and Diffusion Models
Authors: Sabina Martyniak; Joanna Kaleta; Diego Dall'Alba; Michał Naskręt; Szymon Płotka; Przemysław Korzeniowski
Venue: WACV 2025 (main)

Abstract: Computer-assisted surgical (CAS) systems enhance surgical execution and outcomes by providing advanced support to surgeons. These systems often rely on deep learning models trained on complex challenging-to-annotate data. While synthetic data generation can address these challenges enhancing the realism of such data is crucial. This work introduces a multi-stage pipeline for generating realistic synthetic data featuring a fully-fledged surgical simulator that automatically produces all necessary annotations for modern CAS systems. This simulator generates a wide set of annotations that surpass those available in public synthetic datasets. Additionally it offers a more complex and realistic simulation of surgical interactions including the dynamics between surgical instruments and deformable anatomical environments outperforming existing approaches. To further bridge the visual gap between synthetic and real data we propose a lightweight and flexible image-to-image translation method based on Stable Diffusion (SD) and Low-Rank Adaptation (LoRA). This method leverages a limited amount of annotated data enables efficient training and maintains the integrity of annotations generated by our simulator. The proposed pipeline is experimentally validated and can translate synthetic images into images with real-world characteristics which can generalize to real-world context thereby improving both training and CAS guidance. The code and the dataset are available at https://github.com/SanoScience/SimuScope.