GenAI4ELab/papercli-papers huggingface dataset:
Uni-SLAM: Uncertainty-Aware Neural Implicit SLAM for Real-Time Dense Indoor Scene Reconstruction
Authors: Shaoxiang Wang; Yaxu Xie; Chun-Peng Chang; Christen Millerdurai; Alain Pagani; Didier Stricker
Venue: WACV 2025 (main)

Abstract: Neural implicit fields have recently emerged as a powerful representation method for multi-view surface reconstruction due to their simplicity and state-of-the-art performance. However reconstructing thin structures of indoor scenes while ensuring real-time performance remains a challenge for dense visual SLAM systems. Previous methods do not consider varying quality of input RGB-D data and employ fixed-frequency mapping process to reconstruct the scene which could result in the loss of valuable information in some frames. In this paper we propose Uni-SLAM a decoupled 3D spatial representation based on hash grids for indoor reconstruction. We introduce a novel defined predictive uncertainty to reweight the loss function along with strategic local-to-global bundle adjustment. Experiments on synthetic and real-world datasets demonstrate that our system achieves state-of-the-art tracking and mapping accuracy while maintaining real-time performance. It significantly improves over current methods with a 25% reduction in depth L1 error and a 66.86% completion rate within 1 cm on the Replica dataset reflecting a more accurate reconstruction of thin structures. Project page: https://shaoxiang777.github.io/project/uni-slam/
MS-Glance: Bio-Inspired Non-Semantic Context Vectors and their Applications in Supervising Image Reconstruction
Authors: Ziqi Gao; Wendi Yang; Yujia Li; Lei Xing; S. Kevin Zhou
Venue: WACV 2025 (main)

Abstract: Non-semantic context information is crucial for visual recognition as the human visual perception system first uses global statistics to process scenes rapidly before identifying specific objects. However while semantic information is increasingly incorporated into computer vision tasks such as image reconstruction non-semantic information such as global spatial structures is often overlooked. To bridge the gap we propose a biologically informed non-semantic context descriptor MS-Glance along with the Glance Index Measure for comparing two images. A Global Glance vector is formulated by randomly retrieving pixels based on a perception-driven rule from an image to form a vector representing non-semantic global context while a local Glance vector is a flattened local image window mimicking a zoom-in observation. The Glance Index is defined as the inner product of two standardized sets of Glance vectors. We evaluate the effectiveness of incorporating Glance supervision in two reconstruction tasks: image fitting with implicit neural representation (INR) and undersampled MRI reconstruction. Extensive experimental results show that MS-Glance outperforms existing image restoration losses across both natural and medical images. The code is available at https://github.com/Z7Gao/MSGlance.
DAM: Dynamic Adapter Merging for Continual Video QA Learning
Authors: Feng Cheng; Ziyang Wang; Yi-Lin Sung; Yan-Bo Lin; Mohit Bansal; Gedas Bertasius
Venue: WACV 2025 (main)

Abstract: We present a parameter-efficient method for continual video question-answering (VidQA) learning. Our method named DAM uses the proposed Dynamic Adapter Merging to (i) mitigate catastrophic forgetting (ii) enable efficient adaptation to continually arriving datasets (iii) handle inputs from unknown datasets during inference and (iv) enable knowledge sharing across similar dataset domains. Given a set of continually streaming VidQA datasets we sequentially train dataset-specific adapters for each dataset while freezing the parameters of a large pretrained video-language backbone. During inference given a video-question sample from an unknown domain our method first uses the proposed non-parametric router function to compute a probability for each adapter reflecting how relevant that adapter is to the current video-question input instance. Subsequently the proposed dynamic adapter merging scheme aggregates all the adapter weights into a new adapter instance tailored for that particular test sample to compute the final VidQA prediction mitigating the impact of inaccurate router predictions and facilitating knowledge sharing across domains. Our DAM model outperforms prior state-of-the-art continual learning approaches by 9.1% while exhibiting 1.9% less forgetting on 6 VidQA datasets spanning various domains. We further extend DAM to continual image classification and image QA and outperform prior methods by a large margin. The code will be publicly released.
VG-SSL: Benchmarking Self-Supervised Representation Learning Approaches for Visual Geo-Localization
Authors: Jiuhong Xiao; Gao Zhu; Giuseppe Loianno
Venue: WACV 2025 (main)

Abstract: Visual Geo-localization (VG) is a critical research area for identifying geo-locations from visual inputs particularly in autonomous navigation for robotics and vehicles. Current VG methods often learn feature extractors from geo-labeled images to create dense geographically relevant representations. Recent advances in Self-Supervised Learning (SSL) have demonstrated its capability to achieve performance on par with supervised techniques with unlabeled images. This study presents a novel VG-SSL framework designed for versatile integration and benchmarking of diverse SSL methods for representation learning in VG featuring a unique geo-related pair strategy GeoPair. Through extensive performance analysis we adapt SSL techniques to improve VG on datasets from hand-held and car-mounted cameras used in robotics and autonomous vehicles. Our results show that contrastive learning and information maximization methods yield superior geo-specific representation quality matching or surpassing the performance of state-of-the-art VG techniques. To our knowledge This is the first benchmarking study of SSL in VG highlighting its potential in enhancing geo-specific visual representations for robotics and autonomous vehicles. The code is publicly available at https://github.com/arplaboratory/VG-SSL.
Multi-Label Continual Learning for the Medical Domain: A Novel Benchmark
Authors: Marina Ceccon; Davide Dalle Pezze; Alessandro Fabris; Gian Antonio Susto
Venue: WACV 2025 (main)

Abstract: Despite the critical importance of the medical domain in Deep Learning most of the research in this area solely focuses on training models in static environments. It is only in recent years that research has begun to address dynamic environments and tackle the Catastrophic Forgetting problem through Continual Learning (CL) techniques. Previous studies have primarily focused on scenarios such as Domain Incremental Learning and Class Incremental Learning which do not fully capture the complexity of real-world applications. Therefore in this work we propose a novel benchmark combining the challenges of new class arrivals and domain shifts in a single framework by considering the New Instances and New Classes (NIC) scenario. This benchmark aims to model a realistic CL setting for the multi-label classification problem in medical imaging. Additionally it encompasses a greater number of tasks compared to previously tested scenarios. Specifically our benchmark consists of two datasets (NIH and CXP) nineteen classes and seven tasks a stream longer than the previously tested ones. To solve common challenges (e.g. the task inference problem) found in the CIL and NIC scenarios we propose a novel approach called Replay Consolidation with Label Propagation (RCLP). Our method surpasses existing approaches exhibiting superior performance with minimal forgetting.
Uniform Attention Maps: Boosting Image Fidelity in Reconstruction and Editing
Authors: Wenyi Mo; Tianyu Zhang; Yalong Bai; Bing Su; Ji-Rong Wen
Venue: WACV 2025 (main)

Abstract: Text-guided image generation and editing using diffusion models have achieved remarkable advancements. Among these tuning-free methods have gained attention for their ability to perform edits without extensive model adjustments offering simplicity and efficiency. However existing tuning-free approaches often struggle with balancing fidelity and editing precision. Reconstruction errors in DDIM Inversion are partly attributed to the cross-attention mechanism in U-Net which introduces misalignments during the inversion and reconstruction process. To address this we analyze reconstruction from a structural perspective and propose a novel approach that replaces traditional cross-attention with uniform attention maps significantly enhancing image reconstruction fidelity. Our method effectively minimizes distortions caused by varying text conditions during noise prediction. To complement this improvement we introduce an adaptive mask-guided editing technique that integrates seamlessly with our reconstruction approach ensuring consistency and accuracy in editing tasks. Experimental results demonstrate that our approach not only excels in achieving high-fidelity image reconstruction but also performs robustly in real image composition and editing scenarios. This study underscores the potential of uniform attention maps to enhance the fidelity and versatility of diffusion-based image processing methods. Code is available at https://github.com/Mowenyii/Uniform-Attention-Maps.
Design Principles of Multi-Scale J-Invariant Networks for Self-Supervised Image Denoising
Authors: Hayeong Yu; Seungjae Han; Young-Gyu Yoon
Venue: WACV 2025 (main)

Abstract: Recent advancements in image denoising have leveraged neural networks to enhance performance particularly in scenarios where clean-noisy image pairs are unavailable. In this context self-supervised image denoising methods have gained prominence centered around the principle of J-invariance -- ensuring that the output pixel is not influenced by its corresponding input pixel. Traditionally enforcing J-invariance has constrained blind spot network (BSN) designs requiring even core operations such as upsampling or downsampling to follow complex rules. This limitation has led to the exclusion of efficient multi-resolution architectures such as U-net increasing computational complexity. To address these constraints we introduce generalized design principles for multi-scale J-invariant networks that allow for the flexible incorporation of nearly any architectural elements. This approach challenges the prevailing notion that J-invariance must be maintained throughout the entire process. Based on our design principles we present U-BSN a novel J-invariant network design that utilizes the versatile U-Net architecture adapting it to accommodate self-supervised learning effectively. We also propose randomized PD an advanced technique that enhances denoising of real-world images with structured noise. Experimental results validate that U-BSN surpasses existing BSNs in handling real-world noise scenarios and achieves the lowest computational complexity among comparable networks thus confirming the effectiveness of our design principles and proposed methodologies.
FMD: Comprehensive Data Compression in Medical Domain via Fused Matching Distillation
Authors: Ju Heon Son; Jang-Hwan Choi
Venue: WACV 2025 (main)

Abstract: Medical datasets are often large and contain sensitive information presenting significant challenges for data sharing and storage. To address these issues this paper introduces a novel method called Fused Matching Distillation (FMD) which combines multiple dataset distillation techniques to achieve both data compression and enhanced privacy. FMD synthesizes representative subsets that capture the essential information from the original dataset while anonymizing sensitive details during the distillation process. The proposed approach integrates two complementary methods: parameter matching a technique that aligns the training trajectories of a teacher network trained on real data with those of a student network trained on synthetic data and feature distribution matching which ensures that the synthetic dataset closely approximates the feature distribution of the original data. By fusing these techniques FMD maximizes the information density within each pixel of the distilled dataset achieving a balance between compression and performance. Experimental evaluations on medical datasets including COVID chest X-ray and Pancreas cancer CT demonstrate that FMD achieves superior accuracy and privacy compared to existing methods. Furthermore the proposed method is evaluated using metrics for both model performance and anonymity showing that FMD not only maintains high diagnostic accuracy but also effectively anonymizes the data. This makes FMD a promising tool for secure and efficient medical data sharing.
Inferring Past Human Actions in Homes with Abductive Reasoning
Authors: Clement Tan; Chai Kiat Yeo; Cheston Tan; Basura Fernando
Venue: WACV 2025 (main)

Abstract: Abductive reasoning aims to make the most likely inference for a given set of incomplete observations. In this paper we introduce "Abductive Past Action Inference" a novel research task aimed at identifying the past actions performed by individuals within homes to reach specific states captured in a single image using abductive inference. The research explores three key abductive inference problems: past action set prediction past action sequence prediction and abductive past action verification. We introduce several models tailored for abductive past action inference including a relational graph neural network a relational bilinear pooling model and a relational transformer model. Notably the newly proposed object-relational bilinear graph encoder-decoder (BiGED) model emerges as the most effective among all methods evaluated demonstrating good proficiency in handling the intricacies of the Action Genome dataset. The contributions of this research significantly advance the ability of deep learning models to reason about current scene evidence and make highly plausible inferences about past human actions. This advancement enables a deeper understanding of events and behaviors which can enhance decision-making and improve system capabilities across various real-world applications such as Human-Robot Interaction and Elderly Care and Health Monitoring. Code and data available at https://github.com/LUNAProject22/AAR
J-Invariant Volume Shuffle for Self-Supervised Cryo-Electron Tomogram Denoising on Single Noisy Volume
Authors: Xiwei Liu; Mohamad Kassab; Min Xu; Qirong Ho
Venue: WACV 2025 (main)

Abstract: Cryo-Electron Tomography (Cryo-ET) enables detailed 3D visualization of cellular structures in near-native states but suffers from low signal-to-noise ratio due to imaging constraints. Traditional denoising methods and supervised learning approaches often struggle with complex noise patterns and the lack of paired datasets. Self-supervised methods which utilize noisy input itself as a target have been studied; however existing Cryo-ET self-supervised denoising methods face significant challenges due to losing information during training and the learned incomplete noise patterns. In this paper we propose a novel self-supervised learning model that denoises Cryo-ET volumetric images using a single noisy volume. Our method features a U-shape J-invariant blind spot network with sparse centrally masked convolutions dilated channel attention blocks and volume-unshuffle/shuffle technique. The volume-unshuffle/shuffle technique expands receptive fields and utilizes multi-scale representations significantly improving noise reduction and structural preservation. Experimental results demonstrate that our approach achieves superior performance compared to existing methods advancing Cryo-ET data processing for structural biology research.
CACE: Sim-to-Real Indoor 3D Semantic Segmentation via Context-Aware Augmentation and Consistency Enforcement
Authors: Tsung-Yu Chen; Luyu Yang; Tzu-Yu Chuang; Shang-Hong Lai
Venue: WACV 2025 (main)

Abstract: Indoor 3D domain adaptation for semantic segmentation is an understudied task. The first unsupervised sim-to-real benchmark was only proposed recently. Existing methods try to modify the source domain data by simulating the occlusion and noise pattern of the target domain. However this methodology unrealistically demands a clear definition of the real-world data patterns and is highly dependent on the simulation quality. In this paper we propose a novel adaptation framework via Context-aware Augmentation and Consistency Enforcement (CACE). Our CACE framework consists of two modules a space and context-aware augmentation module that is invariant of target data pattern and domain gaps and a carefully designed self-supervision module that maximizes the utility of the augmented data. Our CACE surpasses the state-of-the-art method by over 6% on the indoor 3D sim-to-real benchmark 3D-FRONT - ScanNet.
OmniDiffusion: Reformulating 360 Monocular Depth Estimation using Semantic and Surface Normal Conditioned Diffusion
Authors: Payal Mohadikar; Ye Duan
Venue: WACV 2025 (main)

Abstract: Depth estimation is the fundamental computer vision task for scene analysis. With the emergence of the deep learning era supervised monocular image depth estimation (MDE) became a popular choice for the task. Predominantly MDE methods utilize 360 images as ideal input due to their comprehensive field of view scene content compared to perspective images but they suffer from distortions in polar regions making it a more challenging ill-posed problem to date. Over the years methods using CNNs and/or large transformers taking 360 and/or projected perspective patch inputs have been proposed to solve the 360 MDE problem by formulating it as a regression or a classification task. Nevertheless their performance still suffers from global discrepancy inaccuracy poor details and generalizability. Lately diffusion-generating models have shown state-of-the-art performance in image synthesis that captures exceptionally rich knowledge of the visual world. However their ability to perform omnidirectional perception tasks is still unexplored. In this paper we explore a new approach called OmniDiffusion that reformulates the 360 MDE task as a diffusion denoising process. We present a diffusion-based framework to learn an iterative denoising process that denoises random depth distribution into the required depths. The diffusion process is performed in the latent space and uses the guidance of encoded RGB image visual as a condition. Furthermore to advance the image latent in a geometrically meaningful direction we leverage semantic segmentation and surface normal information to provide a more detailed contextual assistance to the denoising process. The performed experiments on the multiple real-world datasets show that our diffusion-denoising approach with the proposed conditions more appropriately refines depths outperforming the existing MDE and diffusion-based methods with state-of-the-art generalization ability while generating more accurate high-quality and detailed 360 depths.
Pre-Capture Privacy via Adaptive Single-Pixel Imaging
Authors: Yoko Sogabe; Shiori Sugimoto; Ayumi Matsumoto; Masaki Kitahara
Venue: WACV 2025 (main)

Abstract: As cameras become ubiquitous in our living environment invasion of privacy is becoming a significant concern. A common approach to privacy preservation is to remove personally identifiable information from a captured image but there is a risk of the original image being leaked. In this paper we propose a pre-capture privacy-aware imaging method that captures images from which the details of a pre-specified anonymized target have been eliminated. The proposed method applies a single-pixel imaging framework in which we introduce a feedback mechanism called an aperture pattern generator (APG). The introduced APG adaptively outputs the next aperture pattern to avoid sampling the anonymized target by using already acquired data as a clue. Furthermore the anonymized target can be set to any object without changing hardware. Except for the removed detailed features of the anonymized target the captured images are of comparable quality to those captured by a general camera and can be used for various computer vision applications. We target faces and license plates and experimentally show that the proposed method can capture clear images in which detailed features of the anonymized target are eliminated achieving both privacy and utility.
Fairer Analysis and Demographically Balanced Face Generation for Fairer Face Verification
Authors: Alexandre Fournier-Montgieux; Michaël Soumm; Adrian Popescu; Bertrand Luvison; Hervé Le Borgne
Venue: WACV 2025 (main)

Abstract: Face recognition and verification are two computer vision tasks whose performances have advanced with the introduction of deep representations. However ethical legal and technical challenges due to the sensitive nature of face data and biases in real-world training datasets hinder their development. Generative AI addresses privacy by creating fictitious identities but fairness problems remain. Using the existing DCFace SOTA framework we introduce a new controlled generation pipeline that improves fairness. Through classical fairness metrics and a proposed in-depth statistical analysis based on logit models and ANOVA we show that our generation pipeline improves fairness more than other bias mitigation approaches while slightly improving raw performance.
Towards Utilising a Range of Neural Activations for Comprehending Representational Associations
Authors: Laura O'Mahony; Nikola S. Nikolov; David JP O'Sullivan
Venue: WACV 2025 (main)

Abstract: Recent efforts to understand intermediate representations in deep neural networks have commonly attempted to label individual neurons and combinations of neurons that make up linear directions in the latent space by examining extremal neuron activations and the highest direction projections. In this paper we show that this approach although yielding a good approximation for many purposes fails to capture valuable information about the behaviour of a representation. Neural network activations are generally dense and so a more complex but realistic scenario is that linear directions encode information at various levels of stimulation. We hypothesise that non-extremal level activations contain complex information worth investigating such as statistical associations and thus may be used to locate confounding human interpretable concepts. We explore the value of studying a range of neuron activations by taking the case of mid-level output neuron activations and demonstrate on a synthetic dataset how they can inform us about aspects of representations in the penultimate layer not evident through analysing maximal activations alone. We use our findings to develop a method to curate data from mid-range logit samples for retraining to mitigate spurious correlations or confounding concepts in the penultimate layer on real benchmark datasets. The success of our method exemplifies the utility of inspecting non-maximal activations to extract complex relationships learned by models.