GenAI4ELab/papercli-papers huggingface dataset:
Pixel-Wise Shuffling with Collaborative Sparsity for Melanoma Hyperspectral Image Classification
Authors: Favour Ekong; Jun Zhou; Kwabena Sarpong; Yongsheng Gao
Venue: WACV 2025 (main)

Abstract: Hyperspectral imaging has emerged as a promising technology for medical image classification particularly in skin cancer diagnosis. However current methods face significant challenges in accurately and robustly classifying non-cancerous skin lesions especially when melanoma lesions overlap with pigmented regions. Existing methods also lack sensitivity to spectral variations and accumulate excess redundant data leading to inefficiencies misclassifications and overfitting while struggling to integrate spatial and spectral information effectively. To overcome these challenges we propose a novel method featuring collaborative sparse unmixing and an advanced pixel-wise shuffling approach with inter-similarity hybrid attention aiming to improve the accuracy of skin cancer diagnosis in real-world scenarios. Experiments are conducted on a publicly available histology-verified dataset to evaluate the efficacy of the proposed method. The experimental results demonstrate that the proposed method can accurately classify melanoma lesions even in cases where the lesions overlap with pigmented regions. The findings indicate that the proposed method outperforms state-of-the-art methods by obtaining an overall accuracy of 73.34% even when limited to 20% of the training data. The proposed approach has the potential to be a valuable tool for improving the diagnostic accuracy of skin cancer in clinical practice.
WINE : Wavelet-Guided GAN Inversion and Editing for High-Fidelity Refinement
Authors: Chaewon Kim; Seung Jun Moon; Gyeong-Moon Park
Venue: WACV 2025 (main)

Abstract: Recent advanced GAN inversion models aim to convey high-fidelity information from original images to generators through methods using generator tuning or high-dimensional feature learning. Despite these efforts accurately reconstructing image-specific details remains as a challenge due to the inherent limitations both in terms of training and structural aspects leading to a bias towards low-frequency information. In this paper we look into the widely used pixel loss in GAN inversion revealing its predominant focus on the reconstruction of low-frequency features. We then propose WINE a Wavelet-guided GAN inversion and editing model which transfers the high-frequency information through wavelet coefficients via newly proposed wavelet loss and wavelet fusion scheme. Notably WINE is the first attempt to interpret GAN inversion in the frequency domain. Our experimental results showcase the precision of WINE in preserving high-frequency details and enhancing image quality. Even in editing scenarios WINE outperforms existing state-of-the-art GAN inversion models with a fine balance between editability and reconstruction quality. Pre-trained model and codes will be publicized after the review process.
Multi-Spectral Image Color Reproduction
Authors: Jiacheng Li; Chang Chen; Xue Hu; Fenglong Song; Youliang Yan; Zhiwei Xiong
Venue: WACV 2025 (main)

Abstract: From camera to screen researchers have developed a well-established system for capturing and reproducing the color experience of human eyes. In this study we aim to upgrade this process by transiting from conventional RGB to multi-spectral image (MSI) color reproduction. While MSI offers evident advantages in color matching we find out it is not trivial to make good use of more spectral information for color constancy. Therefore we present a regularized color reproduction system that incorporates a spectral prior-guided optimization strategy to establish a sensor-optimized RGB projection for color matching along with a learning-based chromatic adaptation model for color constancy. Specifically we define the RGB projection through an end-to-end optimization under the guidance of sensor spectral sensitivities. Subsequently we devise a chromatic adaptation neural network that estimates the scene illuminance and an illuminance-adaptive matrix for auto white balancing and dynamic color correction respectively. Comprehensive experiments show the superiority of our system compared to alternative solutions.
Low-Frequency Black-Box Backdoor Attack via Evolutionary Algorithm
Authors: Yanqi Qiao; Dazhuang Liu; Rui Wang; Kaitai Liang
Venue: WACV 2025 (main)

Abstract: Convolutional Neural Networks (CNNs) that have excelled in diverse computer vision tasks are vulnerable to backdoor attacks enabling attacker-controlled predictions via specific triggers. Restricted to spatial domains recent research exploits perceptual traits by embedding triggers in the frequency domain yielding pixel-level indistinguishable perturbations. In black-box settings restricted access to model and training process necessitates advanced trigger designs. Current frequency-based attacks manipulate magnitude spectra introducing discrepancies between clean and poisoned data though vulnerable to common image processing operations like compression and filtering. In this paper we propose a robust low-frequency backdoor attack (LFBA) in black-box setup that minimally perturbs spectrum components and maintains the perceptual similarity in spatial space simultaneously. Our methodology capitalizes on the insight that optimal triggers can be located in low-frequency regions to maximize attack effectiveness robustness against image transformation operations and stealthiness in dual space. To effectively explore the discrete frequency space we utilize simulated annealing (SA) a form of evolutionary algorithm to optimize the properties of trigger including the frequency bands to be manipulated and the perturbation of each band under restricted attack scenario. Extensive experiments on both CNNs and Vision Transformers (ViT) confirm the effectiveness and robustness of LFBA against image processing operations and state-of-the-art backdoor defenses. Furthermore LFBA exhibits inherent stealthiness in both spatial and frequency spaces making it resistant to human and frequency inspection.
DepthSSC: Monocular 3D Semantic Scene Completion via Depth-Spatial Alignment and Voxel Adaptation
Authors: Jiawei Yao; Jusheng Zhang; Xiaochao Pan; Tong Wu; Canran Xiao
Venue: WACV 2025 (main)

Abstract: The task of 3D semantic scene completion using monocular cameras is gaining significant attention in the field of autonomous driving. This task aims to predict the occupancy status and semantic labels of each voxel in a 3D scene from partial image inputs. Despite numerous existing methods many face challenges such as inaccurately predicting object shapes and misclassifying object boundaries. To address these issues we propose DepthSSC an advanced method for semantic scene completion using only monocular cameras. DepthSSC integrates the Spatial Transformation Graph Fusion (ST-GF) module with Geometric-Aware Voxelization (GAV) enabling dynamic adjustment of voxel resolution to accommodate the geometric complexity of 3D space. This ensures precise alignment between spatial and depth information effectively mitigating issues such as object boundary distortion and incorrect depth perception found in previous methods. Evaluations on the SemanticKITTI and SSCBench-KITTI-360 dataset demonstrate that DepthSSC not only captures intricate 3D structural details effectively but also achieves state-of-the-art performance.
MDCN-PS: Monocular-Depth-Guided Coarse Normal Attention for Robust Photometric Stereo
Authors: Masahiro Yamaguchi; Takashi Shibata; Shoji Yachida; Keiko Yokoyama; Toshinori Hosoi
Venue: WACV 2025 (main)

Abstract: Photometric Stereo (PS) is a technique for estimating surface normals from images illuminated by multiple light sources. However when the target object has a complex shape or the light sources are not appropriately arranged certain regions may experience severe shadows leading to insufficient information for accurate estimation. In this paper we propose a Monocular-Depth-guided Coarse Normal attention for Photometric Stereo (MDCN-PS). The MDCN-PS can effectively combine monocular depth from a single image with PS with multiple light sources by a Photometric Stereo network Adaptor (PS Adaptor) with Coarse Normal Attention. The key is to use the coarse normals obtained from Monocular Depth Estimation as supplementary information which can improve accuracy in regions where the light source is limited due to severe shadows or inhomogeneous light source distribution. Comprehensive experiments on real-world and synthetic datasets show that the proposed method achieved an accuracy improvement of 1.2 points in real-world datasets when limited to two input images and of 3.1 points in synthetic datasets in mean angular error compared to existing methods. Qualitative results also demonstrated that our method improves accuracy in areas with insufficient lighting patterns due to shadows.
MONAS-ESNN: Multi-Objective Neural Architecture Search for Efficient Spiking Neural Networks
Authors: Esmat Ghasemi Saghand; Susana K. Lai-Yuen
Venue: WACV 2025 (main)

Abstract: Spiking Neural Networks (SNNs) have emerged as a compelling alternative to traditional Artificial Neural Networks (ANNs) due to their energy efficiency and biological plausibility. However current SNN models often rely on ANN architectures that may not fully exploit the unique properties of SNNs. Neural Architecture Search (NAS) approaches have been shown to automate the identification of suitable architectures for various applications. Nevertheless very few works have been presented on NAS for SNNs and particularly for identifying architectures that achieve high accuracy while capitalizing on the energy efficiency property of SNNs. In this paper we present a Multi-Objective Neural Architecture Search for Efficient SNNs (MONAS-ESNN) approach that utilizes a training-free NAS to discover SNN architectures that optimize both accuracy and energy efficiency. The proposed MONAS-ESNN uses the NSGA-II evolutionary algorithm to optimize both objective functions while leveraging the unique temporal dynamics of SNNs. We also introduce a new Adjusted Sparsity-Aware Hamming Distance (ASAHD) that enhances the evaluation of potential architectures by representing diverse spike activation patterns for different types of spiking neurons. Experimental results on CIFAR-10 CIFAR-100 and Tiny-ImageNet-200 datasets demonstrate that MONAS-ESNN identifies SNN architectures that have higher accuracy and are more efficient as measured by the number of generated spikes than existing methods. Therefore the proposed MONAS-ESNN can automate the search and discovery of SNN architectures with higher accuracy fewer generated spikes and faster convergence paving the way for more energy-efficient neural networks.
SPACE: SPAtial-Aware Consistency rEgularization for Anomaly Detection in Industrial Applications
Authors: Daehwan Kim; Hyungmin Kim; Daun Jeong; Sungho Suh; Hansang Cho
Venue: WACV 2025 (main)

Abstract: In this paper we propose SPACE a novel anomaly detection methodology that integrates a Feature Encoder (FE) into the structure of the Student-Teacher method. The proposed method has two key elements: Spatial Consistency regularization Loss (SCL) and Feature converter Module (FM). SCL prevents overfitting in student models by avoiding excessive imitation of the teacher model. Simultaneously it facilitates the expansion of normal data features by steering clear of abnormal areas generated through data augmentation. This dual functionality ensures a robust boundary between normal and abnormal data. The FM prevents the learning of ambiguous information from the FE. This protects the learned features and enables more effective detection of structural and logical anomalies. Through these elements SPACE is available to minimize the influence of the FE while integrating various data augmentations. In this study we evaluated the proposed method on the MVTec LOCO MVTec AD and VisA datasets. Experimental results through qualitative evaluation demonstrate the superiority of detection and efficiency of each module compared to state-of-the-art methods.
Exploring Scalability of Self-Training for Open-Vocabulary Temporal Action Localization
Authors: Jeongseok Hyun; Su Ho Han; Hyolim Kang; Joon-Young Lee; Seon Joo Kim
Venue: WACV 2025 (main)

Abstract: The vocabulary size in temporal action localization (TAL) is limited by the scarcity of large-scale annotated datasets. To overcome this recent works integrate vision-language models (VLMs) such as CLIP for open-vocabulary TAL (OV-TAL). However despite the success of VLMs trained on extensive datasets existing OV-TAL methods still rely on human-labeled TAL datasets of limited size to train action localizers limiting their generalizability. In this paper we explore the scalability of self-training with unlabeled YouTube videos for OV-TAL. Our approach consists of two stages: (1) a class-agnostic action localizer is trained on a human-labeled TAL dataset to generate pseudo-labels for unlabeled videos and (2) the large-scale pseudo-labeled dataset is then used to train the localizer. Extensive experiments demonstrate that leveraging web-scale videos in self-training significantly enhances the generalizability of an action localizer. Additionally we identify limitations in existing OV-TAL evaluation schemes and propose a new benchmark for thorough assessment. Finally we showcase the TAL performance of the large multimodal model Gemini-1.5 on our new benchmark. Code is released at https://github.com/HYUNJS/STOV-TAL.
CISOL: An Open and Extensible Dataset for Table Structure Recognition in the Construction Industry
Authors: David Tschirschwitz; Volker Rodehorst
Venue: WACV 2025 (main)

Abstract: Reproducibility and replicability are critical pillars of empirical research particularly in machine learning where they depend not only on the availability of models but also on the datasets used to train and evaluate those models. In this paper we introduce the Construction Industry Steel Ordering List (CISOL) dataset which was developed with a focus on transparency to ensure reproducibility replicability and extensibility. CISOL provides a valuable new research resource and highlights the importance of having diverse datasets even in niche application domains such as table extraction in civil engineering. CISOL is unique in that it contains real-world civil engineering documents from industry making it a distinctive contribution to the field. The dataset contains more than 120000 annotated instances in over 800 document images positioning it as a medium-sized dataset that provides a robust foundation for Table Structure Recognition (TSR) and Table Detection (TD) tasks. Benchmarking results show that CISOL achieves 67.22 mAP@0.5:0.95:0.05 using the YOLOv8 model outperforming the TSR-specific TATR model. This highlights the effectiveness of CISOL as a benchmark for advancing TSR especially in specialized domains.
NestedMorph: Enhancing Deformable Medical Image Registration with Nested Attention Mechanisms
Authors: Gurucharan Marthi Krishna Kumar; Janine Mendola; Amir Shmuel
Venue: WACV 2025 (main)

Abstract: Deformable image registration is crucial for aligning medical images in a nonlinear fashion across different modalities allowing for precise spatial correspondence between varying anatomical structures. This paper presents NestedMorph a novel network utilizing a Nested Attention Fusion approach to improve intra-subject deformable registration between T1-weighted (T1w) MRI and diffusion MRI (dMRI) data. NestedMorph integrates high-resolution spatial details from an encoder with semantic information from a decoder using a multi-scale framework enhancing both local and global feature extraction. Our model notably outperforms existing methods including CNN-based approaches like VoxelMorph MIDIR and CycleMorph as well as Transformer-based models such as TransMorph and ViT-V-Net and traditional techniques like NiftyReg and SyN. Evaluations using the HCP dataset demonstrate that NestedMorph achieves superior performance across key metrics including SSIM HD95 and SDlogJ with the highest SSIM of 0.89 the lowest HD95 of 2.5 and SDlogJ of 0.22. These results highlight NestedMorph's ability to capture both local and global image features effectively leading to superior registration performance. The promising outcomes of this study underscore NestedMorph's potential to significantly advance deformable medical image registration providing a robust framework for future research and clinical applications. The source code and our implementation are available at: https://github.com/AS-Lab/Marthi-et-al-2024-NestedMorph-Deformable-Medical-Image-Registration
Now You See Me: Context-Aware Automatic Audio Description
Authors: Seon-Ho Lee; Jue Wang; David Fan; Zhikang Zhang; Linda Liu; Xiang Hao; Vimal Bhat; Xinyu Li
Venue: WACV 2025 (main)

Abstract: Audio Description (AD) plays a pivotal role as an application system aimed at guaranteeing accessibility in multimedia content which provides additional narrations at suitable intervals to describe visual elements catering specifically to the needs of visually impaired audiences. In this paper we introduce CA3D the pioneering unified Context-Aware Automatic Audio Description system that provides AD event scripts with precise locations in the long cinematic content. Specifically CA3D system consists of: 1) a Temporal Feature Enhancement Module to efficiently capture longer term dependencies 2) an anchor-based AD event detector with feature suppression module that localizes the AD events and extracts discriminative feature for AD generation and 3) a self-refinement module that leverages the generated output to tweak AD event boundaries from coarse to fine. Unlike conventional methods which rely on metadata and ground truth AD timestamp for AD detection and generation tasks the proposed CA3D is the first end-to-end trainable system that only uses visual cue. Extensive experiments demonstrate that the proposed CA3D improves existing architectures for both AD event detection and script generation metrics establishing the new state-of-the-art performances in the AD automation.
Geometry-Aware Deep Learning for 3D Skeleton-Based Motion Prediction
Authors: Mayssa Zaier; Hazem Wannous; Hassen Drira
Venue: WACV 2025 (main)

Abstract: The field of human motion prediction in computer vision faces challenges especially in 3D Skeleton-based Human Motion. Deep learning models albeit successful in most vision tasks were designed for data characterized by an underlying Euclidean structure which is not always fulfilled as pre-processed data may often reside in a non-linear space. Conventional RNNs struggle with capturing long-term dependencies in motion contexts. Our novel approach focuses on geometry-aware deep learning to predict the motion. We use a compact manifold-valued representation of 3D human skeleton motion integrating self-attention in transformer networks. This representation maps motions to points on a manifold ensuring smooth and coherent long-term predictions. Combining Kendall's shape space for non-rigid deformation and Lie group for rigid deformation provides a complete transformation. Experiments on various datasets demonstrate superiority over state-of-the-art methods in both short and long-term horizons.
Feature Space Perturbation: A Panacea to Enhanced Transferability Estimation
Authors: Prafful Kumar Khoba; Zijian Wang; Chetan Arora; Mahsa Baktashmotlagh
Venue: WACV 2025 (main)

Abstract: Leveraging a transferability estimation metric facilitates the non-trivial challenge of selecting the optimal model for the downstream task from a pool of pre-trained models. Most existing metrics primarily focus on identifying the statistical relationship between feature embeddings and the corresponding labels within the target dataset but overlook crucial aspect of model robustness. This oversight may limit their effectiveness in accurately ranking pre-trained models. To address this limitation we introduce a feature perturbation method that enhances the transferability estimation process by systematically altering the feature space. Our method includes a Spread operation that increases intra-class variability adding complexity within classes and an Attract operation that minimizes the distances between different classes thereby blurring the class boundaries. Through extensive experimentation we demonstrate the efficacy of our feature perturbation method in providing a more precise and robust estimation of model transferability. Notably the existing LogMe method exhibited a significant improvement showing a 28.84% increase in performance after applying our feature perturbation method. The implementation is available at https://github.com/prafful-kumar/enhancing_TE.git
Radiance Field-Based Pose Estimation via Decoupled Optimization Under Challenging Initial Conditions
Authors: Si-Yu Lu; Yung-Yao Chen; Yi-Tong Wu; Hsin-Chun Lin; Sin-Ye Jhong; Wen-Huang Cheng
Venue: WACV 2025 (main)

Abstract: Estimating six-degree-of-freedom poses is essential but difficult especially under challenging initial conditions as existing methods relying on photometric loss often fail catastrophically. To address this we propose a novel radiance field-based pose estimation framework that combines Monte Carlo initialization and decoupled optimization. First we design an initialization stage based on Monte Carlo sampling which screens the current optimal initial pose by generating multiple pose hypotheses within the known scene space. Subsequently our decoupled optimization strategy independently refines the rotational and translational components significantly improving the motion stability of the camera during the convergence process. Comprehensive evaluations on synthetic and real-world datasets demonstrate that our method outperforms existing approaches in terms of pose regression accuracy and robustness even under challenging initial pose.
DN-Splatter: Depth and Normal Priors for Gaussian Splatting and Meshing
Authors: Matias Turkulainen; Xuqian Ren; Iaroslav Melekhov; Otto Seiskari; Esa Rahtu; Juho Kannala
Venue: WACV 2025 (main)

Abstract: High-fidelity 3D reconstruction of common indoor scenes is crucial for VR and AR applications. 3D Gaussian splatting a novel differentiable rendering technique has achieved state-of-the-art novel view synthesis results with high rendering speeds and relatively low training times. However its performance on scenes commonly seen in indoor datasets is poor due to the lack of geometric constraints during optimization. In this work we explore the use of readily accessible geometric cues to enhance Gaussian splatting optimization in challenging ill-posed and textureless scenes. We extend 3D Gaussian splatting with depth and normal cues to tackle challenging indoor datasets and showcase techniques for efficient mesh extraction. Specifically we regularize the optimization procedure with depth information enforce local smoothness of nearby Gaussians and use off-the-shelf monocular networks to achieve better alignment with the true scene geometry. We propose an adaptive depth loss based on the gradient of color images improving depth estimation and novel view synthesis results over various baselines. Our simple yet effective regularization technique enables direct mesh extraction from the Gaussian representation yielding more physically accurate reconstructions of indoor scenes. Our code will be released in https://github.com/maturk/dn-splatter.