GenAI4ELab/papercli-papers huggingface dataset:
Spatially-Adaptive Hash Encodings for Neural Surface Reconstruction
Authors: Thomas Walker; Octave Mariotti; Amir Vaxman; Hakan Bilen
Venue: WACV 2025 (main)

Abstract: Positional encodings are a common component of neural scene reconstruction methods and provide a way to bias the learning of neural fields towards coarser or finer representations. Current neural surface reconstruction methods use a "one-size-fits-all" approach to encoding choosing a fixed set of encoding functions and therefore bias across all scenes. Current state-of-the-art surface reconstruction approaches leverage grid-based multi-resolution hash encoding in order to recover high-detail geometry. We propose a learned approach which allows the network to choose its encoding basis as a function of space by masking the contribution of features stored at separate grid resolutions. The resulting spatially adaptive approach allows the network to fit a wider range of frequencies without introducing noise. We test our approach on standard benchmark surface reconstruction datasets and achieve state-of-the-art performance on two benchmark datasets.
MagicStick: Controllable Video Editing via Control Handle Transformations
Authors: Yue Ma; Xiaodong Cun; Sen Liang; Jinbo Xing; Yingqing He; Chenyang Qi; Siran Chen; Qifeng Chen
Venue: WACV 2025 (main)

Abstract: Text-based video editing has recently attracted considerable interest in changing the style or replacing the objects with a similar structure. Beyond this we demonstrate that properties such as shape size location motion etc. can also be edited in videos. Our key insight is that the keyframe's transformations of the specific internal feature (e.g. edge maps of objects or human pose) can easily propagate to other frames to provide generation guidance. We thus propose MagicStick a controllable video editing method that edits the video properties by utilizing the transformation on the extracted internal control signals. In detail to keep the appearance we inflate both the pre-trained image diffusion model and ControlNet to the temporal dimension and train low-rank adaptions (LoRA) layers to fit the specific scenes. Then in editing we perform an inversion and editing framework. Differently finetuned ControlNet is introduced in both inversion and generation for attention guidance with the proposed attention remix between the spatial attention maps of inversion and editing. Yet succinct our method is the first method to show the ability of video property editing from the pre-trained text-to-image model. We present experiments on numerous examples within our unified framework. We also compare with shape-aware text-based editing and handcrafted motion video generation demonstrating our superior temporal consistency and editing capability than previous works.
D-LUT: Photorealistic Style Transfer via Diffusion Process
Authors: Mujing Li; Guanjie Wang; Xingguang Zhang; Qifeng Liao; Chenxi Xiao
Venue: WACV 2025 (main)

Abstract: Post-editing color in photographs is a crucial process for enhancing a photograph's aesthetic value. Traditionally this process has required a significant investment of time and manual effort. Previous color transfer algorithms achieved through encoder-decoder deep learning architectures have simplified this process. However these techniques may introduce artifacts and decrease image quality. Moreover previous approaches are not explainable making the method less user-friendly. In addition the computational requirements of these models limit their deployment across various devices. To address these challenges we introduce the Diffusion-based Look-Up Table (D-LUT). This approach is artifact-free explainable computationally efficient and does not require pretraining stage. It derives a 3D Look-Up Table (3D LUT) for transitioning between the color styles of different images. Specifically this 3D LUT is obtained using a score-matching algorithm followed by color distribution alignment through Langevin dynamics. Our proposed D-LUT approach has achieved state-of-the-art performance while requiring significantly less GPU memory than previous baselines. Importantly the 3D LUTs explicitly derived from the D-LUT algorithm enable color style transfer across broader visual modalities such as real-time color transfer for videos.
OTCXR: Rethinking Self-Supervised Alignment using Optimal Transport for Chest X-ray Analysis
Authors: Vandan Gorade; Azad Singh; Deepak Mishra
Venue: WACV 2025 (main)

Abstract: Self-supervised learning (SSL) has emerged as a promising technique for analyzing medical modalities such as X-rays due to its ability to learn without annotations. However conventional SSL methods face challenges in achieving semantic alignment and capturing subtle details which limits their ability to accurately represent the underlying anatomical structures and pathological features. To address these limitations we propose OTCXR a novel SSL framework that leverages optimal transport (OT) to learn dense semantic invariance. By integrating OT with our innovative Cross-Viewpoint Semantics Infusion Module (CV-SIM) OTCXR enhances the model's ability to capture not only local spatial features but also global contextual dependencies across different viewpoints. This approach enriches the effectiveness of SSL in the context of chest radiographs. Furthermore OTCXR incorporates variance and covariance regularizations within the OT framework to prioritize clinically relevant information while suppressing less informative features. This ensures that the learned representations are comprehensive and discriminative particularly beneficial for tasks such as thoracic disease diagnosis. We validate OTCXR's efficacy through comprehensive experiments on three publicly available chest X-ray datasets. Our empirical results demonstrate the superiority of OTCXR over state-of-the-art methods across all evaluated tasks confirming its capability to learn semantically rich representations.
Adversarial Attention Deficit: Fooling Deformable Vision Transformers with Collaborative Adversarial Patches
Authors: Quazi Mishkatul Alam; Bilel Tarchoun; Ihsen Alouani; Nael Abu-Ghazaleh
Venue: WACV 2025 (main)

Abstract: Deformable vision transformers reduce the expensive quadratic time-complexity of attention modeling by using sparse attention structures making it possible to use transformers in large-scale vision applications such as multi-view vision systems. We show that existing adversarial attacks against conventional vision transformers do not transfer to deformable transformers primarily due to the data-dependent dynamic nature of sparse attention. In this work we present for the first time adversarial attacks against deformable vision transformers by getting control of their attention-inferring module. We develop a novel collaborative attack where a source patch manipulates attention to point to a target patch containing the adversarial noise which fools the model. We observe that our attack alters less than 1% of the patched area in the input field completely disrupting object detection and resulting in 0% AP in single-view object detection using MS COCO and 0% MODA in multi-view object detection using Wildtrack.
Reflective Teacher: Semi-Supervised Multimodal 3D Object Detection in Bird's-Eye-View via Uncertainty Measure
Authors: Saheli Hazra; Sudip Das; Rohit Choudhary; Arindam Das; Ganesh Sistu; Ciarán Eising; Ujjwal Bhattacharya
Venue: WACV 2025 (main)

Abstract: Applying pseudo labeling techniques has been found to be advantageous in semi-supervised 3D object detection (SSOD) in Bird's-Eye-View (BEV) for autonomous driving particularly where labeled data is limited. In the literature Exponential Moving Average (EMA) has been used for adjustments of the weights of teacher network by the student network. However the same induces catastrophic forgetting in the teacher network. In this work we address this issue by introducing a novel concept of Reflective Teacher where the student is trained by both labeled and pseudo labeled data while its knowledge is progressively passed to the teacher through a regularizer to ensure retention of previous knowledge. Additionally we propose Geometry Aware BEV Fusion (GA-BEVFusion) for efficient alignment of multi-modal BEV features thus reducing the disparity between the modalities - camera and LiDAR. This helps to map the precise geometric information embedded among LiDAR points reliably with the spatial priors for extraction of semantic information from camera images. Our experiments on the nuScenes and Waymo datasets demonstrate: 1) improved performance over state-of-the-art methods in both fully supervised and semi-supervised settings; 2) Reflective Teacher achieves equivalent performance with only 25% and 22% of labeled data for nuScenes and Waymo datasets respectively in contrast to other fully supervised methods that utilize the full labeled dataset.
MaskVD: Region Masking for Efficient Video Object Detection
Authors: Sreetama Sarkar; Gourav Datta; Souvik Kundu; Kai Zheng; Chirayata Bhattacharyya; Peter A. Beerel
Venue: WACV 2025 (main)

Abstract: Video tasks are compute-heavy and thus pose a challenge when deploying in real-time applications particularly for tasks that require state-of-the-art Vision Transformers (ViTs). Several research efforts have tried to address this challenge by leveraging the fact that large portions of the video undergo very little change across frames leading to redundant computations in frame-based video processing. In particular some works leverage pixel or semantic differences across frames however this yields limited latency benefits with significantly increased memory overhead. This paper in contrast presents a strategy for masking regions in video frames that leverages the semantic information in images and the temporal correlation between frames to significantly reduce FLOPs and latency with little to no penalty in performance over baseline models. In particular we demonstrate that by leveraging extracted features from previous frames ViT backbones directly benefit from region masking skipping up to 80% of input regions improving FLOPs and latency by 3.14x and 1.5x. We improve memory and latency over the state-of-the-art (SOTA) by 2.3x and 1.14x while maintaining similar detection performance. Additionally our approach demonstrates promising results on convolutional neural networks (CNNs) and provides latency improvements over the SOTA up to 1.3x using specialized computational kernels. Code is available at: https://github.com/sreetamasarkar/MaskVD
Flowering Time Prediction of Wheat from DIA-MS Data
Authors: Yan Yang; Utpal Bose; James Broadbent; Sally Stockwell; Keren A Byrne; Md Zakir Hossain; Eric A Stone; Shannon Dillon
Venue: WACV 2025 (main)

Abstract: Traditional methods utilising data-independent acquisition mass spectrometry (DIA-MS) data for predictions depend on database searches against predefined spectral libraries for characterisation and quantification of the proteomes limiting scalability and adaptability across various applications. However directly applying existing networks on DIA-MS data represented as images for end-to-end predictions struggles to mine a predictive pattern due to non-uniform region importance across the image and divergences exhibited among different regions of the image. To overcome these limitations we propose a new framework with two modules: i) a dynamic sampling module that identifies regions of interest from the DIA-MS image constraining the network to focus on the most informative regions of the image only; ii) a mixture of experts module that sparsely routes the regions of interest to related expert networks facilitating adaptive computation of region features. The region features are then fused for predictions. Experimentally to benchmark our method we collected a large DIA-MS dataset of wheat for flowering time prediction and our approach significantly outperforms previous end-to-end methods i. e. 0.171 R2 improvements.
Vision-Aware Text Features in Referring Image Segmentation: From Object Understanding to Context Understanding
Authors: Hai Nguyen-Truong; E-Ro Nguyen; Tuan-Anh Vu; Minh-Triet Tran; Binh-Son Hua; Sai-Kit Yeung
Venue: WACV 2025 (main)

Abstract: Referring image segmentation is a challenging task that involves generating pixel-wise segmentation masks based on natural language descriptions. The complexity of this task increases with the intricacy of the sentences provided. Existing methods have relied mostly on visual features to generate the segmentation masks while treating text features as supporting components. However this under-utilization of text understanding limits the model's capability to fully comprehend the given expressions. In this work we propose a novel framework that specifically emphasizes object and context comprehension inspired by human cognitive processes through Vision-Aware Text Features. Firstly we introduce a CLIP Prior module to localize the main object of interest and embed the object heatmap into the query initialization process. Secondly we propose a combination of two components: Contextual Multimodal Decoder and Meaning Consistency Constraint to further enhance the coherent and consistent interpretation of language cues with the contextual understanding obtained from the image. Our method achieves significant performance improvements on three benchmark datasets RefCOCO RefCOCO+ and G-Ref. Project page: https://vatex.hkustvgd.com.
Forensic Iris Image-Based Post-Mortem Interval Estimation
Authors: Rasel Ahmed Bhuiyan; Adam Czajka
Venue: WACV 2025 (main)

Abstract: Post-mortem iris recognition is an emerging application of iris-based human identification in a forensic setup. One factor that may be useful in conditioning iris recognition methods is the tissue decomposition level which is correlated with the post-mortem interval (PMI) i.e. the number of hours that have elapsed since death. PMI however is not always available and its precise estimation remains one of the core challenges in forensic examination. This paper presents the first known to us method of the PMI estimation directly from iris images captured after death. To assess the feasibility of the iris-based PMI estimation we designed models predicting the PMI from (a) near-infrared (NIR) (b) visible (RGB) and (c) multispectral (RGB+NIR) forensic iris images. Models were evaluated following a 10-fold cross-validation in (S1) sample-disjoint (S2) subject-disjoint and (S3) cross-dataset scenarios. We explore two data balancing techniques for S3: resampling-based balancing (S3-real) and synthetic data-supplemented balancing (S3-synthetic). We found that using the multispectral data offers a spectacularly low mean absolute error (MAE) of = 3.5 hours in the scenario (S1) a bit worse MAE = 17.5 hours in the scenario (S2) and MAE = 45.77 hours in the scenario (S3). Additionally supplementing the training set with synthetically-generated forensic iris images (S3-synthetic) significantly enhances the models' ability to generalize to new NIR RGB and multispectral data collected in a different lab. This suggests that if the environmental conditions are favorable (e.g. bodies are kept in low temperatures) forensic iris images provide features that are indicative of the PMI and can be automatically estimated.
MFTrans: A Multi-Resolution Fusion Transformer for Robust Tumor Segmentation in Whole Slide Images
Authors: Sungkyu Yang; Woohyun Park; Kwangil Yim; Mansu Kim
Venue: WACV 2025 (main)

Abstract: Accurate tumor segmentation in whole slide image (WSI) is essential for histopathological diagnosis and research but the traditional manual analysis is labor-intensive and prone to variability. Furthermore many artificial models focus on specific magnification images limiting the detailed information available for segmentation. To address these challenges we propose MFTrans a novel multi-resolution fusion transformer with a CNN-based architecture designed for efficient tumor segmentation in WSI. Inspired by the diagnostic procedures of expert pathologists MFTrans integrates both high- and low-magnification images capturing detailed local features and broader contextual relationships through a dual-branch architecture. The model employs a global token transformer and cross-attention mechanism to fuse hierarchical features from dual branches to improve segmentation performance. We evaluate MFTrans on three real-world WSI datasets: Camelyon16 PAIP2019 and Catholic Uijeongbu St. Mary's hospital dataset demonstrating its superior segmentation performance over state-of-the-art methods in balanced and imbalanced setups. These results highlight MFTrans's effectiveness in medical image analysis and its generalizability across different datasets making it a robust tool for automated cancer diagnostics. Our code is available at https://github.com/aimed-gist/MFTrans.
360PanT: Training-Free Text-Driven 360-Degree Panorama-to-Panorama Translation
Authors: Hai Wang; Jing-Hao Xue
Venue: WACV 2025 (main)

Abstract: Preserving boundary continuity in the translation of 360-degree panoramas remains a significant challenge for existing text-driven image-to-image translation methods. These methods often produce visually jarring discontinuities at the translated panorama's boundaries disrupting the immersive experience. To address this issue we propose 360PanT a training-free approach to text-based 360-degree panorama-to-panorama translation with boundary continuity. Our 360PanT achieves seamless translations through two key components: boundary continuity encoding and seamless tiling translation with spatial control. Firstly the boundary continuity encoding embeds critical boundary continuity information of the input 360-degree panorama into the noisy latent representation by constructing an extended input image. Secondly leveraging this embedded noisy latent representation and guided by a target prompt the seamless tiling translation with spatial control enables the generation of a translated image with identical left and right halves while adhering to the extended input's structure and semantic layout. This process ensures a final translated 360-degree panorama with seamless boundary continuity. Experimental results on both real-world and synthesized datasets demonstrate the effectiveness of our 360PanT in translating 360-degree panoramas. Code is available at https://github.com/littlewhitesea/360PanT
Spk2ImgMamba: Spiking Camera Image Reconstruction with Multi-Scale State Space Models
Authors: Jiaoyang Yin; Bin Fan; Chao Xu; Tiejun Huang; Boxin Shi
Venue: WACV 2025 (main)

Abstract: As a bio-inspired vision sensor the spiking camera has showcased remarkable capability in high-speed imaging with a sampling rate of 40000 Hz. Reconstructing clear images from continuous spike streams which is obtained by each photosensor continuously detecting photons and firing them asynchronously has garnered significant attention. Despite promising results existing spike-to-image reconstruction methods face challenges in balancing global receptive fields and efficient computation due to the inherent limitations of their backbones. Recently due to powerful long-range modeling and linear complexity the state space model (SSM) has emerged as a competitive alternative to CNNs and Transformers. In this paper we propose a lightweight spike-to-image reconstruction network that harnesses Mamba as the backbone. Our approach sequentially executes three core modules: temporal information integration spatial feature enhancement and progressive image reconstruction. The former accumulates cues across diverse temporal windows to explore both long-term and short-term contexts. Subsequently to model global dependencies while heightening local detail perception we develop a multi-scale SSM block characterized by multi-scale multi-direction scanning which effectively boosts spatial feature representations. Finally intensity images are decoded progressively from the enhanced light-intensity features. Extensive experiments on both synthetic and real-captured data demonstrate that our approach achieves state-of-the-art performance with only 10% of the network parameters and nearly two orders of magnitude less computational effort. The code will be available at https://github.com/interstellarH/Spk2ImgMamba.
DARDA: Domain-Aware Real-Time Dynamic Neural Network Adaptation
Authors: Shahriar Rifat; Jonathan Ashdown; Francesco Restuccia
Venue: WACV 2025 (main)

Abstract: Test Time Adaptation (TTA) has emerged as a practical solution to mitigate the performance degradation of Deep Neural Networks (DNNs) in the presence of corrupti noise affecting inputs. Existing approaches in TTA continuously adapt the DNN leading to excessive resource consumption and performance degradation due to accumulation of error stemming from lack of supervision. In this work we propose Domain-Aware Real-Time Dynamic Adaptation (DARDA) to address such issues. Our key approach is to proactively learn latent representations of some corruption types each one associated with a sub-network state tailored to correctly classify inputs affected by that corruption. fter deployment DARDA adapts the DNN to previously unseen corruptions in an unsupervised fashion by (i) estimating the latent representation of the ongoing corruption; (ii) selecting the sub-network whose associated corruption is the closest in the latent space to the ongoing corruption; and (iii) adapting DNN state so that its representation matches the ongoing corruption. This way DARDA is more resource-efficient and can swiftly adapt to new distributions caused by different corruptions without requiring a large variety of input data. Through experiments with two popular mobile edge devices - Raspberry Pi and NVIDIA Jetson Nano - we show that DARDA reduces energy consumption and average cache memory footprint respectively by 1.74x and 2.64x with respect to the state of the art while increasing the performance by 10.4% 5.7% and 4.4% on CIFAR-10 CIFAR-100 and TinyImagenet.
A New Benchmark and Baseline for Real-Time High-Resolution Image Inpainting on Edge Devices
Authors: Marcelo Sánchez Ortega; Gil Triginer Garces; Coloma Ballester; Ignacio Sarasua; Lara Raad
Venue: WACV 2025 (main)

Abstract: Existing image inpainting methods have shown impressive completion results for low-resolution images. However most of these algorithms fail at high resolutions and require powerful hardware limiting their deployment on edge devices. Motivated by this we propose the first baseline for REal-Time High-resolution image INpainting on Edge Devices (RETHINED) that is able to inpaint at ultra-high-resolution and can run in real-time (30ms) in a wide variety of mobile devices. A simple yet effective novel method formed by a lightweight Convolutional Neural Network (CNN) to recover structure followed by a resolution-agnostic patch replacement mechanism to provide detailed texture. Specially our pipeline leverages the structural capacity of CNN and the high-level detail of patch-based methods which is a key component for high-resolution image inpainting. To demonstrate the real application of our method we conduct an extensive analysis on various mobile-friendly devices and demonstrate similar inpainting performance while being 100 times faster than existing state-of-the-art methods. Furthemore we release DF8K-Inpainting the first free-from mask UHD inpainting dataset.