GenAI4ELab/papercli-papers huggingface dataset:
Wavelength- and Depth-Aware Deep Image Prior for Blind Hyperspectral Imagery Deblurring with Coarse Depth Guidance
Authors: Jiahuan Li; Xiaoyu Dong; Wei He; Naoto Yokoya
Venue: WACV 2025 (main)

Abstract: Hyperspectral imagery (HSI) provides detailed spectral information enabling precise analysis of materials. However HSI imaging suffers from blurring degradation which results in the loss of fine details and hinders subsequent applications. The degree of blurriness is highly related to wavelength and depth existing deblurring methods either lack the utilization of spectral correlation or ignore the depth variation since paired HSI and depth data are difficult to acquire and less discussed leading to degraded performance when encountering wide-range HSIs of non-planar scenes. To address these challenges in both data acquisition and algorithm design we propose a novel approach that simultaneously collects both modalities and integrates depth refinement into a blind HSI deblurring model with wavelength- and depth-aware deep image prior. Specifically we capture blurred HSI and coarse depth map with separate devices followed by registration. Our method performs depth-guided deblurring through depth-variant multi-channel kernel estimation and soft-weight map-based layer composition while simultaneously refining the depth. The proposed approach effectively restores fine details with fewer artifacts showing superior performance for both simulated blurred HSIs and real captured HSIs.
Disentangling Subject-Irrelevant Elements in Personalized Text-to-Image Diffusion via Filtered Self-Distillation
Authors: Seunghwan Choi; Jooyeol Yun; Jeonghoon Park; Jaegul Choo
Venue: WACV 2025 (main)

Abstract: Recent research has unveiled the development of customizing large-scale text-to-image models. These models bind a unique subject desired by a user to a specific token using the token to generate the subject in various contexts. However models from previous studies also bind elements unrelated to the subject's identity such as common backgrounds or poses in the reference images. This often leads to conflicts between the token and the context of text prompts during inference causing the model to fail to generate both the subject and the prompted context. In this work we approach this issue from a data scarcity perspective and propose to augment the number of reference images through a novel self-distillation framework. Our framework selects high-quality samples from images generated by a teacher model and uses them in student training. Our framework can be applied to any models that suffer from the conflicts and we demonstrate that our framework most effectively resolves the issue through comprehensive evaluations.
Domain-Guided Weight Modulation for Semi-Supervised Domain Generalization
Authors: Chamuditha Jayanga Galappaththige; Zachary Izzo; Xilin He; Honglu Zhou; Muhammad Haris Khan
Venue: WACV 2025 (main)

Abstract: Unarguably deep learning models capable of generalizing to unseen domain data while leveraging a few labels are of great practical significance due to low developmental costs. In search of this endeavor we study the challenging problem of semi-supervised domain generalization (SSDG) where the goal is to learn a domain-generalizable model while using only a small fraction of labeled data and a relatively large fraction of unlabeled data. Domain generalization (DG) methods show subpar performance under the SSDG setting whereas semi-supervised learning (SSL) methods demonstrate relatively better performance however they are considerably poor compared to the fully-supervised DG methods. Towards handling this new but challenging problem of SSDG we propose a novel method that can facilitate the generation of accurate pseudo-labels under various domain shifts. This is accomplished by retaining the domain-level specialism in the classifier during training corresponding to each source domain. Specifically we first create domain-level information vectors on the fly which are then utilized to learn a domain-aware mask for modulating the classifier's weights. We provide a mathematical interpretation for the effect of this modulation procedure on both pseudo-labeling and model training. Our method is plug-and-play and can be readily applied to different SSL baselines for SSDG. Extensive experiments on six challenging datasets in two different SSDG settings show that our method provides visible gains over the various strong SSL-based SSDG baselines. Our code is available at github.com/DGWM.
ContextIQ: A Multimodal Expert-Based Video Retrieval System for Contextual Advertising
Authors: Ashutosh Chaubey; Anoubhav Agarwaal; Sartaki Sinha Roy; Aayush Agrawal; Susmita Ghose
Venue: WACV 2025 (main)

Abstract: Contextual advertising serves ads that are aligned to the content that the user is viewing. The rapid growth of video content on social platforms and streaming services along with privacy concerns has increased the need for contextual advertising. Placing the right ad in the right context creates a seamless and pleasant ad viewing experience resulting in higher audience engagement and ultimately better ad monetization. From a technology standpoint effective contextual advertising requires a video retrieval system capable of understanding complex video content at a very granular level. Current text-to-video retrieval models based on joint multimodal training demand large datasets and computational resources limiting their practicality and lacking the key functionalities required for ad ecosystem integration. We introduce ContextIQ a multimodal expert-based video retrieval system designed specifically for contextual advertising. ContextIQ utilizes modality-specific experts--video audio transcript (captions) and metadata such as objects actions emotion etc.--to create semantically rich video representations. We show that our system without joint training achieves better or comparable results to state-of-the-art models and commercial solutions on multiple text-to-video retrieval benchmarks. Our ablation studies highlight the benefits of leveraging multiple modalities for enhanced video retrieval accuracy instead of using a vision-language model alone. Furthermore we show how video retrieval systems such as ContextIQ can be used for contextual advertising in an ad ecosystem while also addressing concerns related to brand safety and filtering inappropriate content.
CLASS: Conditional Latent Architecture for Search and Synthesis of Design Layouts
Authors: Dipu Manandhar; Paul Guerrero; Zhaowen Wang; John Collomosse
Venue: WACV 2025 (main)

Abstract: We propose CLASS; a novel unified model for the synthesis and search for design layouts two tasks that are often handled separately by prior works. We propose to learn a compact and coherent latent feature of a layout supporting joint search and synthesis. This allows various operations such style-conditioned layout generation latent space manipulation and provides seamless integration of search and synthesis for an effective design workflow. We train CLASS with a dual decoder: a new transformer-based layout-conditioned decoder and a CNN-based raster decoder. The latent-conditioned decoder explicitly conditions upon a latent vector while generating a layout in an auto-regressive fashion. We train CLASS under variational framework which in conjunction with a raster-decoder enhances the latent representation improving both generation and retrieval performances. We show the effectiveness of CLASS on the RICO and PubLayNet benchmarks and demonstrate that CLASS is capable of high-quality synthesis from scratch as well as performing self-completion interpolation project between design layouts whilst achieving close to or better than state-of-the-art search performance.
High-Pass Kernel Prediction for Efficient Video Deblurring
Authors: Bo Ji; Angela Yao
Venue: WACV 2025 (main)

Abstract: State-of-the-art video deblurring methods use deep network architectures to recover sharpened video frames. Blurring especially degrades high-frequency (HF) information yet this aspect is often overlooked by recent models that focus more on enhancing architectural design. Recovering these fine details is challenging partly due to the spectral bias of neural networks which are inclined towards learning low-frequency functions. To address this we enforce explicit network structures to capture the fine details and edges. We dynamically predict adaptive high-pass kernels from a linear combination of high-pass basis kernels to extract high-frequency features. This strategy is highly efficient resulting in low-memory footprints for training and fast run times for inference all while achieving state-of-the-art when compared to low-budget models. The code is available at https://github.com/jibo27/AHFNet.
AnonyNoise: Anonymizing Event Data with Smart Noise to Outsmart Re-Identification and Preserve Privacy
Authors: Katharina Bendig; René Schuster; Nicole Thiemer; Karen Joisten; Didier Stricker
Venue: WACV 2025 (main)

Abstract: The increasing capabilities of deep neural networks for re-identification combined with the rise in public surveillance in recent years pose a substantial threat to individual privacy. Event cameras were initially considered as a promising solution since their output is sparse and therefore difficult for humans to interpret. However recent advances in deep learning proof that neural networks are able to reconstruct high-quality grayscale images and re-identify individuals using data from event cameras. In our paper we contribute a crucial ethical discussion on data privacy and present the first event anonymization pipeline to prevent re-identification not only by humans but also by neural networks. Our method effectively introduces learnable data-dependent noise to cover personally identifiable information in raw event data reducing attackers' re-identification capabilities by up to 60% while maintaining substantial information for the performing of downstream tasks. Moreover our anonymization generalizes well on unseen data and is robust against image reconstruction and inversion attacks. Code: https://github.com/dfki-av/AnonyNoise
Diffusion-Based Conditional Image Editing through Optimized Inference with Guidance
Authors: Hyunsoo Lee; Minsoo Kang; Bohyung Han
Venue: WACV 2025 (main)

Abstract: We present a simple but effective training-free approach for text-driven image-to-image translation based on a pretrained text-to-image diffusion model. Our goal is to generate an image that aligns with the target task while preserving the structure and background of a source image. To this end we derive the representation guidance with a combination of two objectives: maximizing the similarity to the target prompt based on the CLIP score and minimizing the structural distance to the source latent variable. This guidance improves the fidelity of the generated target image to the given target prompt while maintaining the structure integrity of the source image. To incorporate the representation guidance component we optimize the target latent variable of diffusion model's reverse process with the guidance. Experimental results demonstrate that our method achieves outstanding image-to-image translation performance on various tasks when combined with the pretrained Stable Diffusion model.
Frequency-Domain Refinement of Vision Transformers for Robust Medical Image Segmentation under Degradation
Authors: Sanaz Karimijafarbigloo; Sina Ghorbani Kolahi; Reza Azad; Ulas Bagci; Dorit Merhof
Venue: WACV 2025 (main)

Abstract: Medical image segmentation is crucial for precise diagnosis treatment planning and disease monitoring in clinical settings. While convolutional neural networks (CNNs) have achieved remarkable success they struggle with modeling long-range dependencies. Vision Transformers (ViTs) address this limitation by leveraging self-attention mechanisms to capture global contextual information. However ViTs often fall short in local feature description which is crucial for precise segmentation. To address this issue we reformulate self-attention in the frequency domain to enhance both local and global feature representation. Our approach the Enhanced Wave Vision Transformer (EW-ViT) incorporates wavelet decomposition within the self-attention block to adaptively refine feature representation in low and high-frequency components. We also introduce the Prompt-Guided High-Frequency Refiner (PGHFR) module to handle image degradation which mainly affects high-frequency components. This module uses implicit prompts to encode degradation-specific information and adjust high-frequency representations accordingly. Additionally we apply a contrastive learning strategy to maintain feature consistency and ensure robustness against noise leading to state-of-the-art (SOTA) performance in medical image segmentation especially under various conditions of degradation. Source code is available at GitHub.
Leveraging CLIP Encoder for Multimodal Emotion Recognition
Authors: Yehun Song; Sunyoung Cho
Venue: WACV 2025 (main)

Abstract: Multimodal emotion recognition (MER) aims to identify human emotions by combining data from various modalities such as language audio and vision. Despite the recent advances of MER approaches the limitations in obtaining extensive datasets impede the improvement of performance. To mitigate this issue we leverage a Contrastive Language-Image Pre-training (CLIP)-based architecture and its semantic knowledge from massive datasets that aims to enhance the discriminative multimodal representation. We propose a label encoder-guided MER framework based on CLIP (MER-CLIP) to learn emotion-related representations across modalities. Our approach introduces a label encoder that treats labels as text embeddings to incorporate their semantic information leading to the learning of more representative emotional features. To further exploit label semantics we devise a cross-modal decoder that aligns each modality to a shared embedding space by sequentially fusing modality features based on emotion-related input from the label encoder. Finally the label encoder-guided prediction enables generalization across diverse labels by embedding their semantic information as well as word labels. Experimental results show that our method outperforms the state-of-the-art MER methods on the benchmark datasets CMU-MOSI and CMU-MOSEI.
MAGMA: Manifold Regularization for MAEs
Authors: Alin-Eugen Dondera; Anuj R Singh; Hadi Jamali-Rad
Venue: WACV 2025 (main)

Abstract: Masked Autoencoders (MAEs) are an important divide in self-supervised learning (SSL) due to their independence from augmentation techniques for generating positive (and/or negative) pairs as in contrastive frameworks. Their masking and reconstruction strategy also nicely aligns with SSL approaches in natural language processing. Most MAEs are built upon Transformer-based architectures where visual features are not regularized as opposed to their convolutional neural network (CNN) based counterparts which can potentially hinder their performance. To address this we introduce MAGMA a novel batch-wide layer-wise regularization loss applied to representations of different Transformer layers. We demonstrate that by plugging in the proposed regularization loss one can significantly improve the performance of MAE-based models. We further demonstrate the impact of the proposed loss on optimizing other generic SSL approaches (such as VICReg and SimCLR) broadening the impact of the proposed approach. Our code base can be found at https://github.com/adondera/magma.
DeepMIM: Deep Supervision for Masked Image Modeling
Authors: Sucheng Ren; Fangyun Wei; Samuel Albanie; Zheng Zhang; Han Hu
Venue: WACV 2025 (main)

Abstract: Deep supervision which involves extra supervisions to the intermediate features of a neural network was widely used in image classification in the early deep learning era since it significantly reduces the training difficulty and eases the optimization like avoiding gradient vanish over the vanilla training. Nevertheless with the emergence of normalization techniques and residual connection deep supervision in image classification was gradually phased out. In this paper we revisit deep supervision for masked image modeling (MIM) that pre-trains a Vision Transformer (ViT) via a mask-and-predict scheme. Experimentally we find that deep supervision drives the shallower layers to learn more meaningful representations accelerates model convergence and expands attention diversities. Our approach called DeepMIM significantly boosts the representation capability of each layer. In addition DeepMIM is compatible with many MIM models across a range of reconstruction targets. For instance using ViT-B DeepMIM on MAE achieves 84.2 top-1 accuracy on ImageNet outperforming MAE by +0.6. By combining DeepMIM with a stronger tokenizer CLIP our model achieves state-of-the-art performance on various downstream tasks including image classification (85.6 top-1 accuracy on ImageNet-1K outperforming MAE-CLIP by +0.8) object detection (52.8 AP^box on COCO) and semantic segmentation (53.1 mIoU on ADE20K). Code and models are available at https://github.com/OliverRensu/DeepMIM.
Assessing the Quality of 3D Reconstruction in the Absence of Ground Truth: Application to a Multimodal Archaeological Dataset
Authors: Benjamin Coupry; Baptiste Brument; Antoine Laurent; Jean Mélou; Yvain Quéau; Jean-Denis Durou
Venue: WACV 2025 (main)

Abstract: This paper proposes a new dataset of archaeological artefacts for evaluating 3D reconstruction methods and questions the notion of ground truth. Indeed 3D reconstruction of archaeological objects can be carried out using either scanners or photographic methods. It turns out that multi-view stereo (MVS) faithfully reconstructs the overall shape of an object on a par with a hand-held scanner while calibrated photometric stereo (CPS) reveals relief details. The restitution of low- and high-frequencies is therefore the prerogative of distinct methods which indicates that the ground truth and the metric used for evaluation should be chosen in view of the target frequencies. This observation led us to combine MVS and CPS using MVS to calibrate the illumination used by CPS. We demonstrate on our dataset of archaeological objects that this original 3D reconstruction method indeed combines the advantages of MVS and CPS. Our proposed dataset can be accessed here: https://github.com/BenjaminCoupry/the-MAD-project.
LQ-Adapter: ViT-Adapter with Learnable Queries for Gallbladder Cancer Detection from Ultrasound Images
Authors: Chetan Madan; Mayuna Gupta; Soumen Basu; Pankaj Gupta; Chetan Arora
Venue: WACV 2025 (main)

Abstract: We focus on the problem of Gallbladder Cancer (GBC) detection from Ultrasound (US) images. The problem presents unique challenges to modern Deep Neural Network (DNN) techniques due to low image quality arising from noise textures and viewpoint variations. Tackling such challenges would necessitate precise localization performance by the DNN to identify the discerning features for the downstream malignancy prediction. While several techniques have been proposed in the recent years for the problem all of these methods employ complex custom architectures. Inspired by the success of foundational models for natural image tasks along with the use of adapters to fine-tune such models for the custom tasks we investigate the merit of one such design ViT-Adapter for the GBC detection problem. We observe that ViT-Adapter relies predominantly on a primitive CNN-based spatial prior module to inject the localization information via cross-attention which is inefficient for our problem due to the small pathology sizes and variability in their appearances due to non-regular structure of the malignancy. In response we propose LQ-Adapter a modified Adapter design for ViT which improves localization information by leveraging learnable content queries over the basic spatial prior module. Our method surpasses existing approaches enhancing the mean IoU (mIoU) scores by 5.4% 5.8% and 2.7% over ViT-Adapters DINO and FocalNet-DINO respectively on the US image-based GBC detection dataset and establishing a new state-of-the-art (SOTA). Additionally we validate the applicability and effectiveness of LQ-Adapter on the Kvasir-Seg dataset for polyp detection from colonoscopy images. Superior performance of our design on this problem as well showcases its capability to handle diverse medical imaging tasks across different datasets. Source code and trained models are publicly released.
CE-VAE: Capsule Enhanced Variational AutoEncoderfor Underwater Image Enhancement
Authors: Rita Pucci; Niki Martinel
Venue: WACV 2025 (main)

Abstract: Unmanned underwater image analysis for marine monitoring faces two key challenges: (i) degraded image quality due to light attenuation and (ii) hardware storage constraints limiting high-resolution image collection. Existing methods primarily address image enhancement with approaches that hinge on storing the full-size input. In contrast we introduce the Capsule Enhanced Variational AutoEncoder (CE-VAE) a novel architecture designed to efficiently compress and enhance degraded underwater images. Our attention-aware image encoder can project the input image onto a latent space representation while being able to run online on a remote device. The only information that needs to be stored on the device or sent to a beacon is a compressed representation. There is a dual-decoder module that performs offline full-size enhanced image generation. One branch reconstructs spatial details from the compressed latent space while the second branch utilizes a capsule-clustering layer to capture entity-level structures and complex spatial relationships. This parallel decoding strategy enables the model to balance fine-detail preservation with context-aware enhancements. CE-VAE achieves state-of-the-art performance in underwater image enhancement on six benchmark datasets providing up to 3x higher compression efficiency than existing approaches. Code available at https://github.com/iN1k1/ce-vae-underwater-image-enhancement.
Information Theoretic Pruning of Coupled Channels in Deep Neural Networks
Authors: Peyman Rostami; Nilotpal Sinha; Nidhaleddine Chenni; Anis Kacem; Abd El Rahman Shabayek; Carl Shneider; Djamila Aouada
Venue: WACV 2025 (main)

Abstract: Variational channel pruning approaches have obtained impressive results thanks to their stochastic nature well established foundation in information theory and the practically appealing structured sparsity pattern they offer. Despite their success in pruning Plain Networks (PlainNets) their application has faced certain limitations in networks with structurally coupled channels such as ResNets. In such scenarios not only is it required to prune structurally coupled channels together but it is also necessary to ensure that the whole coupled group is irrelevant before pruning is applied. This is an under-investigated problem as most existing methods are designed without taking these couplings into account. In this paper we propose a novel approach based on Information Theoretic Pruning of structurally Coupled Channels (ITPCC) in neural networks. ITPCC allows for learning the probabilistic distribution of coupled channel set importance and prunes the ones withthe least relevant information to the task at hand. Experimental results for image classification on CIFAR10 CIFAR100 and ImageNet datasets show that the proposed method outperforms the state-of-the-art more significantly at high compression rates.