GenAI4ELab/papercli-papers huggingface dataset:
Context-Aware Optimal Transport Learning for Retinal Fundus Image Enhancement
Authors: Vamsi Krishna S Vasa; Peijie Qiu; Wenhui Zhu; Yujian Xiong; Oana Dumitrascu; Yalin Wang
Venue: WACV 2025 (main)

Abstract: Retinal fundus photography offers a non-invasive way to diagnose and monitor a variety of retinal diseases but is prone to inherent quality glitches arising from systemic imperfections or operator/patient-related factors. However high-quality retinal images are crucial for carrying out accurate diagnoses and automated analyses. The fundus image enhancement is typically formulated as a distribution alignment problem by finding a one-to-one mapping between a low-quality image and its high-quality counterpart. This paper proposes a context-informed optimal transport (OT) learning framework for tackling unpaired fundus image enhancement. In contrast to standard generative image enhancement methods which struggle with handling contextual information (e.g. over-tampered local structures and unwanted artifacts) the proposed context-aware OT learning paradigm better preserves local structures and minimizes unwanted artifacts. Leveraging deep contextual features we derive the proposed context-aware OT using the earth mover's distance and show that the proposed context-OT has a solid theoretical guarantee. Experimental results on a large-scale dataset demonstrate the superiority of the proposed method over several state-of-the-art supervised and unsupervised methods in terms of signal-to-noise ratio structural similarity index as well as two downstream tasks. The code is available at https://github.com/Retinal-Research/Contextual-OT.
Perceive Query & Reason: Enhancing Video QA with Question-Guided Temporal Queries
Authors: Roberto Amoroso; Gengyuan Zhang; Rajat Koner; Lorenzo Baraldi; Rita Cucchiara; Volker Tresp
Venue: WACV 2025 (main)

Abstract: Video Question Answering (Video QA) is a challenging video understanding task that requires models to comprehend entire videos identify the most relevant information based on contextual cues from a given question and reason accurately to provide answers. Recent advancements in Multimodal Large Language Models (MLLMs) have transformed video QA by leveraging their exceptional commonsense reasoning capabilities. This progress is largely driven by the effective alignment between visual data and the language space of MLLMs. However for video QA an additional space-time alignment poses a considerable challenge for extracting question-relevant information across frames. In this work we investigate diverse temporal modeling techniques to integrate with MLLMs aiming to achieve question-guided temporal modeling that leverages pre-trained visual and textual alignment in MLLMs. We propose T-Former a novel temporal modeling method that creates a question-guided temporal bridge between frame-wise visual perception and the reasoning capabilities of LLMs. Our evaluation across multiple video QA benchmarks demonstrates that T-Former competes favorably with existing temporal modeling approaches and aligns with recent advancements in video QA.
Remote Blood Pressure Estimation from Facial Videos using Transfer Learning: Leveraging PPG to rPPG Conversion
Authors: Chun-Hong Cheng; Jing Wei Chin; Kwan Long Wong; Tsz Tai Chan; Hau Ching Lo; Kwan Lok Pang; Richard So; Bryan Yan
Venue: WACV 2025 (main)

Abstract: Blood pressure (BP) monitoring is crucial for health assessment but existing contact-based methods face cost and comfort barriers. Remote photoplethysmography (rPPG) offers a promising contactless solution yet research is hampered by limited rPPG datasets with corresponding BP labels. This paper presents a transfer learning methodology for BP measurement. This approach involves utilizing a base dataset comprising signals produced by PPG (PPG-signals) to acquire knowledge that can be transferred to a target dataset containing signals generated by rPPG (rPPG-signals). In our study we trained diverse deep learning models using publicly available datasets containing PPG-signals. Subsequently these models were fine-tuned and evaluated using a public dataset that specifically consists of rPPG-signals. Additionally we explored the relationship between BP and heart rate and examined different loss functions and normalization approaches to optimize the performance of the deep learning models. The findings of our study demonstrate that our best model achieved a better performance than the state-of-the-art model with mean absolute error (MAE) of 8.721 (reduced by 4.879) mmHg and 8.653 (reduced by 1.647) mmHg for systolic blood pressure (SBP) and diastolic blood pressure (DBP) in a dataset with clinical settings showing promising potential for remote BP estimation.
GStex: Per-Primitive Texturing of 2D Gaussian Splatting for Decoupled Appearance and Geometry Modeling
Authors: Victor Rong; Jingxiang Chen; Sherwin Bahmani; Kiriakos Kutulakos; David Lindell
Venue: WACV 2025 (main)

Abstract: Gaussian splatting has demonstrated excellent performance for view synthesis and scene reconstruction. The representation achieves photorealistic quality by optimizing the position scale color and opacity of thousands to millions of 2D or 3D Gaussian primitives within a scene. However since each Gaussian primitive encodes both appearance and geometry these attributes are strongly coupled - thus high-fidelity appearance modeling requires a large number of Gaussian primitives even when the scene geometry is simple (e.g. for a textured planar surface). We propose to texture each 2D Gaussian primitive so that even a single Gaussian can be used to capture appearance details. By employing per-primitive texturing our appearance representation is agnostic to the topology and complexity of the scene's geometry. We show that our approach GStex yields improved visual quality over prior work in texturing Gaussian splats. Furthermore we demonstrate that our decoupling enables improved novel view synthesis performance compared to 2D Gaussian splatting when reducing the number of Gaussian primitives and that GStex can be used for scene appearance editing and re-texturing.
Composed Image Retrieval for Training-Free Domain Conversion
Authors: Nikos Efthymiadis; Bill Psomas; Zakaria Laskar; Konstantinos Karantzalos; Yannis Avrithis; Ondrej Chum; Giorgos Tolias
Venue: WACV 2025 (main)

Abstract: This work addresses composed image retrieval in the context of domain conversion where the content of a query image is retrieved in the domain specified by the query text. We show that a strong vision-language model provides sufficient descriptive power without additional training. The query image is mapped to the text input space using textual inversion. Unlike common practice that invert in the continuous space of text tokens we use the discrete word space via a nearest-neighbor search in a text vocabulary. With this inversion the image is softly mapped across the vocabulary and is made more robust using retrieval-based augmentation. Database images are retrieved by a weighted ensemble of text queries combining mapped words with the domain text. Our method outperforms prior art by a large margin on standard and newly introduced benchmarks. Code: https://github.com/NikosEfth/freedom
HSDA: High-Frequency Shuffle Data Augmentation for Bird's-Eye-View Map Segmentation
Authors: Calvin Glisson; Qiuxiao Chen
Venue: WACV 2025 (main)

Abstract: Autonomous driving has garnered significant attention in recent research and Bird's-Eye-View (BEV) map segmentation plays a vital role in the field providing the basis for safe and reliable operation. While data augmentation is a commonly used technique for improving BEV map segmentation networks existing approaches predominantly focus on manipulating spatial domain representations. In this work we investigate the potential of frequency domain data augmentation for camera-based BEV map segmentation. We observe that high-frequency information in camera images is particularly crucial for accurate segmentation. Based on this insight we propose High-frequency Shuffle Data Augmentation (HSDA) a novel data augmentation strategy that enhances a network's ability to interpret high-frequency image content. This approach encourages the network to distinguish relevant high-frequency information from noise leading to improved segmentation results for small and intricate image regions as well as sharper edge and detail perception. Evaluated on the nuScenes dataset our method demonstrates broad applicability across various BEV map segmentation networks achieving a new state-of-the-art mean Intersection over Union (mIoU) of 61.3% for camera-only systems. This significant improvement underscores the potential of frequency domain data augmentation for advancing the field of autonomous driving perception. Code has been released: https://github.com/Zarhult/HSDA
Unsupervised Denoising for Signal-Dependent and Row-Correlated Imaging Noise
Authors: Benjamin Salmon; Alexander Krull
Venue: WACV 2025 (main)

Abstract: Accurate analysis of microscopy images is hindered by the presence of noise. This noise is usually signal-dependent and often additionally correlated along rows or columns of pixels. Current self- and unsupervised denoisers can address signal-dependent noise but none can reliably remove noise that is also row- or column-correlated. Here we present the first fully unsupervised deep learning-based denoiser capable of handling imaging noise that is row-correlated as well as signal-dependent. Our approach uses a Variational Autoencoder (VAE) with a specially designed autoregressive decoder. This decoder is capable of modeling row-correlated and signal-dependent noise but is incapable of independently modeling underlying clean signal. The VAE therefore produces latent variables containing only clean signal information and these are mapped back into image space using a proposed second decoder network. Our method does not require a pre-trained noise model and can be trained from scratch using unpaired noisy data. We benchmark our approach on microscopy datatsets from a range of imaging modalities and sensor types each with row- or column-correlated signal-dependent noise and show that it outperforms existing self- and unsupervised denoisers.
Mixed Patch Visible-Infrared Modality Agnostic Object Detection
Authors: Heitor R. Medeiros; David Latortue; Eric Granger; Marco Pedersoli
Venue: WACV 2025 (main)

Abstract: In real-world scenarios using multiple modalities like visible (RGB) and infrared (IR) can greatly improve the performance of a predictive task such as object detection (OD). Multimodal learning is a common way to leverage these modalities where multiple modality-specific encoders and a fusion module are used to improve performance. In this paper we tackle a different way to employ RGB and IR modalities where only one modality or the other is observed by a single shared vision encoder. This realistic setting requires a lower memory footprint and is more suitable for applications such as autonomous driving and surveillance which commonly rely on RGB and IR data. However when learning a single encoder on multiple modalities one modality can dominate the other producing uneven recognition results. This work investigates how to efficiently leverage RGB and IR modalities to train a common transformer-based OD vision encoder while countering the effects of modality imbalance. For this we introduce a novel training technique to Mix Patches (MiPa) from the two modalities in conjunction with a patch-wise modality agnostic module for learning a common representation of both modalities. Our experiments show that MiPa can learn a representation to reach competitive results on traditional RGB/IR benchmarks while only requiring a single modality during inference. Our code is available at: https://github.com/heitorrapela/MiPa.
Differentially Private Integrated Decision Gradients (IDG-DP) for Radar-Based Human Activity Recognition
Authors: Idris Zakariyya; Linda Tran; Kaushik Bhargav Sivangi; Paul Henderson; Fani Deligianni
Venue: WACV 2025 (main)

Abstract: Human motion analysis offers significant potential for healthcare monitoring and early detection of diseases. The advent of radar-based sensing systems has captured the spotlight for they are able to operate without physical contact and they can integrate with pre-existing Wi-Fi networks. They are also seen as less privacy-invasive compared to camera-based systems. However recent research has shown high accuracy in recognizing subjects or gender from radar gait patterns raising privacy concerns. This study addresses these issues by investigating privacy vulnerabilities in radar-based Human Activity Recognition (HAR) systems and proposing a novel method for privacy preservation using Differential Privacy (DP) driven by attributions derived with Integrated Decision Gradient (IDG) algorithm. We investigate Black-box Membership Inference Attack (MIA) Models in HAR settings across various levels of attacker-accessible information. We extensively evaluated the effectiveness of the proposed IDG-DP method by designing a CNN-based HAR model and rigorously assessing its resilience against MIAs. Experimental results demonstrate the potential of IDG-DP in mitigating privacy attacks while maintaining utility across all settings particularly excelling against label-only and shadow model black-box MIA attacks. This work represents a crucial step towards balancing the need for effective radar-based HAR with robust privacy protection in healthcare environments.
GeneralizeFormer: Layer-Adaptive Model Generation across Test-Time Distribution Shifts
Authors: Sameer Ambekar; Zehao Xiao; Xiantong Zhen; Cees Snoek
Venue: WACV 2025 (main)

Abstract: We consider the problem of test-time domain generalization where a model is trained on several source domains and adjusted on target domains never seen during training. Different from the common methods that fine-tune the model or adjust the classifier parameters online we propose to generate multiple layer parameters on the fly during inference by a lightweight meta-learned transformer which we call GeneralizeFormer. The layer-wise parameters are generated per target batch without fine-tuning or online adjustment. By doing so our method is more effective in dynamic scenarios with multiple target distributions and also avoids forgetting valuable source distribution characteristics. Moreover by considering layer-wise gradients the proposed method adapts itself to various distribution shifts. To reduce the computational and time cost we fix the convolutional parameters while only generating parameters of the Batch Normalization layers and the linear classifier. Experiments on six widely used domain generalization datasets demonstrate the benefits and abilities of the proposed method to efficiently handle various distribution shifts generalize in dynamic scenarios and avoid forgetting.
PICASSO: A Feed-Forward Framework for Parametric Inference of CAD Sketches via Rendering Self-Supervision
Authors: Ahmet Serdar Karadeniz; Dimitrios Mallis; Nesryne Mejri; Kseniya Cherenkova; Anis Kacem; Djamila Aouada
Venue: WACV 2025 (main)

Abstract: This work introduces PICASSO a framework for the parameterization of 2D CAD sketches from hand-drawn and precise sketch images. PICASSO converts a given CAD sketch image into parametric primitives that can be seamlessly integrated into CAD software. Our framework leverages rendering self-supervision to enable the pre-training of a CAD sketch parameterization network using sketch renderings only thereby eliminating the need for corresponding CAD parameterization. Thus we significantly reduce reliance on parameter-level annotations which are often unavailable particularly for hand-drawn sketches. The two primary components of PICASSO are (1) a Sketch Parameterization Network (SPN) that predicts a series of parametric primitives from CAD sketch images and (2) a Sketch Rendering Network (SRN) that renders parametric CAD sketches in a differentiable manner and facilitates the computation of a rendering (image-level) loss for self-supervision. We demonstrate that the proposed PICASSO can achieve reasonable performance even when finetuned with only a small number of parametric CAD sketches. Extensive evaluation on the widely used SketchGraphs and CAD as Language datasets validates the effectiveness of the proposed approach on zero- and few-shot learning scenarios.
Counting Guidance for High Fidelity Text-to-Image Synthesis
Authors: Wonjun Kang; Kevin Galim; Hyung Il Koo; Nam Ik Cho
Venue: WACV 2025 (main)

Abstract: Recently there have been significant improvements in the quality and performance of text-to-image generation largely due to the impressive results attained by diffusion models. However text-to-image diffusion models sometimes struggle to create high-fidelity content for the given input prompt. One specific issue is their difficulty in generating the precise number of objects specified in the text prompt. For example when provided with the prompt "five apples and ten lemons on a table" images generated by diffusion models often contain an incorrect number of objects. In this paper we present a method to improve diffusion models so that they accurately produce the correct object count based on the input prompt. We adopt a counting network that performs reference-less class-agnostic counting for any given image. We calculate the gradients of the counting network and refine the predicted noise for each step. To address the presence of multiple types of objects in the prompt we utilize novel attention map guidance to obtain high-quality masks for each object. Finally we guide the denoising process using the calculated gradients for each object. Through extensive experiments and evaluation we demonstrate that the proposed method significantly enhances the fidelity of diffusion models with respect to object count.
AMNCutter: Affinity-Attention-Guided Multi-View Normalized Cutter for Unsupervised Surgical Instrument Segmentation
Authors: Mingyu Sheng; Jianan Fan; Dongnan Liu; Ron Kikinis; Weidong Cai
Venue: WACV 2025 (main)

Abstract: Surgical instrument segmentation (SIS) is pivotal for robotic-assisted minimally invasive surgery assisting surgeons by identifying surgical instruments in endoscopic video frames. Recent unsupervised surgical instrument segmentation (USIS) methods primarily rely on pseudo-labels derived from low-level features such as color and optical flow but these methods show limited effectiveness and generalizability in complex and unseen endoscopic scenarios. In this work we propose a label-free unsupervised model featuring a novel module named Multi-View Normalized Cutter (m-NCutter). Different from previous USIS works our model is trained using a graph-cutting loss function that leverages patch affinities for supervision eliminating the need for pseudo-labels. The framework adaptively determines which affinities from which levels should be prioritized. Therefore the low- and high-level features and their affinities are effectively integrated to train a label-free unsupervised model showing superior effectiveness and generalization ability. We conduct comprehensive experiments across multiple SIS datasets to validate our approach's state-of-the-art (SOTA) performance robustness and exceptional potential as a pre-trained model. Our code is released at https://github.com/MingyuShengSMY/AMNCutter.
Dynamic Attention-Guided Diffusion for Image Super-Resolution
Authors: Brian B. Moser; Stanislav Frolov; Federico Raue; Sebastian Palacio; Andreas Dengel
Venue: WACV 2025 (main)

Abstract: Diffusion models in image Super-Resolution (SR) treat all image regions uniformly which risks compromising the overall image quality by potentially introducing artifacts during denoising of less-complex regions. To address this we propose "You Only Diffuse Areas" (YODA) a dynamic attention-guided diffusion process for image SR. YODA selectively focuses on spatial regions defined by attention maps derived from the low-resolution images and the current denoising time step. This time-dependent targeting enables a more efficient conversion to high-resolution outputs by focusing on areas that benefit the most from the iterative refinement process i.e. detail-rich objects. We empirically validate YODA by extending leading diffusion-based methods SR3 DiffBIR and SRDiff. Our experiments demonstrate new state-of-the-art performances in face and general SR tasks across PSNR SSIM and LPIPS metrics. As a side effect we find that YODA reduces color shift issues and stabilizes training with small batches.
SEMU-Net: A Segmentation-Based Corrector for Fabrication Process Variations of Nanophotonics with Microscopic Images
Authors: Rambod Azimi; Yijian Kong; Dusan Gostimirovic; James J. Clark; Odile Liboiron-Ladouceur
Venue: WACV 2025 (main)

Abstract: Integrated silicon photonic devices which manipulate light to transmit and process information on a silicon-on-insulator chip are highly sensitive to structural variations. Minor deviations during nanofabrication--the precise process of building structures at the nanometer scale--such as over- or under-etching corner rounding and unintended defects can significantly impact performance. To address these challenges we introduce SEMU-Net a comprehensive set of methods that automatically segments scanning electron microscope (SEM) images and uses them to train two deep neural network models based on U-Net and its variants. The predictor model anticipates fabrication-induced variations while the corrector model adjusts the design to address these issues ensuring that the final fabricated structures closely align with the intended specifications. Experimental results show that the segmentation U-Net reaches an average IoU score of 99.30% while the corrector attention U-Net in a tandem architecture achieves an average IoU score of 98.67%.
Frame by Familiar Frame: Understanding Replication in Video Diffusion Models
Authors: Aimon Rahman; Malsha V. Perera; Vishal M. Patel
Venue: WACV 2025 (main)

Abstract: Building on the momentum of image generation diffusion models there is an increasing interest in video-based diffusion models. However video generation poses greater challenges due to its higher-dimensional nature the scarcity of training data and the complex spatiotemporal relationships involved. Image generation models due to their extensive data requirements have already strained computational resources to their limits. There have been instances of these models reproducing elements from the training samples leading to concerns and even legal disputes over sample replication. Video diffusion models which operate with even more constrained datasets and are tasked with generating both spatial and temporal content may be more prone to replicating samples from their training sets. Compounding the issue these models are often evaluated using metrics that inadvertently reward replication. In our paper we present a systematic investigation into the phenomenon of sample replication in video diffusion models. We scrutinize various recent diffusion models for video synthesis assessing their tendency to replicate spatial and temporal content in both unconditional and conditional generation scenarios. Our study identifies strategies that are less likely to lead to replication. Furthermore we propose new evaluation strategies that take replication into account offering a more accurate measure of a model's ability to generate the original content.