GenAI4ELab/papercli-papers huggingface dataset:
TLDR: Text Based Last-Layer Retraining for Debiasing Image Classifiers
Authors: Juhyeon Park; Seokhyeon Jeong; Taesup Moon
Venue: WACV 2025 (main)

Abstract: An image classifier may depend on incidental features stemming from a strong correlation between the feature and the classification target in the training dataset. Recently Last Layer Retraining (LLR) with group-balanced datasets is shown to be efficient in mitigating the spurious correlation of classifiers. However the acquisition of image-based group-balanced datasets is costly which hinders the general applicability of the LLR method. In this work we propose to perform LLR based on text datasets built with large language models to debias a general image classifier. To that end we demonstrate that text can generally be a proxy for its corresponding image beyond the image-text joint embedding space which is achieved with a linear projector that ensures orthogonality between its weight and the modality gap of the joint embedding space. In addition we propose a systematic validation procedure that checks whether the generated words are compatible with the embedding space of CLIP and the image classifier which is shown to be effective for improving debiasing performance. We dub these procedures as TLDR (Text-based Last layer retraining for Debiasing image classifieRs) and show our method achieves the performance that is competitive with the LLR methods that require group-balanced image dataset for retraining. Furthermore TLDR outperforms other baselines that involve training the last layer without any group annotated dataset. Codes: https://github.com/beotborry/TLDR
Fair Domain Generalization with Heterogeneous Sensitive Attributes Across Domains
Authors: Ragja Palakkadavath; Hung Le; Thanh Nguyen-Tang; Sunil Gupta; Svetha Venkatesh
Venue: WACV 2025 (main)

Abstract: Domain generalization(DG) techniques classify data from unseen domains by capitalizing on data from multiple source domains. Most methods in DG focus on improving predictive performance measures on the unseen domain. Recent studies have also attempted to enforce fairness measures on the unseen domain. However these studies assume that every domain has the same sensitive attribute including the unseen domain. In practice each domain may be required to satisfy fairness with respect to its own set of multiple sensitive attributes. Given a set of sensitive attributes (S) current methods need to train 2^n models to ensure fairness with respect to any subset of S where n = |S|. We propose a single-model solution to address this new problem setting. We learn two feature representations one to generalize the model's predictive performance and another to generalize the model's fairness. The first representation is made invariant across domains to generalize predictive performance. The second representation is kept selectively invariant i.e. invariant only across domains having the same sensitive attributes. Our single model exhibits superior predictive performance and fairness measures against the current alternative of 2^n models on unseen domains on multiple real-world vision datasets. Our code is available at https://github.com/ragjapk/SISA.
Infant Action Generative Modeling
Authors: Xiaofei Huang; Elaheh Hatamimajoumerd; Amal Mathew; Sarah Ostadabbas
Venue: WACV 2025 (main)

Abstract: Despite advancements in human motion generation models their performance drops in infant motion generation due to limited data available and lack of 3D skeleton ground truth. To address this we introduce the infant action generation and classification (InfAGenC) pipeline which combines a transformer-based variational autoencoder (VAE) with a spatial-temporal graph convolutional network (STGCN) to create synthetic infant action samples. By iterative refinement of the generative model with diverse and accurate data we improve the realism of synthetic data leading to more precise infant action recognition models. Our results show significant improvements in action recognition performance on real-world data demonstrating that synthetic data can enhance small training datasets and advance infant action recognition. Our pipeline increases action recognition accuracy up to 88.58% on the infant action dataset and up to 98% on an adult action dataset.
CAMS: Convolution and Attention-Free Mamba-Based Cardiac Image Segmentation
Authors: Abbas Khan; Muhammad Asad; Martin Benning; Caroline Roney; Gregory Slabaugh
Venue: WACV 2025 (main)

Abstract: Convolutional Neural Networks (CNNs) and Transformer-based self-attention models have become the standard for medical image segmentation. This paper demonstrates that convolution and self-attention while widely used are not the only effective methods for segmentation. Breaking with convention we present a Convolution and self-attention-free Mamba-based semantic Segmentation Network named CAMS-Net. Specifically we design a Mamba-based Channel Aggregator and Spatial Aggregator which are applied independently in each encoder-decoder stage. The Channel Aggregator extracts information across different channels and the Spatial Aggregator learns features across different spatial locations. We also propose a Linearly Interconnected Factorized Mamba (LIFM) block to reduce the computational complexity of a Mamba block and to enhance its decision function by introducing a non-linearity between two factorized Mamba blocks. Our model outperforms the existing state-of-the-art CNN self-attention and Mamba-based methods on CMR and M&Ms-2 Cardiac segmentation datasets showing how this innovative convolution and self-attention-free method can inspire further research beyond CNN and Transformer paradigms achieving linear complexity and reducing the number of parameters. Source code and pre-trained models are available at: https://github.com/kabbas570/CAMS-Net.
VipDiff: Towards Coherent and Diverse Video Inpainting via Training-Free Denoising Diffusion Models
Authors: Chaohao Xie; Kai Han; Kwan-Yee K. Wong
Venue: WACV 2025 (main)

Abstract: Recent video inpainting methods have achieved encouraging improvements by leveraging optical flow to guide pixel propagation from reference frames either in the image space or feature space. However they would produce severe artifacts when the masked area is too large and no pixel correspondences could be found. Recently denoising diffusion models have demonstrated impressive performance in generating diverse and high-quality images and have been exploited in a number of works for image inpainting. These methods however cannot be applied directly to videos to produce temporal-coherent inpainting results. In this paper we propose a training-free framework named VipDiff for conditioning diffusion model on the reverse diffusion process to produce temporal-coherent inpainting results without requiring any training data or fine-tuning the pre-trained models. VipDiff takes optical flow as guidance to extract valid pixels from reference frames to serve as constraints in optimizing the randomly sampled Gaussian noise and uses the generated results for further pixel propagation and conditional generation. VipDiff also allows for generating diverse video inpainting results over different sampled noise. Experiments demonstrate that our VipDiff outperforms state-of-the-art methods in terms of both spatial-temporal coherence and fidelity.
ColorizeDiffusion: Improving Reference-Based Sketch Colorization with Latent Diffusion Model
Authors: Dingkun Yan; Liang Yuan; Erwin Wu; Yuma Nishioka; Issei Fujishiro; Suguru Saito
Venue: WACV 2025 (main)

Abstract: Diffusion models have achieved great success in dual-conditioned image generation. However they still face significant challenges in image-guided sketch colorization where reference and sketch images usually exhibit different spatial structures and semantics. This mismatch termed "distribution shift" in this paper results in various artifacts and degrades the colorization quality. To address this issue we conducted thorough investigations into the image-prompted latent diffusion model and developed a two-stage training framework to mitigate the effects of distribution shift based on our analysis. Comprehensive quantitative comparisons qualitative evaluations and user studies were performed to demonstrate the superiority of our proposed methods. Additionally an ablation study was conducted to assess the impact of the distribution shift and the selection of reference embeddings. Codes are made publicly available at https://github.com/tellurionkanata/colorizeDiffusion.
My3DGen: A Scalable Personalized 3D Generative Model
Authors: Luchao Qi; Jiaye Wu; Annie N. Wang; Shengze Wang; Roni Sengupta
Venue: WACV 2025 (main)

Abstract: In recent years generative 3D face models (e.g. EG3D) have been developed to tackle the problem of synthesizing photo-realistic faces. However these models are often unable to capture facial features unique to each individual highlighting the importance of personalization. Some prior works have shown promise in personalizing generative face models but these studies primarily focus on 2D settings. Also these methods require both fine-tuning and storing a large number of parameters for each user posing a hindrance to achieving scalable personalization. Another challenge of personalization is the limited number of training images available for each individual which often leads to overfitting when using full fine-tuning methods. Our proposed approach My3DGen generates a personalized 3D prior of an individual using as few as 50 training images. My3DGen allows for novel view synthesis semantic editing of a given face (e.g. adding a smile) and synthesizing novel appearances all while preserving the original person's identity. We decouple the 3D facial features into global features and personalized features by freezing the pre-trained EG3D and training additional personalized weights through low-rank decomposition. As a result My3DGen introduces only 240K personalized parameters per individual leading to a 127x reduction in trainable parameters compared to the 30.6M required for fine-tuning the entire parameter space. Despite this significant reduction in storage our model preserves identity features without compromising the quality of downstream applications both quantitatively and qualitatively.
Discriminative Score Suppression for Weakly Supervised Video Anomaly Detection
Authors: Chen Xu; Chunguo Li; Hongjie Xing
Venue: WACV 2025 (main)

Abstract: Weakly supervised video anomaly detection (WSVAD) often relies on Multiple Instance Learning (MIL). However selecting only the most discriminative segments for training limits the model's ability to comprehensively detect anomalous events particularly hard anomalies. To overcome this limitation we propose the Discriminative Score Suppression (DSS) module. This module suppresses the discriminative scores of the most prominent anomalies shifting the model's attention to less obvious but important hard anomalies. This approach guides the model to learn the critical features of hard anomalies enabling a more comprehensive detection of anomalous events. Additionally the Anomaly Score Refinement (ASR) module constructs a dissimilarity-based classifier by storing normal patterns as prototypes and integrates this with a neural network classifier. By combining the anomaly scores from both classifiers more accurate detection of true hard anomalies is achieved. A score-sensitive inner-bag loss function not only adjusts penalties based on anomaly scores but also ensures that the model avoids erroneous selections. Our method accurately detects various anomalies including challenging and multi-segment anomalies while minimizing false positives for normal events. Extensive experiments show that the proposed framework outperforms state-of-the-art methods on the UCF-Crime and XD-Violence datasets.
Semiotic-Based Construction of a Large Emotional Image Dataset with Neutral Samples
Authors: Marco Blanchini; Giovanna Dimitri; Lydia Abady; Benedetta Tondi; Tarcisio Lancioni; Mauro Barni
Venue: WACV 2025 (main)

Abstract: Image Visual Sentiment Analysis (VSA) requires the availability of large annotated datasets whose construction presents many challenges. The necessity of gathering a large amount of labeled images contrasts with the rigorous but lengthy process required for manual annotation based on psychovisual experiments and with the automatic gathering of large amounts of data roughly labeled based on the sentiment analysis of the text accompanying the images like captions tweets and tags. An additional limitation is the scarcity of high-quality datasets with a neutral class which forces the images to be classified into emotions even when the observers show no emotional activation. In this work we present a scalable methodology rooted in semiotics and art theory for the construction of a 3-class (positive negative and neutral) VSA dataset enabling the downloading of a desired quantity of images while maintaining labeling coherence and accuracy. Based on the proposed methodology we introduce and make publicly available a VSA dataset of over 100000 images. To validate the quality of the dataset we used it to train several classifiers and compared their performance with those of classifiers trained on other datasets. The results show that the classifiers trained on the new dataset provide better performance when tested on independent datasets including those commonly used for psychovisual experiments.
SADDLe: Sharpness-Aware Decentralized Deep Learning with Heterogeneous Data
Authors: Sakshi Choudhary; Sai Aparna Aketi; Kaushik Roy
Venue: WACV 2025 (main)

Abstract: Decentralized training enables learning with distributed datasets generated at different locations without relying on a central server. In realistic scenarios the data distribution across these sparsely connected learning agents can be significantly heterogeneous leading to local model over-fitting and poor global model generalization. Another challenge is the high communication cost of training models in such a peer-to-peer fashion without any central coordination. In this paper we jointly tackle these two-fold practical challenges by proposing SADDLe a set of sharpness-aware decentralized deep learning algorithms. SADDLe leverages Sharpness-Aware Minimization (SAM) to seek a flatter loss landscape during training resulting in better model generalization as well as enhanced robustness to communication compression. We present two versions of our approach and demonstrate its effectiveness through extensive experiments on various Computer Vision datasets (CIFAR-10 CIFAR-100 Imagenette and ImageNet) model architectures and graph topologies. Our results show that SADDLe leads to 1-20% improvement in test accuracy as compared to existing techniques while incurring a minimal accuracy drop (  1%) in the presence of up to 4x compression.
DiffPAD: Denoising Diffusion-Based Adversarial Patch Decontamination
Authors: Jia Fu; Xiao Zhang; Sepideh Pashami; Fatemeh Rahimian; Anders Holst
Venue: WACV 2025 (main)

Abstract: In the ever-evolving adversarial machine learning landscape developing effective defenses against patch attacks has become a critical challenge necessitating reliable solutions to safeguard real-world AI systems. Although diffusion models have shown remarkable capacity in image synthesis and have been recently utilized to counter l_p-norm bounded attacks their potential in mitigating localized patch attacks remains largely underexplored. In this work we propose DiffPAD a novel framework that harnesses the power of diffusion models for adversarial patch decontamination. DiffPAD first performs super-resolution restoration on downsampled input images then adopts binarization dynamic thresholding scheme and sliding window for effective localization of adversarial patches. Such a design is inspired by the theoretically derived correlation between patch size and diffusion restoration error that is generalized across diverse patch attack scenarios. Finally DiffPAD applies inpainting techniques to the original input images with the estimated patch region being masked. By integrating closed-form solutions for super-resolution restoration and image inpainting into the conditional reverse sampling process of a pre-trained diffusion model DiffPAD obviates the need for text guidance or fine-tuning. Through comprehensive experiments we demonstrate that DiffPAD not only achieves state-of-the-art adversarial robustness against patch attacks but also excels in recovering naturalistic images without patch remnants. The source code is available at https://github.com/JasonFu1998/DiffPAD.
Data-Efficient Alignment in Medical Imaging via Reconfigurable Generative Networks
Authors: Divya Saxena; Jiannong Cao; Jiahao Xu; Tarun Kulshrestha
Venue: WACV 2025 (main)

Abstract: Recent advances in deep learning have witnessed many successful medical image translation models that learn correspondences between two visual domains. However building robust mappings between domains is a significant challenge when handling misalignments caused by factors such as respiratory motion and anatomical changes. This issue is further exacerbated in scenarios with limited data availability leading to a significant degradation in translation quality. In this paper we introduce a novel data-efficient framework for aligning medical images via Reconfigurable Generative Network (Reconfig-MIT) for high-quality image translation. The key idea of Reconfig-MIT is to adaptively expand the generative network width within a Generative Adversarial Networks (GAN) architecture initially expanding rapidly to capture low-level features and then slowing to refine high-level complexities. This dynamic network adaptation mechanism allows to adaptively learn at different rates thus the model can better respond to deviations in the data caused by misalignments while maintaining an effective equilibrium with the discriminator (D). We also introduce the Recursive Cycle-Consistency Loss (R-CCL) which extends the cycle consistency loss to effectively preserve key anatomical structures and their spatial relationships improving translation quality. Extensive experiments show that Reconfig-MIT is a generic framework that enables easy integration with existing image translation methods including those incorporating registration networks used for correcting misalignments and provides robust and high-quality translation on paired and unpaired misaligned data in both data-rich and data-limited scenarios. https://github.com/IntellicentAI-Lab/Reconfig-MIT.
A Rapid Test for Accuracy and Bias of Face Recognition Technology
Authors: Manuel Knott; Ignacio Serna; Ethan Mann; Pietro Perona
Venue: WACV 2025 (main)

Abstract: Measuring the accuracy of face recognition (FR) systems is essential for improving performance and ensuring responsible use. Accuracy is typically estimated using large annotated datasets which are costly and difficult to obtain. We propose a novel method for 1:1 face verification that benchmarks FR systems quickly and without manual annotation starting from approximate labels (e.g. from web search results). Unlike previous methods for training set label cleaning ours leverages the embedding representation of the models being evaluated achieving high accuracy in smaller-sized test datasets. Our approach reliably estimates FR accuracy and ranking significantly reducing the time and cost of manual labeling. We also introduce the first public benchmark of five FR cloud services revealing demographic biases particularly lower accuracy for Asian women. Our rapid test method can democratize FR testing promoting scrutiny and responsible use of the technology. Our method is provided as a publicly accessible tool at https://github.com/caltechvisionlab/frt-rapid-test.
FineControlNet: Fine-Level Text Control for Image Generation with Spatially Aligned Text Control Injection
Authors: Hongsuk Choi; Isaac Kasahara; Selim Engin; Moritz A. Graule; Nikhil Chavan-Dafle; Volkan Isler
Venue: WACV 2025 (main)

Abstract: Recently introduced ControlNet has the ability to steer the text-driven image generation process with geometric input such as human 2D pose or edge representations. While ControlNet provides control over the geometric form of the instances in the generated image it lacks the capability to dictate the visual appearance of each instance. We present FineControlNet to provide fine control over each instance's appearance while maintaining the pose control capability. Specifically we develop and demonstrate FineControlNet with geometric control via human pose images and appearance control via instance-level text prompts. The spatial alignment of instance-specific text prompts and 2D poses in latent space enables the fine control capabilities of FineControlNet. We evaluate the performance of FineControlNet with rigorous comparison against state-of-the-art pose-conditioned text-to-image diffusion models. FineControlNet achieves superior performance in generating high quality images that follow instance-specific controls. We will release the code and the dataset.
F2FLDM: Latent Diffusion Models with Histopathology Pre-Trained Embeddings for Unpaired Frozen Section to FFPE Translation
Authors: Man M. Ho; Shikha Dubey; Yosep Chong; Beatrice Knudsen; Tolga Tasdizen
Venue: WACV 2025 (main)

Abstract: Frozen Section (FS) technique is a rapid method taking only 15-30 minutes to prepare slides for pathologists' evaluation during surgery enabling immediate surgical decisions. However the FS process often introduces artifacts and distortions like folds and ice-crystal effects. In contrast these artifacts are absent in higher-quality formalin-fixed paraffin-embedded (FFPE) slides which take 2-3 days to prepare. Generative Adversarial Network (GAN)-based methods have been used to translate FS to FFPE images but they may leave morphological inaccuracies or introduce new artifacts reducing translation quality for clinical assessments. In this study we benchmark recent generative models focusing on GANs and Latent Diffusion Models (LDMs) to address these limitations. We introduce a novel approach combining LDMs with Histopathology Pre-Trained Embeddings to enhance FS image restoration. Our framework uses LDMs conditioned by both text and pre-trained embeddings to learn meaningful features of FS and FFPE images. Through diffusion and denoising processes our approach preserves essential diagnostic attributes like color staining and tissue morphology using DDIM Inversion with FS representation. Additionally it enhances the histological quality of translated images by predicting the targeted FFPE representation via an innovative embedding translation mechanism improving morphological details and reducing FS artifacts. As a result this work significantly improves classification performance with the Area Under the Curve rising from 81.99% to 94.64% and achieves the best results in user study. This work establishes a new benchmark for FS to FFPE image translation quality promising enhanced reliability and accuracy in histopathology FS image analysis. Our work is available at https://minhmanho.github.io/f2f_ldm/.