GenAI4ELab/papercli-papers huggingface dataset:
Are Exemplar-Based Class Incremental Learning Models Victim of Black-Box Poison Attacks?
Authors: Neeresh Kumar Perla; Md. Iqbal Hossain; Afia Sajeeda; Ming Shao
Venue: WACV 2025 (main)

Abstract: Class Incremental Learning (CIL) models are designed to continuously learn new classes without forgetting previously learned ones often relying on an exemplar set to retain a portion of knowledge from previously learned classes. However their vulnerability to adversarial attacks under novel and unexplored conditions remains unstudied. In this work we are the first to evaluate the robustness of exemplar-based CIL models using a non-overlapping dataset where the dataset is independent of the training and test sets of the target model. We propose and implement a novel black-box attack framework targeting the exemplar set of class incremental learning models using zero-overlapping data. Specifically we focus on scenarios where the target model provides only hard-label predictions without interactive access. Our experimental evaluation covers a range of exemplar-based incremental learning algorithms different surrogate models and black-box attack options. Our findings reveal significant vulnerabilities in exemplar-based CIL models to poisoning-based attacks using a non-overlapping dataset.
CLIPping Imbalances: A Novel Evaluation Baseline and PEARL Dataset for Pedestrian Attribute Recognition
Authors: Kamalakar Vijay; Lalit Lohani; Kamakshya Prasad Nayak; Debi Prosad Dogra; Heeseung Choi; Hyungjoo Jung; Ig-Jae Kim
Venue: WACV 2025 (main)

Abstract: Pedestrian Attribute Recognition (PAR) serves as a fundamental task in computer vision and is crucial for upgrading security systems. It helps in precisely identifying and characterizing various attributes of pedestrians. However current PAR datasets have certain issues in representing a wide range of attributes correctly which makes the existing PAR methods less effective in real-world scenarios. Addressing this limitation this paper introduces PEARL a comprehensive dataset comprising of diverse pedestrian images annotated with 146 attributes. These samples have been sourced from surveillance videos across twelve countries. This paper also formulates an image-based PAR using language-image fusion strategy and utilizes CLIP as a new evaluation baseline. Specifically we leverage textual information by transforming sets of attributes into meaningful sentences. Addressing the inherent data imbalance in PAR we provide three types of prompt settings to optimize the training of the CLIP model. Our evaluation encompasses a thorough assessment of the proposed baseline model across various datasets including PEARL dataset as well as established PAR benchmarks such as PA100K RAP and PETA.
Exploring the Stability Gap in Continual Learning: The Role of the Classification Head
Authors: Wojciech Łapacz; Daniel Marczak; Filip Szatkowski; Tomasz Trzciński
Venue: WACV 2025 (main)

Abstract: Continual learning (CL) has emerged as a critical area in machine learning enabling neural networks to learn from evolving data distributions while mitigating catastrophic forgetting. However recent research has identified the stability gap - a phenomenon where models initially lose performance on previously learned tasks before partially recovering during training. Such learning dynamics are contradictory to the intuitive understanding of stability in continual learning where one would expect the performance to degrade gradually instead of rapidly decreasing and then partially recovering later. To better understand and alleviate the stability gap we investigate it at different levels of the neural network architecture particularly focusing on the role of the classification head. We introduce the nearest-mean classifier (NMC) as a tool to attribute the influence of the backbone and the classification head on the stability gap. Our experiments demonstrate that NMC not only improves final performance but also significantly enhances training stability across various continual learning benchmarks including CIFAR100 ImageNet100 CUB-200 and FGVC Aircrafts. Moreover we find that NMC also reduces task-recency bias. Our analysis provides new insights into the stability gap and suggests that the primary contributor to this phenomenon is the linear head rather than the insufficient representation learning.
Bayesian Optimal Latent Projection for Noisy Image Restoration
Authors: Ziqiang Shi; Rujie Liu; Jun Takahashi; Takuma Yamamoto
Venue: WACV 2025 (main)

Abstract: In recent years image restoration using large-scale latent diffusion generative models (DGM) has attracted increasing attention and achieved significant progress. Most of these latent DGM-based image restoration methods require predicting the original clean image in each iteration which is then used to estimate the image for the next iteration. However these predicted original clean images are often inaccurate leading to errors in the subsequent image estimation. In other words there is a significant deviation between the final sampling restoration trajectory and the ground truth trajectory. The purpose of this paper is to narrow the gap between these two trajectories and enhance the performance of image restoration. We propose the Bayesian Optimal Latent Projection (BOLP) algorithm which identifies the optimal random noise within the Gaussian distribution to iteratively correct the estimated image at each step thereby minimizing the distance to the ground truth image. Experiments in deblurring super-resolution and inpainting on FFHQ and ImageNet datasets demonstrate that the BOLP outperforms the previously established best algorithms and sets a new state of the art.
Towards a Training Free Approach for 3D Scene Editing
Authors: Vivek Madhavaram; Shivangana Rawat; Chaitanya Devaguptapu; Charu Sharma; Manohar Kaul
Venue: WACV 2025 (main)

Abstract: Text driven diffusion models have shown remarkable capabilities in editing images. However when editing 3D scenes existing works mostly rely on training a NeRF for 3D editing. Recent NeRF editing methods leverages edit operations by deploying 2D diffusion models and project these edits into 3D space. They require strong positional priors alongside text prompt to identify the edit location. These methods are operational on small 3D scenes and are more generalized to particular scene. They require training for each specific edit and cannot be exploited in real-time edits. To address these limitations we propose a novel method FreeEdit to make edits in training free manner using mesh representations as a substitute for NeRF. Training-free methods are now a possibility because of the advances in foundation model's space. We leverage these models to bring a training-free alternative and introduce solutions for insertion replacement and deletion. We consider insertion replacement and deletion as basic blocks for performing intricate edits with certain combinations of these operations. Given a text prompt and a 3D scene our model is capable of identifying what object should be inserted/replaced or deleted and location where edit should be performed. We also introduce a novel algorithm as part of FreeEdit to find the optimal location on grounding object for placement. We evaluate our model by comparing it with baseline models on a wide range of scenes using quantitative and qualitative metrics and showcase the merits of our method with respect to others. Project page: https://vivekmadhavaram.github.io/FreeEdit_page/1.
Self-Supervised Learning with Probabilistic Density Labeling for Rainfall Probability Estimation
Authors: Junha Lee; Sojung An; Sujeong You; Namik Cho
Venue: WACV 2025 (main)

Abstract: Numerical weather prediction (NWP) models are fundamental in meteorology for simulating and forecasting the behavior of various atmospheric variables. The accuracy of precipitation forecasts and the acquisition of sufficient lead time are crucial for preventing hazardous weather events. However the performance of NWP models is limited by the nonlinear and unpredictable patterns of extreme weather phenomena driven by temporal dynamics. In this regard we propose a Self-Supervised Learning with Probabilistic Density Labeling (SSLPDL) for estimating rainfall probability by post-processing NWP forecasts. Our post-processing method uses self-supervised learning (SSL) with masked modeling for reconstructing atmospheric physics variables enabling the model to learn the dependency between variables. The pre-trained encoder is then utilized in transfer learning to a precipitation segmentation task. Furthermore we introduce a straightforward labeling approach based on probability density to address the class imbalance in extreme weather phenomena like heavy rain events. Experimental results show that SSLPDL surpasses other precipitation forecasting models in regional precipitation post-processing and demonstrates competitive performance in extending forecast lead times. Our code is available at https://github.com/joonha425/SSLPDL.
STAY Diffusion: Styled Layout Diffusion Model for Diverse Layout-to-Image Generation
Authors: Ruyu Wang; Xuefeng Hou; Sabrina Schmedding; Marco Huber
Venue: WACV 2025 (main)

Abstract: In layout-to-image (L2I) synthesis controlled complex scenes are generated from coarse information like bounding boxes. Such a task is exciting to many downstream applications because the input layouts offer strong guidance to the generation process while remaining easily reconfigurable by humans. In this paper we proposed STyled LAYout Diffusion (STAY Diffusion) a diffusion-based model that produces photo-realistic images and provides fine-grained control of stylized objects in scenes. Our approach learns a global condition for each layout and a self-supervised semantic map for weight modulation using a novel Edge-Aware Normalization (EA Norm). A new Styled-Mask Attention (SM Attention) is also introduced to cross-condition the global condition and image feature for capturing the objects' relationships. These measures provide consistent guidance through the model enabling more accurate and controllable image generation. Extensive benchmarking demonstrates that our STAY Diffusion presents high-quality images while surpassing previous state-of-the-art methods in generation diversity accuracy and controllability.
Learning to Count from Pseudo-Labeled Segmentation
Authors: Jingyi Xu; Hieu Le; Dimitris Samaras
Venue: WACV 2025 (main)

Abstract: Class-agnostic counting (CAC) has numerous potential applications across various domains. The goal is to count objects of an arbitrary category during testing based on only a few annotated exemplars. However existing methods often count all objects in the image including those from different categories than the exemplars. To address this issue we propose localizing the area containing the objects of interest via an exemplar-based segmentation model before counting them. To train this model we propose a novel method to obtain pseudo-labeled segmentation masks. Specifically we use an unsupervised image clustering method to generate a set of candidate pseudo object masks from which we select the optimal one using a pre-trained CAC model. We show that the trained segmentation model can effectively localize objects of interest based on the exemplars and prevent the model from counting everything. To properly evaluate the performance of CAC methods in real-world scenarios we introduce two new benchmarks: a synthetic test set and a new test set of real images containing countable objects from multiple classes. Our proposed method shows a significant advantage over previous CAC methods on these two benchmarks.
Text-to-Image Synthesis for Domain Generalization in Face Anti-Spoofing
Authors: Naeun Ko; Yonghyun Jeong; Jong Chul Ye
Venue: WACV 2025 (main)

Abstract: This paper addresses the challenge of developing robust Face Anti-Spoofing (FAS) models for face recognition systems. Traditional FAS protocols are limited by a lack of diversity in subject identities and environmental conditions restricting generalization to real-world scenarios. Recent advancements in spoof image synthesis have mitigated data scarcity but still fail to capture the full range of facial attributes and environmental variability needed for effective domain generalization. To address this we propose a novel framework capable of generating diverse realistic facial images with text-guided control. We fine-tune Stable Diffusion to extract real facial features and specifically train LoRA layers to capture detailed spoof patterns. Additionally the text-guided control of attributes helps overcome the lack of diversity seen in previous methods. Extensive experiments demonstrate that our text-to-image-based synthetic data generation significantly enhances the robustness of FAS models establishing a new benchmark for domain-independent and reliable anti-spoofing systems.
Generalizable Single-Source Cross-Modality Medical Image Segmentation via Invariant Causal Mechanisms
Authors: Boqi Chen; Yuanzhi Zhu; Yunke Ao; Sebastiano Caprara; Reto Sutter; Gunnar Rätsch; Ender Konukoglu; Anna Susmelj
Venue: WACV 2025 (main)

Abstract: Single-source domain generalization (SDG) aims to learn a model from a single source domain that can generalize well on unseen target domains. This is an important task in computer vision particularly relevant to medical imaging where domain shifts are common. In this work we consider a challenging yet practical setting: SDG for cross-modality medical image segmentation. We combine causality-inspired theoretical insights on learning domain-invariant representations with recent advancements in diffusion-based augmentation to improve generalization across diverse imaging modalities. Guided by the "intervention-augmentation equivariant" principle we use controlled diffusion models (DMs) to simulate diverse imaging styles while preserving the content leveraging rich generative priors in large-scale pretrained DMs to comprehensively perturb the multidimensional style variable. Extensive experiments on challenging cross-modality segmentation tasks demonstrate that our approach consistently outperforms state-of-the-art SDG methods across three distinct anatomies and imaging modalities. The source code is available at https://github.com/ratschlab/ICMSeg.
Combining Inherent Knowledge of Vision-Language Models with Unsupervised Domain Adaptation through Strong-Weak Guidance
Authors: Thomas Westfechtel; Dexuan Zhang; Tatsuya Harada
Venue: WACV 2025 (main)

Abstract: Unsupervised domain adaptation (UDA) tries to overcome the tedious work of labeling data by leveraging a labeled source dataset and transferring its knowledge to a similar but different target dataset. Meanwhile current vision-language models exhibit remarkable zero-shot prediction capabilities. In this work we combine knowledge gained through UDA with the inherent knowledge of vision-language models. We introduce a strong-weak guidance learning scheme that employs zero-shot predictions to help align the source and target dataset. For the strong guidance we expand the source dataset with the most confident samples of the target dataset. Additionally we employ a knowledge distillation loss as weak guidance. The strong guidance uses hard labels but is only applied to the most confident predictions from the target dataset. Conversely the weak guidance is employed to the whole dataset but uses soft labels. The weak guidance is implemented as a knowledge distillation loss with (adjusted) zero-shot predictions. We show that our method complements and benefits from prompt adaptation techniques for vision-language models. We conduct experiments and ablation studies on three benchmarks (OfficeHome VisDA and DomainNet) outperforming state-of-the-art methods. Our ablation studies further demonstrate the contributions of different components of our algorithm.
Text Change Detection in Multilingual Documents using Image Comparison
Authors: Doyoung Park; Naresh Reddy Yarram; Sunjin Kim; MinKyu Kim; Seongho Joe; Taehee Lee
Venue: WACV 2025 (main)

Abstract: Document comparison typically relies on optical character recognition (OCR) as its core technology. However OCR requires the selection of appropriate language models for each document and the performance of multilingual or hybrid models remains limited. To overcome these challenges we propose text change detection (TCD) using an image comparison model tailored for multilingual documents. Unlike OCR-based approaches our method employs word-level text image-to-image comparison to detect changes. Our model generates bidirectional change segmentation maps between the source and target documents. To enhance performance without requiring explicit text alignment or scaling preprocessing we employ correlations among multi-scale attention features. We also construct a benchmark dataset comprising actual printed and scanned word pairs in various languages to evaluate our model. We validate our approach using our benchmark dataset and public benchmarks Distorted Document Images and the LRDE Document Binarization Dataset. We compare our model against state-of-the-art semantic segmentation and change detection models as well as to conventional OCR-based models.
ComFace: Facial Representation Learning with Synthetic Data for Comparing Faces
Authors: Yusuke Akamatsu; Terumi Umematsu; Hitoshi Imaoka; Shizuko Gomi; Hideo Tsurushima
Venue: WACV 2025 (main)

Abstract: Daily monitoring of intra-personal facial changes associated with health and emotional conditions has great potential to be useful for medical healthcare and emotion recognition fields. However the approach for capturing intra-personal facial changes is relatively unexplored due to the difficulty of collecting temporally changing face images. In this paper we propose a facial representation learning method using synthetic images for comparing faces called ComFace which is designed to capture intra-personal facial changes. For effective representation learning ComFace aims to acquire two feature representations i.e. inter-personal facial differences and intra-personal facial changes. The key point of our method is the use of synthetic face images to overcome the limitations of collecting real intra-personal face images. Facial representations learned by ComFace are transferred to three extensive downstream tasks for comparing faces: estimating facial expression changes weight changes and age changes from two face images of the same individual. Our ComFace trained using only synthetic data achieves comparable to or better transfer performance than general pre-training and state-of-the-art representation learning methods trained using real images.
Click&Describe: Multimodal Grounding and Tracking for Aerial Objects
Authors: Rupanjali Kukal; Jay Patravali; Fuxun Yu; Simranjit Singh; Nikolaos Karianakis; Rishi Madhok
Venue: WACV 2025 (main)

Abstract: The fusion of multiple modalities such as vision and language has led to significant progress in grounding and tracking tasks. However this success has not yet translated to aerial single-object tracking (SOT) due to the lack of text annotations in existing aerial SOT datasets. To overcome this limitation we provide text annotations for five existing aerial datasets designed to support and promote multi-modal research in the aerial tracking domain. Furthermore to address challenges such as small object dimensions similar-looking objects and target size fluctuations we introduce a third input modality: click (or point prompt). To offer a user-friendly and interactive alternative to precise bounding box annotations we seamlessly integrate click and language information in the model's input. This enables approximate target specification with reduced effort and time. We introduce CLaVi a novel multimodal framework that redefines input interaction by incorporating multiple modalities. This integration improves target localization and tracking efficiency providing a significant advancement in the way input is provided to the model. Furthermore we conduct experiments on the five datasets to provide AerTrack-460 benchmark to validate the effectiveness of our approach. AerTrack-460 benchmark shows competitive performance and in some cases outperforms previous language-based grounding and tracking techniques setting a strong baseline for future research. Code and data will be made available soon.
Disentangling Disentangled Representations: Towards Improved Latent Units via Diffusion Models
Authors: Youngjun Jun; Jiwoo Park; Kyobin Choo; Tae Eun Choi; Seong Jae Hwang
Venue: WACV 2025 (main)

Abstract: Disentangled representation learning (DRL) aims to break down observed data into core intrinsic factors for a profound understanding of the data. In real-world scenarios manually defining and labeling these factors are non-trivial making unsupervised methods attractive. Recently there have been limited explorations of utilizing diffusion models (DMs) which are already mainstream in generative modeling for unsupervised DRL. They implement their own inductive bias to ensure that each latent unit input to the DM expresses only one distinct factor. In this context we design Dynamic Gaussian Anchoring to enforce attribute-separated latent units for more interpretable DRL. This unconventional inductive bias explicitly delineates the decision boundaries between attributes while also promoting the independence among latent units. Additionally we also propose Skip Dropout technique which easily modifies the denoising U-Net to be more DRL-friendly addressing its uncooperative nature with the disentangling feature extractor. Our methods which carefully consider the latent unit semantics and the distinct DM structure enhance the practicality of DM-based disentangled representations demonstrating state-of-the-art disentanglement performance on both synthetic and real data as well as advantages in downstream tasks.
XR-MBT: Multi-Modal Full Body Tracking for XR through Self-Supervision with Learned Depth Point Cloud Registration
Authors: Denys Rozumnyi; Nadine Bertsch; Othman Sbai; Filippo Arcadu; Yuhua Chen; Artsiom Sanakoyeu; Manoj Kumar; Catherine Herold; Robin Kips
Venue: WACV 2025 (main)

Abstract: Tracking the full body motions of users in XR (AR/VR) devices is a fundamental challenge to bring a sense of authentic social presence. Due to the absence of dedicated leg sensors currently available body tracking methods adopt a synthesis approach to generate plausible motions given a 3-point signal from the head and controller tracking. In order to enable mixed reality features modern XR devices are capable of estimating depth information of the headset surroundings using available sensors combined with dedicated machine learning models. Such egocentric depth sensing cannot drive the body directly as it is not registered and is incomplete due to limited field-of-view and body self-occlusions. For the first time we propose to leverage the available depth sensing signal combined with self-supervision to learn a multi-modal pose estimation model capable of tracking full body motions in real time on XR devices. We demonstrate how current 3-point motion synthesis models can be extended to point cloud modalities using a semantic point cloud encoder network combined with a residual network for multi-modal pose estimation. These modules are trained jointly in a self-supervised way leveraging a combination of real unregistered point clouds and simulated data obtained from motion capture. We compare our approach against several state-of-the-art systems for XR body tracking and show that our method accurately tracks a diverse range of body motions. XR-MBT tracks legs in XR for the first time whereas traditional synthesis approaches based on partial body tracking are blind.