GenAI4ELab/papercli-papers huggingface dataset:
PixSwap: High-Resolution Face Swapping for Effective Reflection of Identity via Pixel-Level Supervision with Synthetic Paired Dataset
Authors: Taewoo Kim; Geonsu Lee; Hyukgi Lee; Seongtae Kim; Younggun Lee
Venue: WACV 2025 (main)

Abstract: Face swapping is to interchange identity features such as eyes nose and lips between a source and a target face while preserving the target attributes like expression pose skin color and hair. Despite considerable advancements in quality over the years recent studies conducting high-resolution face swapping still encounter challenges in reflecting the source identity and ensuring robust performance. We identify two primary reasons for these challenges: (1) the absence of pixel-level supervision and (2) limitations in the model architecture and pipeline. To address the first problem we construct a pseudo-ground truth paired dataset and provide essential pixel-level supervision for effective attribute and identity handling during face swapping. Models trained on the paired dataset significantly enhance the source identity reflection compared to those without the paired dataset. On the other hand existing StyleGAN-based approaches often underutilize source latent vectors or heavily rely on pre-trained models during inference resulting in incomplete identity representation or reduced robustness. Therefore we introduce a novel face-swapping model that is adept at leveraging spatial information of the target attributes and fully utilizing features of the source identity. Thanks to the effective paired dataset and network our model achieves state-of-the-art performance in both image and video-level face swapping notably improving the source identity reflection while preserving the target attributes. Extensive experiments validate the superior performance of our model over existing baselines.
Hijacking Vision-and-Language Navigation Agents with Adversarial Environmental Attacks
Authors: Zijiao Yang; Xiangxi Shi; Eric Slyman; Stefan Lee
Venue: WACV 2025 (main)

Abstract: Assistive embodied agents that can be instructed in natural language to perform tasks in open-world environments have the potential to significantly impact labor tasks like manufacturing or in-home care - benefiting the lives of those who come to depend on them. In this work we consider how this benefit might be hijacked by local modifications in the appearance of the agent's operating environment. Specifically we take the popular Vision-andLanguage Navigation (VLN) task as a representative setting and develop a whitebox adversarial attack that optimizes a 3D attack object's appearance to induce desired behaviors in pretrained VLN agents that observe it in the environment. We demonstrate that the proposed attack can cause VLN agents to ignore their instructions and execute alternative actions after encountering the attack object - even for instructions and agent paths not considered when optimizing the attack. For these novel settings we find our attacks can induce early-termination behaviors or divert an agent along an attacker-defined multi-step trajectory. Under both conditions environmental attacks significantly reduce agent capabilities to successfully follow user instructions.
TORE: Token Recycling in Vision Transformers for Efficient Active Visual Exploration
Authors: Jan Olszewski; Dawid Damian Rymarczyk; Piotr Wojcik; Mateusz Pach; Bartosz Zielinski
Venue: WACV 2025 (main)

Abstract: Active Visual Exploration (AVE) optimizes the utilization of robotic resources in real-world scenarios by sequentially selecting the most informative observations. However modern methods require a high computational budget due to processing the same observations multiple times through the autoencoder transformers. As a remedy we introduce a novel approach to AVE called TOken REcycling (TORE). It divides the encoder into extractor and aggregator components. The extractor processes each observation separately enabling the reuse of tokens passed to the aggregator. Moreover to further reduce the computations we decrease the decoder to only one block. Through extensive experiments we demonstrate that TORE outperforms state-of-the-art methods while reducing computational overhead by up to 90%.
CharDiff: Improving Sampling Convergence via Characteristic Function Consistency in Diffusion Models
Authors: Abhishek Kumar Sinha; S. Manthira Moorthi
Venue: WACV 2025 (main)

Abstract: Diffusion models have demonstrated extensive capabilities for generative modelling in both conditional and conditional image synthesis tasks. The reverse sampling has been the center of interest to improve the overall image quality without retraining the model from scratch. In this work we propose a plug-and-play module by utilizing the characteristic function of the distributions to minimize sampling drift. We experiment with existing diffusion solvers with our module in-place during denoising step to provide additional performance gain in image synthesis linear inverse problem tasks and text-conditioned image synthesis. Moreover We theoretically establish the effectiveness of the method in terms of improved Frechet Inception Distance (FID) and second order Tweedie moment for reduced trajectory deviation.
eLIR-Net: An Efficient AI Solution for Image Retouching
Authors: Tingting Zhao; Chenguang Liu; Kamal Jnawali; Chang Su
Venue: WACV 2025 (main)

Abstract: Picture quality serves as a primary differentiator for prominent display panel manufacturers. AI-based solutions have made remarkable progress in delivering expert-level image color remastering operations. However their demand on intensive computation resources heavily impedes the on-device usage in industries where speed and scale are crucial. In this paper we propose an extremely lightweight image-retouching network (eLIR-Net) that can be deployed on resource-restricted hardware like mobile terminals and embedded devices. The eLIR-Net takes in easily computable and intuitive color distributions to represent the macro view of an image which is encoded by a well-designed condition network to guide the generative base network to converge in the desired direction thus generating visually captivating outputs. Both quantitative and qualitative results show the proposed eLIR-Net can achieve equivalent or superior performance compared to the benchmark models at an affordable cost 6.8K parameters which is 24.1% the size of the smallest state-of-the-art network to the best of our knowledge. This work showcases the possibility that compact models deliver competitive performance compared to large models with affordable cost that enables the benefits of AI to be shared across a wider range of industrial applications.
Bandwidth-Efficient Communication Modelling for Autonomous Vehicle Collaborative Perception
Authors: Dinghao Jin; Yuan Zeng; Yi Gong
Venue: WACV 2025 (main)

Abstract: Accurate perception systems are crucial for intelligent vehicle motion planning and control. Cooperative perception can significantly enhance the perception capabilities of autonomous vehicles by sharing information beyond the field of view of individual vehicles. However most recent studies on cooperative perception have overlooked the bandwidth limitations of information sharing. To address this issue this paper presents a novel multi-agent cooperative 3D object detection framework to improve the detection accuracy of connected autonomous vehicles (CAVs) in bandwidth-constrained wireless communication environments. The framework includes an attention-based communication mechanism to select optimal communication partners and a foreground-background separation strategy to prioritize key information. The effectiveness of the proposed framework is evaluated on three benchmark datasets CARLA-3D V2XSet and OPV2V demonstrating the state-of-the-art performance-bandwidth trade-off.
Guardian of the Ensembles: Introducing Pairwise Adversarially Robust Loss for Resisting Adversarial Attacks in DNN Ensembles
Authors: Shubhi Shukla; Subhadeep Dalui; Manaar Alam; Shubhajit Datta; Arijit Mondal; Debdeep Mukhopadhyay; Partha Pratim Chakrabarti
Venue: WACV 2025 (main)

Abstract: Adversarial attacks rely on transferability where an adversarial example (AE) crafted on a surrogate classifier tends to mislead a target classifier. Recent ensemble methods demonstrate that AEs are less likely to mislead multiple classifiers in an ensemble. This paper proposes a new ensemble training using a Pairwise Adversarially Robust Loss (PARL) that by construction produces an ensemble of classifiers with diverse decision boundaries. PARL utilizes outputs and gradients of each layer with respect to network parameters in every classifier within the ensemble simultaneously. PARL is demonstrated to achieve higher robustness against black-box transfer attacks than previous ensemble methods as well as adversarial training without adversely affecting clean example accuracy. Extensive experiments using standard Resnet20 WideResnet28-10 classifiers demonstrate the robustness of PARL against state-of-the-art adversarial attacks. While maintaining similar clean accuracy and lesser training time the proposed architecture has a 24.8% increase in robust accuracy (e = 0.07) from the state-of-the art method. Code is available at: https://github.com/shubhishukla10/PARL
A Conic Transformation Approach for Solving the Perspective-Three-Point Problem
Authors: Haidong Wu; Snehal Bhayani; Janne Heikkilä
Venue: WACV 2025 (main)

Abstract: We propose a conic transformation method to solve the Perspective-Three-Point (P3P) problem. In contrast to the current state-of-the-art solvers which formulate the P3P problem by intersecting two conics and constructing a degenerate conic to find the intersection our approach builds upon a new formulation based on a transformation that maps the two conics to a new coordinate system where one of the conics becomes a standard parabola in a canonical form. This enables expressing one variable in terms of the other variable and as a consequence substantially simplifies the problem of finding the conic intersection. Moreover the polynomial coefficients are fast to compute and we only need to determine the real-valued intersection points which avoids the requirement of using computationally expensive complex arithmetic. While the current state-of-the-art methods reduce the conic intersection problem to solving a univariate cubic equation our approach despite resulting in a quartic equation is still faster thanks to this new simplified formulation. Extensive evaluations demonstrate that our method achieves higher speed while maintaining robustness and stability comparable to state-of-the-art methods.
Robot Instance Segmentation with Few Annotations for Grasping
Authors: Moshe Kimhi; David Vainshtein; Chaim Baskin; Dotan Di Castro
Venue: WACV 2025 (main)

Abstract: The ability of robots to manipulate objects relies heavily on their aptitude for visual perception. In domains characterized by cluttered scenes and high object variability most methods call for vast labeled datasets laboriously hand-annotated with the aim of training capable models. Once deployed the challenge of generalizing to unfamiliar objects implies that the model must evolve alongside its domain. To address this we propose a novel framework that combines Semi-Supervised Learning (SSL) with Learning Through Interaction (LTI) allowing a model to learn by observing scene alterations and leverage visual consistency despite temporal gaps without requiring curated data of interaction sequences. As a result our approach exploits partially annotated data through self-supervision and incorporates temporal context using pseudo-sequences generated from unlabeled still images. We validate our method on two common benchmarks ARMBench mix-object-tote and OCID where it achieves state-of-the-art performance. Notably on ARMBench we attain an AP50 of 86.37 almost a 20% improvement over existing work and obtain remarkable results in scenarios with extremely low annotation achieving an AP50 score of 84.89 with just 1% of annotated data compared to previous state of the art of 82 which targeted the fully annotated dataset.
Volumetric Conditioning Module to Control Pretrained Diffusion Models for 3D Medical Images
Authors: Suhyun Ahn; Wonjung Park; Jihoon Cho; Jinah Park
Venue: WACV 2025 (main)

Abstract: Spatial control methods using additional modules on pretrained diffusion models have gained attention for enabling conditional generation in natural images. These methods guide the generation process with new conditions while leveraging the capabilities of large models. They could be beneficial as training strategies in the context of 3D medical imaging where training a diffusion model from scratch is challenging due to high computational costs and data scarcity. However the potential application of spatial control methods with additional modules to 3D medical images has not yet been explored. In this paper we present a tailored spatial control method for 3D medical images with a novel lightweight module Volumetric Conditioning Module (VCM). Our VCM employs an asymmetric U-Net architecture to effectively encode complex information from various levels of 3D conditions providing detailed guidance in image synthesis. To examine the applicability of spatial control methods and the effectiveness of VCM for 3D medical data we conduct experiments under single and multimodal conditions scenarios across a wide range of dataset sizes from extremely small datasets with 10 samples to large datasets with 500 samples. The experimental results show that the VCM is effective for conditional generation and efficient in terms of requiring less training data and computational resources. We further investigate the potential applications for our spatial control method through axial super-resolution for medical images. Our code is available at https://github.com/SSTDVProject/ VCM.git
Revisiting Machine Unlearning with Dimensional Alignment
Authors: Seonguk Seo; Dongwan Kim; Bohyung Han
Venue: WACV 2025 (main)

Abstract: Machine unlearning an emerging research topic focusing on data privacy compliance enables trained models to erase information learned from specific data. While many existing methods indirectly address this issue by intentionally injecting incorrect supervision they often result in drastic and unpredictable changes to decision boundaries and feature spaces leading to training instability and undesired side effects. To address this challenge more fundamentally we first analyze the changes in latent feature spaces between the original and retrained models and observe that the feature representations of samples not included in training are closely aligned with the feature manifolds of previously seen samples. Building on this insight we introduce a novel evaluation metric for machine unlearning coined dimensional alignment which measures the alignment between the eigenspaces of the forget and retain sets. We incorporate this metric as a regularizer loss to develop a robust and stable unlearning framework which is further enhanced by a self-distillation loss and an alternating training scheme. Our framework effectively eliminates information from the forget set while preserving knowledge from the retain set. Finally we identify critical flaws in existing evaluation metrics for machine unlearning and propose new tools that more accurately capture its fundamental objectives.
OT-VP: Optimal Transport-Guided Visual Prompting for Test-Time Adaptation
Authors: Yunbei Zhang; Akshay Mehra; Jihun Hamm
Venue: WACV 2025 (main)

Abstract: Vision Transformers (ViTs) have demonstrated remarkable capabilities in learning representations but their performance is compromised when applied to unseen domains. Previous methods either engage in prompt learning during the training phase or modify model parameters at test time through entropy minimization. The former often overlooks unlabeled target data while the latter doesn't fully address domain shifts. In this work our approach Optimal Transport-guided Test-Time Visual Prompting (OT-VP) handles these problems by leveraging prompt learning at test time to align the target and source domains without accessing the training process or altering pre-trained model parameters. This method involves learning a universal visual prompt for the target domain by optimizing the Optimal Transport distance. With only four learned prompt tokens OT-VP exceeds state-of-the-art performance across three stylistic datasets--PACS VLCS OfficeHome and one corrupted dataset ImageNet-C. Additionally OT-VP operates efficiently both in terms of memory and computation and is adaptable for extension to online settings. The code is available at https://github.com/zybeich/OT-VP.
Towards Zero-Shot 3D Anomaly Localization
Authors: Yizhou Wang; Kuan-Chuan Peng; Yun Fu
Venue: WACV 2025 (main)

Abstract: 3D anomaly detection and localization is of great significance for industrial inspection. Prior 3D anomaly detection and localization methods focus on the setting that the testing data share the same category as the training data which is normal. However in real-world applications the normal training data for the target 3D objects can be unavailable due to issues like data privacy or export control regulation. To tackle these challenges we identify a new task - zero-shot 3D anomaly detection and localization where the training and testing classes do not overlap. To this end we design 3DzAL a novel patch-level contrastive learning framework based on pseudo anomalies generated using the inductive bias from task-irrelevant 3D xyz data to learn more representative feature representations. Furthermore we train a normalcy classifier network to classify the normal patches and pseudo anomalies and utilize the classification result jointly with feature distance to design anomaly scores. Instead of directly using the patch point clouds we introduce adversarial perturbations to the input patch xyz data before feeding into the 3D normalcy classifier for the classification-based anomaly score. We show that 3DzAL outperforms the state-of-the-art anomaly detection and localization performance.
SegDesicNet: Lightweight Semantic Segmentation in Remote Sensing with Geo-Coordinate Embeddings for Domain Adaptation
Authors: Sachin Verma; Frank Lindseth; Gabriel Kiss
Venue: WACV 2025 (main)

Abstract: Semantic segmentation is essential for analyzing high-definition remote sensing images (HRSIs) because it allows the precise classification of objects and regions at the pixel level. However remote sensing data present challenges owing to geographical location weather and environmental variations making it difficult for semantic segmentation models to generalize across diverse scenarios. Existing methods are often limited to specific data domains and require expert annotators and specialized equipment for semantic labeling. In this study we propose a novel unsupervised domain adaptation technique for remote sensing semantic segmentation by utilizing geographical coordinates that are readily accessible in remote sensing setups as metadata in a dataset. To bridge the domain gap we propose a novel approach that considers the combination of an image's location-encoding trait and the spherical nature of Earth's surface. Our proposed SegDesicNet module regresses the GRID positional encoding of the geocoordinates projected over the unit sphere to obtain the domain loss. Our experimental results demonstrate that the proposed SegDesicNet outperforms state-of-the-art domain adaptation methods in remote sensing image segmentation achieving an improvement of approximately 6% in the mean intersection over union (MIoU) with a  27% drop in parameter count on benchmarked subsets of the publicly available FLAIR #1 dataset. We also benchmarked our method performance on the custom split of the ISPRS Potsdam dataset. Our algorithm seeks to reduce the modeling disparity between artificial neural networks and human comprehension of the physical world making the technology more human-centric and scalable.
HexaGen3D: StableDiffusion is One Step Away from Fast and Diverse Text-to-3D Generation
Authors: Antoine Mercier; Ramin Nakhli; Mahesh Reddy; Rajeev Yasarla; Hong Cai; Fatih Porikli; Guillaume Berger
Venue: WACV 2025 (main)

Abstract: Despite the latest remarkable advances in generative modeling efficient generation of high-quality 3D objects from textual prompts remains a difficult task. A key challenge lies in data scarcity: the most extensive 3D datasets encompass merely millions of samples while their 2D counterparts contain billions of text-image pairs. To address this we propose a novel approach which harnesses the power of large pretrained 2D diffusion models. More specifically our approach HexaGen3D fine-tunes a pretrained text-to-image model to jointly predict 6 orthographic projections and the corresponding 3D latent. We then decode these latents to generate a textured mesh. HexaGen3D does not require per-sample optimization and can infer high-quality and diverse objects from textual prompts in 7 seconds offering significantly better quality-to-latency trade-offs than existing approaches. Furthermore HexaGen3D demonstrates strong generalization to new objects or compositions.
VMAs: Video-to-Music Generation via Semantic Alignment in Web Music Videos
Authors: Yan-Bo Lin; Yu Tian; Linjie Yang; Gedas Bertasius; Heng Wang
Venue: WACV 2025 (main)

Abstract: We present a framework for learning to generate background music from video inputs. Unlike existing works that rely on symbolic musical annotations which are limited in quantity and diversity our method leverages large-scale web videos accompanied by background music. This enables our model to learn to generate realistic and diverse music. To accomplish this goal we develop a generative video-music Transformer with a novel semantic video-music alignment scheme. Our model uses a joint autoregressive and contrastive learning objective which encourages the generation of music aligned with high-level video content. We also introduce a novel video-beat alignment scheme to match the generated music beats with the low-level motions in the video. Lastly to capture fine-grained visual cues in a video needed for realistic background music generation we introduce a new temporal video encoder architecture allowing us to efficiently process videos consisting of many densely sampled frames. We train our framework on our newly curated DISCO-MV dataset consisting of 2.2M video-music samples which is orders of magnitude larger than any prior datasets used for video music generation. Our method outperforms existing approaches on the DISCO-MV and MusicCaps datasets according to various music generation evaluation metrics including human evaluation. Results are available at https://genjib.github.io/project_page/VMAs/index.html