GenAI4ELab/papercli-papers huggingface dataset:
Re-Evaluating Group Robustness via Adaptive Class-Specific Scaling
Authors: Seonguk Seo; Bohyung Han
Venue: WACV 2025 (main)

Abstract: Group distributionally robust optimization which aims to improve robust accuracies--worst-group and unbiased accuracies--is a prominent algorithm used to mitigate spurious correlations and address dataset bias. Although existing approaches have reported improvements in robust accuracies these gains often come at the cost of average accuracy due to inherent trade-offs. To control this trade-off flexibly and efficiently we propose a simple class-specific scaling strategy directly applicable to existing debiasing algorithms with no additional training. We further develop an instance-wise adaptive scaling technique to alleviate this trade-off even leading to improvements in both robust and average accuracies. Our approach reveals that a naive ERM baseline matches or even outperforms the recent debiasing methods by simply adopting the class-specific scaling technique. Additionally we introduce a novel unified metric that quantifies the trade-off between the two accuracies as a scalar value allowing for a comprehensive evaluation of existing algorithms. By tackling the inherent trade-off and offering a performance landscape our approach provides valuable insights into robust techniques beyond just robust accuracy. We validate the effectiveness of our framework through experiments across datasets in computer vision and natural language processing domains.
Ego-VPA: Egocentric Video Understanding with Parameter-Efficient Adaptation
Authors: Tz-Ying Wu; Kyle Min; Subarna Tripathi; Nuno Vasconcelos
Venue: WACV 2025 (main)

Abstract: Video understanding typically requires fine-tuning the large backbone when adapting to new domains. In this paper we leverage the egocentric video foundation models (Ego-VFMs) based on video-language pre-training and propose a parameter-efficient adaptation for egocentric video tasks namely Ego-VPA. It employs a local sparse approximation for each video frame/text feature using the basis prompts and the selected basis prompts are used to synthesize video/text prompts. Since the basis prompts are shared across frames and modalities it models context fusion and cross-modal transfer in an efficient fashion. Experiments show that Ego-VPA excels in lightweight adaptation (with only 0.84% learnable parameters) largely improving over baselines and reaching the performance of full fine-tuning.
Weight Copy and Low-Rank Adaptation for Few-Shot Distillation of Vision Transformers
Authors: Diana-Nicoleta Grigore; Mariana-Iuliana Georgescu; Jon Alvarez Justo; Tor Johansen; Andreea Iuliana Ionescu; Radu Tudor Ionescu
Venue: WACV 2025 (main)

Abstract: Few-shot knowledge distillation recently emerged as a viable approach to harness the knowledge of large-scale pre-trained models using limited data and computational resources. In this paper we propose a novel few-shot feature distillation approach for vision transformers. Our approach is based on two key steps. Leveraging the fact that vision transformers have a consistent depth-wise structure we first copy the weights from intermittent layers of existing pre-trained vision transformers (teachers) into shallower architectures (students) where the intermittence factor controls the complexity of the student transformer with respect to its teacher. Next we employ an enhanced version of Low-Rank Adaptation (LoRA) to distill knowledge into the student in a few-shot scenario aiming to recover the information processing carried out by the skipped teacher layers. We present comprehensive experiments with supervised and self-supervised transformers as teachers on six data sets from various domains (natural medical and satellite images) and tasks (classification and segmentation). The empirical results confirm the superiority of our approach over state-of-the-art competitors. Moreover the ablation results demonstrate the usefulness of each component of the proposed pipeline. We release our code at https://github.com/dianagrigore/WeCoLoRA.
FAIR-TAT: Improving Model Fairness using Targeted Adversarial Training
Authors: Tejaswini Medi; Steffen Jung; Margret Keuper
Venue: WACV 2025 (main)

Abstract: Deep neural networks are susceptible to adversarial attacks and common corruptions which undermine their robustness. In order to enhance model resilience against such challenges Adversarial Training (AT) has emerged as a prominent solution. Nevertheless adversarial robustness is often attained at the expense of model fairness during AT i.e. disparity in class-wise robustness of the model. While distinctive classes become more robust towards such adversaries hard to detect classes suffer. Recently research has focused on improving model fairness specifically for perturbed images overlooking the accuracy of the most likely non-perturbed data. Additionally despite their robustness against the adversaries encountered during model training state-of-the-art adversarial trained models have difficulty maintaining robustness and fairness when confronted with diverse adversarial threats or common corruptions. In this work we address the above concerns by introducing a novel approach called Fair Targeted Adversarial Training (FAIR-TAT). We show that using targeted adversarial attacks for adversarial training (instead of untargeted attacks) can allow for more favorable trade-offs with respect to adversarial fairness. Empirical results validate the efficacy of our approach.
Semantic Prompting with Image Token for Continual Learning
Authors: Jisu Han; Jaemin Na; Wonjun Hwang
Venue: WACV 2025 (main)

Abstract: Continual learning aims to refine model parameters for new tasks while retaining knowledge from previous tasks. Recently prompt-based learning has emerged to leverage pre-trained models to be prompted to learn subsequent tasks without the reliance on the rehearsal buffer. Although this approach has demonstrated outstanding results existing methods depend on preceding task-selection process to choose appropriate prompts. However imperfectness in task-selection may lead to negative impacts on the performance particularly in the scenarios where the number of tasks is large or task distributions are imbalanced. To address this issue we introduce a novel task-agnostic approach that focuses on the visual semantic information of image tokens eliminating the preceding task prediction. By leveraging the ability of the pre-trained model to discriminate between similar tokens our method not only subdivides the prompt but also eliminates the need for additional forward pass. Consequently we achieve competitive performance on four benchmarks while significantly reducing training time compared to state-of-the-art methods. The code is available at https://github.com/pilsHan/I-Prompt
EmoVOCA: Speech-Driven Emotional 3D Talking Heads
Authors: Federico Nocentini; Claudio Ferrari; Stefano Berretti
Venue: WACV 2025 (main)

Abstract: A notable challenge in 3D talking head generation consists in blending speech-related motions with expression dynamics. This is primarily caused by the lack of comprehensive 3D datasets that combine diversity in spoken sentences with a variety of facial expressions. Some literature works attempted to overcome such lack of data by fitting parametric 3D models (3DMMs) to 2D videos and using the reconstructed 3D faces as replacement. However their underlying parametric space limits the precision required to accurately reproduce convincing lip motions and synching which is crucial for the application at hand. In this work we look at the problem from a different perspective and developed a data-driven technique to combine inexpressive 3D talking heads with a set of 3D expressive sequences which we used for creating a synthetic dataset called EmoVOCA. We then designed and trained an emotional 3D talking head generator that accepts a 3D face an audio file an emotion label and an intensity value as inputs and learns to animate the audio-synchronized lip movements with expressive traits of the face. Comprehensive experiments both quantitative and qualitative using our data and generator evidence superior ability in synthesizing convincing animations when compared with the best performing methods in the literature. Our code and pre-trained models are available at https://github.com/miccunifi/EmoVOCA.
Sun Off Lights On: Photorealistic Monocular Nighttime Simulation for Robust Semantic Perception
Authors: Konstantinos Tzevelekakis; Shutong Zhang; Luc Van Gool; Christos Sakaridis
Venue: WACV 2025 (main)

Abstract: Nighttime scenes are hard to semantically perceive with learned models and annotate for humans. Thus realistic synthetic nighttime data become all the more important for learning robust semantic perception at night thanks to their accurate and cheap semantic annotations. However existing data-driven or hand-crafted techniques for generating nighttime images from daytime counterparts suffer from poor realism. The reason is the complex interaction of highly spatially varying nighttime illumination which differs drastically from its daytime counterpart with objects of spatially varying materials in the scene happening in 3D and being very hard to capture with such 2D approaches. The above 3D interaction and illumination shift have proven equally hard to model in the literature as opposed to other conditions such as fog or rain. Our method named Sun Off Lights On (SOLO) is the first to perform nighttime simulation on single images in a photorealistic fashion by operating in 3D. It first explicitly estimates the 3D geometry the materials and the locations of light sources of the scene from the input daytime image and relights the scene by probabilistically instantiating light sources in a way that accounts for their semantics and then running standard ray tracing. Not only is the visual quality and photorealism of our nighttime images superior to competing approaches including diffusion models but the former images are also proven more beneficial for semantic nighttime segmentation in day-to-night adaptation. Code and data are publicly available at https://github.com/ktzevel/SOLO.
Online-LoRA: Task-Free Online Continual Learning via Low Rank Adaptation
Authors: Xiwen Wei; Guihong Li; Radu Marculescu
Venue: WACV 2025 (main)

Abstract: Catastrophic forgetting is a significant challenge in online continual learning (OCL) especially for non-stationary data streams that do not have well-defined task boundaries. This challenge is exacerbated by the memory constraints and privacy concerns inherent in rehearsal buffers. To tackle catastrophic forgetting in this paper we introduce Online-LoRA a novel framework for task-free OCL. Online-LoRA allows to finetune pre-trained Vision Transformer (ViT) models in real-time to address the limitations of rehearsal buffers and leverage pre-trained models' performance benefits. As the main contribution our approach features a novel online weight regularization strategy to identify and consolidate important model parameters. Moreover Online-LoRA leverages the training dynamics of loss values to enable the automatic recognition of the data distribution shifts. Extensive experiments across many task-free OCL scenarios and benchmark datasets (including CIFAR-100 ImageNet-R ImageNet-S CUB-200 and CORe50) demonstrate that Online-LoRA can be robustly adapted to various ViT architectures while achieving better performance compared to SOTA methods.
Improving Zero-Shot Object-Level Change Detection by Incorporating Visual Correspondence
Authors: Hung Huy Nguyen; Pooyan Rahmanzadehgervi; Long Mai; Anh Totti Nguyen
Venue: WACV 2025 (main)

Abstract: Detecting object-level changes between two images across possibly different views (Fig. 1) is a core task in many applications that involve visual inspection or camera surveillance. Existing change-detection approaches suffer from three major limitations: (1) lack of evaluation on image pairs that contain no changes leading to unreported false positive rates; (2) lack of correspondences (i.e. localizing the regions before and after a change); and (3) poor zero-shot generalization across different domains. To address these issues we introduce a novel method that leverages change correspondences (a) during training to improve change detection accuracy and (b) at test time to minimize false positives. That is we harness the supervision labels of where an object is added or removed to supervise change detectors improving their accuracy over previous work [21] by a large margin. Our work is also the first to predict correspondences between pairs of detected changes using estimated homography and the Hungarian algorithm. Our model demonstrates superior performance over existing methods achieving state-of-the-art results in change detection and change correspondence accuracy across both in-distribution and zero-shot benchmarks.
FASTER: A Font-Agnostic Scene Text Editing and Rendering Framework
Authors: Alloy Das; Sanket Biswas; Prasun Roy; Subhankar Ghosh; Umapada Pal; Michael Blumenstein; Josep Lladós; Saumik Bhattacharya
Venue: WACV 2025 (main)

Abstract: Scene Text Editing (STE) is a challenging research problem that primarily aims towards modifying existing texts in an image while preserving the background and the font style of the original text. Despite its utility in numerous real-world applications existing style-transfer-based approaches have shown sub-par editing performance due to (1) complex image backgrounds (2) diverse font attributes and (3) varying word lengths within the text. To address such limitations in this paper we propose a novel font-agnostic scene text editing and rendering framework named FASTER for simultaneously generating text in arbitrary styles and locations while preserving a natural and realistic appearance and structure. A combined fusion of target mask generation and style transfer units with a cascaded self-attention mechanism has been proposed to focus on multi-level text region edits to handle varying word lengths. Extensive evaluation on a real-world database with further subjective human evaluation study indicates the superiority of FASTER in both scene text editing and rendering tasks in terms of model performance and efficiency. The code and pre-trained models have been released in our Github repo.
DragText: Rethinking Text Embedding in Point-Based Image Editing
Authors: Gayoon Choi; Taejin Jeong; Sujung Hong; Seong Jae Hwang
Venue: WACV 2025 (main)

Abstract: Point-based image editing enables accurate and flexible control through content dragging. However the role of text embedding during the editing process has not been thoroughly investigated. A significant aspect that remains unexplored is the interaction between text and image embeddings. During the progressive editing in a diffusion model the text embedding remains constant. As the image embedding increasingly diverges from its initial state the discrepancy between the image and text embeddings presents a significant challenge. In this study we found that the text prompt significantly influences the dragging process particularly in maintaining content integrity and achieving the desired manipulation. Upon these insights we propose DragText which optimizes text embedding in conjunction with the dragging process to pair with the modified image embedding. Simultaneously we regularize the text optimization process to preserve the integrity of the original text prompt. Our approach can be seamlessly integrated with existing diffusion-based drag methods enhancing performance with only a few lines of code.
Reversing the Damage: A QP-Aware Transformer-Diffusion Approach for 8K Video Restoration under Codec Compression
Authors: Ali Mollaahmadi Dehaghi; Reza Razavi; Mohammad Moshirpour
Venue: WACV 2025 (main)

Abstract: In this paper we introduce DiQP; a novel Transformer-Diffusion model for restoring 8K video quality degraded by codec compression. To the best of our knowledge our model is the first to consider restoring the artifacts introduced by various codecs (AV1 HEVC) by Denoising Diffusion without considering additional noise. This approach allows us to model the complex non-Gaussian nature of compression artifacts effectively learning to reverse the degradation. Our architecture combines the power of Transformers to capture long-range dependencies with an enhanced windowed mechanism that preserves spatio-temporal context within groups of pixels across frames. To further enhance restoration the model incorporates auxiliary "Look Ahead" and "Look Around" modules providing both future and surrounding frame information to aid in reconstructing fine details and enhancing overall visual quality. Extensive experiments on different datasets demonstrate that our model outperforms state-of-the-art methods particularly for high-resolution videos such as 4K and 8K showcasing its effectiveness in restoring perceptually pleasing videos from highly compressed sources
Lifting by Gaussians: A Simple Fast and Flexible Method for 3D Instance Segmentation
Authors: Rohan Chacko; Nicolai Häeni; Eldar Khaliullin; Lin Sun; Douglas Lee
Venue: WACV 2025 (main)

Abstract: We introduce Lifting By Gaussians (LBG) a novel approach for open-world instance segmentation of 3D Gaussian Splatted Radiance Fields (3DGS). Recently 3DGS Fields have emerged as a highly efficient and explicit alternative to Neural Field-based methods for high-quality Novel View Synthesis. Our 3D instance segmentation method directly lifts 2D segmentation masks from SAM (alternately FastSAM etc.) together with CLIP and DINOv2 features directly fusing them onto 3DGS (or similar Gaussian radiance fields such as 2DGS). Unlike previous approaches LBG requires no per-scene training allowing it to operate seamlessly on any existing 3DGS reconstruction. Our approach is not only an order of magnitude faster and simpler than existing approaches; it is also highly modular enabling 3D semantic segmentation of existing 3DGS fields without requiring a specific parametrization of the 3D Gaussians. Furthermore our technique achieves superior semantic segmentation for 2D semantic novel view synthesis and 3D asset extraction results while maintaining flexibility and efficiency. We further introduce a novel approach to evaluate individually segmented 3D assets from 3D radiance field segmentation methods.
TimberVision: A Multi-Task Dataset and Framework for Log-Component Segmentation and Tracking in Autonomous Forestry Operations
Authors: Daniel Steininger; Julia Simon; Andreas Trondl; Markus Murschitz
Venue: WACV 2025 (main)

Abstract: Timber represents an increasingly valuable and versatile resource. However forestry operations such as harvesting handling and measuring logs still require substantial human labor in remote environments posing significant safety risks. Progressively automating these tasks has the potential of increasing their efficiency as well as safety but requires an accurate detection of individual logs as well as live trees and their context. Although initial approaches have been proposed for this challenging application domain specialized data and algorithms are still too scarce to develop robust solutions. To mitigate this gap we introduce the TimberVision dataset consisting of more than 2k annotated RGB images containing a total of 51k trunk components including cut and lateral surfaces thereby surpassing any existing dataset in this domain in terms of both quantity and detail by a large margin. Based on this data we conduct a series of ablation experiments for oriented object detection and instance segmentation and evaluate the influence of multiple scene parameters on model performance. We introduce a generic framework to fuse the components detected by our models for both tasks into unified trunk representations. Furthermore we automatically derive geometric properties and apply multi-object tracking to further enhance robustness. Our detection and tracking approach provides highly descriptive and accurate trunk representations solely from RGB image data even under challenging environmental conditions. Our solution is suitable for a wide range of application scenarios and can be readily combined with other sensor modalities.
MimicGait: A Model Agnostic Approach for Occluded Gait Recognition using Correlational Knowledge Distillation
Authors: Ayush Gupta; Rama Chellappa
Venue: WACV 2025 (main)

Abstract: Gait recognition is an important biometric technique over large distances. State-of-the-art gait recognition systems perform very well in controlled environments at close range. Recently there has been an increased interest in gait recognition in the wild prompted by the collection of outdoor more challenging datasets containing variations in terms of illumination pitch angles and distances. An important problem in these environments is that of occlusion where the subject is partially blocked from camera view. While important this problem has received little attention. Thus we propose MimicGait a model-agnostic approach for gait recognition in the presence of occlusions. We train the network using a multi-instance correlational distillation loss to capture both inter-sequence and intra-sequence correlations in the occluded gait patterns of a subject utilizing an auxiliary Visibility Estimation Network to guide the training of the proposed mimic network. We demonstrate the effectiveness of our approach on challenging real-world datasets like GREW Gait3D and BRIAR. The code is available at https://github.com/Ayush-00/mimicgait.
PoolAtnRes: Towards Generalisable Differential Morphing Attack Detection
Authors: Raghavendra Ramachandra; Sushma Krupa Venkatesh; Guoqiang Li
Venue: WACV 2025 (main)

Abstract: Morphing attacks can successfully deceive face recognition systems resulting in unreliable access control especially in the border control scenario. Consequently the development of Morphing Attack Detection (MAD) algorithms is crucial for detecting morphing attacks based on either a single facial image (S-MAD) or two facial images (Differential-MAD or D-MAD). In this work we proposed a novel D-MAD approach PoolAtnRes to reliably detect morphing attacks. The proposed PoolAtnRes architecture is constructed using three main functional blocks namely convolution pooling Hybrid Attention and Residual blocks which are serially connected to detect morphing attacks. Extensive experiments were performed on the newly constructed morphing dataset using nine morphing-generation techniques. The detection performance of the proposed PoolAtnRes model was compared with three state-of-the-art (SOTA) D-MAD techniques with different performance evaluation protocols to benchmark its generalizability to unseen morphing generation. The results obtained indicated the best performance of the proposed PoolAtnRes D-MAD.