GenAI4ELab/papercli-papers huggingface dataset:
Detective Networks: Enhancing Disaster Recognition in Images Through Attention Shifting using Optimal Masking
Authors: Narongthat Thanyawet; Photchara Ratsamee; Yuki Uranishi; Haruo Takemura
Venue: WACV 2025 (main)

Abstract: Aerial investigation is used for surveying damage and identifying post-disaster events through imagery data. However the challenge lies in detecting disaster-related areas within aerial or shipborne images as these can appear as minor regions making recognition difficult. To address this challenge we introduce the Detective Network (DeNet) designed to optimally mask images thereby shifting the attention of machine learning models towards these small yet crucial regions. Utilizing the concepts of patch and anchor box DeNet incorporates a masking candidate layer and a masking layer to facilitate optimal masking. Our experimental findings are compelling; by preprocessing images with DeNet before analysis using an image captioning model we achieved a remarkable accuracy of 92.91% in landslide detection from side-view image captions and 87.50% for shipborne view detection. The result demonstrates the efficacy of DeNet in enhancing the recognition of disaster-related areas in challenging imaging conditions.
CRAAC: Consistency Regularised Active Learning with Automatic Corrections for Real-Life Road Image Annotations
Authors: Percy Lam; Sooyong Park; Weiwei Chen; Lavindra de Silva; Ioannis Brilakis
Venue: WACV 2025 (main)

Abstract: In annotating real-life large noisy and domain-specific images for digitising infrastructure substantial human effort persists despite past advancements. This research provides practical and interpretable scores for human annotators enabling flexible annotation strategies improving automation and reducing the effort required to create and correct image labels. The authors present the CRAAC solution: Consistency Regularised Active learning and Automatic Corrections which builds on Mask R-CNN with three additional modules: consistency regularisation scoring modules for active learning and automatic corrections. Experiments on our pavement image dataset recorded with a low silhouette score of 0.146 and qualitative annotation inconsistencies reduce the human effort of mouse clicks by 5-11% and improve the quality metrics of mAP and AR by approx. 40% from the original Mask R-CNN. The automatic correction further reduces the performance variation.
Navigating Heterogeneity and Privacy in One-Shot Federated Learning with Diffusion Models
Authors: Matias Mendieta; Guangyu Sun; Chen Chen
Venue: WACV 2025 (main)

Abstract: Federated learning (FL) enables multiple clients to train models collectively while preserving data privacy. However FL faces challenges in terms of communication cost and data heterogeneity. One-shot federated learning has emerged as a solution by reducing communication rounds improving efficiency and providing better security against eavesdropping attacks. Nevertheless data heterogeneity remains a significant challenge impacting performance. This work explores the effectiveness of diffusion models in one-shot FL demonstrating their applicability in addressing data heterogeneity and improving FL performance. Additionally we investigate the utility of our diffusion model approach FedDiff compared to other one-shot FL methods under differential privacy (DP). Furthermore to improve generated sample quality under DP settings we propose a pragmatic Fourier Magnitude Filtering (FMF) method enhancing the effectiveness of the generated data for global model training. Code available at https://github.com/mmendiet/FedDiff.
Separating Direct and Global Components from Novel Viewpoints
Authors: Kengo Matsufuji; Lin Shi; Ryo Kawahara; Takahiro Okabe
Venue: WACV 2025 (main)

Abstract: Separating an image of a scene illuminated by a light source into direct components such as specular reflection and diffuse reflection and global components such as inter-reflection and subsurface scattering is important as preprocessing for various computer vision and graphics applications. Conventional methods cannot separate direct and global components from novel viewpoints and have difficulties in robustly separating those components from a small number of images even from known viewpoints. In this paper we propose a method for synthesizing the direct and global components of a scene from novel viewpoints by using a relatively small number of images. Specifically our proposed method uses the multi-view images captured by using a coaxial projector-camera system and then recovers the density and radiance values of each component on the basis of neural radiance fields (NeRF). We conduct a number of experiments using real images captured with a projector-camera system and confirm the effectiveness of our method. In addition we demonstrate that our method is useful for two applications: image-based material editing and 3D shape recovery.
ReMix: Training Generalized Person Re-Identification on a Mixture of Data
Authors: Timur Mamedov; Anton Konushin; Vadim Konushin
Venue: WACV 2025 (main)

Abstract: Modern person re-identification (Re-ID) methods have a weak generalization ability and experience a major accuracy drop when capturing environments change. This is because existing multi-camera Re-ID datasets are limited in size and diversity since such data is difficult to obtain. At the same time enormous volumes of unlabeled single-camera records are available. Such data can be easily collected and therefore it is more diverse. Currently single-camera data is used only for self-supervised pre-training of Re-ID methods. However the diversity of single-camera data is suppressed by fine-tuning on limited multi-camera data after pre-training. In this paper we propose ReMix a generalized Re-ID method jointly trained on a mixture of limited labeled multi-camera and large unlabeled single-camera data. Effective training of our method is achieved through a novel data sampling strategy and new loss functions that are adapted for joint use with both types of data. Experiments show that ReMix has a high generalization ability and outperforms state-of-the-art methods in generalizable person Re-ID. To the best of our knowledge this is the first work that explores joint training on a mixture of multi-camera and single-camera data in person Re-ID.
Phaseformer: Phase-Based Attention Mechanism for Underwater Image Restoration and Beyond
Authors: Raqib Khan; Anshul Negi; Ashutosh Kulkarni; Shruti S. Phutke; Santosh Kumar Vipparthi; Subrahmanyam Murala
Venue: WACV 2025 (main)

Abstract: Quality degradation is observed in underwater images due to the effects of light refraction and absorption by water leading to issues like color cast haziness and limited visibility. This degradation negatively affects the performance of autonomous underwater vehicles used in marine applications. To address these challenges we propose a lightweight phase-based transformer network with 1.77M parameters for underwater image restoration (UIR). Our approach focuses on effectively extracting non-contaminated features using a phase-based self-attention mechanism. We also introduce an optimized phase attention block to restore structural information by propagating prominent attentive features from the input. We evaluate our method on both synthetic (UIEB UFO-120) and real-world (UIEB U45 UCCS SQUID) underwater image datasets. Additionally we demonstrate its effectiveness for low-light image enhancement using the LOL dataset. Through extensive ablation studies and comparative analysis it is clear that the proposed approach outperforms existing state-of-the-art (SOTA) methods. The testing code is included in the supplementary material and will be publicly released upon acceptance.
CamoFA: A Learnable Fourier-Based Augmentation for Camouflage Segmentation
Authors: Minh-Quan Le; Minh-Triet Tran; Trung-Nghia Le; Tam V. Nguyen; Thanh-Toan Do
Venue: WACV 2025 (main)

Abstract: Camouflaged object detection (COD) and camouflaged instance segmentation (CIS) aim to recognize and segment objects that are blended into their surroundings respectively. While several deep neural network models have been proposed to tackle those tasks augmentation methods for COD and CIS have not been thoroughly explored. Augmentation strategies can help improve models' performance by increasing the size and diversity of the training data and exposing the model to a wider range of variations in the data. Besides we aim to automatically learn transformations that help to reveal the underlying structure of camouflaged objects and allow the model to learn to identify better and segment camouflaged objects. To achieve this we propose a learnable augmentation method in the frequency domain for COD and CIS via the Fourier transform approach dubbed CamoFA. Our method leverages a conditional generative adversarial network and cross-attention mechanism to generate a reference image and an adaptive hybrid swapping with parameters to mix the low-frequency component of the reference image and the high-frequency component of the input image. This approach aims to make camouflaged objects more visible for detection and segmentation models. Without bells and whistles our proposed augmentation method boosts the performance of camouflaged object detectors and instance segmenters by large margins.
Federated Voxel Scene Graph for Intracranial Hemorrhage
Authors: Antoine P. Sanner; Jonathan Stieber; Nils F. Grauhan; Suam Kim; Marc A. Brockmann; Ahmed E. Othman; Anirban Mukhopadhyay
Venue: WACV 2025 (main)

Abstract: Intracranial Hemorrhage is a potentially lethal condition whose manifestation is vastly diverse and shifts across clinical centers worldwide. Deep-learning-based solutions are starting to model complex relations between brain structures but still struggle to generalize. While gathering more diverse data is the most natural approach privacy regulations often limit the sharing of medical data. We propose the first application of Federated Scene Graph Generation. We show that our models can leverage the increased training data diversity. For Scene Graph Generation they can recall up to 20% more clinically relevant relations across datasets compared to models trained on a single centralized dataset. Learning structured data representation in a federated setting can open the way to the development of new methods that can leverage this finer information to regularize across clients more effectively.
Hyperdimensional Representation for Adaptive Information Association and Memorization
Authors: Zhuowen Zou; Prathyush Poduval; Narayan Srinivasa; Mohsen Imani
Venue: WACV 2025 (main)

Abstract: Many computer vision applications rely on interpretable machine learning algorithms to analyze the data collected from various sources. We leverage Hyperdimensional Computing (HDC) as an innovative computational model that mimics key brain functionalities to achieve efficient and robust cognitive learning. We propose HDlm a novel HDC-based cognitive representation capable of adaptive information association and memorization. HDlm first theoretically expands HDC mathematics to support selective information association and adaptive memorization. Then it exploits the proposed operations to support cognitive operations including set membership information retrieval and item comparison. We evaluated our solution for a selection of applications related to visual data representation and sequence matching analysis. Our evaluation shows that HDlm provides more adaptive similarity metrics between objects that lead to better task performance.
HandCraft: Anatomically Correct Restoration of Malformed Hands in Diffusion Generated Images
Authors: Zhenyue Qin; Yiqun Zhang; Yang Liu; Dylan Campbell
Venue: WACV 2025 (main)

Abstract: Generative text-to-image models such as Stable Diffusion have demonstrated a remarkable ability to generate diverse high-quality images. However they are surprisingly inept when it comes to rendering human hands which are often anatomically incorrect or reside in the "uncanny valley". In this paper we propose a method HandCraft for restoring such malformed hands. This is achieved by automatically constructing masks and depth images for hands as conditioning signals using a parametric model allowing a diffusion-based image editor to fix the hand's anatomy and adjust its pose while seamlessly integrating the changes into the original image preserving pose color and style. Our plug-and-play hand restoration solution is compatible with existing pretrained diffusion models and the restoration process facilitates adoption by eschewing any fine-tuning or training requirements for the diffusion models. We also contribute MalHand datasets that contain generated images with a wide variety of malformed hands in several styles for hand detector training and hand restoration benchmarking and demonstrate through qualitative and quantitative evaluation that HandCraft not only restores anatomical correctness but also maintains the integrity of the overall image.
SCOT: Self-Supervised Contrastive Pretraining for Zero-Shot Compositional Retrieval
Authors: Bhavin Jawade; João V. B. Soares; Kapil Thadani; Deen Dayal Mohan; Amir Erfan Eshratifar; Benjamin Culpepper; Paloma de Juan; Srirangaraj Setlur; Venu Govindaraju
Venue: WACV 2025 (main)

Abstract: Compositional image retrieval (CIR) is a multimodal learning task where a model combines a query image with a user-provided text modification to retrieve a target image. CIR finds applications in a variety of domains including product retrieval (e-commerce) and web search. Existing methods primarily focus on fully-supervised learning wherein models are trained on datasets of labeled triplets such as FashionIQ and CIRR. This poses two significant challenges: (i) curating such triplet datasets is labor intensive; and (ii) models lack generalization to unseen objects and domains. In this work we propose SCOT (Self-supervised COmpositional Training) a novel zero-shot compositional pretraining strategy that combines existing large image-text pair datasets with the generative capabilities of large language models to contrastively train an embedding composition network. Specifically we show that the text embedding from a large-scale contrastively-pretrained vision-language model can be utilized as proxy target supervision during compositional pretraining replacing the target image embedding. In zero-shot settings this strategy surpasses SOTA zero-shot compositional retrieval methods as well as many fully-supervised methods on standard benchmarks such as FashionIQ and CIRR.
ROADS: Robust Prompt-Driven Multi-Class Anomaly Detection under Domain Shift
Authors: Hossein Kashiani; Niloufar Alipour Talemi; Fatemeh Afghah
Venue: WACV 2025 (main)

Abstract: Recent advancements in anomaly detection have shifted focus towards Multi-class Unified Anomaly Detection (MUAD) offering more scalable and practical alternatives compared to traditional one-class-one-model approaches. However existing MUAD methods often suffer from inter-class interference and are highly susceptible to domain shifts leading to substantial performance degradation in real-world applications. In this paper we propose a novel robust prompt-driven MUAD framework called ROADS to address these challenges. ROADS employs a hierarchical class-aware prompt integration mechanism that dynamically encodes class-specific information into our anomaly detector to mitigate interference among anomaly classes. Additionally ROADS incorporates a domain adapter to enhance robustness against domain shifts by learning domain-invariant representations. Extensive experiments on MVTec-AD and VISA datasets demonstrate that ROADS surpasses state-of-the-art methods in both anomaly detection and localization with notable improvements in out-of-distribution settings.
Memory-Efficient Continual Learning with Neural Collapse Contrastive
Authors: Trung-Anh Dang; Vincent Nguyen; Ngoc-Son Vu; Christel Vrain
Venue: WACV 2025 (main)

Abstract: Contrastive learning has significantly improved representation quality enhancing knowledge transfer across tasks in continual learning (CL). However catastrophic forgetting remains a key challenge as contrastive based methods primarily focus on "soft relationships" or "softness" between samples which shift with changing data distributions and lead to representation overlap across tasks. Recently the newly identified Neural Collapse phenomenon has shown promise in CL by focusing on "hard relationships" or "hardness" between samples and fixed prototypes. However this approach overlooks "softness" crucial for capturing intra-class variability and this rigid focus can also pull old class representations toward current ones increasing forgetting. Building on these insights we propose Focal Neural Collapse Contrastive (FNC^2) a novel representation learning loss that effectively balances both soft and hard relationships. Additionally we introduce the Hardness-Softness Distillation (HSD) loss to progressively preserve the knowledge gained from these relationships across tasks. Our method outperforms state-of-the-art approaches particularly in minimizing memory reliance. Remarkably even without the use of memory our approach rivals rehearsal-based methods offering a compelling solution for data privacy concerns.
Towards Real-Time Open-Vocabulary Video Instance Segmentation
Authors: Bin Yan; Martin Sundermeyer; David Joseph Tan; Huchuan Lu; Federico Tombari
Venue: WACV 2025 (main)

Abstract: In this paper we address the challenge of performing open-vocabulary video instance segmentation (OV-VIS) in real-time. We analyze the computational bottlenecks of state-of-the-art foundation models that performs OV-VIS and propose a new method TROY-VIS that significantly improves processing speed while maintaining high accuracy. We introduce three key techniques: (1) Decoupled Attention Feature Enhancer to speed up information interaction between different modalities and scales; (2) Flash Embedding Memory for obtaining fast text embeddings of object categories; and (3) Kernel Interpolation for exploiting the temporal continuity in videos. Our experiments demonstrate that TROY-VIS achieves the best trade-off between accuracy and speed on two large-scale OV-VIS benchmarks BURST and LV-VIS running 20x faster than GLEE-Lite (25 FPS v.s. 1.25 FPS) with comparable or even better accuracy. These results demonstrate TROY-VIS's potential for real-time applications in dynamic environments such as mobile robotics and augmented reality. Code and model will be released at https://github.com/google-research/troyvis.
Dataset Augmentation by Mixing Visual Concepts
Authors: Md Abdullah Al Rahat Kutubi; Hemanth Venkateswara
Venue: WACV 2025 (main)

Abstract: This paper proposes a dataset augmentation method by fine-tuning pre-trained diffusion models. Generating images using a pre-trained diffusion model with textual conditioning often results in domain discrepancy between real data and generated images. We propose a fine-tuning approach where we adapt the diffusion model by conditioning it with real images and novel text embeddings. We introduce a unique procedure called Mixing Visual Concepts (MVC) where we create novel text embeddings from image captions. The MVC enables us to generate multiple images which are diverse and yet similar to the real data enabling us to perform effective dataset augmentation. We perform comprehensive qualitative and quantitative evaluations with the proposed dataset augmentation approach showcasing both coarse-grained and fine-grained changes in generated images. Our approach outperforms state-of-the-art augmentation techniques on benchmark classification tasks. The code is available at https://github.com/rahatkutubi/MVC
DMPT: Decoupled Modality-Aware Prompt Tuning for Multi-Modal Object Re-Identification
Authors: Minghui Lin; Shu Wang; Xiang Wang; Jianhua Tang; Longbin Fu; Zhengrong Zuo; Nong Sang
Venue: WACV 2025 (main)

Abstract: Current multi-modal object re-identification approaches based on large-scale pre-trained backbones (i.e. ViT) have displayed remarkable progress and achieved excellent performance. However these methods usually adopt the standard full fine-tuning paradigm which requires the optimization of considerable backbone parameters causing extensive computational and storage requirements. In this work we propose an efficient prompt-tuning framework tailored for multi-modal object re-identification dubbed DMPT which freezes the main backbone and only optimizes several newly added decoupled modality-aware parameters. Specifically we explicitly decouple the visual prompts into modality-specific prompts which leverage prior modality knowledge from a powerful text encoder and modality-independent semantic prompts which extract semantic information from multi-modal inputs such as visible near-infrared and thermal-infrared. Built upon the extracted features we further design a Prompt Inverse Bind (PromptIBind) strategy that employs bind prompts as a medium to connect the semantic prompt tokens of different modalities and facilitates the exchange of complementary multi-modal information boosting final re-identification results. Experimental results on multiple common benchmarks demonstrate that our DMPT can achieve competitive results to existing state-of-the-art methods while requiring only 6.5% fine-tuning of the backbone parameters.
Image-Caption Encoding for Improving Zero-Shot Generalization
Authors: Eric Yu; Christopher Liao; Sathvik Ravi; Theodoros Tsiligkaridis; Brian Kulis
Venue: WACV 2025 (main)

Abstract: Recent advances in vision-language models have combined contrastive approaches with generative methods to achieve state-of-the-art (SOTA) on downstream inference tasks like zero-shot image classification. However a persistent issue of these models for image classification is their out-of-distribution (OOD) generalization capabilities. We first show that when an OOD datapoint is misclassified the correct class can be typically found in the Top-K predicted classes. In order to steer the model prediction toward the correct class within the top predicted classes we propose the Image-Caption Encoding (ICE) method a straightforward approach that directly enforces consistency between the image-conditioned and caption-conditioned predictions at evaluation time only. Intuitively we take advantage of unique properties of the generated captions to guide our local search for the correct class label within the Top-K predicted classes. We show that our method can be easily combined with other SOTA methods to enhance Top-1 OOD accuracies by 0.5% on average and up to 3% on challenging datasets.
FT2TF: First-Person Statement Text-To-Talking Face Generation
Authors: Xingjian Diao; Ming Cheng; Wayner Barrios; SouYoung Jin
Venue: WACV 2025 (main)

Abstract: Talking face generation has gained immense popularity in the computer vision community with various applications including AR VR teleconferencing digital assistants and avatars. Traditional methods are mainly audio-driven which have to deal with the inevitable resource-intensive nature of audio storage and processing. To address such a challenge we propose FT2TF - First-Person Statement Text-To-Talking Face Generation a novel one-stage end-to-end pipeline for talking face generation driven by first-person statement text. Different from previous work our model only leverages visual and textual information without any other sources (e.g. audio/landmark/pose) during inference. Extensive experiments are conducted on LRS2 and LRS3 datasets and results on multi-dimensional evaluation metrics are reported. Both quantitative and qualitative results showcase that FT2TF outperforms existing relevant methods and reaches the state-of-the-art. This achievement highlights our model's capability to bridge first-person statements and dynamic face generation providing insightful guidance for future work.