GenAI4ELab/papercli-papers huggingface dataset:
WiGNet: Windowed Vision Graph Neural Network
Authors: Gabriele Spadaro; Marco Grangetto; Attilio Fiandrotti; Enzo Tartaglione; Jhony H. Giraldo
Venue: WACV 2025 (main)

Abstract: In recent years Graph Neural Networks (GNNs) have demonstrated strong adaptability to various real-world challenges with architectures such as Vision GNN (ViG) achieving state-of-the-art performance in several computer vision tasks. However their practical applicability is hindered by the computational complexity of constructing the graph which scales quadratically with the image size. In this paper we introduce a novel Windowed vision Graph neural Network (WiGNet) model for efficient image processing. WiGNet explores a different strategy from previous works by partitioning the image into windows and constructing a graph within each window. Therefore our model uses graph convolutions instead of the typical 2D convolution or self-attention mechanism. WiGNet effectively manages computational and memory complexity for large image sizes. We evaluate our method in the ImageNet-1k benchmark dataset and test the adaptability of WiGNet using the CelebA-HQ dataset as a downstream task with higher-resolution images. In both of these scenarios our method achieves competitive results compared to previous vision GNNs while keeping memory and computational complexity at bay. WiGNet offers a promising solution toward the deployment of vision GNNs in real-world applications. We publicly released the code and pre-trained models at https://github.com/EIDOSLAB/WiGNet.
Transferring Foundation Models for Generalizable Robotic Manipulation
Authors: Jiange Yang; Wenhui Tan; Chuhao Jin; Keling Yao; Bei Liu; Jianlong Fu; Ruihua Song; Gangshan Wu; Limin Wang
Venue: WACV 2025 (main)

Abstract: Improving the generalization capabilities of general-purpose robotic manipulation in real world has long been a significant challenge. Existing approaches often rely on collecting large-scale robotic data which is costly and time-consuming. However due to insufficient diversity of data they typically suffer from limiting their capability in open-domain scenarios with new objects and diverse environments. In this paper we propose a novel paradigm that effectively leverages language-reasoning segmentation mask generated by internet-scale foundation models to condition robot manipulation tasks. By integrating the mask modality which incorporates semantic geometric and temporal correlation priors derived from vision foundation models into the end-to-end policy model our approach can effectively and robustly perceive object pose and enable sample-efficient generalization learning including new object instances semantic categories and unseen backgrounds. We first introduce a series of foundation models to ground natural language demands across multiple tasks. Secondly we develop a two-stream 2D policy model based on imitation learning which processes raw images and object masks to predict robot actions with a local-global perception manner. Extensive real-world experiments conducted on a Franka Emika robot and a low-cost dual-arm robot demonstrate the effectiveness of our proposed paradigm and policy. Demos can be found in link1 or link2 and our code will be released at https://github.com/MCG-NJU/TPM.
Robust Novelty Detection through Style-Conscious Feature Ranking
Authors: Stefan Smeu; Elena Burceanu; Emanuela Haller; Andrei Liviu Nicolicioiu
Venue: WACV 2025 (main)

Abstract: Novelty detection seeks to identify samples deviating from a known distribution yet data shifts in a multitude of ways and only a few consist of relevant changes. Aligned with out-of-distribution generalization literature we advocate for a formal distinction between task-relevant semantic or content changes and irrelevant style changes. This distinction forms the basis for robust novelty detection emphasizing the identification of semantic changes resilient to style distributional shifts. To this end we introduce Stylist a method that utilizes pretrained large-scale model representations to selectively discard environment-biased features. By computing per-feature scores based on feature distribution distances between environments Stylist effectively eliminates features responsible for spurious correlations enhancing novelty detection performance. Evaluations on adapted domain generalization datasets and a synthetic dataset demonstrate Stylist's efficacy in improving novelty detection across diverse datasets with stylistic and content shifts. We make our code available at https://github.com/bit-ml/Stylist.
LowFormer: Hardware Efficient Design for Convolutional Transformer Backbones
Authors: Moritz Nottebaum; Matteo Dunnhofer; Christian Micheloni
Venue: WACV 2025 (main)

Abstract: Research in efficient vision backbones is evolving into models that are a mixture of convolutions and transformer blocks. A smart combination of both architecture-wise and component-wise is mandatory to excel in the speed-accuracy trade-off. Most publications focus on maximizing accuracy and utilize MACs (multiply accumulate operations) as an efficiency metric. The latter however often do not measure accurately how fast a model actually is due to factors like memory access cost and degree of parallelism. We analyzed common modules and architectural design choices for backbones not in terms of MACs but rather in actual throughput and latency as the combination of the latter two is a better representation of the efficiency of models in real applications. We applied the conclusions taken from that analysis to create a recipe for increasing hardware-efficiency in macro design. Additionally we introduce a simple slimmed-down version of Multi-Head Self-Attention that aligns with our analysis. We combine both macro and micro design to create a new family of hardware-efficient backbone networks called LowFormer. LowFormer achieves a remarkable speedup in terms of throughput and latency while achieving similar or better accuracy than current state-of-the-art efficient backbones. In order to prove the generalizability of our hardware-efficient design we evaluate our method on GPU mobile GPU and ARM CPU. We further show that the downstream tasks object detection and semantic segmentation profit from our hardware-efficient architecture. Code and models are available at https://github.com/altair199797/LowFormer.
Towards Unbiased Continual Learning: Avoiding Forgetting in the Presence of Spurious Correlations
Authors: Giacomo Capitani; Lorenzo Bonicelli; Angelo Porrello; Federico Bolelli; Simone Calderara; Elisa Ficarra
Venue: WACV 2025 (main)

Abstract: Continual Learning (CL) has emerged as a paramount area in Artificial Intelligence (AI) because of its ability to learn multiple tasks sequentially without significant performance degradation. Despite the growing interest in CL frameworks a critical aspect must be addressed: the inherent biases within training data. In this work we show that if overlooked these biases can significantly impair the efficacy of continual learning models by inducing reliance on suboptimal shortcuts during data stream and memory retention exacerbating catastrophic forgetting. In response we present Learning without Shortcuts (LwS) which sets forth two primary objectives: (i) to identify and mitigate the exploitation of spurious correlations within the data stream and (ii) to develop a novel mechanism that constructs a fair memory buffer used in replay-based CL strategies. Our buffer construction strategy exploits the model confidence in a given example to balance the portion of samples per class hence their contribution when replay activates. Unlike existing methods LwS is agnostic to protected attributes and results highlight that the proposed solution is indeed resilient to spurious correlations in CL settings. Code is available at https://github.com/aimagelab/mammoth
Denoising Diffusion Models for High-Resolution Microscopy Image Restoration
Authors: Pamela Osuna-Vargas; Maren H. Wehrheim; Lucas Zinz; Johanna Rahm; Ashwin Balakrishnan; Alexandra Kaminer; Mike Heilemann; Matthias Kaschube
Venue: WACV 2025 (main)

Abstract: Advances in microscopy imaging enable researchers to visualize structures at the nanoscale level thereby unraveling intricate details of biological organization. However challenges such as image noise photobleaching of fluorophores and low tolerability of biological samples to high light doses remain restricting temporal resolutions and experiment durations. Reduced laser doses enable longer measurements at the cost of lower resolution and increased noise which hinders accurate downstream analyses. Here we train a denoising diffusion probabilistic model (DDPM) to predict high-resolution images by conditioning the model on low-resolution information. Additionally the probabilistic aspect of the DDPM allows for repeated generation of images that tend to further increase the signal-to-noise ratio. We show that our model achieves a performance that is better or similar to the previously best-performing methods across four highly diverse datasets. Importantly while any of the previous methods show competitive performance for some but not all datasets our method consistently achieves high performance across all four data sets suggesting high generalizability. Our code and datasets are available at https://github.com/kaschube-lab/ddpm_highres_microscopy https://doi.org/10.5281/zenodo.14178277 https://doi.org/10.5281/zenodo.14215837.
Prior2Posterior: Model Prior Correction for Long-Tailed Learning
Authors: S Divakar Bhat; Amit More; Mudit Soni; Surbhi Agrawal
Venue: WACV 2025 (main)

Abstract: Learning-based solutions for long-tailed recognition face difficulties in generalizing on balanced test datasets. Due to imbalanced data prior the learned a posteriori distribution is biased toward the most frequent (head) classes leading to an inferior performance on the least frequent (tail) classes. In general the performance can be improved by removing such a bias by eliminating the effect of imbalanced prior modeled using the number of class samples (frequencies). We first observe that the effective prior on the classes learned by the model at the end of the training can differ from the empirical prior obtained using class frequencies. Thus we propose a novel approach to accurately model the effective prior of a trained model using a posteriori probabilities. We propose to correct the imbalanced prior by adjusting the predicted a posteriori probabilities (Prior2Posterior: P2P) using the calculated prior in a post-hoc manner after the training and show that it can result in improved model performance. We present theoretical analysis showing the optimality of our approach for models trained with naive cross-entropy loss as well as logit adjusted loss. Our experiments show that the proposed approach achieves new state-of-the-art (SOTA) on several benchmark datasets from the long-tail literature in the category of logit adjustment methods. Further the proposed approach can be used to inspect any existing method to capture the effective prior and remove any residual bias to improve its performance post-hoc without model retraining. We also show that by using the proposed post-hoc approach the performance of many existing methods can be improved further.
Domain Generalization using Large Pretrained Models with Mixture-of-Adapters
Authors: Gyuseong Lee; Wooseok Jang; Jinhyeon Kim; Jaewoo Jung; Seungryong Kim
Venue: WACV 2025 (main)

Abstract: Learning robust vision models that perform well in out-of-distribution (OOD) situations is an important task for model deployment in real-world settings. Despite extensive research in this field many proposed methods have only shown minor performance improvements compared to the simplest empirical risk minimization (ERM) approach which was evaluated on a benchmark with a limited hyperparameter search space. Our focus in this study is on leveraging the knowledge of large pretrained models to improve handling of OOD scenarios and tackle domain generalization problems. However prior research has revealed that naively fine-tuning a large pretrained model can impair OOD robustness. Thus we employ parameter-efficient fine-tuning (PEFT) techniques to effectively preserve OOD robustness while working with large models. Our extensive experiments and analysis confirm that the most effective approaches involve ensembling diverse models and increasing the scale of pretraining. As a result we achieve state-of-the-art performance in domain generalization tasks. Our code and project page are available at: https://cvlab-kaist.github.io/MoA
From Visual Explanations to Counterfactual Explanations with Latent Diffusion
Authors: Tung Luu; Nam Le; Duc Le; Bac Le
Venue: WACV 2025 (main)

Abstract: Visual counterfactual explanations are ideal hypothetical images that change the decision-making of the classifier with high confidence toward the desired class while remaining visually plausible and close to the initial image. In this paper we propose a new approach to tackle two key challenges in recent prominent works: i) determining which specific counterfactual features are crucial for distinguishing the "concept" of the target class from the original class and ii) supplying valuable explanations for the non-robust classifier without relying on the support of an adversarially robust model. Our method identifies the essential region for modification through algorithms that provide visual explanations and then our framework generates realistic counterfactual explanations by combining adversarial attacks based on pruning the adversarial gradient of the target classifier and the latent diffusion model. The proposed method outperforms previous state-of-the-art results on various evaluation criteria on ImageNet and CelebA-HQ datasets. In general our method can be applied to arbitrary classifiers highlight the strong association between visual and counterfactual explanations make semantically meaningful changes from the target classifier and provide observers with subtle counterfactual images.
Active Learning with Context Sampling and One-vs-Rest Entropy for Semantic Segmentation
Authors: Fei Wu; Pablo Márquez Neila; Hedyeh Rafii-Tari; Raphael Sznitman
Venue: WACV 2025 (main)

Abstract: Multi-class semantic segmentation remains a cornerstone challenge in computer vision. Yet dataset creation remains excessively demanding in time and effort especially for specialized domains. Active Learning (AL) mitigates this challenge by selecting data points for annotation strategically. However existing patch-based AL methods often overlook boundary pixels' critical information essential for accurate segmentation. We present OREAL a novel patch-based AL method designed for multi-class semantic segmentation. OREAL enhances boundary detection by employing maximum aggregation of pixel-wise uncertainty scores. Additionally we introduce one-vs-rest entropy a novel uncertainty score function that computes class-wise uncertainties while achieving implicit class balancing during dataset creation. Comprehensive experiments across diverse datasets and model architectures validate our hypothesis.
VisualFusion: Enhancing Blog Content with Advanced Infographic Pipeline
Authors: Anurag Deo; Savita Bhat; Shirish Karande
Venue: WACV 2025 (main)

Abstract: Infographics represent a key component of any blog or article facilitating effective communication of ideas while fostering reader engagement. However many content creators possess limited expertise in crafting visually striking infographics. This gap is effectively addressed by our proposed pipeline designed to aid writers in generating compelling infographics tailored to their written content. Our pipeline uses textual content and tabular data from the blog to generate anchor plots. Leveraging LLM for prompt generation the pipeline integrates the generated prompts with these anchor plots through an Image to Image (I2I) generation model. We observe that the majority of the resulting images generated using this approach align with the article's narrative and effectively represent the underlying tabular data. Additionally we introduce our proposed AADaT (Aesthetical Adherence to Data and Text) Score adept at comprehensively assessing aesthetics textual alignment data fidelity and overall image quality concurrently. In comparative evaluations our pipeline has demonstrated around 15% superior performance relative to state-of-the-art models such as DALLE and Stable Diffusion Large by showcasing much better data adherence and aesthetics. While state-of-the-art models excel in some metrics but falter in others our pipeline demonstrates a balanced performance across all metrics.
Class-Agnostic Visio-Temporal Scene Sketch Semantic Segmentation
Authors: Aleyna Kütük; Tevfik Metin Sezgin
Venue: WACV 2025 (main)

Abstract: Scene sketch semantic segmentation is a crucial task for various applications including sketch-to-image retrieval and scene understanding. Existing sketch segmentation methods treat sketches as bitmap images leading to the loss of temporal order among strokes due to the shift from vector to image format. Moreover these methods struggle to segment objects from categories absent in the training data. In this paper we propose a Class-Agnostic Visio-Temporal Network (CAVT) for scene sketch semantic segmentation. CAVT employs a class-agnostic object detector to detect individual objects in a scene and groups the strokes of instances through its post-processing module. This is the first approach that performs segmentation at both the instance and stroke levels within scene sketches. Furthermore there is a lack of free-hand scene sketch datasets with both instance and stroke-level class annotations. To fill this gap we collected the largest Free-hand Instance- and Stroke-level Scene Sketch Dataset (FrISS) that contains 1K scene sketches and covers 403 object classes with dense annotations. Extensive experiments on FrISS and other datasets demonstrate the superior performance of our method over state-of-the-art scene sketch segmentation models. Our code and dataset can be accessed from https://github.com/aleynakutuk6/CAVT.
Can Out-of-Domain Data Help to Learn Domain-Specific Prompts for Multimodal Misinformation Detection?
Authors: Amartya Bhattacharya; Debarshi Brahma; Suraj Nagaje; Anmol Asati; Vikas Verma; Soma Biswas
Venue: WACV 2025 (main)

Abstract: Spread of fake news using out-of-context images and captions has become widespread in this era of information overload. Since fake news can belong to different domains like politics sports etc. with their unique characteristics inference on a test image-caption pair is contingent on how well the model has been trained on similar data. Since training individual models for each domain is not practical we propose a novel framework termed DPOD (Domain-specific Prompt tuning using Out-of-Domain data) which can exploit out-of-domain data during training to improve fake news detection of all desired domains simultaneously. First to compute generalizable features we modify the Vision-Language Model CLIP to extract features that help to align the representations of the images and corresponding captions of both the in-domain and out-of-domain data in a label-aware manner. Further we propose a domain-specific prompt learning technique which leverages training samples of all the available domains based on the extent they can be useful to the desired domain. Extensive experiments on the large-scale NewsCLIPpings and VERITE benchmarks demonstrate that DPOD achieves state of-the-art performance for this challenging task. Code: https://github.com/scviab/DPOD.
Scene-LLM: Extending Language Model for 3D Visual Reasoning
Authors: Rao Fu; Jingyu Liu; Xilun Chen; Yixin Nie; Wenhan Xiong
Venue: WACV 2025 (main)

Abstract: This paper introduces Scene-LLM a 3D-visual-language model that enhances embodied agents' abilities in interactive 3D indoor environments by integrating the reasoning strengths of Large Language Models (LLMs). Scene-LLM adopts a unified 3D visual feature representation that incorporates dense spatial information and supports scene state updates. The model employs a projection layer to efficiently project these features in the pre-trained textual embedding space enabling effective interpretation of 3D visual information. Unique to our approach is the integration of both scene-level and egocentric 3D information with a compact hybrid representation. This combination is pivotal for interactive planning where scene-level data supports global planning and egocentric data is important for localization. Notably we use egocentric 3D frame features for feature alignment an efficient technique that incorporates the model with fine-grained concepts. Our experiments with Scene-LLM demonstrate its strong capabilities in scene captioning question answering and interactive planning. We believe Scene-LLM advances the field of 3D visual understanding and reasoning offering new possibilities for sophisticated agent interactions in indoor settings.
Style-Pro: Style-Guided Prompt Learning for Generalizable Vision-Language Models
Authors: Niloufar Alipour Talemi; Hossein Kashiani; Fatemeh Afghah
Venue: WACV 2025 (main)

Abstract: Pre-trained Vision-language (VL) models such as CLIP have shown significant generalization ability to downstream tasks even with minimal fine-tuning. While prompt learning has emerged as an effective strategy to adapt pre-trained VL models for downstream tasks current approaches frequently encounter severe overfitting to specific downstream data distributions. This overfitting constrains the original behavior of the VL models to generalize to new domains or unseen classes posing a critical challenge in enhancing the adaptability and generalization of VL models. To address this limitation we propose Style-Pro a novel style-guided prompt learning framework that mitigates overfitting and preserves the zero-shot generalization capabilities of CLIP. Style-Pro employs learnable style bases to synthesize diverse distribution shifts guided by two specialized loss functions that ensure style diversity and content integrity. Then to minimize discrepancies between unseen domains and the source domain Style-Pro maps the unseen styles into the known style representation space as a weighted combination of style bases. Moreover to maintain consistency between the style-shifted prompted model and the original frozen CLIP Style-Pro introduces consistency constraints to preserve alignment in the learned embeddings minimizing deviation during adaptation to downstream tasks. Extensive experiments across 11 benchmark datasets demonstrate the effectiveness of Style-Pro consistently surpassing state-of-the-art methods in various settings including base-to-new generalization cross-dataset transfer and domain generalization.
DivAvatar: Diverse 3D Avatar Generation with a Single Prompt
Authors: Weijing Tao; Biwen Lei; Kunhao Liu; Shijian Lu; Miaomiao Cui; Xuansong Xie
Venue: WACV 2025 (main)

Abstract: Text-to-Avatar generation has recently made significant strides due to advancements in diffusion models. However most existing work remains constrained by limited diversity producing avatars with subtle differences in appearance for a given text prompt. We design DivAvatar a novel framework that generates diverse avatars empowering 3D creatives with a multitude of distinct and richly varied 3D avatars from a single text prompt. Different from most existing work that exploits scene-specific 3D representations such as NeRF DivAvatar finetunes a 3D generative model (i.e. EVA3D) allowing diverse avatar generation from simply noise sampling in inference time. DivAvatar has two key designs that help achieve generation diversity and visual quality. The first is a noise sampling technique during training phase which is critical in generating diverse appearances. The second is a semantic-aware zoom mechanism and a novel depth loss the former producing appearances of high textual fidelity by separate fine-tuning of specific body parts and the latter improving geometry quality greatly by smoothing the generated mesh in the features space. Extensive experiments show that DivAvatar is highly versatile in generating avatars of diverse appearances.