GenAI4ELab/papercli-papers huggingface dataset:
Learning Anatomy-Disease Entangled Representation
Authors: Fatemeh Haghighi; Michael B. Gotway; Jianming Liang
Venue: WACV 2025 (main)

Abstract: Human experts demonstrate proficiency not only in disentangling anatomical structures from disease conditions but also in intertwining anatomical and disease information to accurately diagnose a variety of disorders. However deep learning models despite their prowess in acquiring intricate representation often struggle to learn representation where distinct semantic aspects of the data (both anatomy and pathology) are entangled particularly in medical images which present a rich array of anatomical structures and potential pathological conditions. We envision that a deep model when trained to comprehend medical images akin to human perception would offer powerful representation with higher generalizability robustness and interpretability. To realize this vision we have developed LeADER a framework for learning anatomy-disease entangled representation from medical images. As a proof of concept we have trained LeADER on 1M chest radiographs gathered from 10 public datasets. Experimental results across 11 medical tasks compared to 8 baselines in zero-shot linear probing limited data regimes and full fine-tuning settings demonstrate LeADER's superior performance over the Google CXR Foundation Model large-scale medical models and fully/self-supervised baselines across diverse downstream tasks. This enhanced performance is attributed to the significance of entangling anatomy-specific and disease-specific representations via our framework which enables the simultaneous acquisition of both anatomical and disease knowledge yet overlooked in existing supervised/self-supervised learning methods. All code and models are available at GitHub.com/JLiangLab/LeADER.
MemControl: Mitigating Memorization in Diffusion Models via Automated Parameter Selection
Authors: Raman Dutt; Ondrej Bohdal; Pedro Sanchez; Sotirios Tsaftaris; Timothy Hospedales
Venue: WACV 2025 (main)

Abstract: Diffusion models excel in generating images that closely resemble their training data but are also susceptible to data memorization raising privacy ethical and legal concerns particularly in sensitive domains such as medical imaging. We hypothesize that this memorization stems from the overparameterization of deep models and propose that regularizing model capacity during fine-tuning can mitigate this issue. Firstly we empirically show that regulating the model capacity via Parameter-efficient fine-tuning (PEFT) mitigates memorization to some extent however it further requires the identification of the exact parameter subsets to be fine-tuned for high-quality generation. To identify these subsets we introduce a bi-level optimization framework MemControl that automates parameter selection using memorization and generation quality metrics as rewards during fine-tuning. The parameter subsets discovered through MemControl achieve a superior tradeoff between generation quality and memorization. For the task of medical image generation our approach outperforms existing state-of-the-art memorization mitigation strategies by fine-tuning as few as 0.019% of model parameters. Moreover we demonstrate that the discovered parameter subsets are transferable to non-medical domains. Our framework is scalable to large datasets agnostic to reward functions and can be integrated with existing approaches for further memorization mitigation. To the best of our knowledge this is the first study to empirically evaluate memorization in medical images and propose a targeted yet universal mitigation strategy. The code is available at https://github.com/Raman1121/Diffusion_Memorization_HPO
Recognizing Unseen States of Unknown Objects by Leveraging Knowledge Graphs
Authors: Filippos Gouidis; Konstantinos Papoutsakis; Theodore Patkos; Antonis Argyros; Dimitris Plexousakis
Venue: WACV 2025 (main)

Abstract: We investigate the problem of Object State Classification (OSC) in the context of zero-shot learning. Specifically we propose the first method for Zero-shot Object-agnostic State Classification (OaSC) that given an image infers the state of a single object without relying on the knowledge or the estimation of the object class. In that direction we capitalize on Knowledge Graphs (KGs) for structuring and organizing external knowledge which in combination with visual information enable effective inference of the states of objects that have not been encountered in the training set. Having this unique property a significant strength of our method is that it can handle an Open Set of object classes. We investigate the performance of OaSC in various datasets and settings against several hypotheses and in comparison with state-of-the-art approaches for object attribute classification. OaSC outperforms these methods significantly across all benchmarks.
SynDRA: Synthetic Dataset for Railway Applications
Authors: Gianluca D'Amico; Federico Nesti; Giulio Rossolini; Mauro Marinoni; Salvatore Sabina; Giorgio Buttazzo
Venue: WACV 2025 (main)

Abstract: The use of deep learning techniques in railway environments faces significant obstacles especially for computer vision tasks. Such obstacles are mainly due to the inherent safety concerns required for installing the proper equipment on a train and the substantial effort required to precisely annotate large datasets especially for segmentation tasks. Public datasets of real-world images are quite scarce and suffer from severe limitations such as coarse manual annotation or narrow range of scenarios. In addition real-world datasets often do not contain scenes that represent critical situations. To address such limitations this paper introduces SynDRA a synthetic dataset of photo-realistic images generated using a railway simulator built on Unreal Engine 5. SynDRA offers precise pixel-level annotations across diverse scenarios thereby facilitating more effective testing and training of deep learning models for semantic segmentation tasks in railway settings. The advantages of the proposed dataset are validated through a series of experiments that highlight the potential of SynDRA to enhance the performance of deep learning models in scenarios where real-world annotated data is scarce. The dataset is publicly available at the following link: https://syndra.retis.santannapisa.it.
VISIONARY: Novel Spatial-Spectral Attention Mechanism for Hyperspectral Image Denoising
Authors: Aditya Dixit; Nischit Hosamani; Puneet Gupta; Ankur Garg
Venue: WACV 2025 (main)

Abstract: Image denoising mitigates noise from the captured images and thereby enhances the efficacy of high-demand vision applications such as classification and segmentation. Hyperspectral Images (HSIs) with their multiple spectral bands provide valuable information and make them highly applicable in real-world applications. Current Deep Learning methods mainly employ Transformers to denoise HSIs spatially and spectrally through self-attention (SA). However SA focuses on individual samples and overlooks potential correlations within the images indicating room for improvement. Moreover existing Transformer-based denoising methods often fail to appropriately balance the importance of spatial and spectral features. This paper presents a novel method VISIONARY to address these issues by obtaining better HSI feature representation. To this end it introduces the Spatial-Spectral-Cubic Transformer (SSCformer) block to address the shortcomings of Transformers in HSI denoising particularly their inability to capture correlations within images of the same type by introducing Global Feature Attention (GFA). Additionally the SSCformer independently determines the optimal weightage for spatial and spectral features using attention mechanisms leading to more effective denoising. Our method VISIONARY is based on the integration of Transformer U-Net and CNN architecture. Experimental results demonstrate that VISIONARY outperforms well-known methods on publicly available datasets and our SSCformer block can be easily integrated with existing Transformer-based HSI denoising methods to improve their efficacy.
TPD-STR: Text Polygon Detection with Split Transformers
Authors: Sangyeon Kim; Sangkuk Lee; Jeesoo Kim; Nojun Kwak
Venue: WACV 2025 (main)

Abstract: Regressing text in natural scenes with polygonal representations is challenging due to shape prediction difficulties. To address this we introduce Text Polygon Detection with Split Transformers (TPD-STR) which directly regresses polygonal points. TPD-STR incorporates the Decoder Split (DS) architecture to separate polygonal point regression and textness classification and the Positional Information Propagation (PIP) module to enhance classification. Both modules are effective and compatible with existing methods. TPD-STR achieves state-of-the-art (SOTA) performance among regression-based methods surpassing segmentation-based methods on MSRA-TD500 without external data. Adding DS and PIP to existing models further improves performance. Experiments demonstrate the model's ability to detect text instances effectively.
ReC-TTT: Contrastive Feature Reconstruction for Test-Time Training
Authors: Marco Colussi; Sergio Mascetti; Jose Dolz; Christian Desrosiers
Venue: WACV 2025 (main)

Abstract: The remarkable progress in deep learning (DL) showcases outstanding results in various computer vision tasks. However adaptation to real-time variations in data distributions remains an important challenge. Test-Time Training (TTT) was proposed as an effective solution to this issue which increases the generalization ability of trained models by adding an auxiliary task at train time and then using its loss at test time to adapt the model. Inspired by the recent achievements of contrastive representation learning in unsupervised tasks we propose ReC-TTT a test-time training technique that can adapt a DL model to new unseen domains by generating discriminative views of the input data. ReC-TTT uses cross-reconstruction as an auxiliary task between a frozen encoder and two trainable encoders taking advantage of a single shared decoder. This enables at test time to adapt the encoders to extract features that will be correctly reconstructed by the decoder that in this phase is frozen on the source domain. Experimental results show that ReC-TTT achieves better results than other state-of-the-art techniques in most domain shift classification challenges. The code is available at: https://github.com/warpcut/ReC-TTT
Effective Scene Graph Generation by Statistical Relation Distillation
Authors: Thanh-Son Nguyen; Hong Yang; Basura Fernando
Venue: WACV 2025 (main)

Abstract: Annotating scene graphs for images is a time-consuming task resulting in many instances of missing relations within existing datasets. In this paper we introduce the Statistical Relation Distillation (SRD) method to enhance scenegraph datasets. SRD leverages human-annotated relations alongside object-to-object and predicate-to-predicate similarities to reinforce the existence likelihood of scene graph relations. Moreover SRD can augment relational frequency using relations of non-selected object and predicate categories that are usually omitted by scene graph generation (SGG) tasks. The output from SRD derives the prior probability which is combined with model-predicted probabilities to annotate missing relations in training images and sub-sequently re-train SGG models on the augmented dataset. We evaluate our proposed method on Visual Genome and GQA-200 datasets. Experimental results show that training on the augmented dataset enhances the performance of prominent scene-graph generation models. The implementation code is at https://github.com/LUNAProject22/SRD.
Cross Image Feature Perturbation with Pseudo Label Fusion for Semi-Supervised Medical Image Segmentation
Authors: Minxia Xu; Han Yang; Bo Song; Weida Hu; Jinshui Miao; Erkang Cheng
Venue: WACV 2025 (main)

Abstract: Semi-supervised medical image segmentation aims to leverage a limited set of labeled images alongside a substantial volume of unlabeled images to train semantic segmentation models. Existing studies often employ consistency regularization to maximize the utilization of unlabeled data thereby enhancing the model's robustness and accuracy. However the methods for constructing perturbations at image-level on unlabeled data are typically simplistic involving techniques such as color transformations additive noise which do not adequately leverage the precise and reliable supervisory information available from labeled images. To address this limitation in addition to image perturbation we propose a cross-image feature perturbation approach for semi-supervised medical image segmentation. This method utilizes feature information from labeled images to guide the refinement of ambiguous semantic representations in unlabeled images thereby expanding the perturbation space more effectively. Moreover recognizing the limitations of existing consistency regularization frameworks that rely on confidence thresholds to filter pseudo-labels we introduce an uncertainty-based pseudo-label fusion strategy. This strategy mitigates the effects of unreliable predictions caused by perturbations by calculating the uncertainty and using it as a weight during pseudo-label fusion. We have conducted extensive experiments on the 2D ACDC and 3D LA datasets. The results demonstrate that our approach achieves performance comparable to the current state-of-the-art (SOTA) methods.
DeCLIP: Decoding CLIP Representations for Deepfake Localization
Authors: Stefan Smeu; Elisabeta Oneata; Dan Oneata
Venue: WACV 2025 (main)

Abstract: Generative models can create entirely new images but they can also partially modify real images in ways that are undetectable to the human eye. In this paper we address the challenge of automatically detecting such local manipulations. One of the most pressing problems in deepfake detection remains the ability of models to generalize to different classes of generators. In the case of fully manipulated images representations extracted from large self-supervised models (such as CLIP) provide a promising direction towards more robust detectors. Here we introduce DeCLIP-- a first attempt to leverage such large pretrained features for detecting local manipulations. We show that when combined with a reasonably large convolutional decoder pretrained self-supervised representations are able to perform localization and improve generalization capabilities over existing methods. Unlike previous work our approach is able to perform localization on the challenging case of latent diffusion models where the entire image is affected by the fingerprint of the generator. Moreover we observe that this type of data which combines local semantic information with a global fingerprint provides more stable generalization than other categories of generative methods
Instance-Warp: Saliency Guided Image Warping for Unsupervised Domain Adaptation
Authors: Shen Zheng; Anurag Ghosh; Srinivasa Narasimhan
Venue: WACV 2025 (main)

Abstract: Driving is challenging in conditions like night rain and snow. Lack of good labeled datasets has hampered progress in scene understanding under such conditions. Unsupervised Domain Adaptation (UDA) using large labeled clear-day datasets is a promising research direction in such cases. However many UDA methods are trained with dominant scene backgrounds (e.g. roads sky sidewalks) that appear dramatically different across domains. As a result they struggle to learn effective features of smaller and often sparse foreground objects (e.g. people vehicles signs). In this work we improve UDA training by applying in-place image warping to focus on salient objects. We design instance-level saliency guidance to adaptively oversample object regions and undersample background areas which reduces adverse effects from background context and enhances backbone feature learning. Our approach improves adaptation across geographies lighting and weather conditions and is agnostic to the task (segmentation detection) domain adaptation algorithm saliency guidance and underlying model architecture. Result highlights include +6.1 mAP50 for BDD100K Clear to DENSE Foggy +3.7 mAP50 for BDD100K Day to Night +3.0 mAP50 for BDD100K Clear to Rainy and +6.3 mIoU for Cityscapes to ACDC. Besides Our method adds minimal training memory and no additional inference latency. Code is available at https://github.com/ShenZheng2000/Instance-Warp.
ZAHA: Introducing the Level of Facade Generalization and the Large-Scale Point Cloud Facade Semantic Segmentation Benchmark Dataset
Authors: Olaf Wysocki; Yue Tan; Thomas Froech; Yan Xia; Magdalena Wysocki; Ludwig Hoegner; Daniel Cremers; Christoph Holst
Venue: WACV 2025 (main)

Abstract: Facade semantic segmentation is a long-standing challenge in photogrammetry and computer vision. Although the last decades have witnessed the influx of facade segmentation methods there is a lack of comprehensive facade classes and data covering the architectural variability. In ZAHA we introduce Level of Facade Generalization (LoFG) novel hierarchical facade classes designed based on international urban modeling standards ensuring compatibility with real-world challenging classes and uniform methods' comparison. Realizing the LoFG we present to date the largest semantic 3D facade segmentation dataset providing 601 million annotated points at five and 15 classes of LoFG2 and LoFG3 respectively. Moreover we analyze the performance of baseline semantic segmentation methods on our introduced LoFG classes and data complementing it with a discussion on the unresolved challenges for facade segmentation. We firmly believe that ZAHA shall facilitate further development of 3D facade semantic segmentation methods enabling robust segmentation indispensable in creating urban digital twins.
Make-A-Texture: Fast Shape-Aware 3D Texture Generation in 3 Seconds
Authors: Liat Sless Gorelik; Yuchen Fan; Omri Armstrong; Forrest N Iandola; Yilei Li; Ita Lifshitz; Rakesh Ranjan
Venue: WACV 2025 (main)

Abstract: We present Make-A-Texture a new framework that efficiently synthesizes high-resolution texture maps from textual prompts for given 3D geometries. Our approach progressively generates textures that are consistent across multiple viewpoints with a depth-aware inpainting diffusion model in an optimized sequence of viewpoints determined by an automatic view selection algorithm. A significant feature of our method is its remarkable efficiency achieving a full texture generation within an end-to-end runtime of just 3.07 seconds on a single NVIDIA H100 GPU significantly outperforming existing methods. Such an acceleration is achieved by optimizations in the diffusion model and a specialized backprojection method. Moreover our method reduces the artifacts in the backprojection phase by selectively masking out non-frontal faces and internal faces of open-surfaced objects. Experimental results demonstrate that Make-A-Texture matches or exceeds the quality of other state-of-the-art methods. Our work significantly improves the applicability and practicality of texture generation models for real-world 3D content creation including interactive creation and text-guided texture editing.
PRoGS: Progressive Rendering of Gaussian Splats
Authors: Brent Zoomers; Maarten Wijnants; Ivan Molenaers; Joni Vanherck; Jeroen Put; Lode Jorissen; Nick Michiels
Venue: WACV 2025 (main)

Abstract: Over the past year 3D Gaussian Splatting (3DGS) has received significant attention for its ability to represent 3D scenes in a perceptually accurate manner. However it can require a substantial amount of storage since each splat's individual data must be stored. While compression techniques offer a potential solution by reducing the memory footprint they still necessitate retrieving the entire scene before any part of it can be rendered. In this work we introduce a novel approach for progressively rendering such scenes aiming to display visible content that closely approximates the final scene as early as possible without loading the entire scene into memory. This approach benefits both on-device rendering applications limited by memory constraints and streaming applications where minimal bandwidth usage is preferred. To achieve this we approximate the contribution of each Gaussian to the final scene and construct an order of prioritization on their inclusion in the rendering process. Additionally we demonstrate that our approach can be combined with existing compression methods to progressively render (and stream) 3DGS scenes optimizing bandwidth usage by focusing on the most important splats within a scene. Overall our work establishes a foundation for making remotely hosted 3DGS content more quickly accessible to end-users in over-the-top consumption scenarios with our results showing significant improvements in quality across all metrics compared to existing methods.
SynDroneVision: A Synthetic Dataset for Image-Based Drone Detection
Authors: Tamara R. Lenhard; Andreas Weinmann; Kai Franke; Tobias Koch
Venue: WACV 2025 (main)

Abstract: Developing robust drone detection systems is often constrained by the limited availability of large-scale annotated training data and the high costs associated with real-world data collection. However leveraging synthetic data generated via game engine-based simulations provides a promising and cost-effective solution to overcome this issue. Therefore we present SynDroneVision a synthetic dataset specifically designed for RGB-based drone detection in surveillance applications. Featuring diverse backgrounds lighting conditions and drone models SynDroneVision offers a comprehensive training foundation for deep learning algorithms. To evaluate the dataset's effectiveness we perform a comparative analysis across a selection of recent YOLO detection models. Our findings demonstrate that SynDroneVision is a valuable resource for real-world data enrichment achieving notable enhancements in model performance and robustness while significantly reducing the time and costs of real-world data acquisition. SynDroneVision can be accessed at https://zenodo.org/records/13360116.
Distribution Optimization under Gaussian Hypothesis for Domain Adaptive Semantic Segmentation
Authors: Chen Liang; Weihua Chen; Xin Zhao; Junyan Wang; Lijun Cao; Junge Zhang
Venue: WACV 2025 (main)

Abstract: Domain adaptive semantic segmentation aims to transfer a model proficient in dense image classification from a source domain to a target domain. While various transfer methods have been explored in previous studies we argue that the modeling of categories within the model significantly affects its transferability. Building on the Gaussian Hypothesis which posits that each category in the feature space adheres to a multidimensional Gaussian distribution we propose a Class-Aware Variational Inference (CAVI) training method. This approach normalizes features of different categories into distinct multidimensional Gaussian distributions. To further learn domain-independent feature distributions we optimize the feature space using a Gaussian-based alignment strategy and incorporate Gaussian-based contrastive learning. Experimental results demonstrate that our method achieves state-of-the-art performance on the GTAV-to-Cityscapes and Synthia-to-Cityscapes benchmarks.