GenAI4ELab/papercli-papers huggingface dataset:
SHIP: Structural Hierarchies for Instance-Dependent Partial Labels
Authors: Tushar Kadam; Utkarsh Mishra; Aakarsh Malhotra
Venue: WACV 2025 (main)

Abstract: Partial label learning (PLL) aims to train classification models under conditions where each training sample is associated with a candidate set of labels. This set contains multiple labels among which only one is correct. This work addresses instance-dependent noise in PLL by leveraging hierarchical structures within the label space. We introduce a method to derive label hierarchies from instance-dependent partial labels. Subsequently we propose Structural Hierarchies for Instance-dependent Partial label (SHIP). SHIP is a modular component that integrates into deep learning architectures with applications that have intrinsic hierarchies. SHIP harnesses label hierarchy to enhance instance-dependent PLL performance across various deep-learning algorithms with hierarchy in the dataset. We conduct experiments on five publicly available benchmark datasets with four recent PLL algorithms. Experimental results show that incorporating SHIP into state-of-the-art architectures yields up to a 2.6% improvement in accuracy when hierarchies are present in the data. Moreover when the number of classes is high SHIP achieves up to a 2.5% reduction in mean mistake severity highlighting its effectiveness in mitigating error severity.
CEMIL: Contextual Attention Based Efficient Weakly Supervised Approach for Histopathology Image Classification
Authors: Tawsifur Rahman; Alexander S. Baras; Rama Chellappa
Venue: WACV 2025 (main)

Abstract: Multiple Instance Learning (MIL) has shown potential for analyzing Whole Slide Images (WSIs) in digital pathology but it faces challenges related to redundant information learning and generalization due to limited supervision and the computational complexity of Gigapixel WSIs. Many MIL-based methods apply a small weight matrix to all WSI patches. In this study we focus on developing computationally efficient models that improve MIL-based WSI classification by processing fewer patches while improving performance. We propose an attention-based approach using knowledge distillation where a compute-intensive "instructor" model analyzes all WSI patches to train a resource-efficient "learner" model which considers only a subset of patches. Comprehensive evaluations on four cancer subtype datasets--TCGA-BRCA TCGA-NSCLC TCGA-RCC and PANDA--demonstrate that an "observe-everything" instructor can effectively train an "observe-minimally" learner network. Overall our proposed learner network enhances performance by 4% compared to the state-of-the-art while reducing inference time by 45% and FLOPs by approximately 88%.
A Novel Perspective for Multi-Modal Multi-Label Skin Lesion Classification
Authors: Yuan Zhang; Yutong Xie; Hu Wang; Jodie C Avery; M Louise Hull; Gustavo Carneiro
Venue: WACV 2025 (main)

Abstract: The efficacy of deep learning-based Computer-Aided Diagnosis (CAD) methods for skin diseases relies on analyzing multiple data modalities (i.e. clinical+dermoscopic images and patient metadata) and addressing the challenges of multi-label classification. Current approaches tend to rely on limited multi-modal techniques and treat the multi-label problem as a multiple multi-class problem overlooking issues related to imbalanced learning and multi-label correlation. This paper introduces the innovative Skin Lesion Classifier utilizing a Multi-modal Multi-label TransFormer-based model (SkinM2Former). For multi-modal analysis we introduce the Tri-Modal Cross-attention Transformer (TMCT) that fuses the three image and metadata modalities at various feature levels of a transformer encoder. For multi-label classification we introduce a multi-head attention (MHA) module to learn multi-label correlations complemented by an optimisation that handles multi-label and imbalanced learning problems. SkinM2Former achieves a mean average accuracy of 77.27% and a mean diagnostic accuracy of 77.85% on the public Derm7pt dataset outperforming state-of-the-art (SOTA) methods.
Dual-Schedule Inversion: Training- and Tuning-Free Inversion for Real Image Editing
Authors: Jiancheng Huang; Yi Huang; Jianzhuang Liu; Donghao Zhou; Yifan Liu; Shifeng Chen
Venue: WACV 2025 (main)

Abstract: Text-conditional image editing is a practical AIGC task that has recently emerged with great commercial and academic value. For real image editing most diffusion model-based methods use DDIM Inversion as the first stage before editing. However DDIM Inversion often results in reconstruction failure leading to unsatisfactory performance for downstream editing. To address this problem we first analyze why the reconstruction via DDIM Inversion fails. We then propose a new inversion and sampling method named Dual-Schedule Inversion. We also design a classifier to adaptively combine Dual-Schedule Inversion with different editing methods for user-friendly image editing. Our work can achieve superior reconstruction and editing performance with the following advantages: 1) It can reconstruct real images perfectly without fine-tuning and its reversibility is guaranteed mathematically. 2) The edited object/scene conforms to the semantics of the text prompt. 3) The unedited parts of the object/scene retain the original identity.
Mind the Prompt: A Novel Benchmark for Prompt-Based Class-Agnostic Counting
Authors: Luca Ciampi; Nicola Messina; Matteo Pierucci; Giuseppe Amato; Marco Avvenuti; Fabrizio Falchi
Venue: WACV 2025 (main)

Abstract: Recently object counting has shifted towards class-agnostic counting (CAC) which counts instances of arbitrary object classes never seen during model training. With advancements in robust vision-and-language foundation models there is a growing interest in prompt-based CAC where object categories are specified using natural language. However we identify significant limitations in current benchmarks for evaluating this task which hinder both accurate assessment and the development of more effective solutions. Specifically we argue that the current evaluation protocols do not measure the ability of the model to understand which object has to be counted. This is due to two main factors: (i) the shortcomings of CAC datasets which primarily consist of images containing objects from a single class and (ii) the limitations of current counting performance evaluators which are based on traditional class-specific counting and focus solely on counting errors. To fill this gap we introduce the Prompt-Aware Counting (PrACo) benchmark. It comprises two targeted tests coupled with evaluation metrics specifically designed to quantitatively measure the robustness and trustworthiness of existing prompt-based CAC models. We evaluate state-of-the-art methods and demonstrate that although some achieve impressive results on standard class-specific counting metrics they exhibit a significant deficiency in understanding the input prompt indicating the need for more careful training procedures or revised designs. The code for reproducing our results is available at https://github.com/ciampluca/PrACo.
DLCR: A Generative Data Expansion Framework via Diffusion for Clothes-Changing Person Re-ID
Authors: Nyle Siddiqui; Florinel Alin Croitoru; Gaurav Kumar Nayak; Radu Tudor Ionescu; Mubarak Shah
Venue: WACV 2025 (main)

Abstract: With the recent exhibited strength of generative diffusion models an open research question is if images generated by these models can be used to learn better visual representations. While this generative data expansion may suffice for easier visual tasks we explore its efficacy on a more difficult discriminative task: clothes-changing person re-identification (CC-ReID). CC-ReID aims to match people appearing in non-overlapping cameras even when they change their clothes across cameras. Not only are current CC-ReID models constrained by the limited diversity of clothing in current CC-ReID datasets but generating additional data that retains important personal features for accurate identification is a current challenge. To address this issue we propose DLCR a novel data expansion framework that leverages pre-trained diffusion and large language models (LLMs) to accurately generate diverse images of individuals in varied attire. We generate additional data for five benchmark CC-ReID datasets (PRCC CCVID LaST VC-Clothes and LTCC) and increase their clothing diversity by 10X totaling over 2.1M images generated. DLCR employs diffusion-based text-guided inpainting conditioned on clothing prompts constructed using LLMs to generate synthetic data that only modifies a subject's clothes while preserving their personally identifiable features. With this massive increase in data we introduce two novel strategies - progressive learning and test-time prediction refinement - that respectively reduce training time and further boosts CC-ReID performance. On the PRCC dataset we obtain a large top-1 accuracy improvement of 11.3% by training CAL a previous state of the art (SOTA) method with DLCR-generated data. We publicly release our code and generated data for each dataset here: https://github.com/CroitoruAlin/dlcr.
Diffusion-Based Particle-DETR for BEV Perception
Authors: Asen Nachkov; Danda Pani Paudel; Martin Danelljan; Luc Van Gool
Venue: WACV 2025 (main)

Abstract: The Bird-Eye-View (BEV) is one of the most widely-used scene representations for visual perception in Autonomous Vehicles (AVs) due to its well suited compatibility to downstream tasks. For the enhanced safety of AVs modeling perception uncertainty in BEV is crucial. Recent diffusion-based methods offer a promising approach to uncertainty modeling for visual perception but fail to effectively detect small objects in the large coverage of the BEV. Such degradation of performance can be attributed primarily to the specific network architectures and the matching strategy used when training. Here we address this problem by combining the diffusion paradigm with current state-of-the-art 3D object detectors in BEV. We analyze the unique challenges of this approach which do not exist with deterministic detectors and present a simple technique based on object query interpolation that allows the model to learn positional dependencies even in the presence of the diffusion noise. Based on this we present a diffusion-based DETR model for object detection that bears similarities to particle methods. Abundant experimentation on the NuScenes dataset shows equal or better performance for our generative approach compared to deterministic state-of-the-art methods. The source code is at https://github.com/insait-institute/ParticleDETR.
Enhancing Scene Graph Generation with Hierarchical Relationships and Commonsense Knowledge
Authors: Bowen Jiang; Zhijun Zhuang; Shreyas S. Shivakumar; Camillo J. Taylor
Venue: WACV 2025 (main)

Abstract: This work introduces an enhanced approach to generating scene graphs by incorporating both a relationship hierarchy and commonsense knowledge. Specifically we begin by proposing a hierarchical relation head that exploits an informative hierarchical structure. It jointly predicts the relation super-category between object pairs in an image along with detailed relations under each super-category. Following this we implement a robust commonsense validation pipeline that harnesses foundation models to critique the results from the scene graph prediction system removing nonsensical predicates even with a small language-only model. Extensive experiments on Visual Genome and OpenImage V6 datasets demonstrate that the proposed modules can be seamlessly integrated as plug-and-play enhancements to existing scene graph generation algorithms. The results show significant improvements with an extensive set of reasonable predictions beyond dataset annotations. Codes are available at https://github.com/bowenupenn/scene graph_commonsense.
Domain-Generalized Object Anti-Spoofing: Bridging Gaps and Patch Selection for Robust Detection Across Domains
Authors: Geonu Lee; Yonghyun Jeong; Haneol Jang; Youngjoon Yoo
Venue: WACV 2025 (main)

Abstract: In online applications significant risks exist in peer-to-peer transactions due to malicious behaviors of arbitrary users such as taking advantage of manipulated images or impersonating others using recaptured images. Moreover recent advancements in display screens and imaging devices have made it increasingly challenging to distinguish such spoofing images from the naked eye. However a lack of datasets for object anti-spoofing significantly hinders the practical implementation of object anti-spoofing techniques compared to facial anti-spoofing tasks. To address this data scarcity issue for object anti-spoofing we propose a method that utilizes face anti-spoofing images for training. Our approach leverages low-rank adaptation employing fine-tuning with downstream tasks of large language models to facilitate domain transition between faces and generic objects. We also analyze a power spectrum to select useful patches for spoofing detection and introduce a patch-based learning method to effectively capture spoofing patterns. Lastly we present a novel protocol for assessing domain generalization in the generic object anti-spoofing task. Our model demonstrates state-of-the-art generalization performance compared to existing object anti-spoofing models surpassing even those simply augmented with face datasets.
VILLS : Video-Image Learning to Learn Semantics for Person Re-Identification
Authors: Siyuan Huang; Ram Prabhakar Kathirvel; Yuxiang Guo; Rama Chellappa; Cheng Peng
Venue: WACV 2025 (main)

Abstract: Person Re-identification is a research area with significant real world applications. Despite recent progress existing methods face challenges in robust re-identification in the wild e.g. by focusing only on a particular modality and on unreliable patterns such as clothing. A generalized method is highly desired but remains elusive to achieve due to issues such as the trade-off between spatial and temporal resolution and inaccurate feature extraction. We propose VILLS (Video-Image Learning to Learn Semantics) a self-supervised method that jointly learns spatial and temporal features from images and videos. VILLS first designs a local semantic extraction module that adaptively extracts semantically consistent and robust spatial features. Then VILLS designs a unified feature learning and adaptation module to represent image and video modalities in a consistent feature space. By Leveraging self-supervised large-scale pre-training VILLS establishes a new State-of-The-Art that significantly outperforms existing image and video-based methods.
ERUP-YOLO: Enhancing Object Detection Robustness for Adverse Weather Condition by Unified Image-Adaptive Processing
Authors: Yuka Ogino; Yuho Shoji; Takahiro Toizumi; Atsushi Ito
Venue: WACV 2025 (main)

Abstract: We propose an image-adaptive object detection method for adverse weather conditions such as fog and low-light. Our framework employs differentiable preprocessing filters to perform image enhancement suitable for later-stage object detections. Our framework introduces two differentiable filters: a Bezier curve-based pixel-wise (BPW) filter and a kernel-based local (KBL) filter. These filters unify the functions of classical image processing filters and improve performance of object detection. We also propose a domain-agnostic data augmentation strategy using the BPW filter. Our method does not require data-specific customization of the filter combinations parameter ranges and data augmentation. We evaluate our proposed approach called Enhanced Robustness by Unified Image Processing (ERUP)-YOLO by applying it to the YOLOv3 detector. Experiments on adverse weather datasets demonstrate that our proposed filters match or exceed the expressiveness of conventional methods and our ERUP-YOLO achieved superior performance in a wide range of adverse weather conditions including fog and low-light conditions.
Conditional GAN for Enhancing Diffusion Models in Efficient and Authentic Global Gesture Generation from Audios
Authors: Yongkang Cheng; Mingjiang Liang; Shaoli Huang; Gaoge Han; Jifeng Ning; Wei Liu
Venue: WACV 2025 (main)

Abstract: Audio-driven simultaneous gesture generation is vital for human-computer communication AI games and film production. While previous research has shown promise there are still limitations. Methods based on VAEs are accompanied by issues of local jitter and global instability whereas methods based on diffusion models are hampered by low generation efficiency. This is because the denoising process of DDPM in the latter relies on the assumption that the noise added at each step is sampled from a unimodal distribution and the noise values are small. DDIM borrows the idea from the Euler method for solving differential equations disrupts the Markov chain process and increases the noise step size to reduce the number of denoising steps thereby accelerating generation. However simply increasing the step size during the step-by-step denoising process causes the results to gradually deviate from the original data distribution leading to a significant drop in the quality of the generated actions and the emergence of unnatural artifacts. In this paper we break the assumptions of DDPM and achieves breakthrough progress in denoising speed and fidelity. Specifically we introduce a conditional GAN to capture audio control signals and implicitly match the multimodal denoising distribution between the diffusion and denoising steps within the same sampling step aiming to sample larger noise values and apply fewer denoising steps for high-speed generation. In addition to enable the model to generate high-fidelity global gestures and avoid artifacts we introduce an explicit motion geometric loss to enhance the quality and global stability of the generated gestures. Numerous qualitative and quantitative experiments show that compared to contemporary diffusion-based methods our method offers faster generation speed and higher fidelity and compared to non-diffusion methods it provides a more stable global effect and a more natural user experience.
PK-YOLO: Pretrained Knowledge Guided YOLO for Brain Tumor Detection in Multiplanar MRI Slices
Authors: Ming Kang; Fung Fung Ting; Raphael C.-W. Phan; Chee-Ming Ting
Venue: WACV 2025 (main)

Abstract: Brain tumor detection in multiplane Magnetic Resonance Imaging (MRI) slices is a challenging task due to the various appearances and relationships in the structure of the multiplane images. In this paper we propose a new You Only Look Once (YOLO)-based detection model that incorporates Pretrained Knowledge (PK) called PK-YOLO to improve the performance for brain tumor detection in multiplane MRI slices. To our best knowledge PK-YOLO is the first pretrained knowledge guided YOLO-based object detector. The main components of the new method are a pretrained pure lightweight convolutional neural network-based backbone via sparse masked modeling a YOLO architecture with the pretrained backbone and a regression loss function for improving small object detection. The pretrained backbone allows for feature transferability of object queries on individual plane MRI slices into the model encoders and the learned domain knowledge base can improve in-domain detection. The improved loss function can further boost detection performance on small-size brain tumors in multiplanar two-dimensional MRI slices. Experimental results show that the proposed PK-YOLO achieves competitive performance on the multiplanar MRI brain tumor detection datasets compared to state-of-the-art YOLO-like and DETR-like object detectors. The code is available at https://github.com/mkang315/PK-YOLO.
When Cars Meet Drones: Hyperbolic Federated Learning for Source-Free Domain Adaptation in Adverse Weather
Authors: Giulia Rizzoli; Matteo Caligiuri; Donald Shenaj; Francesco Barbato; Pietro Zanuttigh
Venue: WACV 2025 (main)

Abstract: In Federated Learning (FL) multiple clients collaboratively train a global model without sharing private data. In semantic segmentation the Federated source Free Domain Adaptation (FFREEDA) setting is of particular interest where clients undergo unsupervised training after supervised pretraining at the server side. While few recent works address FL for autonomous vehicles intrinsic real-world challenges such as the presence of adverse weather conditions and the existence of different autonomous agents are still unexplored. To bridge this gap we address both problems and introduce a new federated semantic segmentation setting where both car and drone clients co-exist and collaborate. Specifically we propose a novel approach for this setting which exploits a batch-norm weather-aware strategy to dynamically adapt the model to the different weather conditions while hyperbolic space prototypes are used to align the heterogeneous client representations. Finally we introduce FLYAWARE the first semantic segmentation dataset with adverse weather data for aerial vehicles.
I3D-AE-LSTM: A 2-Stream Autoencoder for Action Quality Assessment using a Newly Created Cricket Batsman Video Dataset
Authors: Tevin Moodley; Dustin Terence van der Haar
Venue: WACV 2025 (main)

Abstract: In this study we introduce UJ-AQA-CricketVision a dataset comprising 8540 video clips of cricket strokes each annotated with detailed phase breakdowns. We develop a novel multi-variate approach for Action Quality Assessment (AQA) at a body level that leverages an Autoencoder for extracting sophisticated feature representations from video frames and pose estimated keypoints. These features are subsequently utilised by a multi-layer perceptron regression-based model to accurately predict the quality of cricket actions in terms of their head shoulder hands hips and feet. Our approach is benchmarked against contemporary state-of-the-art AQA methods and achieves a Spearman Rank Correlation score of 0.84. The performance highlights the significance of integrating pose keypoint and frame data for the nuanced analysis of short and complex action sequences in sports such as cricket. This work aims to foster the development of accurate Action Quality Assessment methods on Cricket Video data. The dataset can be found here: https://github.com/dvanderhaar/uj-aqa-cricketvision
DDPM-CD: Denoising Diffusion Probabilistic Models as Feature Extractors for Remote Sensing Change Detection
Authors: Wele Gedara Chaminda Bandara; Nithin Gopalakrishnan Nair; Vishal Patel
Venue: WACV 2025 (main)

Abstract: Remote sensing change detection is crucial for understanding the dynamics of our planet's surface facilitating the monitoring of environmental changes evaluating human impact predicting future trends and supporting decision-making. In this work we introduce a novel approach for change detection that can leverage off-the-shelf unlabeled remote sensing images in the training process by pre-training a Denoising Diffusion Probabilistic Model (DDPM) - a class of generative models used in image synthesis. DDPMs learn the training data distribution by gradually converting training images into a Gaussian distribution using a Markov chain. During inference (i.e. sampling) they can generate a diverse set of samples closer to the training distribution starting from Gaussian noise achieving state-of-the-art image synthesis results. However in this work our focus is not on image synthesis but on utilizing it as a pre-trained feature extractor for the downstream application of change detection. Specifically we fine-tune a lightweight change classifier utilizing the feature representations produced by the pre-trained DDPM alongside change labels. Experiments conducted on the LEVIR-CD WHU-CD DSIFN-CD and CDD datasets demonstrate that the proposed DDPM-CD method significantly outperforms the existing self supervised state-of-the-art change detection methods in terms of F1 score IoU and overall accuracy highlighting the pivotal role of pre-trained DDPM as a feature extractor for downstream applications. Code and pre-trained models available at https://github.com/wgcban/ddpm-cd