GenAI4ELab/papercli-papers huggingface dataset:
Comparative Knowledge Distillation
Authors: Alex Tianyi Xu; Alex Wilf; Paul Pu Liang; Alexander Obolenskiy; Daniel Fried; Louis-Philippe Morency
Venue: WACV 2025 (main)

Abstract: In the era of large-scale pretrained models Knowledge Distillation (KD) serves an important role in transferring the wisdom of computationally-heavy teacher models to lightweight efficient student models while preserving performance. Yet KD settings often assume readily available access to teacher models capable of performing many inferences -- a notion increasingly at odds with the realities of costly large-scale models. Addressing this gap we study an important question: how KD algorithms fare as the number of teacher inferences decreases a setting we term Reduced-Teacher-Inference Knowledge Distillation (RTI-KD). We observe that the performance of prevalent KD techniques and state-of-the-art data augmentation strategies suffers considerably as the number of teacher inferences is reduced. One class of approaches termed "relational" knowledge distillation underperforms the rest yet we hypothesize that they hold promise for reduced dependency on teacher models because they can augment the effective dataset size without additional teacher calls. We find that a simple change -- performing high-dimensional comparisons instead of low-dimensional relations which we term "Comparative Knowledge Distillation" -- vaults performance well over existing KD approaches. We perform empirical evaluation across varied experimental settings and rigorous analysis to understand the learning outcomes of our method. All code is made publicly available.
Noise-Aware Evaluation of Object Detectors
Authors: Jeffri Murrugarra Llerena; Claudio R. Jung
Venue: WACV 2025 (main)

Abstract: Supervised object detection requires annotated datasets for training and evaluation purposes. However human annotation of large datasets is error-prone and frequent mistakes are erroneous labels missing objects and imprecise bounding boxes. The main goals of this work are to quantify the extent of annotation noise in terms of corner-wise discrepancies assess how it impacts evaluation metrics for object detection and propose noise-aware alternatives that serve as upper and lower bounds for a baseline metric. We focus our analysis on the Microsoft COCO dataset and re-evaluate several state-of-the-art object detectors using the proposed metrics. We show that the Average Precision (AP) metric might be considerably over or under-estimated particularly for small objects and restrictive IoU acceptance thresholds. Our code is available at https://github.com/Artcs1/Error-Aware.
On the Importance of Dual-Space Augmentation for Domain Generalized Object Detection
Authors: Hayoung Park; Choongsang Cho; Guisik Kim
Venue: WACV 2025 (main)

Abstract: The distribution gap between training data and real-world data often causes significant performance drops in networks trained via naive supervised learning. To address this domain generalization methods have been developed to gain robust performance in unseen domains. In this paper we propose a single-domain generalized object detection (S-DGOD) method. Unlike previous works we utilize both image-level and feature-level augmentations and experimentally demonstrate their synergistic effects. Image-level augmentations expand the source domain while feature-level augmentations leverage CLIP to incorporate potential domain descriptions. Our method achieves superior performance with 29.2% mAP on the Cityscapes-C and 37.1% mAP on the Diverse-Weather dataset.
Learning Visual Grounding from Generative Vision and Language Model
Authors: Shijie Wang; Dahun Kim; Ali Taalimi; Chen Sun; Weicheng Kuo
Venue: WACV 2025 (main)

Abstract: Visual grounding tasks aim to localize image regions based on natural language references. In this work we explore whether generative VLMs predominantly trained on image-text data could be leveraged to scale up the text annotation of visual grounding data. We find that grounding knowledge already exists in generative VLM and can be elicited by proper prompting. We thus prompt a VLM to generate object-level descriptions by feeding it object regions from existing object detection datasets. We further propose attribute modeling to explicitly capture the important object attributes and spatial relation modeling to capture inter-object relationship both of which are common linguistic pattern in referring expression. Our constructed dataset (500K images 1M objects 16M referring expressions) is one of the largest grounding datasets to date and the first grounding dataset with purely model-generated queries and human-annotated objects. To verify the quality of this data we conduct zero-shot transfer experiments to the popular RefCOCO benchmarks for both referring expression comprehension (REC) and segmentation (RES) tasks. On both tasks our model significantly outperform the state-of-the-art approaches without using human annotated visual grounding data. Our results demonstrate the promise of generative VLM to scale up visual grounding in the real world.
Oriented Cell Dataset: A Dataset and Benchmark for Oriented Cell Detection and Applications
Authors: Lucas Kirsten; Angelo Angonezi; Jose Marques; Fernanda Oliveira; Juliano Faccioni; Camila Cassel; Débora de Sousa; Samlai Vedovatto; Guido Lenz; Claudio Jung
Venue: WACV 2025 (main)

Abstract: This work presents a new public dataset for cell detection in bright-field microscopy images annotated with Oriented Bounding Boxes (OBBs) named Oriented Cell Dataset (OCD). Our dataset also contains a subset of images with five independent expert annotations which allows inter-annotation analysis to determine a suitable IoU acceptance threshold for evaluating cell detectors. We show that OBBs and a derived representation Oriented Ellipses (OEs) provide a more accurate shape representation than standard Horizontal Bounding Boxes (HBBs) with a slight overhead of one extra click in the annotation process. We benchmarked OCD using 14 state-of-the-art oriented object detectors and explored two main problems in cancer biology: cell confluence and polarity determination. Our code and dataset are available at https://github.com/LucasKirsten/Deep-Cell-Tracking-EBB.
Sigma: Siamese Mamba Network for Multi-Modal Semantic Segmentation
Authors: Zifu Wan; Pingping Zhang; Yuhao Wang; Silong Yong; Simon Stepputtis; Katia Sycara; Yaqi Xie
Venue: WACV 2025 (main)

Abstract: Multi-modal semantic segmentation significantly enhances AI agents' perception and scene understanding especially under adverse conditions like low-light or overexposed environments. Leveraging additional modalities (X-modality) like thermal and depth alongside traditional RGB provides complementary information enabling more robust and reliable segmentation. In this work we introduce Sigma a Siamese Mamba network for multi-modal semantic segmentation utilizing the Selective Structured State Space Model Mamba. Unlike conventional methods that rely on CNNs with their limited local receptive fields or Vision Transformers (ViTs) which offer global receptive fields at the cost of quadratic complexity our model achieves global receptive fields coverage with linear complexity. By employing a Siamese encoder and innovating a Mamba fusion mechanism we effectively select essential information from different modalities. A decoder is then developed to enhance the channel-wise modeling ability of the model. Our method Sigma is rigorously evaluated on both RGB-Thermal and RGB-Depth segmentation tasks demonstrating its superiority and marking the first successful application of State Space Models (SSMs) in multi-modal perception tasks. Code is available at https://github.com/zifuwan/Sigma.
DocMatcher: Document Image Dewarping via Structural and Textual Line Matching
Authors: Felix Hertlein; Alexander Naumann; York Sure-Vetter
Venue: WACV 2025 (main)

Abstract: Document image dewarping is a crucial step in the digitization of physical documents as it aims to remove the distortions induced by challenging environment settings and document sheet deformations often encountered when using smartphone cameras for image capture. Recently deep learning-based methods were combined with knowledge about the expected document structure also known as a template at inference time to improve the dewarping results. Our contributions in this work are threefold: (1) we propose a novel document image dewarping approach that leverages the prior knowledge about the document structure effectively by detecting and matching lines from the warped and the template domain and (2) we introduce a novel evaluation metric called matched normalized character error rate (mnCER) to overcome the limitations of existing metrics in evaluating the dewarping process. (3) Finally we evaluate our approach on the Inv3DReal dataset and show that our approach outperforms the state-of-the-art methods in terms of visual and text-based metrics. Our approach improves upon the state-of-the-art methods by 32.6% in Local Distortion and 40.2% in mnCER. Our code and models are available at https://felixhertlein.github.io/doc-matcher.
Design-o-Meter: Towards Evaluating and Refining Graphic Designs
Authors: Sahil Goyal; Abhinav Mahajan; Swasti Mishra; Prateksha Udhayanan; Tripti Shukla; KJ Joseph; Balaji Vasan Srinivasan
Venue: WACV 2025 (main)

Abstract: Graphic designs are an effective medium for visual communication. They range from greeting cards to corporate flyers and beyond. Off-late machine learning techniques are able to generate such designs which accelerates the rate of content production. An automated way of evaluating their quality becomes critical. Towards this end we introduce Design-o-meter a data-driven methodology to quantify the goodness of graphic designs. Further our approach can suggest modifications to these designs to improve its visual appeal. To the best of our knowledge Design-o-meter is the first approach that scores and refines designs in a unified framework despite the inherent subjectivity and ambiguity of the setting. Our exhaustive quantitative and qualitative analysis of our approach against baselines adapted for the task (including recent Multimodal LLM based approaches) brings out the efficacy of our methodology. We hope our work will usher more interest in this important and pragmatic problem setting. Project Page: https://sahilg06.github.io/Design-o-meter/
LumiGauss: Relightable Gaussian Splatting in the Wild
Authors: Joanna Kaleta; Kacper Kania; Tomasz Trzcinski; Marek Kowalski
Venue: WACV 2025 (main)

Abstract: Decoupling lighting from geometry using unconstrained photo collections is notoriously challenging. Solving it would benefit many users as creating complex 3D assets takes days of manual labor. Many previous works have attempted to address this issue often at the expense of output fidelity which questions the practicality of such methods. We introduce LumiGauss - a technique that tackles 3D reconstruction of scenes and environmental lighting through 2D Gaussian Splatting. Our approach yields high-quality scene reconstructions and enables realistic lighting synthesis under novel environment maps. We also propose a method for enhancing the quality of shadows common in outdoor scenes by exploiting spherical harmonics properties. Our approach facilitates seamless integration with game engines and enables the use of fast precomputed radiance transfer. We validate our method on the NeRF-OSR dataset demonstrating superior performance over baseline methods. Moreover LumiGauss can synthesize realistic images for unseen environment maps. Our code: https: //github.com/joaxkal/lumigauss.
3D Understanding of Deformable Linear Objects: Datasets and Transferability Benchmark
Authors: Bare Luka Žagar; Mingyu Liu; Tim Hertel; Ekim Yurtsever; Alois C. Knoll
Venue: WACV 2025 (main)

Abstract: Deformable linear objects are commonly found in our daily lives. Understanding them visually can be challenging even for humans as the same object can become entangled and look completely different. Examples of deformable linear objects include blood vessels and wiring harnesses which are crucial for the proper functioning of systems like the human body and vehicles. Recently some studies have focused on 2D image segmentation of wires. However there are no point cloud datasets available for studying 3D deformable linear objects which are more complex and challenging. To address this gap we introduce two point cloud datasets PointWire and PointVessel generated using our proposed semi-automatic pipeline. We evaluated state-of-the-art methods on these large-scale 3D deformable linear object benchmarks. Additionally we analyzed the generalization capabilities of these methods through transferability experiments on the PointWire and PointVessel datasets.
On Explaining Knowledge Distillation: Measuring and Visualising the Knowledge Transfer Process
Authors: Gereziher Adhane; Mohammad Mahdi Dehshibi; Dennis Vetter; David Masip; Gemma Roig
Venue: WACV 2025 (main)

Abstract: Knowledge distillation (KD) remains challenging due to the opaque nature of the knowledge transfer process from a Teacher to a Student making it difficult to address certain issues related to KD. To address this we proposed UniCAM a novel gradient-based visual explanation method which effectively interprets the knowledge learned during KD. Our experimental results demonstrate that with the guidance of the Teacher's knowledge the Student model becomes more efficient learning more relevant features while discarding those that are not relevant. We refer to the features learned with the Teacher's guidance as distilled features and the features irrelevant to the task and ignored by the Student as residual features. Distilled features focus on key aspects of the input such as textures and parts of objects. In contrast residual features demonstrate more diffused attention often targeting irrelevant areas including the backgrounds of the target objects. In addition we proposed two novel metrics: the feature similarity score (FSS) and the relevance score (RS) which quantify the relevance of the distilled knowledge. Experiments on the CIFAR10 ASIRRA and Plant Disease datasets demonstrate that UniCAM and the two metrics offer valuable insights to explain the KD process.
FLAIR: A Conditional Diffusion Framework with Applications to Face Video Restoration
Authors: Zihao Zou; Jiaming Liu; Shirin Shoushtari; Yubo Wang; Ulugbek S. Kamilov
Venue: WACV 2025 (main)

Abstract: Face video restoration (FVR) is a challenging but important problem where one seeks to recover a perceptually realistic face videos from a low-quality input. While diffusion probabilistic models (DPMs) have been shown to achieve remarkable performance for face image restoration they often fail to preserve temporally coherent high-quality videos compromising the fidelity of reconstructed faces. We present a new conditional diffusion framework called FLAIR for FVR. FLAIR ensures improved temporal alignments across frames in a computationally efficient fashion by converting a traditional image DPM into a video DPM. The proposed conversion uses a recurrent video refinement layer and a temporal self-attention at different scales. FLAIR also uses a conditional iterative refinement process to balance the perceptual and distortion quality during inference. This process consists of two key components: a data-consistency module that analytically ensures that the generated video precisely matches its degraded observation and a coarse-to-fine image enhancement module specifically for facial regions. Our extensive experiments show superiority of FLAIR over the current state-of-the-art (SOTA) for video super-resolution deblurring JPEG restoration and space-time frame interpolation on two high-quality face video datasets.
Texture Shape and Order Matter: A New Transformer Design for Sequential DeepFake Detection
Authors: Yunfei Li; Yuezun Li; Xin Wang; Baoyuan Wu; Jiaran Zhou; Junyu Dong
Venue: WACV 2025 (main)

Abstract: Sequential DeepFake detection is an emerging task that predicts the manipulation sequence in order. Existing methods typically formulate it as an image-to-sequence problem employing conventional Transformer architectures. However these methods lack dedicated design and consequently result in limited performance. As such this paper describes a new Transformer design called TSOM by exploring three perspectives: Texture Shape and Order of Manipulations. Our method features four major improvements: we describe a new texture-aware branch that effectively captures subtle manipulation traces with a Diversiform Pixel Difference Attention module. Then we introduce a Multi-source Cross-attention module to seek deep correlations among spatial and sequential features enabling effective modeling of complex manipulation traces. To further enhance the cross-attention we describe a Shape-guided Gaussian mapping strategy providing initial priors of the manipulation shape. Finally observing that the subsequent manipulation in a sequence may influence traces left in the preceding one we intriguingly invert the prediction order from forward to backward leading to notable gains as expected. Extensive experimental results demonstrate that our method outperforms others by a large margin highlighting the superiority of our method.
Automated Evaluation of Large Vision-Language Models on Self-Driving Corner Cases
Authors: Kai Chen; Yanze Li; Wenhua Zhang; Yanxin Liu; Pengxiang Li; Ruiyuan Gao; Lanqing Hong; Meng Tian; Xinhai Zhao; Zhenguo Li; Dit-Yan Yeung; Huchuan Lu; Xu Jia
Venue: WACV 2025 (main)

Abstract: Large Vision-Language Models (LVLMs) have received widespread attentions for advancing the interpretable self-driving. Existing evaluations of LVLMs primarily focus on multi-faceted capabilities in natural circumstances lacking automated and quantifiable assessment for self-driving let alone the severe road corner cases. In this work we propose CODA-LM the very first benchmark for the automatic evaluation of LVLMs for self-driving corner cases. We adopt a hierarchical data structure and prompt powerful LVLMs to analyze complex driving scenes and generate high-quality pre-annotations for the human annotators while for LVLM evaluation we show that using the text-only large language models (LLMs) as judges reveals even better alignment with human preferences than the LVLM judges. Moreover with our CODA-LM we build CODA-VLM a new driving LVLM surpassing all open-sourced counterparts on CODA-LM. Our CODA-VLM performs comparably with GPT-4V even surpassing GPT-4V by +21.42% on the regional perception task. We hope CODA-LM can become the catalyst to promote interpretable self-driving empowered by LVLMs.
Self-Supervised Pre-Training with Diffusion Model for Few-Shot Landmark Detection in X-Ray Images
Authors: Roberto Di Via; Francesca Odone; Vito Paolo Pastore
Venue: WACV 2025 (main)

Abstract: Deep neural networks have been extensively applied in the medical domain for various tasks including image classification segmentation and landmark detection. However their application is often hindered by data scarcity both in terms of available annotations and images. This study introduces a novel application of denoising diffusion probabilistic models (DDPMs) to the landmark detection task specifically addressing the challenge of limited annotated data in x-ray imaging. Our key innovation lies in leveraging DDPMs for self-supervised pre-training in landmark detection a previously unexplored approach in this domain. This method enables accurate landmark detection with minimal annotated training data (as few as 50 images) surpassing both ImageNet supervised pre-training and traditional self-supervised techniques across three popular x-ray benchmark datasets. To our knowledge this work represents the first application of diffusion models for self-supervised learning in landmark detection which may offer a valuable pre-training approach in few-shot regimes for mitigating data scarcity. To bolster further development and reproducibility we provide open access to our code and pre-trained models for a variety of x-ray related applications: https://github.com/Malga-Vision/DiffusionXray-FewShot-LandmarkDetection.
A Recipe for Geometry-Aware 3D Mesh Transformers
Authors: Mohammad Farazi; Yalin Wang
Venue: WACV 2025 (main)

Abstract: Utilizing patch-based transformers for unstructured geometric data such as polygon meshes presents significant challenges primarily due to the absence of a canonical ordering and variations in input sizes. Prior approaches to handling 3D meshes and point clouds have either relied on computationally intensive node-level tokens for large objects or resorted to resampling to standardize patch size. Moreover these methods generally lack a geometry-aware stable Structural Embedding (SE) often depending on simplistic absolute SEs such as 3D coordinates which compromise isometry invariance essential for tasks like semantic segmentation. In our study we meticulously examine the various components of a geometry-aware 3D mesh transformer from tokenization to structural encoding assessing the contribution of each. Initially we introduce a spectral-preserving tokenization rooted in algebraic multi-grid methods. Subsequently we detail an approach for embedding features at the patch level accommodating patches with variable node counts. Through comparative analyses against a baseline model employing simple point-wise Multi-Layer Perceptrons (MLP) our research highlights critical insights: 1) the importance of structural and positional embeddings facilitated by heat diffusion in general 3D mesh transformers; 2) the effectiveness of novel components such as geodesic masking and feature interaction via cross-attention in enhancing learning; and 3) the superior performance and efficiency of our proposed methods in challenging segmentation and classification tasks.
CabNIR: A Benchmark for In-Vehicle Infrared Monocular Depth Estimation
Authors: Ugo Leone Cavalcanti; Matteo Poggi; Fabio Tosi; Valerio Cambareri; Vladimir Zlokolica; Stefano Mattoccia
Venue: WACV 2025 (main)

Abstract: Accurate in-cabin depth estimation is critical for advancing automotive safety and occupant comfort. However existing datasets for in-vehicle scene understanding tasks often fall short in providing sufficient information and scale needed to evaluate existing depth estimation methods. In this paper we present a novel benchmark tailored for monocular depth estimation in vehicle interiors containing both near-infrared (NIR) images and corresponding ground truth depth data. Featuring over 41000 frames captured across 36 distinct vehicles and 45 different passengers it offers an unprecedented level of variability for this application domain. Evaluation on our testbench of cutting-edge single-view depth models in different flavors including zero-shot affine-invariant depth estimation or indomain specialization reveals that current depth estimation approaches while promising still have a significant performance gap to overcome before achieving the reliability required for downstream safety-critical applications. In light of its diverse range and complex scenarios we believe this benchmark could serve as a common reference for further research concerning in-cabin monocular depth estimation.