GenAI4ELab/papercli-papers huggingface dataset:
DashCop: Automated E-Ticket Generation for Two-Wheeler Traffic Violations using Dashcam Videos
Authors: Deepti Rawat; Keshav Gupta; Aryamaan Basu Roy; Ravi Kiran Sarvadevabhatla
Venue: WACV 2025 (main)

Abstract: Motorized two-wheelers are a prevalent and economical means of transportation particularly in the Asia-Pacific region. However hazardous driving practices such as triple riding and non-compliance with helmet regulations contribute significantly to accident rates. Addressing these violations through automated enforcement mechanisms can enhance traffic safety. In this paper we propose DashCop an end-to-end system for automated E-ticket generation. The system processes vehicle-mounted dashcam videos to detect two-wheeler traffic violations. Our contributions include: (1) a novel Segmentation and Cross-Association (SAC) module to accurately associate riders with their motorcycles (2) a robust cross-association-based tracking algorithm optimized for the simultaneous presence of riders and motorcycles and (3) the RideSafe-400 dataset a comprehensive annotated dashcam video dataset for triple riding and helmet rule violations. Our system demonstrates significant improvements in violation detection validated through extensive evaluations on the RideSafe-400 dataset. Project page: https://dash-cop.github.io/
Enhancing Image Layout Control with Loss-Guided Diffusion Models
Authors: Zakaria Patel; Kirill Serkh
Venue: WACV 2025 (main)

Abstract: Diffusion models are a powerful class of generative models capable of producing high-quality images from pure noise using a simple text prompt. While most methods which introduce additional spatial constraints into the generated images (e.g. bounding boxes) require fine-tuning a smaller and more recent subset of these methods take advantage of the models' attention mechanism and are training-free. These methods generally fall into one of two categories. The first entails modifying the cross-attention maps of specific tokens directly to enhance the signal in certain regions of the image. The second works by defining a loss function over the cross-attention maps and using the gradient of this loss to guide the latent. While previous work explores these as alternative strategies we provide an interpretation for these methods which highlights their complimentary features and demonstrate that it is possible to obtain superior performance when both methods are used in concert.
LiCamPose: Combining Multi-View LiDAR and RGB Cameras for Robust Single-Timestamp 3D Human Pose Estimation
Authors: Zhiyu Pan; Zhicheng Zhong; Wenxuan Guo; Yifan Chen; Jianjiang Feng; Jie Zhou
Venue: WACV 2025 (main)

Abstract: Several methods have been proposed to estimate 3D human pose from multi-view images achieving satisfactory performance on public datasets collected under relatively simple conditions. However there are limited approaches studying extracting 3D human skeletons from multimodal inputs such as RGB and point cloud data. To address this gap we introduce LiCamPose a pipeline that integrates multi-view RGB and sparse point cloud information to estimate robust 3D human poses via single timestamp. We demonstrate the effectiveness of the volumetric architecture in combining these modalities. Furthermore to circumvent the need for manually labeled 3D human pose annotations we develop a synthetic dataset generator for pretraining and design an unsupervised domain adaptation strategy to train a 3D human pose estimator without manual annotations. To validate the generalization capability of our method LiCamPose is evaluated on four datasets including two public datasets one synthetic dataset and one challenging self-collected dataset named BasketBall covering diverse scenarios. The results demonstrate that LiCamPose exhibits great generalization performance and significant application potential. The code generator and datasets are available at https://github.com/Yu-Yy/LiCamPose.
FluoNeRF: Fluorescent Novel-View Synthesis under Novel Light Source Colors
Authors: Lin Shi; Kengo Matsufuji; Ryo Kawahara; Takahiro Okabe
Venue: WACV 2025 (main)

Abstract: Synthesizing photo-realistic images of a scene from arbitrary viewpoints and under arbitrary lighting environments is one of the important research topics in computer vision and graphics. In this paper we propose a method for synthesizing photo-realistic images of a scene with fluorescent objects from novel viewpoints and under novel lighting colors. In general fluorescent materials absorb light with certain wavelengths and then emit light with longer wavelengths than the absorbed ones in contrast to reflective materials which preserve wavelengths of light. Therefore we cannot reproduce the colors of fluorescent objects under arbitrary lighting colors by combining conventional view synthesis techniques with the white balance adjustment of the RGB channels. Accordingly we extend the novel view synthesis based on the neural radiance fields by incorporating the superposition principle of light; our proposed method captures a sparse set of images of a scene from varying viewpoints and under varying light source colors by using a display-camera system and then synthesize photo-realistic images of the scene without explicitly modeling the geometric and photometric models of the scene. We conduct a number of experiments using real images and confirm the effectiveness of our method.
Planar Gaussian Splatting
Authors: Farhad G. Zanjani; Hong Cai; Hanno Ackermann; Leila Mirvakhabova; Fatih Porikli
Venue: WACV 2025 (main)

Abstract: This paper presents Planar Gaussian Splatting (PGS) a novel neural rendering approach to learn the 3D geometry and parse the 3D planes of a scene directly from multiple RGB images. The PGS leverages Gaussian primitives to model the scene and employ a hierarchical Gaussian mixture approach to group them. Similar Gaussians are progressively merged probabilistically in the tree-structured Gaussian mixtures to identify distinct 3D plane instances and form the overall 3D scene geometry. In order to enable the grouping the Gaussian primitives contain additional parameters such as plane descriptors derived by lifting 2D masks from a general 2D segmentation model and surface normals. Experiments show that the proposed PGS achieves state-of-the-art performance in 3D planar reconstruction without requiring either 3D plane labels or depth supervision. In contrast to existing supervised methods that have limited generalizability and struggle under domain shift PGS maintains its performance across datasets thanks to its neural rendering and scene specific optimization mechanism while also being significantly faster than existing optimization-based approaches.
Importance-Guided Interpretability and Pruning for Video Transformers in Driver Action Recognition
Authors: Raquel Panadero Palenzuela; Dominik Schörkhuber; Margrit Gelautz
Venue: WACV 2025 (main)

Abstract: Recently transformers have gained prominence in video action recognition due to their ability to capture spatio-temporal dependencies. Despite their effectiveness the interpretability of their self-attention mechanisms remains limited posing obstacles in understanding model decisions impacting transparency and bias identification. Additionally the computational demands of transformer architectures particularly the self-attention mechanism present practical difficulties. To tackle both challenges we adapt existing interpretability techniques and introduce a layer pruning method guided by importance metrics. In the context of driver action recognition our findings highlight the efficacy of the applied head importance metrics in pinpointing crucial attention heads and identifying key visual cues essential for recognizing driver behavior. Experimental results conducted on three mainstream video transformers demonstrate the effectiveness of the proposed pruning technique with significantly reduced computational costs and only slight performance degradation by removing low-relevance layers. Specifically on our DriverActionInsight (DAI) dataset we achieve a 23.5% FLOPs saving in compressing Video Swin with less than a 1% decrease in Top-1 accuracy.
McCaD: Multi-Contrast MRI Conditioned Adaptive Adversarial Diffusion Model for High-Fidelity MRI Synthesis
Authors: Sanuwani Dayarathna; Kh Tohidul Islam; Bohan Zhuang; Guang Yang; Jianfei Cai; Meng Law; Zhaolin Chen
Venue: WACV 2025 (main)

Abstract: Magnetic Resonance Imaging (MRI) is instrumental in clinical diagnosis offering diverse contrasts that provide comprehensive diagnostic information. However acquiring multiple MRI contrasts is often constrained by high costs long scanning durations and patient discomfort. Current synthesis methods typically focused on single-image contrasts fall short in capturing the collective nuances across various contrasts. Moreover existing methods for multi-contrast MRI synthesis often fail to accurately map feature level information across multiple imaging contrasts. We introduce McCaD (Multi-Contrast MRI Conditioned Adaptive Adversarial Diffusion) a novel framework leveraging an adversarial diffusion model conditioned on multiple contrasts for high-fidelity MRI synthesis. McCaD significantly enhances synthesis accuracy by employing a multiscale feature-guided mechanism incorporating denoising and semantic encoders. An adaptive feature maximization strategy and a spatial feature-attentive loss have been introduced to capture more intrinsic features across multiple contrasts. This facilitates a precise and comprehensive feature-guided denoising process. Extensive experiments on tumor and healthy multi-contrast MRI datasets demonstrated that the McCaD outperforms state-of-the-art baselines quantitively and qualitatively. The code is available at https://github.com/sanuwanihewa/McCaD.
Detecting Wildfires on UAVs with Real-Time Segmentation Trained by Larger Teacher Models
Authors: Julius Pesonen; Teemu Hakala; Väinö Karjalainen; Niko Koivumäki; Lauri Markelin; Anna-Maria Raita-Hakola; Juha Suomalainen; Ilkka Pölönen; Eija Honkavaara
Venue: WACV 2025 (main)

Abstract: Early detection of wildfires is essential to prevent large-scale fires resulting in extensive environmental structural and societal damage. Uncrewed aerial vehicles (UAVs) can cover large remote areas effectively with quick deployment requiring minimal infrastructure and equipping them with small cameras and computers enables autonomous real-time detection. In remote areas however detection methods are limited to onboard computation due to the lack of high-bandwidth mobile networks. For accurate camera-based localisation segmentation of the detected smoke is essential but training data for deep learning-based wildfire smoke segmentation is limited. This study shows how small specialised segmentation models can be trained using only bounding box labels leveraging zero-shot foundation model supervision. The method offers the advantages of needing only fairly easily obtainable bounding box labels and requiring training solely for the smaller student network. The proposed method achieved 63.3 percent mIoU on a manually annotated and diverse wildfire dataset. The used model can perform in real-time at 25 fps with a UAV-carried NVIDIA Jetson Orin NX computer while reliably recognising smoke as demonstrated at real-world forest burning events. Code is available at: https://gitlab.com/fgi_nls/public/wildfire-real-time-segmentation
Multi-Class Textual-Inversion Secretly Yields a Semantic-Agnostic Classifier
Authors: Kai Wang; Fei Yang; Bogdan Raducanu; Joost van de Weijer
Venue: WACV 2025 (main)

Abstract: With the advent of large pre-trained vision-language models such as CLIP prompt learning methods aim to enhance the transferability of the CLIP model. They learn the prompt given few samples from the downstream task given the specific class names as prior knowledge which we term as semantic-aware classification. However in many realistic scenarios we only have access to few samples and knowledge of the class names (e.g. when considering instances of classes). This challenging scenario represents the semantic-agnostic discriminative case. Text-to-Image (T2I) personalization methods aim to adapt T2I models to unseen concepts by learning new tokens and endowing these tokens with the capability of generating the learned concepts. These methods do not require knowledge of class names as a semantic-aware prior. Therefore in this paper we first explore Textual Inversion and reveal that the new concept tokens possess both generation and classification capabilities by regarding each category as a single concept. However learning classifiers from single-concept textual inversion is limited since the learned tokens are suboptimal for the discriminative tasks. To mitigate this issue we propose Multi-Class textual inversion which includes a discriminative regularization term for the token updating process. Using this technique our method MC-TI achieves stronger Semantic-Agnostic Classification while preserving the generation capability of these modifier tokens given only few samples per category. In the experiments we extensively evaluate MC-TI on 12 datasets covering various scenarios which demonstrates that MC-TI achieves superior results in terms of both classification and generation outcomes.
FaVoR: Features via Voxel Rendering for Camera Relocalization
Authors: Vincenzo Polizzi; Marco Cannici; Davide Scaramuzza; Jonathan Kelly
Venue: WACV 2025 (main)

Abstract: Camera relocalization methods range from dense image alignment to direct camera pose regression from a query image. Among these sparse feature matching stands out as an efficient versatile and generally lightweight approach with numerous applications. However feature-based methods often struggle with significant viewpoint and appearance changes leading to matching failures and inaccurate pose estimates. To overcome these limitations we propose a novel approach that leverages a globally sparse but locally dense 3D representation of 2D features. By tracking and triangulating landmarks over a sequence of frames we construct a sparse voxel map optimized to render image patch descriptors observed during tracking. Given an initial pose estimate we first synthesize descriptors from the voxels using volumetric rendering and then perform feature matching to estimate the camera pose. This method enables the generation of descriptors for unseen views enhancing robustness to viewpoint changes. We evaluate our method on the 7-Scenes and Cambridge Landmarks datasets. Our results show that our approach significantly outperforms existing state-of-the-art feature representation techniques in indoor environments achieving up to a 39% improvement in median translation error. Additionally our approach yields comparable results to other methods for outdoor scenes but with lower computational and memory footprints.
Label Augmented Dataset Distillation
Authors: Seoungyoon Kang; Youngsun Lim; Hyunjung Shim
Venue: WACV 2025 (main)

Abstract: Traditional dataset distillation primarily focuses on image representation while often overlooking the important role of labels. In this study we introduce Label-Augmented Dataset Distillation (LADD) a new dataset distillation framework enhancing dataset distillation with label augmentations. LADD sub-samples each synthetic image generating additional dense labels to capture rich semantics. These dense labels require only a 2.5% increase in storage (ImageNet subsets) with significant performance benefits providing strong learning signals. Our label-generation strategy can complement existing dataset distillation methods and significantly enhance their training efficiency and performance. Experimental results demonstrate that LADD outperforms existing methods in terms of computational overhead and accuracy. With three high-performance dataset distillation algorithms LADD achieves remarkable gains by an average of 14.9% in accuracy. Furthermore the effectiveness of our method is proven across various datasets distillation hyperparameters and algorithms. Finally our method improves the cross-architecture robustness of the distilled dataset which is important in the application scenario.
Towards Privacy-Preserving Split Learning for ControlNet
Authors: Dixi Yao
Venue: WACV 2025 (main)

Abstract: With the emerging trend of large generative models ControlNet is introduced to enable users to fine-tune pre-trained models with their own data for various use cases. A natural question arises: how can we train ControlNet models while ensuring users' data privacy across distributed devices? We first propose a new distributed learning structure that eliminates the need for the server to send gradients based on split learning. We discover that in the context of fine-tuning ControlNet with split learning most existing attacks are ineffective except for two mentioned in previous literature. To counter these threats we leverage the properties of diffusion models and design a new timestep sampling policy during forward processes. We also propose a privacy-preserving activation function and a method to prevent private text prompts from leaving clients tailored for image generation with diffusion models. Our experimental results demonstrate that our algorithms and systems greatly enhance the efficiency of distributed fine-tuning for ControlNet while ensuring users' data privacy without compromising image generation quality.
Solar Multimodal Transformer: Intraday Solar Irradiance Predictor using Public Cameras and Time Series
Authors: Yanan Niu; Roy Sarkis; Demetri Psaltis; Mario Paolone; Christophe Moser; Luisa Lambertini
Venue: WACV 2025 (main)

Abstract: Accurate intraday solar irradiance forecasting is crucial for optimizing dispatch planning and electricity trading. For this purpose we introduce a novel and effective approach that includes three distinguishing components from the literature: 1) the uncommon use of single-frame public camera imagery; 2) solar irradiance time series scaled with a proposed normalization step which boosts performance; and 3) a lightweight multimodal model called Solar Multimodal Transformer (SMT) that delivers accurate short-term solar irradiance forecasting by combining images and scaled time series. Benchmarking against Solcast a leading solar forecasting service provider our model improved prediction accuracy by 25.95%. Our approach allows for easy adaptation to various camera specifications offering broad applicability for real-world solar forecasting challenges.
Finding Dino: A Plug-and-Play Framework for Zero-Shot Detection of Out-of-Distribution Objects using Prototypes
Authors: Poulami Sinhamahapatra; Franziska Schwaiger; Shirsha Bose; Huiyu Wang; Karsten Roscher; Stephan Günnemann
Venue: WACV 2025 (main)

Abstract: Detecting and localising unknown or Out-of-distribution (OOD) objects in any scene can be a challenging task in vision particularly in safety-critical cases involving autonomous systems like automated vehicles or trains. Supervised anomaly segmentation or open-world object detection models depend on training on exhaustively annotated datasets for every domain and still struggle in distinguishing between background and OOD objects. In this work we present a plug-and-play framework - PRototype based OOD detection Without Labels (PROWL). It is an inference-based method that does not require training on the domain dataset and relies on extracting relevant features from self-supervised pre-trained models. PROWL can be easily adapted to detect in-domain objects in any operational design domain (ODD) in a zero-shot manner by specifying a list of known classes from this domain. PROWL as a first zero-shot unsupervised method achieves state-of-the-art results on the RoadAnomaly and RoadObstacle datasets provided in road driving benchmarks - SegmentMeIfYouCan (SMIYC) and Fishyscapes as well as comparable performance against existing supervised methods trained without auxiliary OOD data. We also demonstrate its generalisability to other domains such as rail and maritime.
Semantic Prompt Learning for Weakly-Supervised Semantic Segmentation
Authors: Ci-Siang Lin; Chien-Yi Wang; Yu-Chiang Frank Wang; Min-Hung Chen
Venue: WACV 2025 (main)

Abstract: Weakly-Supervised Semantic Segmentation (WSSS) aims to train segmentation models using image data with only image-level supervision. Since precise pixel-level annotations are not accessible existing methods typically focus on producing pseudo masks for training segmentation models by refining CAM-like heatmaps. However the produced heatmaps may capture only the discriminative image regions of object categories or the associated co-occurring backgrounds. To address the issues we propose a Semantic Prompt Learning for WSSS (SemPLeS) framework which learns to effectively prompt the CLIP latent space to enhance the semantic alignment between the segmented regions and the target object categories. More specifically we propose Contrastive Prompt Learning and Prompt-guided Semantic Refinement to learn the prompts that adequately describe and suppress the co-occurring backgrounds associated with each object category. In this way SemPLeS can perform better semantic alignment between object regions and class labels resulting in desired pseudo masks for training segmentation models. The proposed SemPLeS framework achieves competitive performance on standard WSSS benchmarks PASCAL VOC 2012 and MS COCO 2014 and shows compatibility with other WSSS methods. Project page: https://projectdisr.github.io/semples/
Evaluating Sensitivity Consistency of Explanations
Authors: Hanxiao Tan
Venue: WACV 2025 (main)

Abstract: While the performance of deep neural networks is rapidly developing their reliability is increasingly receiving more attention. Explainability methods are one of the most relevant tools to enhance reliability mainly by highlighting important input features for the explanation purpose. Although numerous explainability methods have been proposed their assessment remains challenging due to the absence of ground truth. Several existing studies propose evaluation methods from a certain aspect e.g. fidelity robustness etc. However they typically address only one property of explanations and thus more assessing perspectives contribute to a better explanation evaluating system. This work proposes an evaluation method from a novel perspective called sensitivity consistency where the intuition behind is that features and parameters that strongly impact the predictions and explanations should be highly consistent and vise versa. Extensive experiments on different datasets and models evaluate popular explainability methods while providing qualitative and quantitative results. Our approach further complements the existing evaluation systems and aims to facilitate the proposal of an acknowledged explanation evaluation methodology.
EgoSonics: Generating Synchronized Audio for Silent Egocentric Videos
Authors: Aashish Rai; Srinath Sridhar
Venue: WACV 2025 (main)

Abstract: We introduce EgoSonics a method to generate semantically meaningful and synchronized audio tracks conditioned on silent egocentric videos. Generating audio for silent egocentric videos could open new applications in virtual reality assistive technologies or for augmenting existing datasets. Existing work has been limited to domains like speech music or impact sounds and cannot easily capture the broad range of audio frequencies found in egocentric videos. EgoSonics addresses these limitations by building on the strength of latent diffusion models for conditioned audio synthesis. We first encode and process audio and video data into a form that is suitable for generation. The encoded data is used to train our model to generate audio tracks that capture the semantics of the input video. Our proposed SyncroNet builds on top of ControlNet to provide control signals that enables temporal synchronization to the synthesized audio. Extensive evaluations show that our model outperforms existing work in audio quality and in our newly proposed synchronization evaluation method. Furthermore we demonstrate downstream applications of our model in improving video summarization.