GenAI4ELab/papercli-papers huggingface dataset:
ConvMixFormer- A Resource-Efficient Convolution Mixer for Transformer-Based Dynamic Hand Gesture Recognition
Authors: Mallika Garg; Debashis Ghosh; Pyari Mohan Pradhan
Venue: WACV 2025 (main)

Abstract: Transformer models have demonstrated remarkable success in many domains such as natural language processing (NLP) and computer vision. With the growing interest in transformer-based architectures they are now utilized for gesture recognition. So we also explore and devise a novel ConvMixFormer architecture for dynamic hand gestures. The transformers use quadratic scaling of the attention features with the sequential data due to which these models are computationally complex and heavy. We have considered this drawback of the transformer and designed a resource-efficient model that replaces the self-attention in the transformer with the simple convolutional layer-based token mixer. The computational cost and the parameters used for the convolution-based mixer are comparatively less than the quadratic self-attention. Convolution-mixer helps the model capture the local spatial features that self-attention struggles to capture due to their sequential processing nature. Further an efficient gate mechanism is employed instead of a conventional feed-forward network in the transformer to help the model control the flow of features within different stages of the proposed model. This design uses fewer learnable parameters which is nearly half the vanilla transformer that helps in fast and efficient training. The proposed method is evaluated on NVidia Dynamic Hand Gesture and Briareo datasets and our model has achieved state-of-the-art results on single and multimodal inputs. We have also shown the parameter efficiency of the proposed ConvMixFormer model compared to other methods. The source code is available at https://github.com/mallikagarg/ConvMixFormer.
Deep Joint Unrolling for Deblurring and Low-Light Image Enhancement (JUDE)
Authors: Tu Vo; Chan Y. Park
Venue: WACV 2025 (main)

Abstract: Low-light and blurring issues are prevalent when capturing photos at night often due to the use of long exposure to address dim environments. Addressing these joint problems can be challenging and error-prone if an end-to-end model is trained without incorporating an appropriate physical model. In this paper we introduce JUDE a Deep Joint Unrolling for Deblurring and Low-Light Image Enhancement inspired by the image physical model. Based on Retinex theory and the blurring model the low-light blurry input is iteratively deblurred and decomposed producing sharp low-light reflectance and illuminance through an unrolling mechanism. Additionally we incorporate various modules to estimate the initial blur kernel enhance brightness and eliminate noise in the final image. Comprehensive experiments on LOL-Blur and Real-LOL-Blur demonstrate that our method outperforms existing techniques both quantitatively and qualitatively.
VM-Gait: Multi-Modal 3D Representation Based on Virtual Marker for Gait Recognition
Authors: Zhao-Yang Wang; Jiang Liu; Jieneng Chen; Rama Chellappa
Venue: WACV 2025 (main)

Abstract: Gait recognition plays a vital role in biometric applications by analyzing the unique characteristics of an individual's walking pattern. Methods based on 2D representations such as silhouettes and skeletons are increasingly being developed to learn the shape features and joint dynamic movements. Nevertheless the effectiveness of 2D representation-based methods is impeded by factors such as changes in viewpoint partial occlusion and noisy environments. 3D representation-based methods can complement 2D representation-based approaches by providing more precise dynamic body shapes and motion information along with increased robustness against changes in viewpoint and partial occlusion. However the complexity of acquiring accurate 3D representations and the challenges associated with extracting dynamic topological features from sequences of 3D representations hinder the development of 3D representations-based methods. In this paper we present VM-Gait a novel multi-modal gait recognition framework that harnesses the advantages of integrating both 2D and 3D representations. Furthermore we introduce a new 3D representation Virtual Marker into gait recognition to efficiently learn topological features from 3D representations avoiding the computational complexities inherent in directly learning from 3D representations like 3D meshes or 3D point clouds. Extensive experiments demonstrate that the proposed framework effectively learns and fuses discriminative information from different gait modalities enhancing gait recognition performance.
Point-JEPA: A Joint Embedding Predictive Architecture for Self-Supervised Learning on Point Cloud
Authors: Ayumu Saito; Prachi Kudeshia; Jiju Poovvancheri
Venue: WACV 2025 (main)

Abstract: Recent advancements in self-supervised learning in the point cloud domain have demonstrated significant potential. However these methods often suffer from drawbacks such as lengthy pre-training time the necessity of reconstruction in the input space and the necessity of additional modalities. In order to address these issues we introduce Point-JEPA a joint embedding predictive architecture designed specifically for point cloud data. To this end we introduce a sequencer that orders point cloud patch embeddings to efficiently compute and utilize their proximity based on their indices during target and context selection. The sequencer also allows shared computations of the patch embeddings' proximity between context and target selection further improving the efficiency. Experimentally our method demonstrates state-of-the-art performance while avoiding the reconstruction in the input space or additional modality. In particular Point-JEPA attains a classification accuracy of 93.7% for linear SVM on ModelNet40 surpassing all other self-supervised models. Moreover Point-JEPA also establishes new state-of-the-art performance levels across all four few-shot learning evaluation frameworks. The code is available at https://github.com/Ayumu-JS/Point-JEPA
CM3T: Framework for Efficient Multimodal Learning for Inhomogeneous Interaction Datasets
Authors: Tanay Agrawal; Mohammed Guermal; Michal Balazia; Francois Bremond
Venue: WACV 2025 (main)

Abstract: Challenges in cross-learning involve inhomogeneous or even inadequate amount of training data and lack of resources for retraining large pretrained models. Inspired by transfer learning techniques in NLP adapters and prefix tuning this paper presents a new model-agnostic plugin architecture for cross-learning called CM3T that allows transformer-based models to be able adapt to new or missing information. We introduce two adapter blocks: multi-head vision adapters for transfer learning and cross-attention adapters for multimodal learning. Training becomes substantially efficient as the backbone and other plugins do not need to be finetuned along with these additions. Experiments and ablation studies on three datasets - Epic-Kitchens-100 MPIIGroupInteraction and UDIVA v0.5 - show the efficacy of this framework on different recording settings and tasks. With only 12.8% trainable parameters compared to the backbone to process video input and 22.3% trainable parameters for two additional modalities we achieve comparable and even better results than the state-of-the-art. Compared to similar methods our work achieves this result without any specific requirements for training or pretraining and is a step towards bridging the gap between a general model and specific practical applications in the field of video classification.
CTIP: Towards Accurate Tabular-to-Image Generation for Tire Footprint Generation
Authors: Daeyoung Roh; Donghee Han; Jihyun Nam; Jungsoo Oh; Youngbin You; Jeongheon Park; Mun Yi
Venue: WACV 2025 (main)

Abstract: Generating images directly from tabular data while ensuring an accurate representation of ground truth is a useful application in manufacturing. Simply embedding tabular data to use it as a condition in image generation models often fails to learn the correspondence between tabular features and their impact on the generated image. To overcome this limitation we propose Contrastive TabularImage Pre-training (CTIP) inspired by the CLIP framework. These pre-train methods help improve the quality of the embedding of the tabular encoder on the tabular data which then helps improve the performance of the image generation model. CTIP uses contrastive learning on multiple tabular and image data pairs allowing the model to learn how changes in certain tabular features affect images. This approach is particularly crucial in manufacturing where accurate capture of product outcomes under varying conditions is essential. We demonstrate that applying CTIP enhances image generation performance yielding images that closely match ground truth images even in Feature Fewshot or Feature Zero-shot scenarios where specific features are sparse or novel. We further show the application of CTIP in tire development where tire footprint images are generated based on tire specifications and test conditions. CTIP produces high-quality embeddings that align well with ground truth images and effectively handle the scarcity or sparseness of specific features addressing common challenges in new product development. Our code is available in https://github.com/Noverse0/CTIP.git.
CIRCOD: Co-Saliency Inspired Referring Camouflaged Object Discovery
Authors: Avi Gupta; Koteswar Rao Jerripothula; Tammam Tillo
Venue: WACV 2025 (main)

Abstract: Camouflaged object detection (COD) the task of identifying objects concealed within their surroundings is often quite challenging due to the similarity that exists between the foreground and background. By incorporating an additional referring image where the target object is clearly visible we can leverage the similarities between the two images to detect the camouflaged object. In this paper we propose a novel problem setup: referring camouflaged object discovery (RCOD). In RCOD segmentation occurs only when the object in the referring image is also present in the camouflaged image; otherwise a blank mask is returned. This setup is particularly valuable when searching for specific camouflaged objects. Current COD methods are often generic leading to numerous false positives in applications focused on specific objects. To address this we introduce a new framework called Co-Saliency Inspired Referring Camouflaged Object Discovery (CIRCOD). Our approach consists of two main components: Co-Saliency-Aware Image Transformation (CAIT) and Co-Salient Object Discovery (CSOD). The CAIT module reduces the appearance and structural variations between the camouflaged and referring images while the CSOD module utilizes the similarities between them to segment the camouflaged object provided the images are semantically similar. Covering all semantic categories in current COD benchmark datasets we collected over 1000 referring images to validate our approach. Our extensive experiments demonstrate the effectiveness of our method and show that it achieves superior results compared to existing methods. Code is available at https://github.com/avigupta2798/CIRCOD/
Dense Scene Reconstruction from Light-Field Images Affected by Rolling Shutter
Authors: Hermes McGriff; Renato Martins; Nicolas Andreff; Cedric Demonceaux
Venue: WACV 2025 (main)

Abstract: This paper presents a dense depth estimation approach from light-field (LF) images that is able to compensate for strong rolling shutter (RS) effects. Our method estimates RS compensated views and dense RS compensated disparity maps. We present a two-stage method based on a 2D Gaussians Splatting that allows for a "render and compare" strategy with a point cloud formulation. In the first stage a subset of sub-aperture images is used to estimate an RS agnostic 3D shape that is related to the scene target shape "up to a motion". In the second stage the deformation of the 3D shape is computed by estimating an admissible camera motion. We demonstrate the effectiveness and advantages of this approach through several experiments conducted for different scenes and types of motions. Due to lack of suitable datasets for evaluation we also present a new carefully designed synthetic dataset of RS LF images. The source code trained models and dataset will be made publicly available at: https://github.com/ICB-Vision-AI/DenseRSLF.
GauFRe: Gaussian Deformation Fields for Real-Time Dynamic Novel View Synthesis
Authors: Yiqing Liang; Numair Khan; Zhengqin Li; Thu H Nguyen-Phuoc; Douglas Lanman; James Tompkin; Lei Xiao
Venue: WACV 2025 (main)

Abstract: We propose a method that achieves state-of-the-art rendering quality and efficiency on monocular dynamic scene reconstruction using deformable 3D Gaussians. Implicit deformable representations commonly model motion with a canonical space and time-dependent backward-warping deformation field. Our method GauFRe uses a forward-warping deformation to explicitly model non-rigid transformations of scene geometry. Specifically we propose a template set of 3D Gaussians residing in a canonical space and a time-dependent forward-warping deformation field to model dynamic objects. Additionally we tailor a 3D Gaussian-specific static component supported by an inductive bias-aware initialization approach which allows the deformation field to focus on moving scene regions improving the rendering of complex real-world motion. The differentiable pipeline is optimized end-to-end with a self-supervised rendering loss. Experiments show our method achieves competitive results and higher efficiency than both previous state-of-the-art NeRF and Gaussian-based methods. For real-world scenes GauFRe can train in  20 mins and offer 96 FPS real-time rendering on an RTX 3090 GPU.
AC-IND: Sparse CT Reconstruction Based on Attenuation Coefficient Estimation and Implicit Neural Distribution
Authors: Wangduo Xie; Richard Schoonhoven; Tristan van Leeuwen; Matthew B. Blaschko
Venue: WACV 2025 (main)

Abstract: Computed tomography (CT) reconstruction plays a crucial role in industrial nondestructive testing and medical diagnosis. Sparse view CT reconstruction aims to reconstruct high-quality CT images while only using a small number of projections which helps to improve the detection speed of industrial assembly lines and is also meaningful for reducing radiation in medical scenarios. Sparse CT reconstruction methods based on implicit neural representations (INRs) have recently shown promising performance but still produce artifacts because of the difficulty of obtaining useful prior information. In this work we incorporate a powerful prior: the total number of material categories of objects. To utilize the prior we design AC-IND a self-supervised method based on Attenuation Coefficient Estimation and Implicit Neural Distribution. Specifically our method first transforms the traditional INR from scalar mapping to probability distribution mapping. Then we design a compact attenuation coefficient estimator initialized with values from a rough reconstruction and fast segmentation. Finally our algorithm finishes the CT reconstruction by jointly optimizing the estimator and the generated distribution. Through experiments we find that our method not only outperforms the comparative methods in sparse CT reconstruction but also can automatically generate semantic segmentation maps. Code is available at https://github.com/AIIAAI/AC-IND.
ORID: Organ-Regional Information Driven Framework for Radiology Report Generation
Authors: Tiancheng Gu; Kaicheng Yang; Xiang An; Ziyong Feng; Dongnan Liu; Weidong Cai
Venue: WACV 2025 (main)

Abstract: The objective of Radiology Report Generation (RRG) is to automatically generate coherent textual analyses of diseases based on radiological images thereby alleviating the workload of radiologists. Current AI-based methods for RRG primarily focus on modifications to the encoder-decoder model architecture. To advance these approaches this paper introduces an Organ-Regional Information Driven (ORID) framework which can effectively integrate multi-modal information and reduce the influence of noise from unrelated organs. Specifically based on the LLaVA-Med we first construct an RRG-related instruction dataset to improve organ-regional diagnosis description ability and get the LLaVA-Med-RRG. After that we propose an organ-based cross-modal fusion module to effectively combine the information from the organ-regional diagnosis description and radiology image. To further reduce the influence of noise from unrelated organs on the radiology report generation we introduce an organ importance coefficient analysis module which leverages Graph Neural Network (GNN) to examine the interconnections of the cross-modal information of each organ region. Extensive experiments and comparisons with state-of-the-art methods across various evaluation metrics demonstrate the superior performance of our proposed method.
Rubric-Constrained Figure Skating Scoring
Authors: Arushi Rai; Adriana Kovashka
Venue: WACV 2025 (main)

Abstract: Figure skating automatic scoring is the task of estimating the competition score of a performance video. The technical element score (TES) aggregates the technical quality (grade of execution) and difficulty (base value) scores for each element. Most prior work adapted from short-term action quality assessment entangle difficulty and quality and compute TES for the entire video reducing interpretability for athletes. This is mainly due to a lack of element segmentation and difficulty annotations in existing datasets. Motivated by increasing interpretability we propose a novel method that implicitly segments a video to produce element-level representations and uses adherence with a natural language rubric to score each element without needing additional annotations. We compute element-level representations using learnable element queries in a transformer and propose implicit segmentation regularization to encourage element queries to attend to elements rather than background transitions between elements (most of video). Additionally we use the element list (sequence of elements) to isolate difficulty just like judges who receive the routine list in advance so we can focus on the more critical problem of how well elements are done. These components significantly improve interpretability scoring precision and ranking capability. Code is released at https://arushirai1.github.io/rcs-project.
A Reality Check on Pre-training for Exemplar-free Class-Incremental Learning
Authors: Eva Feillet; Adrian Popescu; Céline Hudelot
Venue: WACV 2025 (main)

Abstract: Exemplar-free class-incremental learning (EFCIL) aims to classify streaming data without storing examples from the past. Recent EFCIL works suggest that (i) models pre-trained with large amounts of data should be used to initialize learning (ii) self-supervised learned transformers generalize better than supervised convolutional models (iii) adding generated data to the pre-training dataset can improve incremental accuracy. In this article we question the above assertions by comprehensively evaluating various initial training strategies combined with four EFCIL algorithms using four large-scale datasets. Our results indicate that: (i) pre-trained models are preferable when the domain of the incremental classification task is well represented in the pre-training datasets but training with initial data remains useful when the domain shift is significant (ii) supervised convolutional networks remain competitive particularly when improving representation transferability using data augmentation or a projector (iii) adding classes from an external dataset to train the initial model boosts performance when the initial set of classes is small but has a limited effect otherwise (iv) additional classes generated with a diffusion model are not necessarily more useful than a well-chosen set of ImageNet classes to improve model transferability. We provide a nuanced analysis of these results and formulate recommendations to facilitate the practical adoption of EFCIL algorithms.
Polarization as Texture: Microscale 3D Shape from Polarized Light Focus
Authors: Ren Matsumoto; Takahiro Okabe; Ryo Kawahara
Venue: WACV 2025 (main)

Abstract: Defocus is a crucial cue for image-based microscale depth estimation yet its measurement depends on spatial appearance changes such as texture. We show that passively observed polarization is responsive to small irregularities of the surface visible in the microscopic world and can be leveraged for focus measure as a strong texture. Our key idea is to leverage texture from polarization for blur analysis and accurately estimate the focus level of microscopic polarization images. We further utilize normal cues from polarization to create a prior distribution of the focus level between neighboring pixels. We then interpolatively propagate the focus level of discrete image slices at different focus depths while denoising. We implement our method with a single polarization camera with a microscope and recover the per-pixel depth from the multi-focus images. The reconstructed results demonstrate the effectiveness of our method for various microscale objects regardless of the surface texture.
Fine-Grained Controllable Video Generation via Object Appearance and Context
Authors: Hsin-Ping Huang; Yu-Chuan Su; Deqing Sun; Lu Jiang; Xuhui Jia; Yukun Zhu; Ming-Hsuan Yang
Venue: WACV 2025 (main)

Abstract: While text-to-video generation shows state-of-the-art results fine-grained output control remains challenging for users relying solely on natural language prompts. In this work we present FACTOR for fine-grained controllable video generation. FACTOR provides an intuitive interface where users can manipulate the trajectory and appearance of individual objects in conjunction with a text prompt. We propose a unified framework to integrate these control signals into an existing text-to-video model. Our approach involves a multimodal condition module with a joint encoder control-attention layers and an appearance augmentation mechanism. This design enables FACTOR to generate videos that closely align with detailed user specifications. Extensive experiments on standard benchmarks and user-provided inputs demonstrate a notable improvement in controllability by FACTOR over competitive baselines.
Exploiting Inter-Sample Information for Long-Tailed Out-of-Distribution Detection
Authors: Nimeshika Udayangani; Hadi Mohaghegh Dolatabadi; Sarah Erfani; Christopher Leckie
Venue: WACV 2025 (main)

Abstract: Detecting out-of-distribution (OOD) data is essential for safe deployment of deep neural networks (DNNs). This problem becomes particularly challenging in the presence of long-tailed in-distribution (ID) datasets often leading to high false positive rates (FPR) and low tail-class ID classification accuracy. In this paper we demonstrate that exploiting inter-sample relationships using a graph-based representation can significantly improve OOD detection in long-tailed recognition of vision datasets. To this end we use the feature space of a pre-trained model to initialize our graph structure. We account for the differences between the activation layer distribution of the pre-training vs. training data and actively introduce Gaussianization to alleviate any deviations from a standard normal distribution in the activation layers of the pre-trained model. We then refine this initial graph representation using graph convolutional networks (GCNs) to arrive at a feature space suitable for long-tailed OOD detection. This leads us to address the inferior performance observed in ID tail-classes within existing OOD detection methods. Experiments over three benchmarks CIFAR10-LT CIFAR100-LT and ImageNet-LT demonstrate that our method outperforms the state-of-the-art approaches by a large margin in terms of FPR and tail-class ID classification accuracy.