GenAI4ELab/papercli-papers huggingface dataset:
Autoregressive Adaptive Hypergraph Transformer for Skeleton-Based Activity Recognition
Authors: Abhisek Ray; Ayush Raj; Maheshkumar H. Kolekar
Venue: WACV 2025 (main)

Abstract: Extracting multiscale contextual information and higher-order correlations among skeleton sequences using Graph Convolutional Networks (GCNs) alone is inadequate for effective action classification. Hypergraph convolution addresses the above issues but cannot harness the long-range dependencies. The transformer proves to be effective in capturing these dependencies and making complex contextual features accessible. We propose an Autoregressive Adaptive HyperGraph Transformer (AutoregAd-HGformer) model for in-phase (autoregressive and discrete) and out-phase (adaptive) hypergraph generation. The vector quantized in-phase hypergraph equipped with powerful autoregressive learned priors produces a more robust and informative representation suitable for hyperedge formation. The out-phase hypergraph generator provides a model-agnostic hyperedge learning technique to align the attributes with input skeleton embedding. The hybrid (supervised and unsupervised) learning in AutoregAd-HGformer explores the action-dependent feature along spatial temporal and channel dimensions. The extensive experimental results and ablation study indicate the superiority of our model over state-of-the-art hypergraph architectures on the NTU RGB+D NTU RGB+D 120 and NW-UCLA datasets.
Stratified Domain Adaptation: A Progressive Self-Training Approach for Scene Text Recognition
Authors: Kha Nhat Le; Hoang-Tuan Nguyen; Hung Tien Tran; Thanh Duc Ngo
Venue: WACV 2025 (main)

Abstract: Unsupervised domain adaptation (UDA) has become increasingly prevalent in scene text recognition (STR) especially where training and testing data reside in different domains. The efficacy of existing UDA approaches tends to degrade when there is a large gap between the source and target domains. To deal with this problem gradually shifting or progressively learning to shift from domain to domain is the key issue. In this paper we introduce the Stratified Domain Adaptation (StrDA) approach which examines the gradual escalation of the domain gap for the learning process. The objective is to partition the target data into subsets so that the progressively self-trained model can adapt to gradual changes. We stratify the target data by evaluating the proximity of each data sample to both the source and target domains. We propose a novel method for employing domain discriminators to estimate the out-of-distribution and domain discriminative levels of data samples. Extensive experiments on benchmark scene-text datasets show that our approach significantly improves the performance of baseline (source-trained) STR models. The source code is available at https://github.com/KhaLee2307/StrDA.
Contrastive Sequential-Diffusion Learning: Non-Linear and Multi-Scene Instructional Video Synthesis
Authors: Vasco Ramos; Yonatan Bitton; Michal Yarom; Idan Szpektor; Joao Magalhaes
Venue: WACV 2025 (main)

Abstract: Generated video scenes for action-centric sequence descriptions such as recipe instructions and do-it-yourself projects often include non-linear patterns where the next video may need to be visually consistent not with the immediately preceding video but with earlier ones. Current multi-scene video synthesis approaches fail to meet these consistency requirements. To address this we propose a contrastive sequential video diffusion method that selects the most suitable previously generated scene to guide and condition the denoising process of the next scene. The result is a multi-scene video that is grounded in the scene descriptions and coherent w.r.t. the scenes that require visual consistency. Experiments with action-centered data from the real world demonstrate the practicality and improved consistency of our model compared to previous work. Code and examples available at https://github.com/novasearch/CoSeD
HeightLane: BEV Heightmap Guided 3D Lane Detection
Authors: Chaesong Park; Eunbin Seo; Jongwoo Lim
Venue: WACV 2025 (main)

Abstract: Accurate 3D lane detection from monocular images presents significant challenges due to depth ambiguity and imperfect ground modeling. Previous attempts to model the ground have often used a planar ground assumption with limited degrees of freedom making them unsuitable for complex road environments with varying slopes. Our study introduces HeightLane an innovative method that predicts a height map from monocular images by creating anchors based on a multi-slope assumption. This approach provides a detailed and accurate representation of the ground. HeightLane employs the predicted heightmap along with a deformable attention-based spatial feature transform framework to efficiently convert 2D image features into 3D bird's eye view (BEV) features enhancing spatial understanding and lane structure recognition. Additionally the heightmap is used for the positional encoding of BEV features further improving their spatial accuracy. This explicit view transformation bridges the gap between front-view perceptions and spatially accurate BEV representations significantly improving detection performance. To address the lack of the necessary ground truth height map in the original OpenLane dataset we leverage the Waymo dataset and accumulate its LiDAR data to generate a height map for the drivable area of each scene. The GT heightmaps are used to train the heightmap extraction module from monocular images. Extensive experiments on the OpenLane validation set show that HeightLane achieves state-of-the-art performance in terms of F-score highlighting its potential in real-world applications.
DASC-SPT: Towards Self-Supervised Panoramic Semantic Segmentation
Authors: Tianlong Tan; Bin Chen; Hongliang Cao; Chenggang Yan; Yike Ma; Feng Dai
Venue: WACV 2025 (main)

Abstract: Self-Supervised Semantic Segmentation aiming to leverage masses of unlabeled data for boosting semantic segmentation has been rapidly emerging as an active task in recent years. However existing self-supervised semantic segmentation approaches mainly focus on planar images leaving multiple distorted objects encountered in panoramic images unexplored due to the formidable challenge of handling heterogeneous degrees of distortions across different locations. In this paper we propose a novel Self-Supervised Panoramic Semantic Segmentation model termed DASC-SPT built upon the mainstream contrastive learning framework. Towards distortions in panoramic images we present two structures to better learn from distorted features by applying planar images. For the input images of self-supervision we design a Spherical Projection Transformation (SPT) strategy that involves randomly projecting planar images onto various locations of the sphere to introduce the distortions. For pixel-wise distorted features we construct a Deformation-aware Sampling Consistency (DASC) framework to further utilize the shared content and discrepancies caused by different distortions of paired views where the deformation-aware consistency can be quantified on pixel-wise features. Both of the two components facilitate the model to adapt to distortions and boost panoramic semantic segmentation. Extensive comprehensive experiments on three panoramic datasets demonstrate the effectiveness and superiority of DASC-SPT approach.
Point Cloud Color Upsampling with Attention-Based Coarse Colorization and Refinement
Authors: Kohei Matsuzaki; Keisuke Nonaka
Venue: WACV 2025 (main)

Abstract: Point cloud color upsampling is an important and less explored research topic. State-of-the-art methods colorize points based on the colors of neighboring points and geometric distances. However these methods often suffer from blurring and noise at color boundaries since object textures can have large color variations even between geometrically neighboring positions. In this paper we propose a point cloud color upsampling method with attention weights for neighboring points. The proposed method first performs coarse colorization with the colors of low-resolution points neighboring the high-resolution points and predicted weights. Then it refines the colors by predicting offsets for high-resolution points with aggregate features obtained from the low-resolution points. Both quantitative and qualitative experimental results on datasets acquired in real-world environments demonstrate that the proposed method achieves significantly superior color upsampling performance compared to state-of-the-art methods.
U-MixFormer: UNet-Like Transformer with Mix-Attention for Efficient Semantic Segmentation
Authors: Seul-Ki Yeom; Julian von Klitzing
Venue: WACV 2025 (main)

Abstract: Semantic segmentation has witnessed remarkable advancements with the adaptation of the Transformer architecture. Parallel to the strides made by the Transformer CNN-based U-Net has seen significant progress especially in high-resolution medical imaging and remote sensing. This dual success inspired us to merge both strengths leading to the inception of a U-Net-based vision transformer decoder tailored for efficient contextual encoding. Here we propose a novel transformer decoder U-MixFormer built upon the U-Net structure designed for efficient semantic segmentation. Our approach distinguishes itself from the previous transformer methods by leveraging lateral connections between the encoder and decoder stages as feature queries for the attention modules apart from the traditional reliance on skip connections. Moreover we innovatively mix hierarchical feature maps from various encoder and decoder stages to form a unified representation for keys and values giving rise to our unique mix-attention module. Our approach demonstrates state-of-the-art performance across various configurations. Extensive experiments show that U-MixFormer outperforms SegFormer FeedFormer and SegNeXt by a large margin. For example U-MixFormer-B0 surpasses SegFormer-B0 and FeedFormer-B0 with 3.8% and 2.0% higher mIoU and 27.3% and 21.8% less computation and outperforms SegNext with 3.3% higher mIoU with MSCAN-T encoder on ADE20K. Code available at https://github.com/julian-klitzing/u-mixformer.
Efficient Progressive Image Compression with Variance-Aware Masking
Authors: Alberto Presta; Enzo Tartaglione; Attilio Fiandrotti; Marco Grangetto; Pamela Cosman
Venue: WACV 2025 (main)

Abstract: Learned progressive image compression is gaining momentum as it allows improved image reconstruction as more bits are decoded at the receiver. We propose a progressive image compression method in which an image is first represented as a pair of base-quality and top-quality latent representations. Next a residual latent representation is encoded as the element-wise difference between the top and base representations. Our scheme enables progressive image compression with element-wise granularity by introducing a masking system that ranks each element of the residual latent representation from most to least important dividing it into complementary components which can be transmitted separately to the decoder in order to obtain different reconstruction quality. The masking system does not add further parameters nor complexity. At the receiver any elements of the top latent representation excluded from the transmitted components can be independently replaced with the mean predicted by the hyperprior architecture ensuring reliable reconstructions at any intermediate quality level. We also introduced Rate Enhancement Modules (REMs) which refine the estimation of entropy parameters using already decoded components. We obtain results competitive with state-of-the-art competitors while significantly reducing computational complexity decoding time and number of parameters.
SegBuilder: A Semi-Automatic Annotation Tool for Segmentation
Authors: Md Alimoor Reza; Eric Manley; Sean Chen; Sameer Chaudhary; Jacob Elafros
Venue: WACV 2025 (main)

Abstract: This paper addresses the problem of image annotation for segmentation tasks. Semantic segmentation involves labeling each pixel in an image with predefined categories such as sky cars roads and humans. Deep learning models require numerous annotated images for effective training but manual annotation is slow and time-consuming. To mitigate this challenge we leverage the Segment Anything Model (SAM)- a vision foundation model. We introduce SegBuilder a framework that incorporates SAM to automatically generate segments which are then tagged by human annotators using a quick selection list. To demonstrate SegBuilder's effectiveness we introduced a novel dataset for image segmentation in underwater environments featuring animals such as sea lions beavers and jellyfish. Experiments on this dataset showed that SegBuilder significantly speeds up the annotation process compared to the publicly available tool Label Studio. SegBuilder also includes a free-form drawing tool allowing users to create correct segments missed by SAM. This feature is particularly useful for scenes with shadows camouflaged objects and part-based segmentation tasks where SAM falls short. Experimentally we demonstrated SegBuilder's efficacy in these scenarios showcasing its potential for generating pixel-wise annotations crucial for training robust deep learning models for semantic segmentation.
Cross-View Meets Diffusion: Aerial Image Synthesis with Geometry and Text Guidance
Authors: Ahmad Arrabi; Xiaohan Zhang; Waqas Sultani; Chen Chen; Safwan Wshah
Venue: WACV 2025 (main)

Abstract: Aerial imagery analysis is critical for many research fields. However obtaining frequent high-quality aerial images is not always accessible due to its high effort and cost requirements. One solution is to use the Ground-to-Aerial (G2A) technique to synthesize aerial images from easily collectible ground images. However G2A is rarely studied because of its challenges including but not limited to the drastic view changes occlusion and range of visibility. In this paper we present a novel Geometric Preserving Ground-to-Aerial (G2A) image synthesis (GPG2A) model that can generate realistic aerial images from ground images. GPG2A consists of two stages. The first stage predicts the Bird's Eye View (BEV) segmentation (referred to as the BEV layout map) from the ground image. The second stage synthesizes the aerial image from the predicted BEV layout map and text descriptions of the ground image. To train our model we present a new multi-modal cross-view dataset namely VIGORv2 built upon VIGOR with newly collected aerial images maps and text descriptions. Our extensive experiments illustrate that GPG2A synthesizes better geometry-preserved aerial images than existing models. We also present two applications data augmentation for cross-view geo-localization and sketch-based region search to further verify the effectiveness of our GPG2A. The code and dataset are available at https://github.com/AhmadArrabi/GPG2A
Multi-View Factorizing and Disentangling: A Novel Framework for Incomplete Multi-View Multi-Label Classification
Authors: Wulin Xie; Lian Zhao; Jiang Long; Xiaohuan Lu; Bingyan Nie
Venue: WACV 2025 (main)

Abstract: Multi-view multi-label classification (MvMLC) has recently garnered significant research attention due to its wide range of real-world applications. However incompleteness in views and labels is a common challenge often resulting from data collection oversights and uncertainties in manual annotation. Furthermore the task of learning robust multi-view representations that are both view-consistent and view-specific from diverse views still a challenge problem in MvMLC. To address these issues we propose a novel framework for incomplete multi-view multi-label classification (iMvMLC). Our method factorizes multi-view representations into two independent sets of factors: view-consistent and view-specific and we correspondingly design a graph disentangling loss to fully reduce redundancy between these representations. Additionally our framework innovatively decomposes consistent representation learning into three key sub-objectives: (i) how to extract view-shared information across different views (ii) how to eliminate intra-view redundancy in consistent representations and (iii) how to preserve task-relevant information. To this end we design a robust task-relevant consistency learning module that collaboratively learns high-quality consistent representations leveraging a masked cross-view prediction (MCP) strategy and information theory. Notably all modules in our framework are developed to function effectively under conditions of incomplete views and labels making our method adaptable to various multi-view and multi-label datasets. Extensive experiments on five datasets demonstrate that our method outperforms other leading approaches.
UCDR-Adapter: Exploring Adaptation of Pre-Trained Vision-Language Models for Universal Cross-Domain Retrieval
Authors: Haoyu Jiang; Zhi-Qi Cheng; Gabriel Moreira; Jiawen Zhu; Jingdong Sun; Bukun Ren; Jun-Yan He; Qi Dai; Xian-Sheng Hua
Venue: WACV 2025 (main)

Abstract: Universal Cross-Domain Retrieval (UCDR) retrieves relevant images from unseen domains and classes without semantic labels ensuring robust generalization. Existing methods commonly employ prompt tuning with pre-trained vision-language models but are inherently limited by static prompts reducing adaptability. We propose UCDR-Adapter which enhances pre-trained models with adapters and dynamic prompt generation through a two-phase training strategy. First Source Adapter Learning integrates class semantics with domain-specific visual knowledge using a Learnable Textual Semantic Template and optimizes Class and Domain Prompts via momentum updates and dual loss functions for robust alignment. Second Target Prompt Generation creates dynamic prompts by attending to masked source prompts enabling seamless adaptation to unseen domains and classes. Unlike prior approaches UCDR-Adapter dynamically adapts to evolving data distributions enhancing both flexibility and generalization. During inference only the image branch and generated prompts are used eliminating reliance on textual inputs for highly efficient retrieval. Extensive benchmark experiments show that UCDR-Adapter consistently outperforms ProS in most cases and other state-of-the-art methods on UCDR U^cCDR and U^dCDR settings.
User-in-the-Loop Evaluation of Multimodal LLMs for Activity Assistance
Authors: Mrinal Verghese; Brian Chen; Hamid Eghbalzadeh; Tushar Nagarajan; Ruta P Desai
Venue: WACV 2025 (main)

Abstract: Our research investigates the capability of modern multimodal reasoning models powered by Large Language Models (LLMs) to facilitate vision-powered assistants for multi-step daily activities. Such assistants must be able to 1) encode relevant visual history from the assistant's sensors e.g. camera 2) forecast future actions for accomplishing the activity and 3) replan based on the user in the loop. To evaluate the first two capabilities grounding visual history and forecasting in short and long horizons we conduct benchmarking of two prominent classes of multimodal LLM approaches - Socratic Models and Vision Conditioned Language Models (VCLMs) on video-based action anticipation tasks using offline datasets. These offline benchmarks however do not allow us to close the loop with the user which is essential to evaluate the replanning capabilities and measure successful activity completion in assistive scenarios. To that end we conduct a first-of-its-kind user study with 18 participants performing 3 different multi-step cooking activities while wearing an egocentric observation device called Aria and following assistance from multimodal LLMs. We find that the Socratic approach outperforms VCLMs in both offline and online settings. We further highlight how grounding long visual history common in activity assistance remains challenging in current models especially for VCLMs and demonstrate that offline metrics do not indicate online performance.
CycleCrash: A Dataset of Bicycle Collision Videos for Collision Prediction and Analysis
Authors: Nishq Poorav Desai; Ali Etemad; Michael Greenspan
Venue: WACV 2025 (main)

Abstract: Self-driving research often underrepresents cyclist collisions and safety. To address this we present CycleCrash a novel dataset consisting of 3000 dashcam videos with 436347 frames that capture cyclists in a range of critical situations from collisions to safe interactions. This dataset enables 9 different cyclist collision prediction and classification tasks focusing on potentially hazardous conditions for cyclists and is annotated with collision-related cyclistrelated and scene-related labels. Next we propose Vid- NeXt a novel method that leverages a ConvNeXt spatial encoder and a non-stationary transformer to capture the temporal dynamics of videos for the tasks defined in our dataset. To demonstrate the effectiveness of our method and create additional baselines on CycleCrash we apply and compare 7 models along with a detailed ablation. We release the dataset and code at https://github.com/ DeSinister/CycleCrash/.
3D Edge Sketch from Multiview Images
Authors: Yilin Zheng; Chiang-Heng Chien; Ricardo Fabbri; Benjamin Kimia
Venue: WACV 2025 (main)

Abstract: The semantic reconstruction of a scene relies in part on the curvilinear structure inherent in images. The recovery of curvilinear structure is not only key to the representation of objects via ridges and other object curves but is also critical to the reconstruction from texture-poor images which lack a sufficient number of features. Prior methods advocate for the recovery of curve segments from images and reconstructing these into an organized collection of 3D curve segments often referred to as the 3D curve sketch which serves as the basis for further reconstruction of curves and surfaces. Observing that the process of edge grouping can lead to fictitious curves or missing veridical groupings this paper advocates for a reconstruction of curvilinear structure directly from image edges in the form of a 3D edge sketch. The multiview reconstruction of edges faces significant combinatorial challenges which are effectively addressed in this paper. We demonstrate through experiments that the 3D edge sketch recovers a vast majority of the curvilinear structure and is a reliable substrate from which 3D curves can be constructed.
DTA: Dual Temporal-Channel-Wise Attention for Spiking Neural Networks
Authors: Minje Kim; Minjun Kim; Xu Yang
Venue: WACV 2025 (main)

Abstract: Spiking Neural Networks (SNNs) present a more energy-efficient alternative to Artificial Neural Networks (ANNs) by harnessing spatio-temporal dynamics and event-driven spikes. Effective utilization of temporal information is crucial for SNNs leading to the exploration of attention mechanisms to enhance this capability. Conventional attention operations either apply identical operation or employ non-identical operations across target dimensions. We identify that these approaches provide distinct perspectives on temporal information. To leverage the strengths of both operations we propose a novel Dual Temporal-channel-wise Attention (DTA) mechanism that integrates both identical/non-identical attention strategies. To the best of our knowledge this is the first attempt to concentrate on both the correlation and dependency of temporal-channel using both identical and non-identical attention operations. Experimental results demonstrate that the DTA mechanism achieves state-of-the-art performance on both static datasets (CIFAR10 CIFAR100 ImageNet-1k) and dynamic dataset (CIFAR10-DVS) elevating spike representation and capturing complex temporal-channel relationship. We open-source our code: https://github.com/MnJnKIM/DTA-SNN.