GenAI4ELab/papercli-papers huggingface dataset:
SpecGen: Neural Spectral BRDF Generation via Spectral-Spatial Tri-plane Aggregation
Authors: Zhenyu Jin; Wenjie Li; Zhanyu Ma; Heng Guo
Venue: WACV 2026 (main)

Abstract: Synthesizing spectral images across different wavelengths is essential for photorealistic rendering. Unlike conventional spectral uplifting methods that convert RGB images into spectral ones, we introduce SpecGen, a novel method that generates spectral bidirectional reflectance distribution functions (BRDFs) from a single RGB image of a sphere. This enables spectral image rendering under arbitrary illuminations and shapes covered by the corresponding material. A key challenge in spectral BRDF generation is the scarcity of measured spectral BRDF data. To address this, we propose the Spectral-Spatial Tri-plane Aggregation (SSTA) network, which models reflectance responses across wavelengths and incident-outgoing directions, allowing the training strategy to leverage abundant RGB BRDF data to enhance spectral BRDF generation. Experiments show that our method accurately reconstructs spectral BRDFs from limited spectral data and surpasses state-of-the-art methods in hyperspectral image reconstruction, achieving an improvement of 8 dB in PSNR. Codes link: https://github.com/sosjzy/SpecGen
TimeRefine: Temporal Grounding with Time Refining Video LLM
Authors: Xizi Wang; Feng Cheng; Ziyang Wang; Huiyu Wang; Md Mohaiminul Islam; Lorenzo Torresani; Mohit Bansal; Gedas Bertasius; David Crandall
Venue: WACV 2026 (main)

Abstract: Video temporal grounding aims to localize relevant temporal boundaries in a video given a textual prompt. Recent work has focused on enabling Video LLMs to perform video temporal grounding via next-token prediction of temporal timestamps. However, accurately localizing timestamps in videos remains challenging for Video LLMs when relying solely on temporal token prediction. Our proposed TimeRefine addresses this challenge in two ways. First, instead of directly predicting the start and end timestamps, we reformulate the temporal grounding task as a temporal refining task: the model first makes rough predictions and then refines them by predicting offsets to the target segment. This refining process is repeated multiple times, through which the model progressively self-improves its temporal localization accuracy. Second, to enhance the model's temporal perception capabilities, we incorporate an auxiliary prediction head that penalizes the model more if a predicted segment deviates further from the ground truth, thus encouraging the model to make closer and more accurate predictions. Our plug-and-play method can be integrated into most LLM-based temporal grounding approaches. The experimental results demonstrate that TIMEREFINE achieves 3.6% and 5.0% mIoU improvements on the ActivityNet and Charades-STA datasets, respectively. Code and pretrained models are available at https://github.com/SJTUwxz/TimeRefine_code.
Towards High-Fidelity, Identity-Preserving Real-Time Makeup Transfer: Decoupling Style Generation
Authors: Lydia Chau; Zhi Yu; Ruowei Jiang
Venue: WACV 2026 (main)

Abstract: We propose a framework for real-time virtual makeup transfer that achieves high-fidelity, identity-preserving results with strong temporal consistency. Existing methods often struggle to disentangle semi-transparent makeup from skin tones and other identity features, leading to identity shifts and fairness concerns. Furthermore, they also lack real-time capabilities and fail to maintain temporal consistency, limiting adoption in practical virtual try-on applications. To address these challenges, we decouple makeup transfer into two stages: transparent makeup mask extraction and graphics-based real-time makeup rendering. Once extracted, makeup masks can be applied in real-time for live video try-on. We generate pseudo-ground-truth data via a hybrid graphics-based rendering pipeline and an unsupervised clustering method, enabling robust training without real paired before-and-after makeup data. To further enhance transparency estimation and color fidelity, we propose transparency-aware reconstruction and lip color objectives. Our method consistently transfers fine-grained makeup details across diverse skin tones and expressions while maintaining temporal stability. Experiments demonstrate superior accuracy, stability, and efficiency over state-of-the-art baselines, making our approach practical for live virtual try-on applications. Video demonstrations are available in supplementary material.
OMeGa: Joint Optimization of Explicit Meshes and Gaussian Splats for Robust Scene-Level Surface Reconstruction
Authors: Yuhang Cao; Haojun Yan; Danya Yao
Venue: WACV 2026 (main)

Abstract: Neural rendering with Gaussian splatting has advanced novel view synthesis, and most methods reconstruct surfaces via post-hoc mesh extraction. However, existing methods suffer from two limitations: (i) inaccurate geometry in texture-less indoor regions, and (ii) the decoupling of mesh extraction from optimization, thereby missing the opportunity to leverage mesh geometry to guide splat optimization. In this paper, we present OMeGa, an end-to-end framework that jointly optimizes an explicit triangle mesh and 2D Gaussian splats via a flexible binding strategy, where spatial attributes of Gaussian Splats are expressed in the mesh frame and texture attributes are retained on splats. To further improve reconstruction accuracy, we integrate mesh constraints and monocular normal supervision into the optimization, thereby regularizing geometry learning. In addition, we propose a heuristic, iterative mesh-refinement strategy that splits high-error faces and prunes unreliable ones to further improve the detail and accuracy of the reconstructed mesh. OMeGa achieves state-of-the-art performance on challenging indoor reconstruction benchmarks, reducing Chamfer-L_1 by 47.3% over the 2DGS baseline while maintaining competitive novel-view rendering quality. The experimental results demonstrate that OMeGa effectively addresses prior limitations in indoor texture-less reconstruction.
Feedback Alignment Meets Low-Rank Manifolds: A Structured Recipe for Local Learning
Authors: Arani Roy; Marco P. Apolinario; Shristi Das Biswas; Kaushik Roy
Venue: WACV 2026 (main)

Abstract: Training deep neural networks (DNNs) with backpropagation (BP) achieves state-of-the-art accuracy but requires global error propagation and full parameterization, leading to substantial memory and computational overhead. Direct Feedback Alignment (DFA) enables local, parallelizable updates with lower memory requirements but is limited by unstructured feedback and poor scalability in deeper architectures, specially convolutional neural networks. To address these limitations, we propose a structured local learning framework that operates directly on low-rank manifolds defined by the Singular Value Decomposition (SVD) of weight matrices. Each layer is trained in its decomposed form, with updates applied to the SVD components using a composite loss that integrates cross-entropy, subspace alignment, and orthogonality regularization. Feedback matrices are constructed to match the SVD structure, ensuring consistent alignment between forward and feedback pathways. Our method reduces the number of trainable parameters relative to the original DFA model, without relying on pruning or post hoc compression. Experiments on CIFAR-10, CIFAR-100, and ImageNet show that our method achieves accuracy comparable to that of BP. Ablation studies confirm the importance of each loss term in the low-rank setting. These results establish local learning on low-rank manifolds as a principled and scalable alternative to full-rank gradient-based training.
Edge-Aware Image Manipulation via Diffusion Models with a Novel Structure-Preservation Loss
Authors: Minsu Gong; Nuri Ryu; Jungseul Ok; Sunghyun Cho
Venue: WACV 2026 (main)

Abstract: Recent advances in image editing leverage latent diffusion models (LDMs) for versatile, text-prompt-driven edits across diverse tasks. Yet, maintaining pixel-level edge structures--crucial for tasks such as photorealistic style transfer or image tone adjustment--remains as a challenge for latent-diffusion-based editing. To overcome this limitation, we propose a novel Structure Preservation Loss (SPL) that leverages local linear models to quantify structural differences between input and edited images. Our training-free approach integrates SPL directly into the diffusion model's generative process to ensure structural fidelity. This core mechanism is complemented by a post-processing step to mitigate LDM decoding distortions, a masking strategy for precise edit localization, and a color preservation loss to preserve hues in unedited areas. Experiments confirm SPL enhances structural fidelity, delivering state-of-the-art performance in latent-diffusion-based image editing. Our code will be publicly released at https://github.com/gongms00/SPL.
Comp4D: Compositional 4D Scene Generation
Authors: Hanwen Liang; Dejia Xu; Neel P. Bhatt; Hezhen Hu; Hanxue Liang; Konstantinos N. Plataniotis
Venue: WACV 2026 (main)

Abstract: The advancements in diffusion models for 2D and 3D content creation have sparked a surge of interest in generating 4D content. However, the scarcity of 3D scene datasets constrains current methodologies to primarily object-centric generation. To overcome this limitation, we present Comp4D, a novel framework for text-to-compositional 4D scene generation. Unlike conventional methods that generate a singular 4D representation of the entire scene, Comp4D innovatively employs a decompose-then-recompose strategy, constructing each 4D component within the scene separately.The framework first decomposes a textual input prompt into multiple object components and delineates their moving trajectories. After initializing the static 3D objects, we construct the compositional 4D scene by accurately positioning these objects along their designated paths. To refine the scene and motion, our method proposes a novel compositional score distillation technique involving trajectory-guided and object-centric sampling, utilizing pre-trained diffusion models across text-to-image, text-to-video, and text-to-3D domains for optimization. Extensive experiments demonstrate our superior 4D content creation capability compared to prior arts, showcasing superior visual quality, motion fidelity, and enhanced object interactions.
One-Shot Fine-Grained Re-Identification of Paint Marked Honey Bees using Vision Foundation Models
Authors: Luke Meyers; Josué A. Rodríguez-Cordero; Rémi Mégret
Venue: WACV 2026 (main)

Abstract: Accurate re-identification (ReID) of individual insects is crucial for quantitative studies of pollinator behavior, with key applications in biological research and ecological monitoring.This work leverages vision foundation models to enable the fine-grained ReID of honey bees from video with a single paint marking and a single reference track. Such marking avoids the disruption required for gluing tags or painting color codes. We present a new challenging dataset of 9495 images and 45 identities, obtained at outdoor bee feeders with significant pose and illumination changes. Our one-shot capable approach first pre-processes video footage to extract pose-normalized bee crops and remove the background using a segmentation foundation model (e.g., SAM2). It then uses a self-supervised visual foundation model (e.g., DINOv3) for image and patch embeddings, coupled with contrastive metric learning and track information to generate robust ID embeddings for ReID. Compared to existing methods, our approach significantly reduces training data requirements.Our extensive studies show how the choices of the different steps of the pipeline impact performance, offering practical insights for future animal re-identification.
PaRaChute: Pathology-Radiology Cross-Modal Fusion for Missing-Modality-Robust Survival Prediction
Authors: Pietro Caforio; Isabella Poles; Marco D. Santambrogio
Venue: WACV 2026 (main)

Abstract: Survival prediction from medical imaging is a critical challenge in computational oncology, with high clinical relevance for patient stratification and treatment planning. However, current Deep Learning methods suffer from three core limitations: they assume complete modality availability, overlook local-to-global cross-modal interactions, and disregard modality-specific signal reliability during optimization. To address these issues, we introduce PaRaChute, a novel Deep Learning framework for robust multimodal survival prediction from heterogeneous and partially missing imaging data. PaRaChute integrates modality-specific pretrained encoders with adapter networks that align radiology and histopathology features into a shared latent space. A Dynamic Contextual Embedding mechanism captures biologically grounded local correlations between pathology and radiology and channels them through a multi-head cross-attention fusion module to guide global survival prediction, while adaptively handling missing modality scenarios. Furthermore, a Gradient Curvature Steering module improves convergence in incomplete data regimes by regularizing gradients via local curvature alignment. Experiments on three CPTAC and TCGA derived cancer cohorts show that PaRaChute achieves a C-index of 0.8367 with full modality input, and it retains strong performance under missing modality conditions (0.7488) while producing clinically meaningful risk stratifications, as confirmed by Kaplan-Meier analysis.
Zero-Shot Table Extraction in Business Documents: A Unified Benchmark with Error Taxonomy and Ecological Analysis
Authors: Eliott Thomas; Mickael Coustaty; Aurélie Joseph; Tri-Cong Pham; Gaspar Deloin; Elodie Carel; Vincent Poulain D'andecy; Jean-Marc Ogier
Venue: WACV 2026 (main)

Abstract: Tables in business documents power analytics and compliance, yet task-specific datasets are costly to build. Practitioners therefore turn to zero-shot vision-language models (VLMs). We study zero-shot realism for table detection (TD) and table structure recognition (TSR) under a unified protocol on DocILE-QUEST and a private STM154 corpus. We report TD with GIoU, Purity, and Completeness, and TSR with TEDS and TEDS-S, evaluating commercial VLMs (GPT-4o, GPT-5-mini), compact detectors, and supervised YOLO/DETR baselines. Zero-shot VLMs are strong for TSR and competitive for TD, while fine-tuned or from-scratch detectors lead when box quality and robustness to clutter matter. We add an automated error taxonomy that isolates actionable failures (missed, merged/split tables, header-body confusions, cell topology). Finally, we quantify emissions, finding a 10^4 gap between the lightest and heaviest systems.
A Unified Diffusion-Based Framework for Multi-Agent Trajectory Prediction Integrating Structured Multi-Modal Representations
Authors: Chenxi Yang; Suyang Xi; Hong Ding; Yiqing Shen; Yunhao Liu
Venue: WACV 2026 (main)

Abstract: Autonomous multi-agent trajectory prediction in open-world scenarios presents persistent challenges, including high behavioral uncertainty, long-horizon dependencies, and the lack of structured guidance during generation. Existing generative approaches often compromise behavioral fidelity in favor of accuracy or diversity, resulting in predictions that are either unrealistic or difficult to control. We propose M^2Traj, a unified framework that couples a closed-loop conditional diffusion model with structured trajectory reasoning and behavior-driven constraints. M^2Traj features a history-guided encoder that captures long-range cross-agent dependencies and scene semantics, and a dynamic closed-loop rollout mechanism that refines predictions through goal-conditioned denoising with iterative feedback. To enable fine-grained control, we introduce a learnable behavior guidance module that softly enforces constraints on velocity, collision risk, comfort, and traffic rule adherence. By jointly modeling agent interactions, future constraints, and uncertainty within a structured generative process, M^2Traj delivers controllable and reliable predictions across diverse urban scenarios. Extensive experiments on three large-scale benchmarks--Waymo, HighD, and MoCAD--demonstrate that M^2Traj achieves balanced and reliable performance across standard accuracy, diversity, and behavior-sensitive metrics, highlighting its potential as a generalizable solution for controllable, structure-aware trajectory prediction in complex multi-agent environments.
From Street to Orbit: Training-Free Cross-View Retrieval via Location Semantics and LLM Guidance
Authors: Jeongho Min; Dongyoung Kim; Jaehyup Lee
Venue: WACV 2026 (main)

Abstract: Cross-view image retrieval, particularly street-to-satellite matching, is a critical task for applications such as autonomous navigation, urban planning, and localization in GPS-denied environments. However, existing approaches often require supervised training on curated datasets and rely on panoramic or UAV-based images, which limits real-world deployment. In this paper, we present a simple yet effective cross-view image retrieval framework that leverages a pretrained vision encoder and a large language model (LLM), requiring no additional training. Given a monocular street-view image, our method extracts geographic cues through web-based image search and LLM-based location inference, generates a satellite query via geocoding API, and retrieves matching tiles using a pretrained vision encoder (e.g., DINOv2) with PCA-based whitening feature refinement. Despite using no ground-truth supervision or finetuning, our proposed method outperforms prior learning-based approaches on the benchmark dataset under zero-shot settings. Moreover, our pipeline enables automatic construction of semantically aligned street-to-satellite datasets, which is offering a scalable and cost-efficient alternative to manual annotation. All source codes will be made publicly available at https://street2orbit.github.io.
Training-free Detection of Text-to-video Generations via Over-coherence
Authors: Jonathan Brokman; Oren Rachmil; Omer Hofman; Roy Betser; Amit Giloni; Roman Vainshtein; Hisashi Kojima
Venue: WACV 2026 (main)

Abstract: Text-to-video generative models have emerged as powerful tools in content creation, capable of synthesizing highly realistic videos from textual prompts. However, the rapid advancement of these models introduces significant security and trust concerns, as traditional detection methods struggle to generalize to unseen generative techniques. Existing approaches commonly rely on supervised learning, requiring continuous dataset curation and retraining, which is impractical given the fast-paced evolution of generative models. In this work, we introduce the first training-free detection method for AI-generated videos, eliminating the need for labeled training data or prior exposure to generation techniques. Our approach exploits a fundamental weakness in text-to-video models: Unnatural temporal over-coherence in frame transitions. By leveraging a novel time-coherence detection criterion, our method identifies distinct artifacts in video embeddings, which are absent in real videos. We extensively evaluate our approach, demonstrating that it significantly outperforms existing baselines - maintaining robustness to unseen generative models. This work establishes a new direction for training-free detection of text-to-video generated content, providing a scalable and time resilient solution.
Countering Multi-modal Representation Collapse through Rank-targeted Fusion
Authors: Seulgi Kim; Kiran Kokilepersaud; Mohit Prabhushankar; Ghassan AlRegib
Venue: WACV 2026 (main)

Abstract: Multi-modal fusion methods often suffer from two types of representation collapse: feature collapse where individual dimensions lose their discriminative power (as measured by eigenspectra), and modality collapse where one dominant modality overwhelms the other. Applications like human action anticipation that require fusing multifarious sensor data are hindered by both feature and modality collapse. However, existing methods attempt to counter feature collapse and modality collapse separately. This is because there is no unifying framework that efficiently addresses feature and modality collapse in conjunction. In this paper, we posit the utility of effective rank as an informative measure that can be utilized to quantify and counter both the representation collapses. We propose Rank-enhancing Token Fuser, a theoretically grounded fusion framework that selectively blends less informative features from one modality with complementary features from another modality. We show that our method increases the effective rank of the fused representation. To address modality collapse, we evaluate modality combinations that mutually increase each others' effective rank. We show that depth maintains representational balance when fused with RGB, avoiding modality collapse. We validate our method on action anticipation, where we present \texttt R3D , a depth-informed fusion framework. Extensive experiments on NTURGBD, UTKinect, and DARai demonstrate that our approach significantly outperforms prior state-of-the-art methods by up to 3.74%. Our code is available at: \href https://github.com/olivesgatech/R3D  https://github.com/olivesgatech/R3D .
MSRTrack: LLM-Powered Object Tracking with Motion and Semantic Reasoning
Authors: Tong Shen; Di Wang; José M. F. Moura
Venue: WACV 2026 (main)

Abstract: State-of-the-art object trackers primarily model appearance relations between the image template and the search region with Siamese networks. However, this well-established approach has a limited ability to leverage both motion and semantic cues of the target object, leading to increasing errors in challenging scenarios like drastic appearance changes and similar-looking distractors. To address the above weaknesses, we propose a novel tracking framework with Motion and Semantic Reasoning (MSRTrack), integrating short-term motion modeling and distinctive semantic features for robust tracking across diverse conditions. Powered by vision large language models (VLLMs) and the Segment Anything Model 2 (SAM2), MSRTrack identifies unique semantic attributes of the target, exploits motion cues across consecutive frames, and complements appearance-based trackers with strong semantic and dynamic reasoning capabilities. Unlike previous vision language tracking (VLT) methods that rely on broad captioning, MSRTrack automatically focuses on a concise set of key semantic attributes of the target, substantially improving target lost recovery and distractor rejection. MSRTrack achieves state-of-the-art performance across multiple tracking benchmarks, with 2.2% improvement on the LaSOT dataset, 9.5% improvement on the VastTrack dataset, and 1.4% on the TNL2K dataset.