GenAI4ELab/papercli-papers huggingface dataset:
Seeing is Believing (and Predicting): Context-Aware Multi-Human Behavior Prediction with Vision Language Models
Authors: Utsav Panchal; Yuchen Liu; Luigi Palmieri; Ilche Georgievski; Marco Aiello
Venue: WACV 2026 (main)

Abstract: Accurately predicting human behaviors is crucial for mobile robots operating in human-populated environments. While prior research primarily focuses on predicting actions in single-human scenarios from an egocentric view, several robotic applications require understanding multiple human behaviors from a third-person perspective. To this end, we present CAMP-VLM (Context-Aware Multi-human behavior Prediction): a Vision Language Model (VLM)-based framework that incorporates contextual features from visual input and spatial awareness from scene graphs to enhance prediction of humans-scene interactions. Due to the lack of suitable datasets for multi-human behavior prediction from an observer view, we perform fine-tuning of CAMP-VLM with synthetic human behavior data generated by a photorealistic simulator, and evaluate the resulting models on both synthetic and real-world sequences to assess their generalization capabilities. Leveraging Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO), CAMP-VLM outperforms the best-performing baseline by up to 66.9% in prediction accuracy.
MAESTRO: Masked AutoEncoders for Multimodal, Multitemporal, and Multispectral Earth Observation Data
Authors: Antoine Labatie; Michael Vaccaro; Nina Lardiere; Anatol Garioud; Nicolas Gonthier
Venue: WACV 2026 (main)

Abstract: Self-supervised learning holds great promise for remote sensing, but standard self-supervised methods must be adapted to the unique characteristics of Earth observation data. We take a step in this direction by conducting a comprehensive benchmark of fusion strategies and normalization schemes of reconstruction targets for multimodal, multitemporal, and multispectral Earth observation data. Based on our findings, we introduce MAESTRO, a novel adaptation of the Masked Autoencoder with optimized fusion mechanisms and a normalization scheme that incorporates a spectral prior as a self-supervisory signal. Evaluated on four Earth observation datasets in both intra- and cross-dataset settings, MAESTRO achieves state-of-the-art performance on tasks that strongly rely on multitemporal dynamics, while also remaining competitive on others.
ASC: Learning Augmentation Severity-Consistent Representations Improves Generalization via Augmentation Search
Authors: Amirhossein Alamdar; Hossein Jafarinia; Mahdi Noori; Mohammad Hossein Rohban
Venue: WACV 2026 (main)

Abstract: Whole Slide Image (WSI) classification is hindered by limited data availability, resulting in weak generalization. Recent efforts leverage data augmentation to address this, but methods adapted from natural images often fail on WSIs--either degrading performance or offering marginal gains. A central challenge lies in tuning augmentation parameters to match WSI-specific characteristics, a task rendered impractical by the computational demands of current WSI pipelines, where feature extraction is frozen and prohibitively expensive. This work introduces two key contributions. First, it proposes DINOASC, an enhanced self-supervised learning framework that modifies DINO to produce embeddings with AugSev Consistency--a property ensuring that linear interpolations across augmentation severities yield semantically coherent representations. Second, it presents the first automatic augmentation search strategy for WSI classification, built on top of TrivialAugment, which efficiently discovers augmentation strength ranges suited to histopathology by exploiting the structured embedding space induced by DINOASC. Together, these components enable augmentation-based generalization improvements without incurring excessive computational overhead. The proposed method achieves state-of-the-art performance on CAMELYON16 and SICAP-MIL.
View-aware Cross-modal Distillation for Multi-view Action Recognition
Authors: Trung Thanh Nguyen; Yasutomo Kawanishi; Vijay John; Takahiro Komamizu; Ichiro Ide
Venue: WACV 2026 (main)

Abstract: The widespread use of multi-sensor systems has increased research in multi-view action recognition. While existing approaches in multi-view setups with fully overlapping sensors benefit from consistent view coverage, partially overlapping settings where actions are visible in only a subset of views remain underexplored. This challenge becomes more severe in real-world scenarios, as many systems provide only limited input modalities and rely on sequence-level annotations instead of dense frame-level labels. In this study, we propose View-aware Cross-modal Knowledge Distillation (ViCoKD), a framework that distills knowledge from a fully supervised multi-modal teacher to a modality- and annotation-limited student. ViCoKD employs a cross-modal adapter with cross-modal attention, allowing the student to exploit multi-modal correlations while operating with incomplete modalities. Moreover, we propose a View-aware Consistency module to address view misalignment, where the same action may appear differently or only partially across viewpoints. It enforces prediction alignment when the action is co-visible across views, guided by human-detection masks and confidence-weighted Jensen-Shannon divergence between their predicted class distributions. Experiments on the real-world MultiSensor-Home dataset show that ViCoKD consistently outperforms competitive distillation methods across multiple backbones and environments, delivering significant gains and surpassing the teacher model under limited conditions.
Autoregressive Styled Text Image Generation, but Make it Reliable
Authors: Carmine Zaccagnino; Fabio Quattrini; Vittorio Pippi; Silvia Cascianelli; Alessio Tonioni; Rita Cucchiara
Venue: WACV 2026 (main)

Abstract: Generating faithful and readable styled text images (especially for Styled Handwritten Text generation - HTG) is an open problem with several possible applications across graphic design, document understanding, and image editing.A lot of research effort in this task is dedicated to developing strategies that reproduce the stylistic characteristics of a given writer, with promising results in terms of style fidelity and generalization achieved by the recently proposed Autoregressive Transformer paradigm for HTG. However, this method requires additional inputs, lacks a proper stop mechanism, and might end up in repetition loops, generating visual artifacts.In this work, we rethink the autoregressive formulation by framing HTG as a multimodal prompt-conditioned generation task, and tackle the content controllability issues by introducing special textual input tokens for better alignment with the visual ones. Moreover, we devise a Classifier-Free-Guidance-based strategy for our autoregressive model. Through extensive experimental validation, we demonstrate that our approach, dubbed Eruku, compared to previous solutions requires fewer inputs, generalizes better to unseen styles, and follows more faithfully the textual prompt, improving content adherence.
FedEFC: Federated Learning Using Enhanced Forward Correction Against Noisy Labels
Authors: Seunghun Yu; Jin-Hyun Ahn; Joonhyuk Kang
Venue: WACV 2026 (main)

Abstract: Federated Learning (FL) is a powerful framework for privacy-preserving distributed learning. It enables multiple clients to collaboratively train a global model without sharing raw data. However, handling noisy labels in FL remains a major challenge due to heterogeneous data distributions and communication constraints, which can severely degrade model performance. To address this issue, we propose FedEFC, a novel method designed to tackle the impact of noisy labels in FL. FedEFC mitigates this issue through two key techniques: (1) prestopping, which prevents overfitting to mislabeled data by dynamically halting training at an optimal point, and (2) loss correction, which adjusts model updates to account for label noise. In particular, we develop an effective loss correction tailored to the unique challenges of FL, including data heterogeneity and decentralized training. Furthermore, we provide a theoretical analysis, leveraging the composite proper loss property, to demonstrate that the FL objective function under noisy label distributions can be aligned with the clean label distribution. Extensive experimental results validate the effectiveness of our approach, showing that it consistently outperforms existing FL techniques in mitigating the impact of noisy labels, particularly under heterogeneous data settings (e.g., achieving up to 41.64% relative performance improvement over the existing loss correction method).
More Than Memory Savings: Zeroth-Order Optimization Mitigates Forgetting in Continual Learning
Authors: Wanhao Yu; Zheng Wang; Shuteng Niu; Sen Lin; Li Yang
Venue: WACV 2026 (main)

Abstract: Zeroth-order (ZO) optimization has gained attention as a memory-efficient alternative to first-order (FO) methods, particularly in settings where gradient computation is expensive or even impractical. Beyond its memory efficiency, in this work, we investigate ZO optimization for continual learning (CL) as a novel approach to address the plasticity-stability-efficiency trilemma.Through theoretical analysis and empirical evidence, we show that ZO optimization naturally leads to flatter loss landscapes, which in turn helps reduce forgetting in continual learning. However, this stability comes at a cost of plasticity: ZO optimization converges more slowly than first-order methods, which limits its ability to acquire new task-specific knowledge, particularly under the constrained training budgets.To better understand this trade-off, we conduct a holistic evaluation of ZO optimization applied to various existing CL methods. Our findings reveal that ZO optimization enhances stability but often undermines plasticity, particularly when used to learnable classifiers.Motivated by this insight, we propose ZO-FC, a simple but effective approach that applies ZO optimization to a single adapter-based PEFT module with FO optimized classifier. This design leverages the stability benefits of ZO while preserving the adaptability of FO updates with negligible memory overhead. Experiments demonstrate that ZO-FC achieves an effective balance between stability and plasticity, offering a practical and memory-efficient solution for
Learning to Animate Images from A Few Videos to Portray Delicate Human Actions
Authors: Haoxin Li; Yingchen Yu; Qilong Wu; Hanwang Zhang; Song Bai; Boyang Li
Venue: WACV 2026 (main)

Abstract: Despite recent progress, video generative models still struggle to generate delicate human actions (e.g., gymnastics), particularly when they are required to start from a user-provided reference image. In this paper, we explore the task of learning to animate images into videos that portray delicate human actions using a small number of videos --- 16 or fewer --- which reduces the need for extensive data collection and enhances practicality for real-world applications. Learning generalizable motion patterns that smoothly transition from user-provided reference images in such a fewshot setting is highly challenging. We propose FLASH (Few-shot Learning to Animate and Steer Humans), which enhances generalization of motion by training the model to reconstruct a video using the motion features and cross-frame correspondences extracted from another video with the same motion but different appearance. This encourages the learning of transferable motion and mitigates overfitting to the appearance in limited training data. Additionally, FLASH extends the decoder with additional layers to propagate details from the reference image to generated frames, improving transition smoothness. Human judges significantly favor FLASH, with 65.78% of 488 responses prefer FLASH over baselines.
UniCalib: Targetless LiDAR-camera Calibration via Probabilistic Flow on Unified Depth Representations
Authors: Shu Han; Xubo Zhu; Ji Wu; Ximeng Cai; Wen Yang; Huai Yu; Gui-Song Xia
Venue: WACV 2026 (main)

Abstract: Online targetless extrinsic LiDAR-camera calibration is essential for robust perception in computer vision applications such as autonomous driving. However, existing methods struggle with the significant modality gap between heterogeneous sensors and fail to handle unreliable correspondences arising from real-world challenges like occlusions and dynamic objects. To address these issues, we introduce UniCalib, a novel method that performs calibration by estimating a probabilistic flow on unified depth representations. UniCalib first bridges the modality gap by converting both the camera images and the sparse LiDAR points into unified, dense depth maps, enabling a unified encoder to learn consistent features. Subsequently, it learns a probabilistic flow field that captures the correspondence uncertainty to improve robustness. This probabilistic approach is reinforced by a reliability map and a perceptually weighted sparse flow loss, which guide the model to suppress the influence of unreliable regions. Experimental results on three datasets validate the accuracy and generalization of UniCalib. In particular, it achieves a mean translation error of 0.55 cm and a rotation error of 0.044deg on the KITTI dataset. The code is available at https://github.com/han-15/UniCalib.
StreetView-Waste: A Multi-Task Dataset for Urban Waste Management
Authors: Diogo J. Paulo; João Martins; Hugo Proença; João C. Neves
Venue: WACV 2026 (main)

Abstract: Urban waste management remains a critical challenge for the development of smart cities. Despite the growing number of litter detection datasets, the problem of monitoring overflowing waste containers, particularly from images captured by garbage trucks, has received little attention. While existing datasets are valuable, they often lack annotations for specific container tracking or are captured in static, decontextualized environments, limiting their utility for real-world logistics. To address this gap, we present StreetView-Waste, a comprehensive dataset of urban scenes featuring litter and waste containers. The dataset supports three key evaluation tasks: (1) waste container detection, (2) waste container tracking, and (3) waste overflow segmentation. Alongside the dataset, we provide baselines for each task by benchmarking state-of-the-art models in object detection, tracking, and segmentation. Additionally, we enhance baseline performance by proposing two complementary strategies: a heuristic-based method for improved waste container tracking and a model-agnostic framework that leverages geometric priors to refine litter segmentation. Our experimental results show that while fine-tuned object detectors achieve reasonable performance in detecting waste containers, baseline tracking methods struggle to accurately estimate their number; however, our proposed heuristics reduce the mean absolute counting error by 79.6%. Similarly, while segmenting amorphous litter is challenging, our geometry-aware strategy improves segmentation mAP@0.5 by 27% on lightweight models, demonstrating the value of multimodal inputs for this task. Ultimately, StreetView-Waste provides a challenging benchmark to encourage research into real-world perception systems for urban waste management.
Beyond the Highlights: Video Retrieval with Salient and Surrounding Contexts
Authors: Jaehun Bang; Moon Ye-Bin; Tae-Hyun Oh; Kyungdon Joo
Venue: WACV 2026 (main)

Abstract: When searching for videos, users often rely on surrounding context such as background elements or temporal details beyond salient content. However, existing video models struggle with fine-grained spatio-temporal understanding particularly surrounding contexts, and there are no datasets that effectively evaluate their performance. We introduce SS Datasets, three video retrieval datasets with detailed salient and surrounding captions. To capture rich, temporally localized contexts aligned with meaningful scene changes, we segment videos by scene transitions and generate captions with a vision-language model. Analyzing current models reveals difficulties in handling surrounding queries and temporally complex videos. To address this, we propose simple yet effective baselines that improve retrieval across diverse query types, enabling more robust generalization to real-world scenarios.
DF-Mamba: Deformable State Space Modeling for 3D Hand Pose Estimation in Interactions
Authors: Yifan Zhou; Takehiko Ohkawa; Guwenxiao Zhou; Kanoko Goto; Takumi Hirose; Yusuke Sekikawa; Nakamasa Inoue
Venue: WACV 2026 (main)

Abstract: Modeling daily hand interactions often struggles with severe occlusions, such as when two hands overlap, which highlights the need for robust feature learning in 3D hand pose estimation (HPE). To handle such occluded hand images, it is vital to effectively learn the relationship between local image features (e.g., for occluded joints) and global context (e.g., cues from inter-joints, inter-hands, or the scene). However, most current 3D HPE methods still rely on ResNet for feature extraction, and such CNN's inductive bias may not be optimal for 3D HPE due to its limited capability to model the global context. To address this limitation, we propose an effective and efficient framework for visual feature extraction in 3D HPE using recent state space modeling (i.e., Mamba), dubbed Deformable Mamba (DF-Mamba). DF-Mamba is designed to capture global context cues beyond standard convolution through Mamba's selective state modeling and the proposed deformable state scanning. Specifically, for local features after convolution, our deformable scanning aggregates these features within an image while selectively preserving useful cues that represent the global context. This approach significantly improves the accuracy of structured 3D HPE, with comparable inference speed to ResNet-50. Our experiments involve extensive evaluations on five divergent datasets including single-hand and two-hand scenarios, hand-only and hand-object interactions, as well as RGB and depth-based estimation. DF-Mamba outperforms the latest image backbones, including VMamba and Spatial-Mamba, on all datasets and achieves state-of-the-art performance.
Eye-for-an-eye: Appearance Transfer with Dense Semantic Correspondence in Diffusion Models
Authors: Sooyeon Go; Kyungmook Choi; Minjung Shin; Youngjung Uh
Venue: WACV 2026 (main)

Abstract: As pre-trained text-to-image diffusion models have become a useful tool for image synthesis, people want to specify the results in various ways. This paper tackles training-free appearance transfer, which produces an image with the structure of a target image from the appearance of a reference image. Existing methods usually do not reflect semantic correspondence, as they rely on query-key similarity within the self-attention layer to establish correspondences between images. To this end, we propose explicitly rearranging the features according to the dense semantic correspondences. Extensive experiments show the superiority of our method in various aspects: preserving the structure of the target and reflecting the correct color from the reference, even when the two images are not aligned.
PiSA: A Self-Augmented Data Engine and Training Strategy for 3D Understanding with Large Models
Authors: Zilu Guo; Hongbin Lin; Zhihao Yuan; Chaoda Zheng; Pengshuo Qiu; Dongzhi Jiang; Renrui Zhang; Chun-Mei Feng; Zhen Li
Venue: WACV 2026 (main)

Abstract: 3D Multimodal Large Language Models (MLLMs) have recently made substantial advancements. However, their potential remains untapped, primarily due to the limited quantity and suboptimal quality of 3D datasets. Current approaches attempt to transfer knowledge from 2D MLLMs to expand 3D instruction data, but still face modality and domain gaps. To this end, we introduce PiSA-Engine (Point-Self-Augmented-Engine), a new framework for generating instruction point-language datasets enriched with 3D spatial semantics. We observe that existing 3D MLLMs offer a comprehensive understanding of point clouds for annotation, while 2D MLLMs excel at cross-validation by providing complementary information. By integrating holistic 2D and 3D insights from off-the-shelf MLLMs, PiSA-Engine enables a continuous cycle of high-quality data generation. We select PointLLM as the baseline and adopt this co-evolution training framework to develop an enhanced 3D MLLM, termed PointLLM-PiSA. Additionally, we identify limitations in previous 3D benchmarks, which often feature coarse language captions and insufficient category diversity, resulting in inaccurate evaluations. To address this gap, we further introduce PiSA-Bench, a comprehensive 3D benchmark covering six key aspects with detailed and diverse labels. Experimental results demonstrate PointLLM-PiSA's state-of-the-art performance in zero-shot 3D object captioning and generative classification on our PiSA-Bench, achieving significant improvements of 46.45% (+8.33%) and 63.75% (+16.25%), respectively. We will release the code, datasets, and benchmark. Project page: https://github.com/CG-ops/PiSA.
Analysis of Text Accuracy and Visual Alignment in Vision-Language Models for Artistic Text Generation
Authors: Fatima Alderazi; Motaz Alfarraj
Venue: WACV 2026 (main)

Abstract: Artistic text generation involves rendering textual content in visually creative and contextually appropriate designs, such as floral or geometric patterns. Despite advancements in Vision-Language Models (VLMs) like DALL*E and Qwen, maintaining both text accuracy and aesthetic alignment is a persistent challenge in this domain. This paper presents an analysis of the performance of several VLMs in generating artistic text, aiming to diagnose performance gaps and guide improvements in text accuracy and multimodal alignment. The objective of this study is to propose an analysis framework to evaluate and improve artistic text generation by assessing models across three key dimensions: character-level text accuracy, semantic similarity, and visual-context alignment. A custom dataset of 1,000 prompts, each specifying a word and artistic style to support systematic evaluation was created. We benchmarked the performance of DALL*E, Qwen-VL, and Qwen-2.5B on text accuracy by comparing the generated text to the original prompt and measuring perceptual alignment using CLIP embeddings. The results show that the DALL-E model outperforms the other models, in balancing text accuracy and style alignment. This study highlights the potential of adopting VLMs for artistic text generation. Furthermore, we present exploratory results using ControlNet, focusing on its potential to improve text accuracy in generated images. Our findings expose critical limitations in current VLMs and offer actionable insights for advancing multimodal generation architectures, datasets, and evaluation strategies, as well as provide a framework for improving text accuracy and aesthetic coherence in creative applications, paving the way for advancements in multimodal AI systems.