GenAI4ELab/papercli-papers huggingface dataset:
MarineEval: Assessing the Marine Intelligence of Vision-Language Models
Authors: Yuk Kwan Wong; Tuan-An To; Jipeng Zhang; Ziqiang Zheng; Sai-Kit Yeung
Venue: WACV 2026 (main)

Abstract: We have witnessed promising progress led by large language models (LLMs) and further visual language models (VLMs) in handling various queries as a general-purpose assistant. VLMs, as a bridge to connect the visual world and language corpus, receive both visual content and various text-only user instructions to generate corresponding responses. Though great success has been achieved by VLMs in various fields, in this work, we ask whether the existing VLMs can act as domain experts, accurately answering marine questions, which require significant domain expertise and address special domain challenges/requirements. To comprehensively evaluate the effectiveness and explore the boundary of existing VLMs, we construct the first large-scale marine VLM dataset and benchmark called MarineEval, with 2,000 image-based question-answering pairs. During our dataset construction, we ensure the diversity and coverage of the constructed data: 7 task dimensions and 20 capacity dimensions. The domain requirements are specially integrated into the data construction and further verified by the corresponding marine domain experts. We comprehensively benchmark 17 existing VLMs on our MarineEval and also investigate the limitations of existing models in answering marine research questions. The experimental results reveal that existing VLMs cannot effectively answer the domain-specific questions, and there is still a large room for further performance improvements. We hope our new benchmark and observations will facilitate future research.
MMHOI: Modeling Complex 3D Multi-Human Multi-Object Interactions
Authors: Kaen Kogashi; Anoop Cherian; Meng-Yu Jennifer Kuo
Venue: WACV 2026 (main)

Abstract: Real-world scenes often feature multiple humans interacting with multiple objects in ways that are causal, goal-oriented, or cooperative. Yet existing 3D human-object interaction (HOI) benchmarks consider only a fraction of these complex interactions. To close this gap, we present MMHOI -- a large-scale, Multi-human Multi-object Interaction dataset consisting of images from 12 everyday scenarios. MMHOI offers complete 3D shape and pose annotations for every person and object, along with labels for 78 action categories and 14 interaction-specific body parts, providing a comprehensive testbed for next-generation HOI research.Building on MMHOI, we present MMHOI-Net, an end-to-end transformer-based neural network for jointly estimating human-object 3D geometries, their interactions, and associated actions. A key innovation in our framework is a structured dual-patch representation for modeling objects and their interactions, combined with action recognition to enhance the interaction prediction. Experiments on MMHOI and the recently proposed CORE4D datasets demonstrate that our approach achieves state-of-the-art performance in multi-HOI modeling, excelling in both accuracy and reconstruction quality.
CLIP's Visual Embedding Projector is a Few-shot Cornucopia
Authors: Mohammad Fahes; Tuan-Hung Vu; Andrei Bursuc; Patrick Perez; Raoul De Charette
Venue: WACV 2026 (main)

Abstract: We introduce ProLIP, a simple and architecture-agnostic method for adapting contrastively pretrained vision-language models, such as CLIP, to few-shot classification. ProLIP fine-tunes the vision encoder's projection matrix with Frobenius norm regularization on its deviation from the pretrained weights. It achieves state-of-the-art performance on 11 few-shot classification benchmarks under both "few-shot validation" and "validation-free" settings. Moreover, by rethinking the non-linear CLIP-Adapter through ProLIP's lens, we design a Regularized Linear Adapter (RLA) that performs better, requires no hyperparameter tuning, is less sensitive to learning rate values, and offers an alternative to ProLIP in black-box scenarios where model weights are inaccessible. Beyond few-shot classification, ProLIP excels in cross-dataset transfer, domain generalization, base-to-new class generalization, and test-time adaptation--where it outperforms prompt tuning while being an order of magnitude faster to train. Code is available at https://github.com/astra-vision/ProLIP .
MomentMix Augmentation with Length-Aware DETR for Temporally Robust Moment Retrieval
Authors: Seojeong Park; Jiho Choi; Kyungjune Baek; Hyunjung Shim
Venue: WACV 2026 (main)

Abstract: Video Moment Retrieval (MR) aims to localize moments within a video based on a given natural language query. Given the prevalent use of platforms like YouTube for information retrieval, the demand for MR techniques is significantly growing. Recent DETR-based models have made notable advances in performance but still struggle with accurately localizing short moments. Through data analysis, we identified limited feature diversity in short moments, which motivated the development of MomentMix. MomentMix generates new short-moment samples by employing two augmentation strategies: ForegroundMix and BackgroundMix, each enhancing the ability to understand the query-relevant and irrelevant frames, respectively. Additionally, our analysis of prediction bias revealed that short moments particularly struggle with accurately predicting their center positions and length of moments. To address this, we propose a Length-Aware Decoder, which conditions length through a novel bipartite matching process. Our extensive studies demonstrate the efficacy of our length-aware approach, especially in localizing short moments, leading to improved overall performance. Our method surpasses state-of-the-art DETR-based methods on benchmark datasets, achieving the highest R1 and mAP on QVHighlights and the highest R1@0.7 on TACoS and Charades-STA (such as a 9.62% gain in R1@0.7 and an 16.9% gain in mAP average for QVHighlights). The code is available at https://github.com/sjpark5800/LA-DETR.
IMPACT: Interpretable Most Important Person Analysis and Classification using Transformer-based Models
Authors: Akshat Rampuria; Kamakshya Prasad Nayak; Kamalakar Vijay Thakare; Tushar Joshi; Aditya Dhananjay Singh; Haesol Park; Heeseung Choi; Debi Prosad Dogra; Ig-Jae Kim
Venue: WACV 2026 (main)

Abstract: Identifying the Most Important Person (MIP) in complex social and sports events remains a challenging problem due to the dynamic nature of group interactions, subtle visual cues, and context-dependent semantics. Traditional methods often struggle to accurately capture the interplay between individuals and the overarching activity, especially in unstructured real-world environments. In addition, the lack of strong supervision and the need for a deeper contextual understanding further complicate the task. In this work, we propose IMPACT, a novel multi-modal framework that leverages recent advances in vision language models to bridge the gap between visual perception and semantic reasoning. Our approach integrates structured scene understanding, natural language generation, and cross-modal learning to jointly model activity recognition and MIP localization. The method integrates language, vision, and spatial reasoning to improve scene interpretability as well as accuracy in group activity recognition tasks. By incorporating language-based representations, the proposed method enables interpretable and robust performance in sports-centric group activity scenarios. Comprehensive experiments on C-Sports and NCAA datasets demonstrate that the framework significantly enhances the localization of key individuals as well as the accuracy of activity prediction, laying the groundwork for a holistic scene understanding in human-centric video and image analysis. Our proposed method achieves an accuracy of 81.6% when compared with human annotator markings and an increase in mAP scores by ~ 5% for MIP identification.
HOLO: Holistic Lightweight Optimization for Scene Understanding with Auto-Annotation and Multimodal Learning
Authors: Xiaoyun Hu; Xiaohan Yan; Nan Wang; Gang Wei; Zhicheng Wang
Venue: WACV 2026 (main)

Abstract: Vision-language models (VLMs) have achieved remarkable success in various domains. However, their application to 3D scene understanding remains largely underexplored. Existing 3D VLMs predominantly focus on object-level tasks and often emphasize instance-centric representations within a scene, lacking holistic scene-level descriptions. In this work, we propose an automated annotation framework that leverages multi-view images to partition 3D scenes into localized point cloud sub-regions, which are then enriched with precise semantic information-all without any manual intervention. We processed ScanNet v2 and ScanNet++ to construct SceneCap, a large-scale dataset designed for scene-level description. To demonstrate the benefits of our framework for scene understanding , we introduce NIUMO-LLM, a lightweight yet high-performing model training on SceneCap. We further demonstrate that NIUMO-LLM achieves state-of-the-art (SOTA) performance on both scene description benchmarks and object-level tasks, requiring only 12 hours of training on a single NVIDIA A800 GPU. This design significantly reduces computational demands, lowering the barrier for MLLM-related research.
UniVid: Unifying Vision Tasks with Pre-trained Video Generation Models
Authors: Lan Chen; Yuchao Gu; Qi Mao
Venue: WACV 2026 (main)

Abstract: Large language models, trained on extensive corpora, successfully unify diverse linguistic tasks within a single generative framework. Inspired by this, recent works like Large Vision Model (LVM) extend this paradigm to vision by organizing tasks into sequential visual sentences, where visual prompts serve as the context to guide outputs. However, such modeling requires task-specific pre-training across modalities and sources, which is costly and limits scalability to unseen tasks. Given that pre-trained video generation models inherently capture temporal sequence dependencies, we explore a more unified and scalable alternative: can a pre-trained video generation model adapt to diverse image and video tasks? To answer this, we propose UniVid, a framework that fine-tunes a video diffusion transformer to handle various vision tasks without task-specific modifications. Tasks are represented as visual sentences, where the context sequence defines both the task and the expected output modality. We evaluate the generalization of UniVid from two perspectives: (1) cross-modal inference with contexts composed of both images and videos, extending beyond LVM's uni-modal setting; (2) cross-source tasks from natural to annotated data, without multi-source pre-training. Despite being trained solely on natural video data, UniVid generalizes well in both settings. Notably, understanding and generation tasks can easily switch by simply reversing the visual sentence order in this paradigm. These findings highlight the potential of pre-trained video generation models to serve as a scalable and unified foundation for vision modeling. Our code is released at https://github.com/CUC-MIPG/UniVid.
Sea-CLIP: Mining Semantic-Aware Representations for Few-Shot Anomaly Detection with CLIP
Authors: Xiao Guo; Zhimin Chen; Carlos D. Castillo; Hongcheng Wang; Xiaoming Liu
Venue: WACV 2026 (main)

Abstract: Few-shot Anomaly Detection (FSAD) is a classic computer vision task, and recent FSAD methods utilize the pre-trained Vision-Language model, i.e., CLIP, to achieve remarkable performance. However, existing CLIP-based approaches disregard object semantics, a crucial factor for enhancing FSAD by guiding comparisons between semantically corresponding patches. To address this limitation, we propose Sea-CLIP, a novel method that integrates semantic-aware representations from DINOv2 to enhance FSAD representation learning. Specifically, Sea-CLIP first leverages a Patch Matching module that uses semantic-aware representations to obtain coarse anomaly segmentation masks. These anomaly masks guide a lightweight Anomaly Matching Decoder (AMD) to utilize CLIP and DINOv2 features for FSAD jointly, and AMD innovatively formulates FSAD as a feature-matching task. Also, unlike prior patch-matching works that directly compute anomaly scores, our method utilizes the AMD to refine coarse predictions into a precise anomaly mask. Our Sea-CLIP achieves state-of-the-art FSAD performance on MVTec and VisA datasets, and we provide a detailed analysis of contributions from semantic-aware representations in identifying anomaly patterns.
CasTex: Cascaded Text-to-Texture Synthesis via Explicit Texture Maps and Physically-Based Shading
Authors: Mishan Aliev; Dmitry Baranchuk; Kirill Struminsky
Venue: WACV 2026 (main)

Abstract: This work investigates text-to-texture synthesis using diffusion models to generate physically-based texture maps. We aim to achieve realistic model appearances under varying lighting conditions. A prominent solution for the task is score distillation sampling. It allows recovering a complex texture using gradient guidance given a differentiable rasterization and shading pipeline. However, in practice, the aforementioned solution in conjunction with the widespread latent diffusion models produces severe visual artifacts and requires additional regularization such as implicit texture parameterization. As a more direct alternative, we propose an approach using cascaded diffusion models for texture synthesis (CasTex). In our setup, score distillation sampling yields high-quality textures out-of-the box. In particular, we were able to omit implicit texture parameterization in favor of an explicit parameterization to improve the procedure. In the experiments, we show that our approach significantly outperforms state-of-the-art optimization-based solutions on public texture synthesis benchmarks.
SOPHY: Generating Simulation-Ready Objects with Physical Materials
Authors: Junyi Cao; Evangelos Kalogerakis
Venue: WACV 2026 (main)

Abstract: We present SOPHY, a generative model for 3D physics-aware shape synthesis. Unlike existing 3D generative models that focus solely on static geometry or 4D models that produce physics-agnostic animations, our method jointly synthesizes shape, texture, and material properties related to physics-grounded dynamics, making the generated objects ready for simulations and interactive, dynamic environments. To train our model, we introduce a dataset of 3D objects annotated with detailed physical material attributes, along with an efficient pipeline for material annotation. Our method enables applications such as text-driven generation of interactive, physics-aware 3D objects and single-image reconstruction of physically plausible shapes. Furthermore, our experiments show that jointly modeling shape and material properties enhances the realism and fidelity of the generated shapes, improving performance on both generative geometry and physical plausibility. Project page: https://xjay18.github.io/SOPHY.
SasMamba: A Lightweight Structure-Aware Stride State Space Model for 3D Human Pose Estimation
Authors: Hu Cui; Wenqiang Hua; Renjing Huang; Shurui Jia; Tessai Hayama
Venue: WACV 2026 (main)

Abstract: Recently, the Mamba architecture based on State Space Models (SSMs) has gained attention in 3D human pose estimation due to its linear complexity and strong global modeling capability. However, existing SSM-based methods typically apply manually designed scan operations to flatten detected 2D pose sequences into purely temporal sequences, either locally or globally. This approach disrupts the inherent spatial structure of human poses and entangles spatial and temporal features, making it difficult to capture complex pose dependencies.To address these limitations, we propose the Skeleton Structure-Aware Stride SSM (SAS-SSM), which first employs a structure-aware spatiotemporal convolution to dynamically capture essential local interactions between joints, and then applies a stride-based scan strategy to construct multi-scale global structural representations. This enables flexible modeling of both local and global pose information while maintaining linear computational complexity.Built upon SAS-SSM, our model SasMamba achieves competitive 3D pose estimation performance with significantly fewer parameters compared to existing hybrid models.The source code will be available at https://hucui2022.github.io/sasmamba_proj/.
MedPEFT-CL: Dual-Phase Parameter-Efficient Continual Learning with Medical Semantic Adapter and Bidirectional Memory Consolidation
Authors: Ziyuan Gao; Philippe Morel
Venue: WACV 2026 (main)

Abstract: Medical vision-language segmentation models suffer from catastrophic forgetting when adapting to new anatomical structures, requiring complete retraining that limits their clinical deployment. Although continual learning approaches have been studied for various applications, targeted research on continual learning approaches specifically designed for medical vision-language tasks remains underexplored.We propose MedPEFT-CL, a parameter-efficient continual learning framework that addresses both efficient learning of new tasks and preservation of previous knowledge through a dual-phase architecture based on CLIPSeg.Our dual-phase architecture features an adaptive learning phase that employs semantic similarity-based adapter allocation and parameter-efficient fine-tuning for medical tasks through prompt similarity analysis, and a knowledge consolidation phase employing bi-directional Fisher-memory coordination. This creates a reinforcing cycle: consolidation directs replay priorities while new tasks provide challenging samples that improve retention strategies.Our key contributions are: (1) a semantic-driven adapter allocation mechanism that enables efficient learning of new medical tasks, (2) a bi-modal LoRA adaptation that significantly reduces trainable parameters while maintaining cross-modal learning, and (3) bidirectional Fisher-memory coordination that prevents catastrophic forgetting from previous medical tasks.Extensive experiments across diverse medical datasets demonstrate superior forgetting mitigation and performance retention with minimal parameter overhead, making the framework effective for continual learning in medical vision-language scenarios.
ZebraPose: Zebra Detection and Pose Estimation using only Synthetic Data
Authors: Elia Bonetto; Aamir Ahmad
Venue: WACV 2026 (main)

Abstract: Collecting and labeling large real-world wild animal datasets is impractical, costly, error-prone, and labor-intensive. For animal monitoring tasks, as detection, tracking, and pose estimation, out-of-distribution viewpoints (e.g. aerial) are also typically needed but rarely found in publicly available datasets. To solve this, existing approaches synthesize data with simplistic techniques that then necessitate strategies to bridge the synthetic-to-real gap. Therefore, real images, style constraints, complex animal models, or pre-trained networks are often leveraged. In contrast, we generate a fully synthetic dataset using a 3D photorealistic simulator and demonstrate that it can eliminate such needs for detecting and estimating 2D poses of wild zebras. Moreover, existing top-down 2D pose estimation approaches using synthetic data assume reliable detection models. However, these often fail in out-of-distribution scenarios, e.g. those that include wildlife or aerial imagery. Our method overcomes this by enabling the training of both tasks using the same synthetic dataset. Through extensive benchmarks, we show that models trained from scratch exclusively on our synthetic data generalize well to real images. We perform these using multiple real-world and synthetic datasets, pre-trained and randomly initialized backbones, and different image resolutions. Code, results, models, and data can be found at https://zebrapose.is.tue.mpg.de/.
Action Anticipation at a Glimpse: To What Extent Can Multimodal Cues Replace Video?
Authors: Manuel Benavent-Lledo; Konstantinos Bacharidis; Victoria Manousaki; Konstantinos Papoutsakis; Antonis Argyros; Jose Garcia-Rodriguez
Venue: WACV 2026 (main)

Abstract: Anticipating actions before they occur is a core challenge in action understanding research. While conventional methods rely on extracting and aggregating temporal information from videos, as humans we can often predict upcoming actions by observing a single moment from a scene, when given sufficient context. Can a model achieve this competence? The short answer is yes, although its effectiveness depends on the complexity of the task. In this work, we investigate to what extent can video aggregation be replaced with alternative modalities. To this end, based on recent advances in visual feature extraction and language-based reasoning, we introduce AAG, a method for Action Anticipation at a Glimpse. AAG combines RGB features with depth cues from a single frame for enhanced spatial reasoning, and incorporates prior action information to provide long-term context. This context is obtained either through textual summaries from Vision-Language Models, or from predictions generated by a single-frame action recognizer. Our results demonstrate that multimodal single-frame action anticipation using AAG can perform competitively compared to both temporally aggregated video baselines and state-of-the-art methods across three instructional activity datasets: IKEA-ASM, Meccano, and Assembly101.
Systematic Analysis of the Unintentional CSAM-Generation-Potential of Text-to-Image Models
Authors: Nicolas Göller; Martin Steinebach
Venue: WACV 2026 (main)

Abstract: The rapid advancements of generative text-to-image (T2I) models and their open accessibility have enabled users to generate high-quality, photorealistic images of humans. Ethical challenges, particularly the deliberate generation of child sexual abuse material (CSAM), have been widely recognized. By contrast, the unintentional creation of such content has received little scholarly attention. The legal risks associated with this phenomenon nevertheless pose a significant threat to the increasing number of users of generative models. To investigate this issue, we conduct a comprehensive systematic evaluation of the potential of state-of-the-art T2I models to generate CSAM against users' intentions. We systematically generate datasets with prompts specifying adult subjects. Using age estimation models, we analyze the datasets regarding age compliance across different visual demographic properties and prompt variations. Our findings show that the six examined prominent T2I models generate images depicting underage individuals despite explicit adult-oriented prompts. Across various dataset settings, Stable Diffusion 3.5 Large and Qwen-Image generate the highest proportion of persons classified as underage in our experiments. We share insights and strategies to mitigate the risk of generating CSAM.
Inpainting of Sparse Depth Maps from Monocular Depth-from-Focus on Pixel Processor Arrays
Authors: Maciej Lewandowski; Piotr Dudek
Venue: WACV 2026 (main)

Abstract: Depth estimation is essential for robotics and effective navigation. While many recent methods attempt to estimate dense depth maps from a single RGB image or a combination of an RGB image and sparse depth measurements, our work leverages the in-pixel computing capabilities of a pixel processor array (PPA), combined with an electrically tunable liquid lens, to capture semi-sparse depth maps via a depth-from-focus approach. We consider the problem of reconstructing dense depth maps from such measurements.We simulate a PPA-based depth-from-focus algorithm on a synthetic focal stack derived from a monocular RGB-D dataset, demonstrating competitive dense depth map reconstruction from depth frames containing as few as 10% non-zero pixels, with a 5-bit resolution. Furthermore, we enhance the performance of semi-sparse depth completion by fusing these PPA-captured depth cues with concurrently acquired RGB images. We also use belief propagation, allowing for highly localized and parallel computation without access to global memory, offering a promising solution from the perspective of PPAs. We show the performance of these algorithms on semi-sparse image depth reconstruction tasks.