GenAI4ELab/papercli-papers huggingface dataset:
Learning Beyond Labels: Self-Supervised Handwritten Text Recognition
Authors: Shree Mitra; Ajoy Mondal; C.V. Jawahar
Venue: WACV 2026 (main)

Abstract: This paper addresses a key challenge in Handwritten Text Recognition (HTR): the dependence on large volumes of labeled data. To overcome this, we propose a self-supervised learning (SSL) framework, LoGo-HTR, that minimizes labeling requirements while achieving strong recognition performance. We introduce a large-scale dataset, SSL-HWD of 10 million word-level handwritten images from diverse scanned documents, partitioned into a small labeled subset and a much larger unlabeled subset. LoGo-HTR combines a local contrastive loss for spatial consistency and a global decorrelation loss to enhance feature diversity. This dual objective enables robust, invariant, and spatially discriminative feature learning. After self-supervised pretraining, we fine-tune a transformer-based decoder using limited labeled data. Extensive experiments on standard HTR benchmarks, which include multilingual and historical data, demonstrate that, after SSL pretraining on our unlabeled dataset, our method consistently outperforms state-of-the-art approaches, even when fine-tuned using only 80% and 20% of the available labeled training data from the respective benchmarks. Ablation studies highlight the effectiveness of our dual loss design and demonstrate the potential of scalable, label-efficient handwritten text recognition. The SSL-HWD dataset and LoGo-HTR model with code are publicly available at https://logo-ssl.github.io/.
A Woman with a Knife or A Knife with a Woman? Measuring Directional Bias Amplification in Image Captions
Authors: Rahul Nair; Bhanu Tokas; Hannah Kerner
Venue: WACV 2026 (main)

Abstract: When we train models on biased datasets, they not only reproduce data biases, but can worsen them at test time --- a phenomenon called bias amplification. Many of the current bias amplification metrics (e.g., BA, DPA) measure bias amplification only in classification datasets. These metrics are ineffective for image captioning datasets, as they cannot capture the language semantics of a caption. Recent work introduced Leakage in Captioning (LIC), a language-aware bias amplification metric that understands caption semantics. However, LIC has a crucial limitation: it cannot identify the source of bias amplification in captioning models. We propose Directional Bias Amplification in Captioning (DBAC), a language-aware and directional metric that can identify when captioning models amplify biases. DBAC has two more improvements over LIC: (1) it is less sensitive to sentence encoders (a hyperparameter in language-aware metrics), and (2) it provides a more accurate estimate of bias amplification in captions. Our experiments on gender and race attributes in the COCO captions dataset show that DBAC is the only reliable metric to measure bias amplification in captions.
Vision-informed Semantic Text Alignment for Open-set Recognition in Remote Sensing
Authors: Siddhant Gole; Akash Pal; Ankit Jha; Subhasis Chaudhuri; Biplab Banerjee
Venue: WACV 2026 (main)

Abstract: Existing Open-Set Recognition (OSR) methods struggle in remote sensing (RS) as their reliance on unimodal visual features fails to resolve the severe inter-class similarity inherent in overhead imagery. To address this, we propose ViSTA-RS, a novel multimodal framework that leverages semantic context from language to disambiguate visually similar scenes. Our approach first constructs semantically-rich class prototypes by jointly encoding images with generated text captions using a Vision-Language Model. We then introduce a reconstruction-based mechanism where an image's visual embedding is expressed as a weighted combination of these semantic prototypes. The magnitude of the reconstruction error serves as a robust novelty score, with a statistically principled threshold determined by Extreme Value Theory (EVT). This alignment of multimodal semantics with prototype reconstruction is uniquely suited for the fine-grained nature of RS data. On four challenging benchmarks, ViSTA-RS sets a new state-of-the-art, improving the AUROC for unknown detection by a significant 6.7% over leading baselines while maintaining high accuracy on known classes.
Guided Model Merging for Hybrid Data Learning: Leveraging Centralized Data to Refine Decentralized Models
Authors: Junyi Zhu; Ruicong Yao; Taha Ceritli; Savas Ozkan; Matthew B. Blaschko; Eunchung Noh; Jeongwon Min; Cho Jung Min; Mete Ozay
Venue: WACV 2026 (main)

Abstract: Current network training paradigms primarily focus on either centralized or decentralized data regimes. However, in practice, data availability often exhibits a hybrid nature, where both regimes coexist. This hybrid setting presents new opportunities for model training, as the two regimes offer complementary trade-offs: decentralized data is abundant but subject to heterogeneity and communication constraints, while centralized data--though limited in volume and potentially unrepresentative--enables better curation and high-throughput access. Despite its potential, effectively combining these paradigms remains challenging, and few frameworks are tailored to hybrid data regimes. To address this, we propose a novel framework that constructs a model atlas from decentralized models and leverages centralized data to refine a global model within this structured space. The refined model is then used to reinitialize the decentralized models. Our method synergizes federated learning (to exploit decentralized data) and model merging (to utilize centralized data), enabling effective training under hybrid data availability. Theoretically, we show that our approach achieves faster convergence than methods relying solely on decentralized data, due to variance reduction in the merging process. Extensive experiments demonstrate that our framework consistently outperforms purely centralized, purely decentralized, and existing hybrid-adaptable methods. Notably, our method remains robust even when the centralized and decentralized data domains differ or when decentralized data contains noise, significantly broadening its applicability.
EndoPBR: Photorealistic Synthetic Data for Surgical 3D Vision via Physically-based Rendering
Authors: John J. Han; Jie Ying Wu
Venue: WACV 2026 (main)

Abstract: Synthetic data has played a pivotal role in developing large-scale 3D vision models due to its high-quality annotations and ease of curation. In domains where labeled data collection is difficult, such as endoscopy, synthetic data holds promise as a means to generate the large-scale annotated datasets required to train modern neural networks. In this work, we address a core question for data-scarce applications in 3D vision: how can we generate synthetic labeled data, and how useful would the data be for training downstream vision models? First, we introduce a novel data generation module that takes images with known geometry and camera poses as input and estimates the material and lighting conditions of the scene. To stabilize training, we leverage domain-specific properties like non-stationary lighting and anatomical material priors. We model the material properties as a bidirectional reflectance distribution function, parameterized by a neural network. Via the rendering equation, we can generate photorealistic images at arbitrary camera poses. We demonstrate that this method produces competitive novel view synthesis results compared to previous work while being more lightweight, flexible, and efficient. Second, we use our synthetic data to train models on various downstream 3D vision tasks and find that models trained solely on our synthetic data generally outperform those trained on real data across various metrics and tasks. Our experiments show that synthetic data is a promising avenue towards robust 3D vision in surgical scenes.
KMOPS: Keypoint-Driven Method for Multi-Object Pose and Metric Size Estimation from Stereo Images
Authors: Ying-Kun Wu; Yi Shen; Tzuhsuan Huang; I-Sheng Fang; Jun-Cheng Chen
Venue: WACV 2026 (main)

Abstract: The six-degree-of-freedom (6-DoF) pose and metric size estimation of multiple objects from RGB images alone remains a challenging task, particularly due to significant variations in object shape, appearance, and frequent occlusions in complex scenes. To address these challenges, we introduce KMOPS, a Keypoint-driven method tailored specifically for Multi-Object Pose and metric Size estimation from a single calibrated stereo image pair. Leveraging the stereo input, our approach first extracts the 2D keypoints of the enclosing bounding boxes of the objects across both views, and subsequently triangulates them to acquire metric 3D positions. Then, we obtain each object's rotation, translation, and dimensions by aligning the triangulated 3D keypoints to the canonical ones using a closed-form solution. Our formulation eliminates the need for predefined 3D search spaces or volumetric anchors, which are often required by other methods to constrain the vast 3D solution space. With extensive experiments on the challenging dataset Transparent Object Dataset (TOD) and StereOBJ-1M, we show that our method outperforms all competing methods with a simple and effective architecture.
Referring Change Detection in Remote Sensing Imagery
Authors: Yilmaz Korkmaz; Jay N. Paranjape; Celso M. de Melo; Vishal M. Patel
Venue: WACV 2026 (main)

Abstract: Change detection in remote sensing imagery is essential for applications such as urban planning, environmental monitoring, and disaster management. Traditional change detection methods typically identify all changes between two temporal images without distinguishing the types of transitions, which can lead to results that may not align with specific user needs. Although semantic change detection methods have attempted to address this by categorizing changes into predefined classes, these methods rely on rigid class definitions and fixed model architectures, making it difficult to mix datasets with different label sets or reuse models across tasks, as the output channels are tightly coupled with the number and type of semantic classes. To overcome these limitations, we introduce Referring Change Detection (RCD), which leverages natural language prompts to detect specific classes of changes in remote sensing images. By integrating language understanding with visual analysis, our approach allows users to specify the exact type of change they are interested in. However, training models for RCD is challenging due to the limited availability of annotated data and severe class imbalance in existing datasets. To address this, we propose a two-stage framework consisting of (I) RCDNet, a cross-modal fusion network designed for referring change detection, and (II) RCDGen, a diffusion-based synthetic data generation pipeline that produces realistic post-change images and change maps for a specified category using only pre-change image, without relying on semantic segmentation masks and thereby significantly lowering the barrier to scalable data creation. Experiments across multiple datasets show that our framework enables scalable and targeted change detection. Code will be made publicly available on \href https://github.com/yilmazkorkmaz1/referring_change_detection  Github .
Isolating the Role of Temporal Information in Video Saliency: A Controlled Experimental Analysis
Authors: Peter El-Jiz; Matthias Kuemmerer; Matthias Tangemann; Matthias Bethge; Andreas Bartels; Michael Mario Bannert
Venue: WACV 2026 (main)

Abstract: The role of temporal information in predicting human gaze in dynamic scenes remains a critical open question, underscored by the paradoxical finding that strong static models can outperform complex video-based models. This suggests that the true contribution of temporal cues has been obscured by confounding architectural variables. To resolve this, we present a rigorous, controlled experiment centered on a minimal architectural pair: a spatio-temporal saliency model (UniformerSal-ST) and its identical spatial-only counterpart (UniformerSal-S), designed to unambiguously isolate the impact of temporal feature integration. Our results demonstrate that principled temporal fusion yields a substantial Information Gain (IG) of +0.20 bits on temporally coherent datasets like LEDOV. Crucially, our controlled comparison also uncovers a key failure mode: on datasets with frequent hard cuts like DIEM, the same mechanism degrades performance, incurring a 0.07 bits IG deficit. We provide a mechanistic explanation for this dichotomy, revealing how certain visual scenarios (scene discontinuities, rapid camera zooms) can disrupt current temporal fusion approaches. By precisely quantifying both the benefits and drawbacks of temporal processing, our work provides the community with clear, actionable insights into when and why temporal information should be modeled for more robust and accurate video saliency prediction.
Data-Driven Loss Functions for Inference-Time Optimization in Text-to-Image
Authors: Sapir Esther Yiflach; Yuval Atzmon; Gal Chechik
Venue: WACV 2026 (main)

Abstract: Text-to-image diffusion models can generate stunning visuals, yet they often fail at tasks children find trivial--like placing a dog to the right of a teddy bear rather than to the left. When combinations get more unusual--a giraffe above an airplane--these failures become even more pronounced. Existing methods attempt to fix these spatial reasoning failures through model fine-tuning or test-time optimization with handcrafted losses that are suboptimal. Rather than imposing our assumptions about spatial encoding, we propose learning these objectives directly from the model's internal representations. We introduce Learn-to-Steer, a novel framework that learns data-driven objectives for test-time optimization rather than handcrafting them. Our key insight is to train a lightweight classifier that decodes spatial relationships from the diffusion model's cross-attention maps, then deploy this classifier as a learned loss function during inference. Training such classifiers poses a surprising challenge: they can take shortcuts by detecting linguistic traces in the cross-attention maps, rather than learning true spatial patterns. We solve this by augmenting our training data with samples generated using prompts with incorrect relation words, which encourages the classifier to avoid linguistic shortcuts and learn spatial patterns from the attention maps. Our method dramatically improves spatial accuracy: from 20% to 61% on FLUX.1-dev and from 7% to 54% on SD2.1 across standard benchmarks. It also generalizes to multiple relations with significantly improved accuracy.
ScoreNet: Netting Lightweight Quality Scores for Better Visual Assessment with Large Multi-Modality Models
Authors: Bahador Rashidi; Kiarash Aghakasiri; Shupei Zhang; Amirmohsen Sattarifard; Yue Zhang; Chao Gao
Venue: WACV 2026 (main)

Abstract: The advancement of general large multi-modal models (LMMs) has transformed many computer vision tasks, shifting image quality assessment (IQA) from specialized algorithms to models built on pre-trained LMM backbones. This evolution raises the question of whether dedicated IQA metrics remain relevant or are becoming obsolete in the age of LMMs. In this paper, we address this challenge by introducing ScoreNet, a novel framework that fuses the strengths of traditional metrics to elevate the IQA capabilities of LMMs. ScoreNet employs a soft prompting mechanism, learning prompts from a curated set of lightweight IQA scores and image embeddings. This context-driven learning strategy enhances the adaptability of LMMs for IQA tasks with a small additional computation cost. We show that ScoreNet serves as a general-purpose extension applicable to modern LMM-based IQA models. We integrate ScoreNet into two high-performing methods--CLIP-IQA and Q-Align--and observe consistent improvements. Experimental results show that ScoreNet not only boosts both models but also surpasses other state-of-the-art IQA approaches. The code is available at https://github.com/AISmartPerception/ScoreNet
Digital Forensic AI You Can Explain: A Case Study on Video Source Camera Identification
Authors: Maryna Veksler; Kemal Akkaya; Selcuk Uluagac
Venue: WACV 2026 (main)

Abstract: In recent years, artificial intelligence (AI) has significantly impacted digital forensics, yet its broader deployment remains limited due to the difficulty of explaining AI decisions. Explainable AI (XAI) presents a promising solution to increase transparency and trust, but its application in digital forensics is still underexplored. In this work, we present a practical and structured explainable digital forensics AI (xDFAI) approach tailored to the forensic task of video source camera identification (VSCI). Our method enables forensic examiners to interpret the behavior of AI models, assess whether decisions are driven by intended logic or arise from random or content-dependent artifacts, and establish the integrity and reliability of explanations. We implement and evaluate this approach on two state-of-the-art VSCI models, providing step-by-step analysis of explanation quality, spatial consistency of high-impact features, and content dependence. Our results reveal that although models achieve strong classification accuracy, their explanations lack spatial stability and are impacted by video content, raising concerns about forensic reliability. To support reproducibility and future research, we provide an open-source implementation. This work underscores the potential of XAI to improve transparency in digital forensics and highlights the challenges of interpreting and presenting results. Our study takes an important step toward the operational deployment of xDFAI in multimedia forensics.
CoreCaption: Core Caption based Text-to-Video Retrieval
Authors: Junkyu Jang
Venue: WACV 2026 (main)

Abstract: The explosive growth of online video content has heightened the need for efficient text-to-video retrieval systems. These systems heavily rely on accurate video-text representations for effective retrieval. However, challenges such as the multi-thematic nature of videos and inconsistent caption quality hinder performance by making it difficult to capture comprehensive video-text relationships. To address these issues, we propose CoreCaption, a novel framework that extracts and leverages core captions--the most representative captions capturing essential video themes. Our approach includes a unique core caption extraction method based on similarity-based density estimation and clustering, and introduces the Core Caption Guided Attention (CCGA) mechanism to integrate video-specific semantic information into text queries while preserving their original intent. Furthermore, we employ a teacher-student architecture for efficient inference without reliance on core captions during deployment. Extensive experiments on benchmark datasets like MSR-VTT, VATEX, and MSVD demonstrate that CoreCaption outperforms state-of-the-art methods. These results validate the effectiveness and robustness of our framework across diverse video-text datasets.
M4U: Evaluating Multilingual Understanding and Reasoning for Large Multimodal Models
Authors: Hongyu Wang; Jiayu Xu; Senwei Xie; Ruiping Wang; Jialin Li; Zhaojie Xie; Bin Zhang; Chuyan Xiong; Xilin Chen
Venue: WACV 2026 (main)

Abstract: Multilingual capability is a crucial requirement for large multimodal models, which are increasingly deployed across diverse countries and languages. However, most existing benchmarks for multilingual multimodal reasoning fail to effectively distinguish models of different strengths; in fact, even text-only language models without visual capabilities can often achieve high scores. As a result, the comprehensive evaluation of state-of-the-art multilingual multimodal models remains underexplored. In this work, we present M4U, a novel and challenging benchmark designed to evaluate multilingual, multi-discipline multimodal understanding and reasoning. M4U comprises 10k samples spanning 64 disciplines across 16 subfields in Science, Engineering, and Healthcare, covering six languages. Using this benchmark, we conduct extensive evaluations of leading Large Multimodal Models (LMMs) and Large Language Models (LLMs) augmented with external tools. Our results reveal that even the strongest LMMs exhibit pronounced language preferences and struggle with reasoning tasks that require integrating multilingual information across visual and textual modalities. In particular, performance drops markedly when models are prompted with cross-lingual multimodal questions, highlighting significant gaps in current multilingual multimodal reasoning capabilities.
Snapmoji: Instant Generation of Animatable Dual-Stylized Avatars
Authors: Eric Ming Chen; Di Liu; Sizhuo Ma; Michael Vasilkovsky; Bing Zhou; Qiang Gao; Wenzhou Wang; Jiahao Luo; Dimitris N. Metaxas; Vincent Sitzmann; Jian Wang
Venue: WACV 2026 (main)

Abstract: Despite the increasing popularity of avatar systems such as Snapchat Bitmojis, existing production avatar platforms face several limitations, such as a limited number of predefined assets, tedious customization processes, and inefficient rendering requirements. Addressing these shortcomings, we introduce Snapmoji, an avatar generation system that instantly creates 3D avatars, and enables customization in a process we call dual-stylization. Snapmoji first maps a selfie of a user to a primary avatar (e.g., Bitmoji style) using a new technique we name Gaussian Domain Adaptation (GDA), then applies a secondary style (e.g., skeleton, yarn, toy) to the primary avatar, all while preserving the user's identity. The generated 3D avatars can then be rendered an animated on mobile devices at 30--40 FPS.
BanglaProtha: Evaluating Vision Language Models in Underrepresented Long-tail Cultural Contexts
Authors: Md Fahim; Md Sakib Ul Rahman; Akm Moshiur Rahman; Md Farhan Ishmam; Md Tasmim Rahman; Fariha Tanjim Shifat; Fabiha Haider; Md Farhad Alam Bhuiyan
Venue: WACV 2026 (main)

Abstract: The advanced multimodal processing of current vision language models (VLMs) has prompted rigorous benchmarking in multicultural settings, revealing a clear inclination toward Western culture. While the bias likely stems from the predominance of Western-centric images in the VLM pretraining data, the resulting long-tail distribution problem is only exacerbated in underrepresented cultural settings, such as Bengali. Our work explores this problem through an aspect-based evaluation of several classes of VLMs on the rich Bengali culture. Our BanglaProtha dataset is a VQA dataset, containing images that encapsulate Bengali cultural elements, questions in native Bengali, and semantically similar multiple-choice answer options. Our experiments provide behavioral insights of VLMs across prompting & fine-tuning strategies, cultural aspects, model size, and augmentation methods. Our work serves as a diagnostic tool for addressing and mitigating inequalities in multicultural and multilingual settings, thereby bringing efforts to democratize AI systems. Our code and data are available at https://github.com/farhanishmam/BanglaProtha.