GenAI4ELab/papercli-papers huggingface dataset:
Cross-modal Identity Mapping: Minimizing Information Loss in Modality Conversion via Reinforcement Learning
Authors: Haonan Jia; Shichao Dong; Xin Dong; Zenghui Sun; Jin Wang; Jinsong Lan; Xiaoyong Zhu; Bo Zheng; Kaifu Zhang
Venue: CVPR 2026 (main)

Abstract: Large Vision-Language Models (LVLMs) often omit or misrepresent critical visual content in generated image captions. Minimizing such information loss will force LVLMs to focus on image details to generate precise descriptions. However, measuring information loss during modality conversion is inherently challenging due to the modal gap between visual content and text output. In this paper, we argue that the quality of an image caption is positively correlated with the similarity between images retrieved via text search using that caption. Based on this insight, we further propose Cross-modal Identity Mapping (CIM), a reinforcement learning framework that enhances image captioning without requiring additional annotations. Specifically, the method quantitatively evaluates the information loss from two perspectives: Gallery Representation Consistency and Query-gallery Image Relevance. Supervised under these metrics, LVLM minimizes information loss and aims to achieve identity mapping from images to captions. The experimental results demonstrate the superior performance of our method in image captioning, even when compared with Supervised Fine-Tuning. Particularly, on the COCO-LN500 benchmark, CIM achieves a 20% improvement in relation reasoning on Qwen2.5-VL-7B. The codes are released on GitHub.
Graph Attention Prototypical Network for Robust Few-Shot Classification
Authors: Tingyun Liu; Licheng Liu; Qibin Zhang; Qiying Feng; C. L. Philip Chen
Venue: CVPR 2026 (main)

Abstract: Few-shot learning has attracted extensive attention, with metric-based approaches such as Prototypical Networks establishing strong baselines. These methods construct class prototypes from support samples and classify query samples via distance metrics, but their performance is highly sensitive to label noise. To tackle this challenge, we propose a novel Graph Attention Prototypical Network (GAPNet) for robust few-shot classification. GAPNet first extracts local and global features via a classic CNN backbone and a group attention broad learning module, respectively. To mitigate the impact of label noise, the intra-class and inter-class relationships between support and query samples are explicitly modeled via a pseudo-label guided graph constructor, and then processed by an edge-aware graph attention module to capture topological correlations. Furthermore, an adaptive noise-robust prototype generator is introduced to dynamically suppress the contributions of noisy samples, substantially improving the reliability of class prototypes. Extensive experiments demonstrate the effectiveness and robustness of GAPNet to label noise. Compared to state-of-the-art approaches, GAPNet improves accuracy in the 5-way 5-shot setting by 3% ~ 8% on three general image benchmarks and one fine-grained classification dataset.
Towards Highly Transferable Vision-Language Attack via Semantic-Augmented Dynamic Contrastive Interaction
Authors: Yuanbo Li; Tianyang Xu; Cong Hu; Tao Zhou; Xiaojun Wu; Josef Kittler
Venue: CVPR 2026 (main)

Abstract: With the rapid advancement and widespread application of vision-language pre-training (VLP) models, their vulnerability to adversarial attacks has become a critical concern. In general, the adversarial examples can typically be designed to exhibit transferable power, attacking not only different models but also across diverse tasks. However, existing attacks on language-vision models mainly rely on static cross-modal interactions and focus solely on disrupting positive image-text pairs, resulting in limited cross-modal disruption and poor transferability. To address this issue, we propose a Semantic-Augmented Dynamic Contrastive Attack (SADCA) that enhances adversarial transferability through progressive and semantically guided perturbation. SADCA progressively disrupts cross-modal alignment through dynamic interactions between adversarial images and texts. This is accomplished by SADCA establishing a contrastive learning mechanism involving adversarial, positive and negative samples, to reinforce the semantic inconsistency of the obtained perturbations. Moreover, we empirically find that input transformations commonly used in traditional transfer-based attacks also benefit VLPs, which motivates a semantic augmentation module that increases the diversity and generalization of adversarial examples. Extensive experiments on multiple datasets and models demonstrate that SADCA significantly improves adversarial transferability and consistently surpasses state-of-the-art methods. The code is released at https://github.com/LiYuanBoJNU/SADCA.
CC-VQA: Conflict- and Correlation-Aware Method for Mitigating Knowledge Conflict in Knowledge-Based Visual Question Answering
Authors: Yuyang Hong; Jiaqi Gu; Yujing Lou; Lubin Fan; Qi Yang; Ying Wang; Kun Ding; Yue Wu; Shiming Xiang; Jieping Ye
Venue: CVPR 2026 (main)

Abstract: Knowledge-based visual question answering (KB-VQA) demonstrates significant potential for handling knowledge-intensive tasks. However, conflicts arise between static parametric knowledge in vision language models (VLMs) and dynamically retrieved information due to the static model knowledge from pre-training. The outputs either ignore retrieved contexts or exhibit inconsistent integration with parametric knowledge, posing substantial challenges for KB-VQA. Current knowledge conflict mitigation methods primarily adapted from language-based approaches, focusing on context-level conflicts through engineered prompting strategies or context-aware decoding mechanisms. However, these methods neglect the critical role of visual information in conflicts and suffer from redundant retrieved contexts, which impair accurate conflict identification and effective mitigation. To address these limitations, we propose CC-VQA: a novel training-free, conflict- and correlation-aware method for KB-VQA. Our method comprises two core components: (1) Vision-Centric Contextual Conflict Reasoning, which performs visual-semantic conflict analysis across internal and external knowledge contexts; and (2) Correlation-Guided Encoding and Decoding, featuring positional encoding compression for low-correlation statements and adaptive decoding using correlation-weighted conflict scoring. Extensive evaluations on E-VQA, InfoSeek, and OK-VQA benchmarks demonstrate that CC-VQA achieves state-of-the-art performance, yielding absolute accuracy improvements of 3.3% to 6.4% compared to existing methods. Code will be released upon acceptance.
Ghosts in the Point Clouds: De-glaring LiDAR in the Transient Domain
Authors: Avery Gump; Connor Henley; Sungjin Cheong; Akarsh Prabhakara; Mohit Gupta
Venue: CVPR 2026 (main)

Abstract: Modern LiDARs are rapidly transitioning from bulky, mechanically scanned systems to ultra-compact, low-cost, solid-state arrays. This miniaturization--while enabling scalability, affordability, and camera-like data structures--introduces a new and severe failure mode: internal-multipath glare. When light from a bright or retroreflective surface reflects and scatters within the LiDAR, light that should reach a single pixel spreads across the pixel array. The resulting artifacts create phantom objects, obscure real ones, and produce safety-critical "ghosts in the point clouds." This paper introduces a physically grounded sensing model and algorithmic techniques for addressing this effect. We show that internal glare can be represented as a linear, scene-independent operator--the Transient Glare Spread Function (TGSF)--acting on the transient measurements. Building on this model, we develop a training-free approach that operates on low-level LiDAR detections (or echoes) prior to point-cloud formation, leveraging knowledge of the glare spread function to reason about the likelihood of each detection arising from glare. The resulting approach is compatible with existing LiDAR signal-processing pipelines, and deployable on unmodified commercial sensors. Using experiments with real single-photon LiDAR hardware, we demonstrate substantial suppression of severe glare artifacts while preserving true scene structure.
OneOcc: Semantic Occupancy Prediction for Legged Robots with a Single Panoramic Camera
Authors: Hao Shi; Ze Wang; Shangwei Guo; Mengfei Duan; Song Wang; Teng Chen; Kailun Yang; Lin Wang; Kaiwei Wang
Venue: CVPR 2026 (main)

Abstract: Robust 3D semantic occupancy is essential for legged and humanoid robots, yet most Semantic Scene Completion (SSC) systems are built for wheeled platforms with forward-facing sensors. We present OneOcc, a vision-only panoramic SSC framework tailored to severe body jitter and 360deg continuity. OneOcc integrates four complementary modules: (i) Dual-Projection fusion (DP-ER), which jointly exploits the raw annular panorama and its equirectangular unfolding to preserve true 360deg continuity while enabling grid-aligned feature extraction and seam-aware context; (ii) Bi-Grid Voxelization (BGV), which reasons in Cartesian and polar/cylindrical voxel spaces to reduce discretization bias and better align with panoramic geometry, yielding sharper free/occupied boundaries; (iii) a lightweight decoder with Hierarchical AMoE-3D fusion that dynamically routes multi-scale 3D features to specialized experts, improving long-range context and occlusion handling; and (iv) a plug-and-play Gait Displacement Compensation (GDC) module that learns feature-level motion correction from gait, stabilizing representations without extra sensors. We also release two panoramic occupancy benchmarks: QuadOcc (real quadruped, first-person 360deg) and Human360Occ (H3O) (CARLA human-ego 360deg with RGB/Depth/semantic-occupancy and standardized within-/cross-city splits). OneOcc sets new SOTA: on QuadOcc it exceeds strong vision baselines and even popular LiDAR methods, and on H3O it improves within-city by +3.83 mIoU and cross-city by +8.08. The modules are lightweight, enabling deployable full-surround semantic perception for legged and humanoid robots.
ModularAgent: A Task-Aware Modular Framework for Joint Optimization of Multimodal Large Language Models and World Models
Authors: Yu-Wei Zhan; Xin Wang; Pengzhe Mao; Tongtong Feng; Ren Wang; Wenwu Zhu
Venue: CVPR 2026 (main)

Abstract: Building generalist embodied agents requires a unified system that can interpret multimodal goals, model environment dynamics, and execute reliable actions across diverse real-world tasks. Multimodal large language models (MLLMs) offer strong semantic priors and cross-modal generalization, while world models (WMs) provide actionable latent dynamics for prediction and control. Their combination holds promise for open-ended embodied intelligence, yet introduces two key challenges: (1) establishing a tight coupling between the semantic intent from MLLMs and the dynamic state representations within the WM's latent space, and (2) achieving task-aware adaptability that supports multi-task learning and cross-environment generalization. To address these limitations, we propose ModularAgent, a task-aware dynamic joint framework that enables bidirectional coupling between MLLMs and WMs. ModularAgent establishes two complementary pathways: a forward path that injects MLLM representations into the WMs latent space for semantically guided imagination, and a backward path where WM-generated feedback refines the MLLMs semantic space via dense text-conditioned rewards. This bidirectional interaction is realized through three synergistic components: Task-Aware Dynamic Joint Learning, Task-Aware Behavior Learning, and MLLM-WM Joint Optimization, which together harmonize semantic reasoning and dynamic prediction. Extensive experiments across multi-task and cross-environment settings demonstrate superior stability and generalization over state-of-the-art baselines, marking a step toward open-ended embodied learning.
Bridging Domains through Subspace-Aware Model Merging
Authors: Levy Chaves; Chao Zhou; Rebekka Burkholz; Eduardo Valle; Sandra Avila
Venue: CVPR 2026 (main)

Abstract: Model merging integrates multiple task-specific models into a single consolidated one. Recent research has made progress in improving merging performance for in-distribution or multi-task scenarios, but domain generalization in model merging remains underexplored. We investigate how merging models fine-tuned on distinct domains affects generalization to unseen domains. Through an analysis of parameter competition in the task matrix using singular value decomposition, we show that merging models trained under different distribution shifts induces stronger conflicts between their subspaces compared to traditional multi-task settings. To mitigate this issue, we propose SCORE (Subspace COnflict-Resolving mErging), a method designed to alleviate such singular subspace conflicts. SCORE finds a shared orthogonal basis by computing the principal components of the concatenated leading singular vectors of all models. It then projects each task matrix into the shared basis, pruning off-diagonal components to remove conflicting singular directions. SCORE consistently outperforms, on average, existing model merging approaches in domain generalization settings across a variety of architectures and model scales, demonstrating its effectiveness and scalability.
Dynamic Magic: Unleashing Restricted Knowledge for Lifelong Person Re-Identification
Authors: Jinjia Peng; Jican Tan; Jiazuo Yu; Zeze Tao; Huibing Wang
Venue: CVPR 2026 (main)

Abstract: Lifelong Person Re-Identification aims to adapt to new domains while preserving old knowledge. Existing methods, whether distillation-based or rehearsal-based, attempt to consolidate diverse knowledge within a fixed model architecture. However, the limited adaptability of such architectures often leads to catastrophic forgetting by overwriting previously acquired knowledge. To overcome this limitation, we propose Versatile Incremental Adaptation (VIA), a novel dynamical expansion framework for LReID, which unleashes restricted knowledge during continuous learning by large pre-trained models. Specifically, Unseen-domain person Adapter (UnA) is embedded in VIA, which employs incremental modular learning to capture the domain's specific knowledge, thereby reducing cross-domain interference and releasing task-specific capacity that is previously limited by static parameter sharing. Meanwhile, considering the substantial amount of sharing knowledge across domains in LReID, we design the Domain-aware Dispatch module to enable inter-domain collaboration and knowledge reuse through adaptive cooperation among multiple shared adapters. Besides, a Holistic Domain Controller is designed to dynamically regulate the learning capacity for new domains based on knowledge similarity, thereby effectively unleashing the generalization potential of pre-trained models. Furthermore, a lightweight Similarity-Guided Auto-Selector is proposed to assign inputs to relevant adapters during inference automatically. Extensive experiments validate VIA's effectiveness, surpassing state-of-the-art methods in both seen and unseen domains.
Dynamic Black-hole Emission Tomography with Physics-informed Neural Fields
Authors: Berthy T. Feng; Andrew A. Chael; David Bromley; Aviad Levis; William T. Freeman; Katherine L. Bouman
Venue: CVPR 2026 (main)

Abstract: With the success of static black-hole imaging, the next frontier is the dynamic and 3D imaging of black holes. Recovering the dynamic 3D gas near a black hole would reveal previously-unseen parts of the universe and inform new physics models. However, only sparse radio measurements from a single viewpoint are possible, making the dynamic 3D reconstruction problem significantly ill-posed. Previously, BH-NeRF addressed the ill-posed problem by assuming Keplerian dynamics of the gas, but this assumption breaks down near the black hole, where the strong gravitational pull of the black hole and increased electromagnetic activity complicate fluid dynamics. To overcome the restrictive assumptions of BH-NeRF, we propose PI-DEF, a physics-informed approach that uses differentiable neural rendering to fit a 4D (time + 3D) emissivity field given EHT measurements. Our approach jointly reconstructs the 3D velocity field with the 4D emissivity field and enforces the velocity as a soft constraint on the dynamics of the emissivity. In experiments on simulated data, we find significantly improved reconstruction accuracy over both BH-NeRF and a physics-agnostic approach. We demonstrate how our method may be used to estimate other physics parameters of the black hole, such as its spin.
Spatial-Aware VLA Pretraining through Visual-Physical Alignment from Human Videos
Authors: Yicheng Feng; Wanpeng Zhang; Ye Wang; Hao Luo; Haoqi Yuan; Sipeng Zheng; Zongqing Lu
Venue: CVPR 2026 (main)

Abstract: Vision-Language-Action (VLA) models provide a promising paradigm for robot learning by integrating visual perception with language-guided policy learning. However, most existing approaches rely on 2D visual inputs to perform actions in 3D physical environments, creating a significant gap between perception and action grounding. To bridge this gap, we propose a Spatial-Aware VLA Pretraining paradigm that enables models to acquire 3D spatial understanding before robot policy learning. Starting from pretrained vision-language models, we leverage large-scale human demonstration videos to extract 3D visual and 3D action annotations, forming a new source of supervision that aligns 2D visual observations with 3D spatial reasoning. We instantiate this paradigm with VIPA-VLA, a dual-encoder architecture that incorporates a 3D visual encoder to augment semantic visual representations with 3D-aware features, and aligns the two through visual-physical alignment pretraining. When adapted to downstream robot tasks, VIPA-VLA achieves significantly improved grounding between 2D vision and 3D action, resulting in more robust and generalizable robotic policies.
PnP-CM: Consistency Models as Plug-and-Play Priors for Inverse Problems
Authors: Merve Gulle; Junno Yun; Yasar Utku Alcalar; Mehmet Akcakaya
Venue: CVPR 2026 (main)

Abstract: Diffusion models have found extensive use in solving inverse problems, by sampling from an approximate posterior distribution of data given the measurements. Recently, consistency models (CMs) have been proposed to directly predict the final output from any point on the diffusion ODE trajectory, enabling high-quality sampling in just a few neural function evaluations (NFEs). CMs have also been utilized for inverse problems, but existing CM-based solvers either require additional task-specific training or utilize data fidelity operations with slow convergence, limiting their applicability to large-scale problems and making them difficult to extend to nonlinear settings. In this work, we reinterpret CMs as proximal operators of a prior, enabling their integration into plug-and-play (PnP) frameworks. Specifically, we propose PnP-CM, an ADMM-based PnP solver that provides a unified framework for solving a wide range of inverse problems, and incorporates noise perturbations and momentum-based updates to improve performance in the low-NFE regime. We evaluate our approach on a diverse set of linear and nonlinear inverse problems. We also train and apply CMs to MRI data for the first time. Our results show that PnP-CM achieves high-quality reconstructions in as few as 4 NFEs, and produces meaningful results in 2 steps, highlighting its effectiveness in real-world inverse problems while outperforming existing  CM-based approaches.
Remote Sensing Image Super-Resolution for Imbalanced Textures: A Texture-Aware Diffusion Framework
Authors: Enzhuo Zhang; Sijie Zhao; Dilxat Muhtar; Zhenshi Li; Xueliang Zhang; Pengfeng Xiao
Venue: CVPR 2026 (main)

Abstract: Generative diffusion priors have recently achieved state-of-the-art performance in natural image super-resolution, demonstrating a powerful capability to synthesize photorealistic details. However, their direct application to remote sensing image super-resolution (RSISR) reveals significant shortcomings. Unlike natural images, remote sensing images exhibit a unique texture distribution where ground objects are globally stochastic yet locally clustered, leading to highly imbalanced textures. This imbalance severely hinders the model's spatial perception. To address this, we propose TexADiff, a novel framework that begins by estimating a Relative Texture Density Map (RTDM) to represent the texture distribution. TexADiff then leverages this RTDM in three synergistic ways: as an explicit spatial conditioning to guide the diffusion process, as a loss modulation term to prioritize texture-rich regions, and as a dynamic adapter for the sampling schedule. These modifications are designed to endow the model with explicit texture-aware capabilities. Experiments demonstrate that TexADiff achieves superior or competitive quantitative metrics. Furthermore, qualitative results show that our model generates faithful high-frequency details while effectively suppressing texture hallucinations. This improved reconstruction quality also results in significant gains in downstream task performance. The source code of our method can be found at https://github.com/ZezFuture/TexAdiff.
Hist2Style: Histogram-Guided Stylization with Bilateral Grids
Authors: Dekel Galor; Adam Pikielny; Zhoutong Zhang; Ke Wang; Laura Waller; Jiawen Chen; Ilya Chugunov
Venue: CVPR 2026 (main)

Abstract: Photorealistic style transfer aims to match the color and tone of an input image to that of a style target while preserving the content and details of the original scene. Although existing large image models can facilitate these kinds of appearance edits, their high computational demands, potential for hallucinations, and limited user control make them unsuitable for high-resolution, real-time workflows. We introduce Hist2Style, a bilateral-grid formulation for fast, edge-aware stylization that preserves visual fidelity by constraining operations to locally affine transforms in bilateral space. Our model distills a large image editing model into a lightweight network by training on a large supervised corpus generated with language and vision-language models, targeting spatially varying color edits. The network conditions on a histogram-based embedding of the style target to provide an interpretable interface for adjusting the output style by modifying the target color distribution. Overall, Hist2Style maintains content structure by construction, avoids hallucinations, and supports real-time, high-resolution photorealistic stylization with interactive user-controllable color and tone adjustments. Our project page is available at \href https://dgalor.github.io/hist2style/  dgalor.github.io/hist2style/ .