GenAI4ELab/papercli-papers huggingface dataset:
CodeV: Code with Images for Faithful Visual Reasoning via Tool-Aware Policy Optimization
Authors: Xinhai Hou; Shaoyuan Xu; Manan Biyani; Moyan Li; Jia Liu; Todd C Hollon; Bryan Wang
Venue: CVPR 2026 (main)

Abstract: Agentic vision-language models are increasingly trained to "think with images" by calling image operations. However, we show that high final-answer accuracy often hides unfaithful visual reasoning: models may invoke tools on irrelevant regions or ignore tool outputs entirely, yet still guess the correct answer. In this work, we first propose a faithfulness evaluation protocol that measures whether intermediate visual tool outputs (e.g., crops) actually contain the queried evidence. This reveals that recent visual agents achieve high final-answer accuracy but exhibit low rates of faithful tool-use on visual search benchmarks. We then introduce CodeV, a code-based visual agent trained with Tool-Aware Policy Optimization (TAPO). TAPO is a process-level RL framework that augments GRPO with dense rewards defined directly on visual tool inputs and outputs, rather than on chain-of-thought tokens, making supervision easier to verify and less susceptible to reward hacking. CodeV represents visual tools as executable Python code, and TAPO assigns step-wise rewards based solely on the question and tool output, encouraging both necessary and evidence-consistent tool use. In a two-stage SFT+RL pipeline, CodeV achieves competitive or superior accuracy while substantially increasing faithful tool-use rates on related visual search benchmarks. Beyond visual search, CodeV attains strong performance on a range of multimodal reasoning and math benchmarks, suggesting that explicitly supervising intermediate tool behavior is crucial for building trustworthy, agentic visual reasoning systems.
Neural Differentiation in Deep Networks: A Theoretical Framework for Expressivity and Representational Diversity
Authors: Boyuan Wang; Richard Jiang
Venue: CVPR 2026 (main)

Abstract: We begin by developing a mathematical framework of neural differentiation, formulated at the level of individual neurons. This framework formalizes the principle that each neuron should acquire a distinct representational role within the network, thereby avoiding redundancy and maximizing collective expressivity. Differentiation is quantified through the Neural Differentiation Index (NDI), a loss-aware measure that characterizes neuron significance from geometric, informational, and curvature-based perspectives within a unified framework. The NDI enables a rigorous characterization of how strongly a neuron diverges from its peers in both function and importance, and supports theoretical guarantees: we establish formal bounds on the error increase under NDI-guided elimination, thereby providing provable safety margins for network compression. Building on this foundation, we introduce Neural Differentiation Pruning (NDP) as a practical instantiation. NDP leverages NDI to perform adaptive, training-time neuron sparsification, followed by targeted fine-tuning, guiding networks toward compact yet highly differentiated backbones. Although the terminology draws loose intuition from biological differentiation, the framework is fully mathematical and architecture-agnostic. Experiments on modern vision benchmarks and architectures show that NDP achieves substantial structured sparsity while maintaining--or even improving--accuracy and robustness, underscoring the practical impact of the differentiation framework.
STAR-R1: Multi-View Spatial TrAnsformation Reasoning by Reinforcing Multimodal LLMs
Authors: Zongzhao Li; Zongyang Ma; Mingze Li; Songyou Li; Yu Rong; Tingyang Xu; Ziqi Zhang; Deli Zhao; Wenbing Huang
Venue: CVPR 2026 (main)

Abstract: Multimodal Large Language Models (MLLMs) remain far from human-level performance in multi-view spatial reasoning, where models must establish object correspondences across view and infer coherent scene semantics. We analyze this limitation through the Transformation-Driven Visual Reasoning (TVR) task and find that Supervised Fine-Tuning (SFT) fails to capture cross-view consistency, whereas reinforcement learning (RL) fails to reliably identify key referential objects. To bridge this gap, we introduce multi-View Spatial TrAnsformation Reasoning (STAR-R1), a two-stage framework that combines process-supervised SFT with a referential-aware RL paradigm. STAR-R1 first learns structured spatial reasoning trajectories from high-quality CoTs and then uses fine-grained rewards on referential selection and answer correctness to encourage effective exploration and robust scene interpretation. Despite using only a small amount of high-quality training data, STAR-R1 surpasses state-of-the-art models with far more training data on the multi-view spatial understanding benchmarks TVR, MMSI-Bench, MindCube-Bench, and SPAR-Bench. Our study reveals the overlooked potential of RL in multi-view spatial understanding and points a way toward potentially achieving more human-like spatial reasoning in MLLMs.
Mario: Multimodal Graph Reasoning with Large Language Models
Authors: Yuanfu Sun; Kang Li; Pengkang Guo; Jiajin Liu; Qiaoyu Tan
Venue: CVPR 2026 (main)

Abstract: Recent advances in large language models (LLMs) have opened new avenues for multimodal reasoning. Yet, most existing methods still rely on pretrained vision-language models (VLMs) to encode image-text pairs in isolation, ignoring the relational structure that real-world multimodal data naturally form. This motivates reasoning on multimodal graphs (MMGs), where each node has textual and visual attributes and edges provide structural cues. Enabling LLM-based reasoning on such heterogeneous multimodal signals while preserving graph topology introduces two key challenges: resolving weak cross-modal consistency and handling heterogeneous modality preference. To address this, we propose Mario, a unified framework that simultaneously resolves the two above challenges and enables effective LLM-based reasoning over MMGs. Mario consists of two innovative stages. Firstly, a graph-conditioned VLM design that jointly refines textual and visual features through fine-grained cross-modal contrastive learning guided by graph topology. Secondly, a modality-adaptive graph instruction tuning mechanism that organizes aligned multimodal features into graph-aware instruction views and employs a learnable router to surface, for each node and its neighborhood, the most informative modality configuration to the LLM. Extensive experiments across diverse MMG benchmarks demonstrate that Mario consistently outperforms state-of-the-art graph models in both supervised and zero-shot scenarios for node classification and link prediction. The code will be made available at https://github.com/sunyuanfu/Mario.
ShapeAR: Generating Editable Shape Layers via Autoregressive Diffusion
Authors: Souymodip Chakraborty; Ankur Singh; Amit Vikram Singh; Vineet Batra; Ankit Phogat
Venue: CVPR 2026 (main)

Abstract: We present ShapeAR, a novel autoregressive latent diffusion framework that decomposes raster images into editable, artist-like vector shape layers. Unlike conventional raster-to-SVG methods that rely on boundary tracing or joint path optimisation, ShapeAR generates non-overlapping RGBA shape layers directly in latent space via flow-matching diffusion. To scale generation to complex scenes with many shapes, we formulate the process auto-regressively, conditioning each step on both the input image (global context) and partial composition of previously generated layers (local context). In addition, we propose geometry-aware evaluation metrics that quantify aesthetic and structural quality of the generated shapes.
MRI Contrast Enhancement Kinetics World Model
Authors: Jindi Kong; Yuting He; Cong Xia; Rongjun Ge; Shuo Li
Venue: CVPR 2026 (main)

Abstract: Clinical MRI contrast acquisition suffers from inefficient information yield, which presents as a mismatch between the risky and costly acquisition protocol and fixed and sparse acquisition sequence. Applying world models to simulate the contrast enhancement kinetics in human body enables continuous contrast-free dynamics. However, the low temporal resolution in MRI acquisition restricts the training of world models, leading to the sparsely sampled dataset. Directly training a generative model to capture the kinetics leads to two limitations: (a) Due to the absence data on missing time, the model tends to overfit to irrelevant features, leading to content distortion. (b) Due to the lack of continuous temporal supervision, the model fails to learn the continuous kinetics law over time, causing temporal discontinuities. For the first time, we propose MRI Contrast Enhancement Kinetics World model (MRI CEKWorld) with SpatioTemporal Consistency Learning (STCL). For (a), guided by spatial law that patient-level structures remain consistent during enhancement, we propose Latent Alignment Learning (LAL) that constructs a patient-specific template to constrain contents to align with this template. For (b), guided by the temporal law that the kinetics follows a consistent smooth trend, we propose Latent Difference Learning (LDL) which extends the unobserved intervals by interpolation and constrain smooth variations in the latent space among interpolated sequence. Extensive experiments on two datasets show our MRI CEKWorld achieves better realistic contents and kinetics. Codes will be available.
L3DR: 3D-aware LiDAR Diffusion and Rectification
Authors: Quan Liu; Xiaoqin Zhang; Ling Shao; Shijian Lu
Venue: CVPR 2026 (main)

Abstract: Range-view (RV) based LiDAR diffusion has recently made huge strides towards 2D photo-realism. However, it neglects 3D geometry realism and often generates various RV artifacts such as depth bleeding and wavy surfaces. We design L3DR, a 3D-aware LiDAR Diffusion and Rectification framework that can regress and cancel RV artifacts in 3D space and restore local geometry accurately. Our theoretical and empirical analysis reveals that 3D models are inherently superior to 2D models in generating sharp and authentic boundaries. Leveraging such analysis, we design a 3D residual regression network that rectifies RV artifacts and achieves superb geometry realism by predicting point-level offsets in 3D space. On top of that, we design a Welsch Loss that helps focus on local geometry and ignore anomalous regions effectively. Extensive experiments over multiple benchmarks including KITTI, KITTI360, nuScenes and Waymo show that the proposed L3DR achieves state-of-the-art generation and superior geometry-realism consistently. In addition, L3DR is generally applicable to different LiDAR diffusion models with little computational overhead.
Semantic Derivative Flow: Graph-Guided Diffusion for Controllable Instance Interactions
Authors: Shibin Mei; Hang Wang; Bingbing Ni
Venue: CVPR 2026 (main)

Abstract: Despite remarkable progress in text-to-image diffusion models, controlling the semantic and spatial relationships between interacting instances remains a fundamental challenge. Current methods that inject spatial constraints often fail to model the intrinsic functional dependencies between entities, leading to implausible interactions. In this paper, we introduce Semantic Derivative Flow (SDF), a novel graph-guided framework that structures the diffusion process within a directed acyclic interaction graph. Our core innovation is a theoretically-motivated derivative attention mechanism, which explicitly enforces the semantic representation of a predicate to be derived from its subject, and the object from the predicate, formalizing a differentiable semantic graph. This principled approach compels the generative process to adhere to the logical chain of interaction. We further integrate a global context node and a real-time regional refinement module to ground the graph in the visual domain holistically. Extensive experiments demonstrate that our model, an instantiation of SDF, establishes a new state-of-the-art in fidelity and controllability on the HICODet benchmark. We complement our empirical results with a theoretical analysis, framing our method as structured message passing on interaction graphs, which provides a rigorous justification for its efficacy and generalization benefits.
Towards Stable Federated Continual Test-Time Adaptation in Wild World
Authors: Liwen Wang; Xingbo Dong; Iman Yi Liao; Zhe Jin
Venue: CVPR 2026 (main)

Abstract: Federated Learning (FL) enables collaborative model training while preserving privacy, but faces challenges with client data heterogeneity and domain shifts during deployment. Although Personalized Federated Learning (PFL) mitigates heterogeneity, it typically requires labelled data from target clients, which is an impractical assumption. Test-Time Adaptation (TTA) offers label-free adaptation, yet its direct use in a continual federated setting risks destabilizing the global model and causing catastrophic forgetting. To address this, we consider the Federated Continual Test-Time Adaptation (FedCTTA) setting, where unlabeled clients arrive sequentially, requiring online adaptation and continuous global model updates. We propose BPFedCTTA, a framework that employs Bayesian Prior-guided Adaptation (BPA) for stable local adaptation via Maximum a Posteriori estimation, and Uncertainty-Gated Single-client Aggregation (UGSA) to selectively integrate updates based on client uncertainty. This approach balances adaptation with knowledge retention, thereby mitigating forgetting. Extensive experiments on cross-domain classification and segmentation show BPFedCTTA outperforms existing FL, PFL, and TTA methods in sequential adaptation and global model improvement. The source code will be made public upon acceptance.
Beyond Ground-Truth: Leveraging Image Quality Priors for Real-World Image Restoration
Authors: Fengyang Xiao; Peng Hu; Lei Xu; XingE Guo; Guanyi Qin; Yuqi Shen; Chengyu Fang; Rihan Zhang; Chunming He; Sina Farsiu
Venue: CVPR 2026 (main)

Abstract: Real-world image restoration aims to restore high-quality (HQ) images from degraded low-quality (LQ) inputs captured under uncontrolled conditions. Existing methods typically depend on ground-truth (GT) supervision, assuming that GT provides perfect reference quality. However, GT can still contain images with inconsistent perceptual fidelity, causing models to converge to the average quality level of the training data rather than achieving the highest perceptual quality attainable. To address these problems, we propose a novel framework, termed IQPIR, that introduces an Image Quality Prior (IQP)--extracted from pre-trained No-Reference Image Quality Assessment (NR-IQA) models--to guide the restoration process toward perceptually optimal outputs explicitly. Our approach synergistically integrates IQP with a learned codebook prior through three key mechanisms:(1) a quality-conditioned Transformer, where NR-IQA-derived scores serve as conditioning signals to steer the predicted representation toward maximal perceptual quality. This design provides a plug-and-play enhancement compatible with existing restoration architectures without structural modification; and(2) a dual-branch codebook structure, which disentangles common and HQ-specific features, ensuring a comprehensive representation of both generic structural information and quality-sensitive attributes; and (3)  a discrete representation-based quality optimization strategy, which mitigates over-optimization effects commonly observed in continuous latent spaces. Extensive experiments on real-world image restoration demonstrate that our method not only surpasses cutting-edge methods but also serves as a generalizable quality-guided enhancement strategy for existing methods. The code is available at https://github.com/fengyang1399-pixel/IQPIR.
MindDriver: Introducing Progressive Multimodal Reasoning for Autonomous Driving
Authors: Lingjun Zhang; Yujian Yuan; Changjie Wu; Xinyuan Chang; Xin Cai; Shuang Zeng; Linzhe Shi; Sijin Wang; Hang Zhang; Mu Xu
Venue: CVPR 2026 (main)

Abstract: Vision-Language Models (VLM) exhibit strong reasoning capabilities, showing promise for end-to-end autonomous driving systems. Chain-of-Thought (CoT), as VLM's widely used reasoning strategy, is facing critical challenges. Existing textual CoT has a large gap between text semantic space and trajectory physical space. Although the recent approach utilizes future image to replace text as CoT process, it lacks clear planning-oriented objective guidance to generate images with accurate scene evolution. To address these, we innovatively propose MindDriver, a progressive multimodal reasoning framework that enables VLM to imitate human-like progressive thinking for autonomous driving.MindDriver presents semantic understanding, semantic-to-physical space imagination, and physical-space trajectory planning.To achieve aligned reasoning processes in MindDriver, we develop a feedback-guided automatic data annotation pipeline to generate aligned multimodal reasoning training data. Furthermore, we develop a progressive reinforcement fine-tuningmethod to optimize the alignment through progressive high-level reward-based learning.MindDriver demonstrates superior performance in both nuScences open-loop and Bench2Drive closed-loop evaluation. Codes:  https://github.com/hotdogcheesewhite/MindDriver.
Hybrid Token Compression for Vision-Language Models
Authors: Jusheng Zhang; Xiaoyang Guo; Kaitong Cai; Qinhan Lv; Yijia Fan; Wenhao Chai; Jian Wang; Keze Wang
Venue: CVPR 2026 (main)

Abstract: Vision-language models (VLMs) have transformed multimodal reasoning, but feeding hundreds of visual patch tokens to LLMs incurs quadratic computational costs, straining memory and context windows. Traditional approaches face a trade-off: continuous compression dilutes high-level semantics like object identities, while discrete quantization loses granular details such as textures. We challenge this by introducing **HTC-VLM**, a hybrid framework that disentangles semantics and appearance through dual channels, i.e., a continuous pathway for fine-grained details via ViT patches and a discrete pathway for symbolic anchors using MGVQ quantization projected to four tokens. These are fused into a 580-token hybrid sequence and compressed to one token via a disentanglement attention mask and a `<voco>` bottleneck, ensuring efficient, grounded representations.HTC-VLM achieves an average performance retention of **87.2%** across seven benchmarks (GQA, VQAv2, MMBench, MME, POPE, SEED-Bench, ScienceQA-Image), outperforming the leading continuous baseline at **81.0%** with a 580-to-1 compression ratio. Attention analyses show the compressed token prioritizes the discrete anchor, validating its semantic guidance. Our work demonstrates that a minimalist hybrid can resolve the efficiency-fidelity dilemma, advancing scalable VLMs.
Trust-calibrated Collaborative Learning for Long-Tailed Visual Recognition
Authors: Hao Zhou; Tingjin Luo
Venue: CVPR 2026 (main)

Abstract: Real-world visual recognition faces the fundamental challenge of long-tailed distributions. While state-of-the-art methods often employ multi-expert models to address different frequency categories, we find that the mutual knowledge distillation used in these models enhances collaboration at the cost of introducing two critical limitations: indiscriminate knowledge transfer leads to bias propagation, where a single expert's error can spread and contaminate others, and error consolidation, where mutual reinforcement of incorrect predictions solidifies erroneous consensus. To overcome these issues, we propose Trust-calibrated Collaborative Learning (TCL). Our framework introduces the trustworthy knowledge orchestration module, which enables reliable distillation and precise collaboration through a knowledge quality gate that blocks erroneous information and a tail-class compensation mechanism that alleviates knowledge scarcity for tail categories. Furthermore, we design a consensus error calibration module that suppresses consensus high-confidence negative classes to correct collective misjudgments and steer optimization in the right direction. Extensive experiments on five long-tailed benchmarks demonstrate that TCL achieves the best performance, raising Top-1 accuracy on CIFAR100-LT to 58.7%, a gain of 2.4% over previous SOTA methods.
STCast: Adaptive Boundary Alignment for Global and Regional Weather Forecasting
Authors: Hao Chen; Tao Han; Jie Zhang; Song Guo; Lei Bai
Venue: CVPR 2026 (main)

Abstract: To gain finer regional forecasts, many works have explored the regional integration from the global atmosphere, e.g., by solving boundary equations in physics-based methods or cropping regions from global forecasts in data-driven methods. However, the effectiveness of these methods is often constrained by static and imprecise regional boundaries, resulting in poor generalization ability. To address this issue, we propose Spatial-Temporal Weather Forecasting (STCast), a novel AI-driven framework for adaptive regional boundary optimization and dynamic monthly forecast allocation. Specifically, our approach employs a Spatial-Aligned Attention (SAA) mechanism, which aligns global and regional spatial distributions to initialize boundaries and adaptively refines them based on attention-derived alignment patterns. Furthermore, we design a Temporal Mixture-of-Experts (TMoE) module, where atmospheric variables from distinct months are dynamically routed to specialized experts using a discrete Gaussian distribution, enhancing the model's ability to capture temporal patterns. Beyond global and regional forecasting, STCast is evaluated on extreme event prediction and ensemble forecasting. Experimental results demonstrate consistent superiority over other methods across all four tasks. Code: https://github.com/chenhao-zju/STCast
Parameter-Efficient Adaptation for MLLMs via Implicit Modality Decomposition
Authors: Mingfang Zhang; Yunhong Wang; Lu Wang; Jiaxin Chen
Venue: CVPR 2026 (main)

Abstract: Parameter-efficient fine-tuning (PEFT) has become a compelling approach for adapting large language models (LLMs) into multimodal large language models (MLLMs), enabling them to handle diverse modalities with substantially lower memory and computational costs. However, most existing PEFT methods neglect the issue of modality-imbalanced learning, which is characterized by the excessive dominance of text modality in updating parameters, thus incurring insufficient learning of non-text modalities and leading to performance degradation. To address this issue, we propose a novel parameter-efficient adaptation method for MLLMs, namely Implicit Modality Decomposition (IMoD), based on LoRA. It firstly decomposes the learnable parameters into the non-overlapped text-specific, non-text-specific and modality-sharing components, thereby alleviating modality imbalance. To further guide the optimization of these components toward specific modalities, we propose Modality-Specific Decoupling Constraint that suppresses cross-modal interference among modality-specific parameters, and Modality-Agnostic Alignment Constraint that encourages modality-sharing component to capture well-aligned, modality-invariant semantics. Extensive experiments across diverse multimodal settings and LLM architectures demonstrate that our method consistently delivers significant performance gains, particularly achieving an averaged 3.3% improvements on the audio-visual-text tasks without sacrificing the parameter and inference efficiency. Code is available at https://github.com/mmffzzz/IMoD.git.