GenAI4ELab/papercli-papers huggingface dataset:
MMCR: Benchmarking Cross-Source Reasoning in Scientific Papers
Authors: Yang Tian; Zheng Lu; Mingqi Gao; Zheng Liu; Bo Zhao
Venue: ICCV 2025 (main)

Abstract: Fully comprehending scientific papers by machines reflects a high level of Artificial General Intelligence, requiring the ability to reason across fragmented and heterogeneous sources of information, presenting a complex and practically significant challenge. While Vision-Language Models (VLMs) have made remarkable strides in various tasks, particularly those involving reasoning with evidence source from single image or text page, their ability to use cross-source information for reasoning remains an open problem. This work presents MMCR, a high-difficulty benchmark designed to evaluate VLMs' capacity for reasoning with cross-source information from scientific papers. The benchmark comprises 276 high-quality questions, meticulously annotated by humans across 7 subjects and 10 task types. Experiments with 18 VLMs demonstrate that cross-source reasoning presents a substantial challenge for existing models. Notably, even the top-performing model, GPT-4o, achieved only 48.55% overall accuracy, with just 20% accuracy in multi-table comprehension tasks, while the second-best model, Qwen2.5-VL-72B, reached 39.86% overall accuracy. These results highlight the pressing need to develop VLMs capable of effectively utilizing cross-source information for reasoning.
ZeroKey: Point-Level Reasoning and Zero-Shot 3D Keypoint Detection from Large Language Models
Authors: Bingchen Gong; Diego Gomez; Abdullah Hamdi; Abdelrahman Eldesokey; Ahmed Abdelreheem; Peter Wonka; Maks Ovsjanikov
Venue: ICCV 2025 (main)

Abstract: We propose a novel zero-shot approach for keypoint detection on 3D shapes. Point-level reasoning on visual data is challenging as it requires precise localization capability, posing problems even for powerful models like DINO or CLIP. Traditional methods for 3D keypoint detection rely heavily on annotated 3D datasets and extensive supervised training, limiting their scalability and applicability to new categories or domains. In contrast, our method utilizes the rich knowledge embedded within Multi-Modal Large Language Models (MLLMs). Specifically, we demonstrate, for the first time, that pixel-level annotations used to train recent MLLMs can be exploited for both extracting and naming salient keypoints on 3D models without any ground truth labels or supervision. Experimental evaluations demonstrate that our approach achieves competitive performance on standard benchmarks compared to supervised methods, despite not requiring any 3D keypoint annotations during training. Our results highlight the potential of integrating language models for localized 3D shape understanding. This work opens new avenues for cross-modal learning and underscores the effectiveness of MLLMs in contributing to 3D computer vision challenges.
Audio-visual Controlled Video Diffusion with Masked Selective State Spaces Modeling for Natural Talking Head Generation
Authors: Fa-Ting Hong; Zunnan Xu; Zixiang Zhou; Jun Zhou; Xiu Li; Qin Lin; Qinglin Lu; Dan Xu
Venue: ICCV 2025 (main)

Abstract: Talking head synthesis is vital for virtual avatars and human-computer interaction. However, most existing methods are typically limited to accepting control from a single primary modality, restricting their practical utility. To this end, we introduce ACTalker, an end-to-end video diffusion framework that supports both multi-signals control and single-signal control for talking head video generation. For multiple control, we design a parallel mamba structure with multiple branches, each utilizing a separate driving signal to control specific facial regions. A gate mechanism is applied across all branches, providing flexible control over video generation. To ensure natural coordination of the controlled video both temporally and spatially, we employ the mamba structure, which enables driving signals to manipulate feature tokens across both dimensions in each branch. Additionally, we introduce a mask-drop strategy that allows each driving signal to independently control its corresponding facial region within the mamba structure, preventing control conflicts. Experimental results demonstrate that our method produces natural-looking facial videos driven by diverse signals and that the mamba layer seamlessly integrates multiple driving modalities without conflict.
DreamCube: RGB-D Panorama Generation via Multi-plane Synchronization
Authors: Yukun Huang; Yanning Zhou; Jianan Wang; Kaiyi Huang; Xihui Liu
Venue: ICCV 2025 (main)

Abstract: 3D panorama synthesis is a promising yet challenging task that demands high-quality and diverse visual appearance and geometry of the generated omnidirectional content. Existing methods leverage rich image priors from pre-trained 2D foundation models to circumvent the scarcity of 3D panoramic data, but the incompatibility between 3D panoramas and 2D single views limits their effectiveness. In this work, we demonstrate that by applying multi-plane synchronization to the operators from 2D foundation models, their capabilities can be seamlessly extended to the omnidirectional domain. Based on this design, we further introduce DreamCube, a multi-plane RGB-D diffusion model for 3D panorama generation, which maximizes the reuse of 2D foundation model priors to achieve diverse appearances and accurate geometry while maintaining multi-view consistency. Extensive experiments demonstrate the effectiveness of our approach in panoramic image generation, panoramic depth estimation, and 3D scene generation.
SILO: Solving Inverse Problems with Latent Operators
Authors: Ron Raphaeli; Sean Man; Michael Elad
Venue: ICCV 2025 (main)

Abstract: Plug-and-play methods for solving inverse problems have continuously improved over the years by incorporating more advanced image priors.Latent diffusion models are among the most powerful priors, making them a natural choice for solving inverse problems. However, existing approaches require multiple applications of an Autoencoder to transition between pixel and latent spaces during restoration, leading to high computational costs and degraded restoration quality. In this work, we introduce a new plug-and-play paradigm that operates entirely in the latent space of diffusion models. By emulating pixel-space degradations directly in the latent space through a short learning phase, we eliminate the need for the Autoencoder during restoration, enabling faster inference and improved restoration fidelity. We validate our method across various image restoration tasks and datasets, achieving significantly higher perceptual quality than previous methods while being 2.6-10x faster in inference and 1.7-7x faster when accounting for the learning phase of the latent operator.
Entropy-Adaptive Diffusion Policy Optimization with Dynamic Step Alignment
Authors: RenYe Yan; Jikang Cheng; Yaozhong Gan; Shikun Sun; You Wu; Yunfan Yang; Liang Ling; Jinlong Lin; Yeshuang Zhu; Jie Zhou; Jinchao Zhang; Junliang Xing; Yimao Cai; Ru Huang
Venue: ICCV 2025 (main)

Abstract: While fine-tuning diffusion models with reinforcement learning (RL) has demonstrated effectiveness in directly optimizing downstream objectives, existing RL frameworks are prone to overfitting the rewards, leading to outputs that deviate from the true data distribution and exhibit reduced diversity. To address this issue, we introduce entropy as a quantitative measure to enhance the exploratory capacity of diffusion models' denoising policies. We propose an adaptive mechanism that dynamically adjusts the application and magnitude of entropy and regularization, guided by real-time quality estimation of intermediate noised states. Theoretically, we prove the convergence of our entropy-enhanced policy optimization and establish two critical properties: 1) global entropy increases through training, ensuring robust exploration capabilities, and 2) entropy systematically decreases during the denoising process, enabling a phase transition from early-stage diversity promotion to late-stage distributional fidelity. Building on this foundation, we propose a plug-and-play RL module that adaptively controls entropy and optimizes denoising steps. Extensive evaluations demonstrate the theoretical soundness and empirical robustness of our method, achieving state-of-the-art quality-diversity trade-offs across benchmarks. Notably, our framework significantly improves the rewards and reduces denoising steps in training by up to 40%.
ScanEdit: Hierarchically-Guided Functional 3D Scan Editing
Authors: Mohamed El Amine Boudjoghra; Ivan Laptev; Angela Dai
Venue: ICCV 2025 (main)

Abstract: With the growing ease of capture of real-world 3D scenes, effective editing becomes essential for the use of captured 3D scan data in various graphics applications.We present ScanEdit, which enables functional editing of complex, real-world 3D scans from natural language text prompts.By leveraging the high-level reasoning capabilities of large language models (LLMs), we construct a hierarchical scene graph representation for an input 3D scan given its instance decomposition. We develop a hierarchically-guided, multi-stage prompting approach using LLMs to decompose general language instructions (that can be vague, without referencing specific objects) into specific, actionable constraints that are applied to our scene graph. Our scene optimization integrates LLM-guided constraints along with 3D-based physical plausibility objectives, enabling the generation of edited scenes that align with a variety of input prompts, from abstract, functional-based goals to more detailed, specific instructions. This establishes a foundation for intuitive, text-driven 3D scene editing in real-world scenes.
DeFSS: Image-to-Mask Denoising Learning for Few-shot Segmentation
Authors: Zishu Qin; Junhao Xu; Weifeng Ge
Venue: ICCV 2025 (main)

Abstract: Deep learning algorithms are highly data-intensive, particularly for tasks requiring pixel-level annotations, such as semantic segmentation, which makes achieving pixel-level image understanding costly. Few-shot segmentation seeks to address this challenge by enabling models to segment novel objects using only a limited number of labeled support images as references. In this paper, we argue that the traditional image-to-mask decoding framework places excessive reliance on the quality of the support sample, which is prone to errors when encountering class bias. Thus, we propose a novel image-to-mask denoising learning paradigm for few-shot segmentation, transforming mask decoding into a denoising process to reduce the support reliance problem with the help of denoising diffusion models. We formulate our image-to-mask denoising learning process in two stages: an image corruption stage and a mask denoising stage. In the first stage, we introduce an adaptive image corruption method that perturbs the image based on regional semantics, motivated by the insight of perturbing data to populate low data density regions. In the second stage, we employ an in-model denoising paradigm, designing a network to facilitate support-to-query semantic propagation and mask denoising in a single forward pass. To enhance categorical discrimination for the denoising network, we incorporate discriminative attribute learning, which leverages base classes to train the model in distinguishing object categories and generalizing to novel classes. Extensive experiments and ablation studies validate the effectiveness of our approach, demonstrating that the proposed method achieves competitive performance across various benchmarks.
Bokehlicious: Photorealistic Bokeh Rendering with Controllable Apertures
Authors: Tim Seizinger; Florin-Alexandru Vasluianu; Marcos V. Conde; Zongwei Wu; Radu Timofte
Venue: ICCV 2025 (main)

Abstract: Bokeh rendering methods play a key role in creating the visually appealing, softly blurred backgrounds seen in professional photography. While recent learning-based approaches show promising results, generating realistic Bokeh with controllable strength remains challenging. Existing methods require additional inputs and suffer from unrealistic Bokeh reproduction due to reliance on synthetic data. In this work, we propose Bokehlicious, a highly efficient network that provides intuitive control over Bokeh strength through an Aperture-Aware Attention mechanism, mimicking the physical lens aperture. To further address the lack of high-quality real-world data, we present RealBokeh, a novel dataset featuring 23,000 high-resolution (24-MP) images captured by professional photographers, covering diverse scenes with varied aperture and focal length settings. Evaluations on both our new RealBokeh and established Bokeh rendering benchmarks show that Bokehlicious consistently outperforms SOTA methods while significantly reducing computational cost and exhibiting strong zero-shot generalization. Our method and dataset further extend to defocus deblurring, achieving competitive results on the RealDOF benchmark. Our code and data will be public.
Turbo2K: Towards Ultra-Efficient and High-Quality 2K Video Synthesis
Authors: Jingjing Ren; Wenbo Li; Zhongdao Wang; Haoze Sun; Bangzhen Liu; Haoyu Chen; Jiaqi Xu; Aoxue Li; Shifeng Zhang; Bin Shao; Yong Guo; Lei Zhu
Venue: ICCV 2025 (main)

Abstract: Demand for 2K video synthesis is rising with increasing consumer expectations for ultra-clear visuals.While diffusion transformers (DiTs) have demonstrated remarkable capabilities in high-quality video generation, scaling them to 2K resolution remains computationally prohibitive due to quadratic growth in memory and processing costs.In this work, we propose Turbo2K, an efficient and practical framework for generating detail-rich 2K videos while significantly improving training and inference efficiency. First, Turbo2K operates in a highly compressed latent space, reducing computational complexity and memory footprint, making high-resolution video synthesis feasible. However, the high compression ratio of the VAE and limited model size impose constraints on generative quality. To mitigate this, we introduce a knowledge distillation strategy that enables a smaller student model to inherit the generative capacity of a larger, more powerful teacher model. Our analysis reveals that, despite differences in latent spaces and architectures, DiTs exhibit structural similarities in their internal representations, facilitating effective knowledge transfer.Second, we design a hierarchical two-stage synthesis framework that first generates multi-level feature at lower resolutions before guiding high-resolution video generation. This approach ensures structural coherence and fine-grained detail refinement while eliminating redundant encoding-decoding overhead, further enhancing computational efficiency.Turbo2K achieves state-of-the-art efficiency, generating 5-second, 24fps, 2K videos with significantly reduced computational cost. Compared to existing methods, Turbo2K is up to 20xfaster for inference, making high-resolution video generation more scalable and practical for real-world applications.
Video2BEV: Transforming Drone Videos to BEVs for Video-based Geo-localization
Authors: Hao Ju; Shaofei Huang; Si Liu; Zhedong Zheng
Venue: ICCV 2025 (main)

Abstract: Existing approaches to drone visual geo-localization predominantly adopt the image-based setting, where a single drone-view snapshot is matched with images from other platforms. Such task formulation, however, underutilizes the inherent video output of the drone and is sensitive to occlusions and viewpoint disparity. To address these limitations, we formulate a new video-based drone geo-localization task and propose the Video2BEV paradigm. This paradigm transforms the video into a Bird's Eye View (BEV), simplifying the subsequent inter-platform matching process. In particular, we employ Gaussian Splatting to reconstruct a 3D scene and obtain the BEV projection. Different from the existing transform methods, e.g., polar transform, our BEVs preserve more fine-grained details without significant distortion. To facilitate the discriminative intra-platform representation learning, our Video2BEV paradigm also incorporates a diffusion-based module for generating hard negative samples. To validate our approach, we introduce UniV, a new video-based geo-localization dataset that extends the image-based University-1652 dataset. UniV features flight paths at 30^\circ and 45^\circ elevation angles with increased frame rates of up to 10 frames per second (FPS). Extensive experiments on the UniV dataset show that our Video2BEV paradigm achieves competitive recall rates and outperforms conventional video-based methods. Compared to other competitive methods, our proposed approach exhibits robustness at lower elevations with more occlusions. The code is available at: https://github.com/HaoDot/Video2BEV-Open.
Puppet-Master: Scaling Interactive Video Generation as a Motion Prior for Part-Level Dynamics
Authors: Ruining Li; Chuanxia Zheng; Christian Rupprecht; Andrea Vedaldi
Venue: ICCV 2025 (main)

Abstract: We introduce Puppet-Master, an interactive video generator that captures the internal, part-level motion of objects, serving as a proxy for modeling object dynamics universally. Given an image of an object and a set of "drags" specifying the trajectory of a few points on the object, the model synthesizes a video where the object's parts move accordingly. To build Puppet-Master, we extend a pre-trained image-to-video generator to encode the input drags. We also propose all-to-first attention, an alternative to conventional spatial attention that mitigates artifacts caused by fine-tuning a video generator on out-of-domain data. The model is fine-tuned on Objaverse-Animation-HQ, a new dataset of curated part-level motion clips obtained by rendering synthetic 3D animations. Unlike real videos, these synthetic clips avoid confounding part-level motion with overall object and camera motion. We extensively filter sub-optimal animations and augment the synthetic renderings with meaningful drags that emphasize the internal dynamics of objects. We demonstrate that Puppet-Master learns to generate part-level motions, unlike other motion-conditioned video generators that primarily move the object as a whole. Moreover, Puppet-Master generalizes well to out-of-domain real images, outperforming existing methods on real-world benchmarks in a zero-shot manner.
MMAT-1M: A Large Reasoning Dataset for Multimodal Agent Tuning
Authors: Tianhong Gao; Yannian Fu; Weiqun Wu; Haixiao Yue; Shanshan Liu; Gang Zhang
Venue: ICCV 2025 (main)

Abstract: Large Language Models (LLMs), enhanced through agent tuning, have demonstrated remarkable capabilities in Chain-of-Thought (CoT) and tool utilization, significantly surpassing the performance of standalone models. However, the multimodal domain still lacks a large-scale, high-quality agent tuning dataset to unlock the full potential of multimodal large language models. To bridge this gap, we introduce MMAT-1M, the first million-scale multimodal agent tuning dataset designed to support CoT, reflection, and dynamic tool usage. Our dataset is constructed through a novel four-stage data engine: 1) We first curate publicly available multimodal datasets containing question-answer pairs; 2) Then, leveraging GPT-4o, we generate rationales for the original question-answer pairs and dynamically integrate API calls and Retrieval Augmented Generation (RAG) information through a multi-turn paradigm; 3) Furthermore, we refine the rationales through reflection to ensure logical consistency and accuracy, creating a multi-turn dialogue dataset with both Rationale and Reflection (RR); 4) Finally, to enhance efficiency, we optionally compress multi-turn dialogues into a One-turn Rationale and Reflection (ORR) format. By fine-tuning open-source multimodal models on the MMAT-1M, we observe significant performance gains. For instance, the InternVL2.5-8B-RR model achieves an average improvement of 2.7% across eight public benchmarks and 8.8% on the RAG benchmark Dyn-VQA, demonstrating the dataset's effectiveness in enhancing multimodal reasoning and tool-based capabilities. The dataset is publicly available at https://github.com/VIS-MPU-Agent/MMAT-1M.
FlowChef: Steering of Rectified Flow Models for Controlled Generations
Authors: Maitreya Patel; Song Wen; Dimitris N. Metaxas; Yezhou Yang
Venue: ICCV 2025 (main)

Abstract: Despite recent advances in Rectified Flow Models (RFMs), unlocking their full potential for controlled generation tasks--such as inverse problems and image editing--remains a significant hurdle. Although RFMs and Diffusion Models (DMs) represent state-of-the-art approaches in generative modeling, their reliance on computationally demanding backpropagation through ODE solvers and inversion strategies often undermines efficiency and precision. In this paper, we present FlowChef, a novel training, inversion, and gradient-free inference-time steering strategy for RFMs that deterministically guides the denoising process. We first develop a theoretical and empirical understanding of the vector-field dynamics of RFMs in efficiently guiding the denoising trajectory. Specifically, leveraging the straightness and smooth Jacobian properties, we derive the mathematical relationship between gradients of rectified flow ODEs. We extend our theoretical findings to solve linear-inverse problems, image editing, classifier guidance, and many more tasks. We perform extensive evaluations and show that FlowChef significantly exceeds baselines in terms of performance, memory, and time requirements, achieving new state-of-the-art results. Remarkably, for the first time, it scales effortlessly to billion-parameter models such as Flux. Project Page: https://flowchef.github.io/
Teaching VLMs to Localize Specific Objects from In-context Examples
Authors: Sivan Doveh; Nimrod Shabtay; Eli Schwartz; Hilde Kuehne; Raja Giryes; Rogerio Feris; Leonid Karlinsky; James Glass; Assaf Arbelle; Shimon Ullman; M. Jehanzeb Mirza
Venue: ICCV 2025 (main)

Abstract: Vision-Language Models (VLMs) have shown remarkable capabilities across diverse visual tasks, including image recognition, video understanding, and Visual Question Answering (VQA) when explicitly trained for these tasks. Despite these advances, we find that present-day VLMs (including the proprietary GPT-4o) lack a fundamental cognitive ability: learning to localize specific objects in a scene by taking into account the context.In this work, we focus on the task of few-shot personalized localization, where a model is given a small set of annotated images (in-context examples) -- each with a category label and bounding box -- and is tasked with localizing the same object type in a query image. Personalized localization can be particularly important in cases of ambiguity of several related objects that can respond to a text or an object that is hard to describe with words.To provoke personalized localization abilities in models, we present a data-centric solution that fine-tunes them using carefully curated data from video object tracking datasets. By leveraging sequences of frames tracking the same object across multiple shots, we simulate instruction-tuning dialogues that promote context awareness. To reinforce this, we introduce a novel regularization technique that replaces object labels with pseudo-names, ensuring the model relies on visual context rather than prior knowledge. Our method significantly enhances few-shot localization performance of recent VLMs ranging from 7B to 72B in size, without sacrificing generalization, as demonstrated on several benchmarks tailored towards evaluating personalized localization abilities. This work is the first to explore and benchmark personalized few-shot localization for VLMs -- exposing critical weaknesses in present-day VLMs, and lays a foundation for future research in context-driven vision-language applications.