Orionnnn/hub-stats huggingface dataset:
Thinking with Images for Multimodal Reasoning: Foundations, Methods, and
  Future Frontiers
Authors: name: Zhaochen Su, name: Peng Xia, name: Hangyu Guo, name: Zhenhua Liu, name: Yan Ma, name: Xiaoye Qu, name: Jiaqi Liu, name: Yanshu Li, name: Kaide Zeng, name: Zhengyuan Yang, name: Linjie Li, name: Yu Cheng, name: Heng Ji, name: Junxian He, name: Yi R. Fung
Published: 2025-06-30 14:48:35

Recent progress in multimodal reasoning has been significantly advanced by
textual Chain-of-Thought (CoT), a paradigm where models conduct reasoning
within language. This text-centric approach, however, treats vision as a
static, initial context, creating a fundamental "semantic gap" between rich
perceptual data and discrete symbolic thought. Human cognition often transcends
language, utilizing vision as a dynamic mental sketchpad. A similar evolution
is now unfolding in AI, marking a fundamental paradigm shift from models that
merely think about images to those that can truly think with images. This
emerging paradigm is characterized by models leveraging visual information as
intermediate steps in their thought process, transforming vision from a passive
input into a dynamic, manipulable cognitive workspace. In this survey, we chart
this evolution of intelligence along a trajectory of increasing cognitive
autonomy, which unfolds across three key stages: from external tool
exploration, through programmatic manipulation, to intrinsic imagination. To
structure this rapidly evolving field, our survey makes four key contributions.
(1) We establish the foundational principles of the think with image paradigm
and its three-stage framework. (2) We provide a comprehensive review of the
core methods that characterize each stage of this roadmap. (3) We analyze the
critical landscape of evaluation benchmarks and transformative applications.
(4) We identify significant challenges and outline promising future directions.
By providing this structured overview, we aim to offer a clear roadmap for
future research towards more powerful and human-aligned multimodal AI.
AsyncFlow: An Asynchronous Streaming RL Framework for Efficient LLM
  Post-Training
Authors: name: Zhenyu Han, name: Ansheng You, name: Haibo Wang, name: Kui Luo, name: Guang Yang, name: Wenqi Shi, name: Menglong Chen, name: Sicheng Zhang, name: Zeshun Lan, name: Chunshi Deng, name: Huazhong Ji, name: Wenjie Liu, name: Yu Huang, name: Yixiang Zhang, name: Chenyi Pan, name: Jing Wang, name: Xin Huang, name: Chunsheng Li, name: Jianping Wu
Published: 2025-07-02 12:45:34

Reinforcement learning (RL) has become a pivotal technology in the
post-training phase of large language models (LLMs). Traditional task-colocated
RL frameworks suffer from significant scalability bottlenecks, while
task-separated RL frameworks face challenges in complex dataflows and the
corresponding resource idling and workload imbalance. Moreover, most existing
frameworks are tightly coupled with LLM training or inference engines, making
it difficult to support custom-designed engines. To address these challenges,
we propose AsyncFlow, an asynchronous streaming RL framework for efficient
post-training. Specifically, we introduce a distributed data storage and
transfer module that provides a unified data management and fine-grained
scheduling capability in a fully streamed manner. This architecture inherently
facilitates automated pipeline overlapping among RL tasks and dynamic load
balancing. Moreover, we propose a producer-consumer-based asynchronous workflow
engineered to minimize computational idleness by strategically deferring
parameter update process within staleness thresholds. Finally, the core
capability of AsynFlow is architecturally decoupled from underlying training
and inference engines and encapsulated by service-oriented user interfaces,
offering a modular and customizable user experience. Extensive experiments
demonstrate an average of 1.59 throughput improvement compared with
state-of-the-art baseline. The presented architecture in this work provides
actionable insights for next-generation RL training system designs.
Bourbaki: Self-Generated and Goal-Conditioned MDPs for Theorem Proving
Authors: name: Matthieu Zimmer, name: Xiaotong Ji, name: Rasul Tutunov, name: Anthony Bordg, name: Jun Wang, name: Haitham Bou Ammar
Published: 2025-07-03 15:41:38

Reasoning remains a challenging task for large language models (LLMs),
especially within the logically constrained environment of automated theorem
proving (ATP), due to sparse rewards and the vast scale of proofs. These
challenges are amplified in benchmarks like PutnamBench, which contains
university-level problems requiring complex, multi-step reasoning. To address
this, we introduce self-generated goal-conditioned MDPs (sG-MDPs), a new
framework in which agents generate and pursue their subgoals based on the
evolving proof state. Given this more structured generation of goals, the
resulting problem becomes more amenable to search. We then apply Monte Carlo
Tree Search (MCTS)-like algorithms to solve the sG-MDP, instantiating our
approach in Bourbaki (7B), a modular system that can ensemble multiple 7B LLMs
for subgoal generation and tactic synthesis. On PutnamBench, Bourbaki (7B)
solves 26 problems, achieving new state-of-the-art results with models at this
scale.
LangScene-X: Reconstruct Generalizable 3D Language-Embedded Scenes with
  TriMap Video Diffusion
Authors: name: Fangfu Liu, name: Hao Li, name: Jiawei Chi, name: Hanyang Wang, name: Minghui Yang, name: Fudong Wang, name: Yueqi Duan
Published: 2025-07-03 17:21:23

Recovering 3D structures with open-vocabulary scene understanding from 2D
images is a fundamental but daunting task. Recent developments have achieved
this by performing per-scene optimization with embedded language information.
However, they heavily rely on the calibrated dense-view reconstruction
paradigm, thereby suffering from severe rendering artifacts and implausible
semantic synthesis when limited views are available. In this paper, we
introduce a novel generative framework, coined LangScene-X, to unify and
generate 3D consistent multi-modality information for reconstruction and
understanding. Powered by the generative capability of creating more consistent
novel observations, we can build generalizable 3D language-embedded scenes from
only sparse views. Specifically, we first train a TriMap video diffusion model
that can generate appearance (RGBs), geometry (normals), and semantics
(segmentation maps) from sparse inputs through progressive knowledge
integration. Furthermore, we propose a Language Quantized Compressor (LQC),
trained on large-scale image datasets, to efficiently encode language
embeddings, enabling cross-scene generalization without per-scene retraining.
Finally, we reconstruct the language surface fields by aligning language
information onto the surface of 3D scenes, enabling open-ended language
queries. Extensive experiments on real-world data demonstrate the superiority
of our LangScene-X over state-of-the-art methods in terms of quality and
generalizability. Project Page: https://liuff19.github.io/LangScene-X.
WebSailor: Navigating Super-human Reasoning for Web Agent
Authors: name: Kuan Li, name: Zhongwang Zhang, name: Huifeng Yin, name: Liwen Zhang, name: Litu Ou, name: Jialong Wu, name: Wenbiao Yin, name: Baixuan Li, name: Zhengwei Tao, name: Xinyu Wang, name: Weizhou Shen, name: Junkai Zhang, name: Dingchu Zhang, name: Xixi Wu, name: Yong Jiang, name: Ming Yan, name: Pengjun Xie, name: Fei Huang, name: Jingren Zhou
Published: 2025-07-03 12:59:07

Transcending human cognitive limitations represents a critical frontier in
LLM training. Proprietary agentic systems like DeepResearch have demonstrated
superhuman capabilities on extremely complex information-seeking benchmarks
such as BrowseComp, a feat previously unattainable. We posit that their success
hinges on a sophisticated reasoning pattern absent in open-source models: the
ability to systematically reduce extreme uncertainty when navigating vast
information landscapes. Based on this insight, we introduce WebSailor, a
complete post-training methodology designed to instill this crucial capability.
Our approach involves generating novel, high-uncertainty tasks through
structured sampling and information obfuscation, RFT cold start, and an
efficient agentic RL training algorithm, Duplicating Sampling Policy
Optimization (DUPO). With this integrated pipeline, WebSailor significantly
outperforms all opensource agents in complex information-seeking tasks,
matching proprietary agents' performance and closing the capability gap.
Fast and Simplex: 2-Simplicial Attention in Triton
Authors: name: Aurko Roy, name: Timothy Chou, name: Sai Surya Duvvuri, name: Sijia Chen, name: Jiecao Yu, name: Xiaodong Wang, name: Manzil Zaheer, name: Rohan Anil
Published: 2025-07-03 16:16:34

Recent work has shown that training loss scales as a power law with both
model size and the number of tokens, and that achieving compute-optimal models
requires scaling model size and token count together. However, these scaling
laws assume an infinite supply of data and apply primarily in compute-bound
settings. As modern large language models increasingly rely on massive
internet-scale datasets, the assumption that they are compute-bound is becoming
less valid. This shift highlights the need for architectures that prioritize
token efficiency.
  In this work, we investigate the use of the 2-simplicial Transformer, an
architecture that generalizes standard dot-product attention to trilinear
functions through an efficient Triton kernel implementation. We demonstrate
that the 2-simplicial Transformer achieves better token efficiency than
standard Transformers: for a fixed token budget, similarly sized models
outperform their dot-product counterparts on tasks involving mathematics,
coding, reasoning, and logic. We quantify these gains by demonstrating that
2-simplicial attention changes the exponent in the scaling laws for knowledge
and reasoning tasks compared to dot product attention.
Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergy
Authors: name: Chris Yuhao Liu, name: Liang Zeng, name: Yuzhen Xiao, name: Jujie He, name: Jiacai Liu, name: Chaojie Wang, name: Rui Yan, name: Wei Shen, name: Fuxiang Zhang, name: Jiacheng Xu, name: Yang Liu, name: Yahui Zhou
Published: 2025-07-02 04:40:29

Despite the critical role of reward models (RMs) in reinforcement learning
from human feedback (RLHF), current state-of-the-art open RMs perform poorly on
most existing evaluation benchmarks, failing to capture the spectrum of nuanced
and sophisticated human preferences. Even approaches that incorporate advanced
training techniques have not yielded meaningful performance improvements. We
hypothesize that this brittleness stems primarily from limitations in
preference datasets, which are often narrowly scoped, synthetically labeled, or
lack rigorous quality control. To address these challenges, we present a
large-scale preference dataset comprising 40 million preference pairs, named
SynPref-40M. To enable data curation at scale, we design a human-AI synergistic
two-stage pipeline that leverages the complementary strengths of human
annotation quality and AI scalability. In this pipeline, humans provide
verified annotations, while large language models perform automatic curation
based on human guidance. Training on this preference mixture, we introduce
Skywork-Reward-V2, a suite of eight reward models ranging from 0.6B to 8B
parameters, trained on a carefully curated subset of 26 million preference
pairs from SynPref-40M. We demonstrate that Skywork-Reward-V2 is versatile
across a wide range of capabilities, including alignment with human
preferences, objective correctness, safety, resistance to stylistic biases, and
best-of-N scaling, achieving state-of-the-art performance across seven major
reward model benchmarks. Ablation studies confirm that the effectiveness of our
approach stems not only from data scale but also from high-quality curation.
The Skywork-Reward-V2 series represents substantial progress in open reward
models, highlighting the untapped potential of existing preference datasets and
demonstrating how human-AI curation synergy can unlock significantly higher
data quality.
Decoupled Planning and Execution: A Hierarchical Reasoning Framework for
  Deep Search
Authors: name: Jiajie Jin, name: Xiaoxi Li, name: Guanting Dong, name: Yuyao Zhang, name: Yutao Zhu, name: Yang Zhao, name: Hongjin Qian, name: Zhicheng Dou
Published: 2025-07-03 14:18:08

Complex information needs in real-world search scenarios demand deep
reasoning and knowledge synthesis across diverse sources, which traditional
retrieval-augmented generation (RAG) pipelines struggle to address effectively.
Current reasoning-based approaches suffer from a fundamental limitation: they
use a single model to handle both high-level planning and detailed execution,
leading to inefficient reasoning and limited scalability. In this paper, we
introduce HiRA, a hierarchical framework that separates strategic planning from
specialized execution. Our approach decomposes complex search tasks into
focused subtasks, assigns each subtask to domain-specific agents equipped with
external tools and reasoning capabilities, and coordinates the results through
a structured integration mechanism. This separation prevents execution details
from disrupting high-level reasoning while enabling the system to leverage
specialized expertise for different types of information processing.
Experiments on four complex, cross-modal deep search benchmarks demonstrate
that HiRA significantly outperforms state-of-the-art RAG and agent-based
systems. Our results show improvements in both answer quality and system
efficiency, highlighting the effectiveness of decoupled planning and execution
for multi-step information seeking tasks. Our code is available at
https://github.com/ignorejjj/HiRA.
Answer Matching Outperforms Multiple Choice for Language Model
  Evaluation
Authors: name: Nikhil Chandak, name: Shashwat Goel, name: Ameya Prabhu, name: Moritz Hardt, name: Jonas Geiping
Published: 2025-07-03 17:59:02

Multiple choice benchmarks have long been the workhorse of language model
evaluation because grading multiple choice is objective and easy to automate.
However, we show multiple choice questions from popular benchmarks can often be
answered without even seeing the question. These shortcuts arise from a
fundamental limitation of discriminative evaluation not shared by evaluations
of the model's free-form, generative answers. Until recently, there appeared to
be no viable, scalable alternative to multiple choice--but, we show that this
has changed. We consider generative evaluation via what we call answer
matching: Give the candidate model the question without the options, have it
generate a free-form response, then use a modern language model with the
reference answer to determine if the response matches the reference. To compare
the validity of different evaluation strategies, we annotate MMLU-Pro and
GPQA-Diamond to obtain human grading data, and measure the agreement of each
evaluation approach. We find answer matching using recent models--even small
ones--achieves near-perfect agreement, in the range of inter-annotator
agreement. In contrast, both multiple choice evaluation and using
LLM-as-a-judge without reference answers aligns poorly with human grading.
Improving evaluations via answer matching is not merely a conceptual concern:
the rankings of several models change significantly when evaluating their
free-form responses with answer matching. In light of these findings, we
discuss how to move the evaluation ecosystem from multiple choice to answer
matching.
μ^2Tokenizer: Differentiable Multi-Scale Multi-Modal Tokenizer for
  Radiology Report Generation
Authors: name: Siyou Li, name: Pengyao Qin, name: Huanan Wu, name: Dong Nie, name: Arun J. Thirunavukarasu, name: Juntao Yu, name: Le Zhang
Published: 2025-06-30 23:14:49

Automated radiology report generation (RRG) aims to produce detailed textual
reports from clinical imaging, such as computed tomography (CT) scans, to
improve the accuracy and efficiency of diagnosis and provision of management
advice. RRG is complicated by two key challenges: (1) inherent complexity in
extracting relevant information from imaging data under resource constraints,
and (2) difficulty in objectively evaluating discrepancies between
model-generated and expert-written reports. To address these challenges, we
propose mu^2LLM, a textbf{mu}ltiscale
textbf{mu}ltimodal large language models for RRG tasks. The
novel {mu}^2Tokenizer, as an intermediate layer, integrates multi-modal
features from the multiscale visual tokenizer and the text tokenizer, then
enhances report generation quality through direct preference optimization
(DPO), guided by GREEN-RedLlama. Experimental results on four large CT
image-report medical datasetdemonstrate that our method outperforms existing
approaches, highlighting the potential of our fine-tuned mu^2LLMs on limited
data for RRG tasks.
ARIG: Autoregressive Interactive Head Generation for Real-time
  Conversations
Authors: name: Ying Guo, name: Xi Liu, name: Cheng Zhen, name: Pengfei Yan, name: Xiaoming Wei
Published: 2025-07-01 06:38:14

Face-to-face communication, as a common human activity, motivates the
research on interactive head generation. A virtual agent can generate motion
responses with both listening and speaking capabilities based on the audio or
motion signals of the other user and itself. However, previous clip-wise
generation paradigm or explicit listener/speaker generator-switching methods
have limitations in future signal acquisition, contextual behavioral
understanding, and switching smoothness, making it challenging to be real-time
and realistic. In this paper, we propose an autoregressive (AR) based
frame-wise framework called ARIG to realize the real-time generation with
better interaction realism. To achieve real-time generation, we model motion
prediction as a non-vector-quantized AR process. Unlike discrete codebook-index
prediction, we represent motion distribution using diffusion procedure,
achieving more accurate predictions in continuous space. To improve interaction
realism, we emphasize interactive behavior understanding (IBU) and detailed
conversational state understanding (CSU). In IBU, based on dual-track
dual-modal signals, we summarize short-range behaviors through
bidirectional-integrated learning and perform contextual understanding over
long ranges. In CSU, we use voice activity signals and context features of IBU
to understand the various states (interruption, feedback, pause, etc.) that
exist in actual conversations. These serve as conditions for the final
progressive motion prediction. Extensive experiments have verified the
effectiveness of our model.
MARVIS: Modality Adaptive Reasoning over VISualizations
Authors: name: Benjamin Feuer, name: Lennart Purucker, name: Oussama Elachqar, name: Chinmay Hegde
Published: 2025-07-02 09:56:24

Scientific applications of machine learning often rely on small, specialized
models tuned to particular domains. Such models often achieve excellent
performance, but lack flexibility. Foundation models offer versatility, but
typically underperform specialized approaches, especially on non-traditional
modalities and long-tail domains. We propose MARVIS (Modality Adaptive
Reasoning over VISualizations), a training-free method that enables even small
vision-language models to predict any data modality with high accuracy. MARVIS
transforms latent embedding spaces into visual representations and then
leverages the spatial and fine-grained reasoning skills of VLMs to successfully
interpret and utilize them. MARVIS achieves competitive performance on vision,
audio, biological, and tabular domains using a single 3B parameter model,
achieving results that beat Gemini by 16\% on average and approach specialized
methods, without exposing personally identifiable information (P.I.I.) or
requiring any domain-specific training. We open source our code and datasets at
https://github.com/penfever/marvis
Locality-aware Parallel Decoding for Efficient Autoregressive Image
  Generation
Authors: name: Zhuoyang Zhang, name: Luke J. Huang, name: Chengyue Wu, name: Shang Yang, name: Kelly Peng, name: Yao Lu, name: Song Han
Published: 2025-07-02 17:59:23

We present Locality-aware Parallel Decoding (LPD) to accelerate
autoregressive image generation. Traditional autoregressive image generation
relies on next-patch prediction, a memory-bound process that leads to high
latency. Existing works have tried to parallelize next-patch prediction by
shifting to multi-patch prediction to accelerate the process, but only achieved
limited parallelization. To achieve high parallelization while maintaining
generation quality, we introduce two key techniques: (1) Flexible Parallelized
Autoregressive Modeling, a novel architecture that enables arbitrary generation
ordering and degrees of parallelization. It uses learnable position query
tokens to guide generation at target positions while ensuring mutual visibility
among concurrently generated tokens for consistent parallel decoding. (2)
Locality-aware Generation Ordering, a novel schedule that forms groups to
minimize intra-group dependencies and maximize contextual support, enhancing
generation quality. With these designs, we reduce the generation steps from 256
to 20 (256times256 res.) and 1024 to 48 (512times512 res.) without
compromising quality on the ImageNet class-conditional generation, and
achieving at least 3.4times lower latency than previous parallelized
autoregressive models.
STR-Match: Matching SpatioTemporal Relevance Score for Training-Free
  Video Editing
Authors: name: Junsung Lee, name: Junoh Kang, name: Bohyung Han
Published: 2025-06-28 12:36:19

Previous text-guided video editing methods often suffer from temporal
inconsistency, motion distortion, and-most notably-limited domain
transformation. We attribute these limitations to insufficient modeling of
spatiotemporal pixel relevance during the editing process. To address this, we
propose STR-Match, a training-free video editing algorithm that produces
visually appealing and spatiotemporally coherent videos through latent
optimization guided by our novel STR score. The score captures spatiotemporal
pixel relevance across adjacent frames by leveraging 2D spatial attention and
1D temporal modules in text-to-video (T2V) diffusion models, without the
overhead of computationally expensive 3D attention mechanisms. Integrated into
a latent optimization framework with a latent mask, STR-Match generates
temporally consistent and visually faithful videos, maintaining strong
performance even under significant domain transformations while preserving key
visual attributes of the source. Extensive experiments demonstrate that
STR-Match consistently outperforms existing methods in both visual quality and
spatiotemporal consistency.