Orionnnn/hub-stats huggingface dataset:
NaViL: Rethinking Scaling Properties of Native Multimodal Large Language
  Models under Data Constraints
Authors: name: Changyao Tian, name: Hao Li, name: Gen Luo, name: Xizhou Zhu, name: Weijie Su, name: Hanming Deng, name: Jinguo Zhu, name: Jie Shao, name: Ziran Zhu, name: Yunpeng Liu, name: Lewei Lu, name: Wenhai Wang, name: Hongsheng Li, name: Jifeng Dai
Published: 2025-10-09 17:59:37

Compositional training has been the de-facto paradigm in existing Multimodal
Large Language Models (MLLMs), where pre-trained vision encoders are connected
with pre-trained LLMs through continuous multimodal pre-training. However, the
multimodal scaling property of this paradigm remains difficult to explore due
to the separated training. In this paper, we focus on the native training of
MLLMs in an end-to-end manner and systematically study its design space and
scaling property under a practical setting, i.e., data constraint. Through
careful study of various choices in MLLM, we obtain the optimal
meta-architecture that best balances performance and training cost. After that,
we further explore the scaling properties of the native MLLM and indicate the
positively correlated scaling relationship between visual encoders and LLMs.
Based on these findings, we propose a native MLLM called NaViL, combined with a
simple and cost-effective recipe. Experimental results on 14 multimodal
benchmarks confirm the competitive performance of NaViL against existing MLLMs.
Besides that, our findings and results provide in-depth insights for the future
study of native MLLMs.
TRAVL: A Recipe for Making Video-Language Models Better Judges of
  Physics Implausibility
Authors: name: Saman Motamed, name: Minghao Chen, name: Luc Van Gool, name: Iro Laina
Published: 2025-10-08 21:03:46

Despite impressive visual fidelity, modern video generative models frequently
produce sequences that violate intuitive physical laws, such as objects
floating, teleporting, or morphing in ways that defy causality. While humans
can easily detect such implausibilities, there remains no robust method for
quantitatively assessing physical realism in video. In this work, we explore
whether Video-Language Models (VLMs) can be trained to serve as reliable judges
of physical plausibility. We find that existing VLMs struggle to identify
physics violations, exposing fundamental limitations in their temporal and
causal reasoning. To address this, we introduce TRAVL, a fine-tuning recipe
that combines a balanced training dataset with a trajectory-aware attention
module to improve motion encoding and discrimination in VLMs. To evaluate
physical reasoning more rigorously, we propose ImplausiBench, a benchmark of
300 videos (150 real, 150 generated) that removes linguistic biases and
isolates visual-temporal understanding. Performance is reported both with
gold-standard human judgments and stricter LLM-as-judge metrics. Together,
TRAVL and ImplausiBench offer a unified framework for probing and improving
physical plausibility in multimodal models, shedding light on a challenging and
underexplored aspect of visual-temporal understanding.
Reinforcement Mid-Training
Authors: name: Yijun Tian, name: Shaoyu Chen, name: Zhichao Xu, name: Yawei Wang, name: Jinhe Bi, name: Peng Han, name: Wei Wang
Published: 2025-09-29 07:21:24

The development of state-of-the-art large language models is commonly
understood as a two-stage process involving pre-training and post-training. We
point out the need for an additional intermediate stage called reinforcement
mid-training with potential for strong performance gains. In this paper, we
formally define the problem and identify three key challenges: (1) inefficient
training due to excessive reasoning steps, (2) disregard of the imbalanced
token entropy distribution, and (3) underutilization of token information. To
address these challenges, we propose RMT, a framework for efficient, adaptive,
and unified reinforcement mid-training with various innovative components. In
particular, we first introduce a dynamic token budget mechanism that constrains
unnecessary reasoning steps and mitigates model overthinking. Next, we design a
curriculum-based adaptive sampling method that fosters a progressive learning
trajectory from easy to hard tokens. Finally, we present a dual training
strategy that combines reinforcement learning with next-token prediction,
ensuring targeted learning on key tokens and full exploitation of all token
information. Extensive experiments demonstrate the superiority of RMT over
state-of-the-art methods, achieving up to +64.91% performance improvement with
only 21% of the reasoning length in language modeling. We also show that
checkpoints obtained after reinforcement mid-training can benefit the
subsequent post-training, yielding up to +18.76% improvement in the
mathematical domain.
M3Retrieve: Benchmarking Multimodal Retrieval for Medicine
Authors: name: Arkadeep Acharya, name: Akash Ghosh, name: Pradeepika Verma, name: Kitsuchart Pasupa, name: Sriparna Saha, name: Priti Singh
Published: 2025-10-08 11:08:47

With the increasing use of RetrievalAugmented Generation (RAG), strong
retrieval models have become more important than ever. In healthcare,
multimodal retrieval models that combine information from both text and images
offer major advantages for many downstream tasks such as question answering,
cross-modal retrieval, and multimodal summarization, since medical data often
includes both formats. However, there is currently no standard benchmark to
evaluate how well these models perform in medical settings. To address this
gap, we introduce M3Retrieve, a Multimodal Medical Retrieval Benchmark.
M3Retrieve, spans 5 domains,16 medical fields, and 4 distinct tasks, with over
1.2 Million text documents and 164K multimodal queries, all collected under
approved licenses. We evaluate leading multimodal retrieval models on this
benchmark to explore the challenges specific to different medical specialities
and to understand their impact on retrieval performance. By releasing
M3Retrieve, we aim to enable systematic evaluation, foster model innovation,
and accelerate research toward building more capable and reliable multimodal
retrieval systems for medical applications. The dataset and the baselines code
are available in this github page https://github.com/AkashGhosh/M3Retrieve.
Code Agent can be an End-to-end System Hacker: Benchmarking Real-world
  Threats of Computer-use Agent
Authors: name: Weidi Luo, name: Qiming Zhang, name: Tianyu Lu, name: Xiaogeng Liu, name: Bin Hu, name: Hung-Chun Chiu, name: Siyuan Ma, name: Yizhe Zhang, name: Xusheng Xiao, name: Yinzhi Cao, name: Zhen Xiang, name: Chaowei Xiao
Published: 2025-10-08 03:35:23

Computer-use agent (CUA) frameworks, powered by large language models (LLMs)
or multimodal LLMs (MLLMs), are rapidly maturing as assistants that can
perceive context, reason, and act directly within software environments. Among
their most critical applications is operating system (OS) control. As CUAs in
the OS domain become increasingly embedded in daily operations, it is
imperative to examine their real-world security implications, specifically
whether CUAs can be misused to perform realistic, security-relevant attacks.
Existing works exhibit four major limitations: Missing attacker-knowledge model
on tactics, techniques, and procedures (TTP), Incomplete coverage for
end-to-end kill chains, unrealistic environment without multi-host and
encrypted user credentials, and unreliable judgment dependent on
LLM-as-a-Judge. To address these gaps, we propose AdvCUA, the first benchmark
aligned with real-world TTPs in MITRE ATT&CK Enterprise Matrix, which comprises
140 tasks, including 40 direct malicious tasks, 74 TTP-based malicious tasks,
and 26 end-to-end kill chains, systematically evaluates CUAs under a realistic
enterprise OS security threat in a multi-host environment sandbox by hard-coded
evaluation. We evaluate the existing five mainstream CUAs, including ReAct,
AutoGPT, Gemini CLI, Cursor CLI, and Cursor IDE based on 8 foundation LLMs. The
results demonstrate that current frontier CUAs do not adequately cover OS
security-centric threats. These capabilities of CUAs reduce dependence on
custom malware and deep domain expertise, enabling even inexperienced attackers
to mount complex enterprise intrusions, which raises social concern about the
responsibility and security of CUAs.
PuzzlePlex: Benchmarking Foundation Models on Reasoning and Planning
  with Puzzles
Authors: name: Yitao Long, name: Yuru Jiang, name: Hongjun Liu, name: Yilun Zhao, name: Jingchen Sun, name: Yiqiu Shen, name: Chen Zhao, name: Arman Cohan, name: Dennis Shasha
Published: 2025-10-07 21:24:29

This work investigates the reasoning and planning capabilities of foundation
models and their scalability in complex, dynamic environments. We introduce
PuzzlePlex, a benchmark designed to assess these capabilities through a diverse
set of puzzles. PuzzlePlex consists of 15 types of puzzles, including
deterministic and stochastic games of varying difficulty, as well as
single-player and two-player scenarios. The PuzzlePlex framework provides a
comprehensive environment for each game, and supports extensibility to generate
more challenging instances as foundation models evolve. Additionally, we
implement customized game-playing strategies for comparison. Building on this
benchmark, we develop fine-grained metrics to measure performance and conduct
an in-depth analysis of frontier foundation models across two settings:
instruction-based and code-based. Furthermore, we systematically investigate
their scaling limits. Our findings show that reasoning models outperform others
in instruction-based settings, while code-based execution presents greater
challenges but offers a scalable and efficient alternative. PuzzlePlex enables
targeted evaluation and guides future improvements in reasoning, planning, and
generalization for foundation models.
FinLFQA: Evaluating Attributed Text Generation of LLMs in Financial
  Long-Form Question Answering
Authors: name: Yitao Long, name: Tiansheng Hu, name: Yilun Zhao, name: Arman Cohan, name: Chen Zhao
Published: 2025-10-07 20:06:15

Large Language Models (LLMs) frequently hallucinate to long-form questions,
producing plausible yet factually incorrect answers. A common mitigation
strategy is to provide attribution to LLM outputs. However, existing benchmarks
primarily focus on simple attribution that retrieves supporting textual
evidence as references. We argue that in real-world scenarios such as financial
applications, attribution goes beyond reference retrieval. We introduce
FinLFQA, a benchmark designed to evaluate the ability of LLMs to generate
long-form answers to complex financial questions with reliable and nuanced
attributions. FinLFQA evaluates three critical aspects of attribution through
human annotations: (1) supporting evidence extracted from financial reports,
(2) intermediate numerical reasoning steps, and (3) domain-specific financial
knowledge that informs the reasoning process. We further provide an automatic
evaluation framework covering both answer quality and attribution quality.
Through extensive experiments on eight LLMs across multiple
attribution-generation paradigms, we find that fine-grained metrics are
important to distinguish model capabilities, that end-to-end generation
achieves comparable performance to post-hoc approaches, and that iterative
refinement only helps when guided by external feedback.
A Single Character can Make or Break Your LLM Evals
Authors: name: Jingtong Su, name: Jianyu Zhang, name: Karen Ullrich, name: Léon Bottou, name: Mark Ibrahim
Published: 2025-10-02 13:27:28

Common Large Language model (LLM) evaluations rely on demonstration examples
to steer models' responses to the desired style. While the number of examples
used has been studied and standardized, the choice of how to format examples is
less investigated. In evaluation protocols and real world usage, users face the
choice how to separate in-context examples: use a comma? new line? semi-colon?
hashtag? etc.? Surprisingly, we find this seemingly minor choice can
dramatically alter model response quality. Across leading model families
(Llama, Qwen, Gemma), performance on MMLU for example can vary by pm 23%
depending on the choice of delimiter. In fact, one can manipulate model
rankings to put any model in the lead by only modifying the single character
separating examples. We find LLMs' brittleness pervades topics, model families,
and doesn't improve with scale. By probing attention head scores, we find that
good-performing delimiters steer attention towards key tokens in the input.
Finally, we explore methods to improve LLMs' robustness to the choice of
delimiter. We find specifying the selected delimiter in the prompt boosts
robustness and offer practical recommendations for the best-performing
delimiters to select.
Glocal Information Bottleneck for Time Series Imputation
Authors: name: Jie Yang, name: Kexin Zhang, name: Guibin Zhang, name: Philip S. Yu, name: Kaize Ding
Published: 2025-10-06 15:24:44

Time Series Imputation (TSI), which aims to recover missing values in
temporal data, remains a fundamental challenge due to the complex and often
high-rate missingness in real-world scenarios. Existing models typically
optimize the point-wise reconstruction loss, focusing on recovering numerical
values (local information). However, we observe that under high missing rates,
these models still perform well in the training phase yet produce poor
imputations and distorted latent representation distributions (global
information) in the inference phase. This reveals a critical optimization
dilemma: current objectives lack global guidance, leading models to overfit
local noise and fail to capture global information of the data. To address this
issue, we propose a new training paradigm, Glocal Information Bottleneck
(Glocal-IB). Glocal-IB is model-agnostic and extends the standard IB framework
by introducing a Global Alignment loss, derived from a tractable mutual
information approximation. This loss aligns the latent representations of
masked inputs with those of their originally observed counterparts. It helps
the model retain global structure and local details while suppressing noise
caused by missing values, giving rise to better generalization under high
missingness. Extensive experiments on nine datasets confirm that Glocal-IB
leads to consistently improved performance and aligned latent representations
under missingness. Our code implementation is available in
https://github.com/Muyiiiii/NeurIPS-25-Glocal-IB.
CALM Before the STORM: Unlocking Native Reasoning for Optimization
  Modeling
Authors: name: Zhengyang Tang, name: Zihan Ye, name: Chenyu Huang, name: Xuhan Huang, name: Chengpeng Li, name: Sihang Li, name: Guanhua Chen, name: Ming Yan, name: Zizhuo Wang, name: Hongyuan Zha, name: Dayiheng Liu, name: Benyou Wang
Published: 2025-10-05 13:38:31

Large Reasoning Models (LRMs) have demonstrated strong capabilities in
complex multi-step reasoning, opening new opportunities for automating
optimization modeling. However, existing domain adaptation methods, originally
designed for earlier instruction-tuned models, often fail to exploit the
advanced reasoning patterns of modern LRMs -- In particular, we show that
direct fine-tuning on traditional non-reflective datasets leads to
limited gains. To fully leverage LRMs' inherent reasoning abilities, we propose
CALM (Corrective Adaptation with Lightweight Modification), a
framework that progressively refines LRMs within their native reasoning modes
for optimization modeling tasks. In CALM, an expert intervener identifies
reasoning flaws and provides concise corrective hints, which the LRM
incorporates to produce improved reasoning trajectories. These interventions
modify fewer than 2.6\% of generated tokens, but generate high-quality data for
soft adaptation through supervised fine-tuning. The adapted model is then
further improved through reinforcement learning. Building on CALM, we develop
STORM (Smart Thinking Optimization Reasoning Model), a
4B-parameter LRM that achieves a new state-of-the-art average accuracy of
68.9\% across five popular optimization modeling benchmarks, matching the
performance of a 671B LRM. These results demonstrate that dynamic, hint-based
data synthesis both preserves and amplifies the native reasoning patterns of
modern LRMs, offering a more effective and scalable path towards expert-level
performance on challenging optimization modeling tasks.
Pushing on Multilingual Reasoning Models with Language-Mixed
  Chain-of-Thought
Authors: name: Guijin Son, name: Donghun Yang, name: Hitesh Laxmichand Patel, name: Amit Agarwal, name: Hyunwoo Ko, name: Chanuk Lim, name: Srikant Panda, name: Minhyuk Kim, name: Nikunj Drolia, name: Dasol Choi, name: Kyong-Ha Lee, name: Youngjae Yu
Published: 2025-10-05 14:39:41

Recent frontier models employ long chain-of-thought reasoning to explore
solution spaces in context and achieve stonger performance. While many works
study distillation to build smaller yet capable models, most focus on English
and little is known about language-specific reasoning. To bridge this gap, we
first introduct **Language-Mixed CoT**, a reasoning schema that switches
between English and a target language, using English as an anchor to excel in
reasoning while minimizing translation artificats. As a Korean case study, we
curate **Yi-Sang**: 5.79M native-Korean prompts from web Q&A, exams, STEM, and
code; 3.7M long reasoning traces generated from Qwen3-32B; and a targeted 260k
high-yield subset. We train ninve models (4B-35B) across six families (Qwen2.5,
Llama-3.1, Gemma-3, etc). Our best model, **KO-REAson-35B**, achieves
state-of-the-art performance, with the highest overall average score (64.0 \pm
25), ranking first on 5/9 benchmarks and second on the remainder. Samller and
mid-sized models also benefit substantially, with an average improvement of
+18.6 points across teh evaluated nine benchmarks. Ablations show
**Language-Mixed CoT** is more effective than monolingual CoT, also resulting
in cross-lingual and mult-modal performance gains. We release our data-curation
pipeline, evaluation system, datasets, and models to advance research on
language-specific reasoning. Data and model collection:
https://huggingface.co/KOREAson.
Revisiting the Uniform Information Density Hypothesis in LLM Reasoning
  Traces
Authors: name: Minju Gwak, name: Guijin Son, name: Jaehyung Kim
Published: 2025-10-08 12:37:04

The Uniform Information Density (UID) hypothesis suggests that effective
communication maintains a stable flow of information. In this work, we revisit
this principle in the context of large language model (LLM) reasoning traces,
asking whether step-level uniformity reflects reasoning quality. To this end,
we propose an entropy-based stepwise information density metric and introduce
two complementary measures of uniformity, local and global uniformity scores.
Across the experiments on six different reasoning benchmarks, we find that
step-level uniformity not only provides a strong theoretical lens but also
yields practical performance benefits; for example, selecting reasoning traces
with more uniform information density at the step-level improves accuracy by
10-32\% relative gains over baselines at AIME2025. Our analysis further reveals
that correct reasoning traces tend to avoid sharp information density spikes,
while incorrect traces exhibit irregular information bursts. These results
demonstrate that UID-inspired information density measures outperform
alternative internal signals as predictors of reasoning quality. Results
highlight the uniformity of the information density as a robust diagnostic and
selection criterion for building more reliable and accurate reasoning systems.
The Markovian Thinker
Authors: name: Milad Aghajohari, name: Kamran Chitsaz, name: Amirhossein Kazemnejad, name: Sarath Chandar, name: Alessandro Sordoni, name: Aaron Courville, name: Siva Reddy
Published: 2025-10-08 01:18:13

Reinforcement learning (RL) has recently become a strong recipe for training
reasoning LLMs that produce long chains of thought (LongCoT). Yet the standard
RL "thinking environment", where the state is the prompt plus all prior
reasoning tokens, makes the state unbounded and forces attention-based policies
to pay quadratic compute as thoughts lengthen. We revisit the environment
itself. We propose Markovian Thinking, a paradigm in which the policy advances
reasoning while conditioning on a constant-size state, decoupling thinking
length from context size. As an immediate consequence this yields linear
compute with constant memory. We instantiate this idea with Delethink, an RL
environment that structures reasoning into fixed-size chunks. Within each
chunk, the model thinks as usual; at the boundary, the environment resets the
context and reinitializes the prompt with a short carryover. Through RL, the
policy learns to write a textual state near the end of each chunk sufficient
for seamless continuation of reasoning after reset. Trained in this
environment, an R1-Distill 1.5B model reasons in 8K-token chunks yet thinks up
to 24K tokens, matching or surpassing LongCoT-RL trained with a 24K budget.
With test-time scaling, Delethink continues to improve where LongCoT plateaus.
The effect of linear compute is substantial: we empirically estimate at 96K
average thinking length LongCoT-RL costs 27 H100-months vs. 7 for Delethink.
Analysis at RL initialization shows off-the-shelf reasoning models (1.5B-120B)
often sample Markovian traces zero-shot across diverse benchmarks, providing
positive samples that make RL effective at scale. Our results show that
redesigning the thinking environment is a powerful lever: it enables very long
reasoning without quadratic overhead and opens a path toward efficient,
scalable reasoning LLMs.
DeepTravel: An End-to-End Agentic Reinforcement Learning Framework for
  Autonomous Travel Planning Agents
Authors: name: Yansong Ning, name: Rui Liu, name: Jun Wang, name: Kai Chen, name: Wei Li, name: Jun Fang, name: Kan Zheng, name: Naiqiang Tan, name: Hao Liu
Published: 2025-09-26 04:03:52

Travel planning (TP) agent has recently worked as an emerging building block
to interact with external tools and resources for travel itinerary generation,
ensuring enjoyable user experience. Despite its benefits, existing studies rely
on hand craft prompt and fixed agent workflow, hindering more flexible and
autonomous TP agent. This paper proposes DeepTravel, an end to end agentic
reinforcement learning framework for building autonomous travel planning agent,
capable of autonomously planning, executing tools, and reflecting on tool
responses to explore, verify, and refine intermediate actions in multi step
reasoning. To achieve this, we first construct a robust sandbox environment by
caching transportation, accommodation and POI data, facilitating TP agent
training without being constrained by real world APIs limitations (e.g.,
inconsistent outputs). Moreover, we develop a hierarchical reward modeling
system, where a trajectory level verifier first checks spatiotemporal
feasibility and filters unsatisfied travel itinerary, and then the turn level
verifier further validate itinerary detail consistency with tool responses,
enabling efficient and precise reward service. Finally, we propose the reply
augmented reinforcement learning method that enables TP agent to periodically
replay from a failures experience buffer, emerging notable agentic capacity. We
deploy trained TP agent on DiDi Enterprise Solutions App and conduct
comprehensive online and offline evaluations, demonstrating that DeepTravel
enables small size LLMs (e.g., Qwen3 32B) to significantly outperform existing
frontier LLMs such as OpenAI o1, o3 and DeepSeek R1 in travel planning tasks.