Orionnnn/hub-stats huggingface dataset:
Spectral Scaling Laws in Language Models: How Effectively Do
  Feed-Forward Networks Use Their Latent Space?
Authors: name: Nandan Kumar Jha, name: Brandon Reagen
Published: 2025-10-01 05:38:21

As large language models (LLMs) scale, the question is not only how large
they become, but how much of their capacity is effectively utilized. Existing
scaling laws relate model size to loss, yet overlook how components exploit
their latent space. We study feed-forward networks (FFNs) and recast width
selection as a spectral utilization problem. Using a lightweight diagnostic
suite -- Hard Rank (participation ratio), Soft Rank (Shannon rank), Spectral
Concentration, and the composite Spectral Utilization Index (SUI) -- we
quantify how many latent directions are meaningfully activated across LLaMA,
GPT-2, and nGPT families. Our key finding is an asymmetric spectral scaling
law: soft rank follows an almost perfect power law with FFN width, while hard
rank grows only sublinearly and with high variance. This asymmetry suggests
that widening FFNs mostly adds low-energy tail directions, while dominant-mode
subspaces saturate early. Moreover, at larger widths, variance further
collapses into a narrow subspace, leaving much of the latent space
under-utilized. These results recast FFN width selection as a principled
trade-off between tail capacity and dominant-mode capacity, offering concrete
guidance for inference-efficient LLM design.
TimeSeriesScientist: A General-Purpose AI Agent for Time Series Analysis
Authors: name: Haokun Zhao, name: Xiang Zhang, name: Jiaqi Wei, name: Yiwei Xu, name: Yuting He, name: Siqi Sun, name: Chenyu You
Published: 2025-10-02 00:18:59

Time series forecasting is central to decision-making in domains as diverse
as energy, finance, climate, and public health. In practice, forecasters face
thousands of short, noisy series that vary in frequency, quality, and horizon,
where the dominant cost lies not in model fitting, but in the labor-intensive
preprocessing, validation, and ensembling required to obtain reliable
predictions. Prevailing statistical and deep learning models are tailored to
specific datasets or domains and generalize poorly. A general, domain-agnostic
framework that minimizes human intervention is urgently in demand. In this
paper, we introduce TimeSeriesScientist (TSci), the first LLM-driven agentic
framework for general time series forecasting. The framework comprises four
specialized agents: Curator performs LLM-guided diagnostics augmented by
external tools that reason over data statistics to choose targeted
preprocessing; Planner narrows the hypothesis space of model choice by
leveraging multi-modal diagnostics and self-planning over the input; Forecaster
performs model fitting and validation and, based on the results, adaptively
selects the best model configuration as well as ensemble strategy to make final
predictions; and Reporter synthesizes the whole process into a comprehensive,
transparent report. With transparent natural-language rationales and
comprehensive reports, TSci transforms the forecasting workflow into a
white-box system that is both interpretable and extensible across tasks.
Empirical results on eight established benchmarks demonstrate that TSci
consistently outperforms both statistical and LLM-based baselines, reducing
forecast error by an average of 10.4% and 38.2%, respectively. Moreover, TSci
produces a clear and rigorous report that makes the forecasting workflow more
transparent and interpretable.
Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generation
Authors: name: Chetwin Low, name: Weimin Wang, name: Calder Katyal
Published: 2025-09-30 21:03:50

Audio-video generation has often relied on complex multi-stage architectures
or sequential synthesis of sound and visuals. We introduce Ovi, a unified
paradigm for audio-video generation that models the two modalities as a single
generative process. By using blockwise cross-modal fusion of twin-DiT modules,
Ovi achieves natural synchronization and removes the need for separate
pipelines or post hoc alignment. To facilitate fine-grained multimodal fusion
modeling, we initialize an audio tower with an architecture identical to that
of a strong pretrained video model. Trained from scratch on hundreds of
thousands of hours of raw audio, the audio tower learns to generate realistic
sound effects, as well as speech that conveys rich speaker identity and
emotion. Fusion is obtained by jointly training the identical video and audio
towers via blockwise exchange of timing (via scaled-RoPE embeddings) and
semantics (through bidirectional cross-attention) on a vast video corpus. Our
model enables cinematic storytelling with natural speech and accurate,
context-matched sound effects, producing movie-grade video clips. All the
demos, code and model weights are published at https://aaxwaz.github.io/Ovi
ExGRPO: Learning to Reason from Experience
Authors: name: Runzhe Zhan, name: Yafu Li, name: Zhi Wang, name: Xiaoye Qu, name: Dongrui Liu, name: Jing Shao, name: Derek F. Wong, name: Yu Cheng
Published: 2025-10-02 17:31:30

Reinforcement learning from verifiable rewards (RLVR) is an emerging paradigm
for improving the reasoning ability of large language models. However, standard
on-policy training discards rollout experiences after a single update, leading
to computational inefficiency and instability. While prior work on RL has
highlighted the benefits of reusing past experience, the role of experience
characteristics in shaping learning dynamics of large reasoning models remains
underexplored. In this paper, we are the first to investigate what makes a
reasoning experience valuable and identify rollout correctness and entropy as
effective indicators of experience value. Based on these insights, we propose
ExGRPO (Experiential Group Relative Policy Optimization), a framework that
organizes and prioritizes valuable experiences, and employs a mixed-policy
objective to balance exploration with experience exploitation. Experiments on
five backbone models (1.5B-8B parameters) show that ExGRPO consistently
improves reasoning performance on mathematical/general benchmarks, with an
average gain of +3.5/7.6 points over on-policy RLVR. Moreover, ExGRPO
stabilizes training on both stronger and weaker models where on-policy methods
fail. These results highlight principled experience management as a key
ingredient for efficient and scalable RLVR.
Go with Your Gut: Scaling Confidence for Autoregressive Image Generation
Authors: name: Harold Haodong Chen, name: Xianfeng Wu, name: Wen-Jie Shu, name: Rongjin Guo, name: Disen Lan, name: Harry Yang, name: Ying-Cong Chen
Published: 2025-09-30 15:08:25

Test-time scaling (TTS) has demonstrated remarkable success in enhancing
large language models, yet its application to next-token prediction (NTP)
autoregressive (AR) image generation remains largely uncharted. Existing TTS
approaches for visual AR (VAR), which rely on frequent partial decoding and
external reward models, are ill-suited for NTP-based image generation due to
the inherent incompleteness of intermediate decoding results. To bridge this
gap, we introduce ScalingAR, the first TTS framework specifically designed for
NTP-based AR image generation that eliminates the need for early decoding or
auxiliary rewards. ScalingAR leverages token entropy as a novel signal in
visual token generation and operates at two complementary scaling levels: (i)
Profile Level, which streams a calibrated confidence state by fusing intrinsic
and conditional signals; and (ii) Policy Level, which utilizes this state to
adaptively terminate low-confidence trajectories and dynamically schedule
guidance for phase-appropriate conditioning strength. Experiments on both
general and compositional benchmarks show that ScalingAR (1) improves base
models by 12.5% on GenEval and 15.2% on TIIF-Bench, (2) efficiently reduces
visual token consumption by 62.0% while outperforming baselines, and (3)
successfully enhances robustness, mitigating performance drops by 26.0% in
challenging scenarios.
Parallel Scaling Law: Unveiling Reasoning Generalization through A
  Cross-Linguistic Perspective
Authors: name: Wen Yang, name: Junhong Wu, name: Chong Li, name: Chengqing Zong, name: Jiajun Zhang
Published: 2025-10-02 17:49:49

Recent advancements in Reinforcement Post-Training (RPT) have significantly
enhanced the capabilities of Large Reasoning Models (LRMs), sparking increased
interest in the generalization of RL-based reasoning. While existing work has
primarily focused on investigating its generalization across tasks or
modalities, this study proposes a novel cross-linguistic perspective to
investigate reasoning generalization. This raises a crucial question:
Does the reasoning capability achieved from English RPT effectively
transfer to other languages? We address this by systematically evaluating
English-centric LRMs on multilingual reasoning benchmarks and introducing a
metric to quantify cross-lingual transferability. Our findings reveal that
cross-lingual transferability varies significantly across initial model, target
language, and training paradigm. Through interventional studies, we find that
models with stronger initial English capabilities tend to over-rely on
English-specific patterns, leading to diminished cross-lingual generalization.
To address this, we conduct a thorough parallel training study. Experimental
results yield three key findings: First-Parallel Leap, a substantial
leap in performance when transitioning from monolingual to just a single
parallel language, and a predictable Parallel Scaling Law, revealing
that cross-lingual reasoning transfer follows a power-law with the number of
training parallel languages. Moreover, we identify the discrepancy between
actual monolingual performance and the power-law prediction as
Monolingual Generalization Gap, indicating that English-centric LRMs
fail to fully generalize across languages. Our study challenges the assumption
that LRM reasoning mirrors human cognition, providing critical insights for the
development of more language-agnostic LRMs.
Group-Relative REINFORCE Is Secretly an Off-Policy Algorithm:
  Demystifying Some Myths About GRPO and Its Friends
Authors: name: Chaorui Yao, name: Yanxi Chen, name: Yuchang Sun, name: Yushuo Chen, name: Wenhao Zhang, name: Xuchen Pan, name: Yaliang Li, name: Bolin Ding
Published: 2025-09-29 02:34:54

Off-policy reinforcement learning (RL) for large language models (LLMs) is
attracting growing interest, driven by practical constraints in real-world
applications, the complexity of LLM-RL infrastructure, and the need for further
innovations of RL methodologies. While classic REINFORCE and its modern
variants like Group Relative Policy Optimization (GRPO) are typically regarded
as on-policy algorithms with limited tolerance of off-policyness, we present in
this work a first-principles derivation for group-relative REINFORCE without
assuming a specific training data distribution, showing that it admits a native
off-policy interpretation. This perspective yields two general principles for
adapting REINFORCE to off-policy settings: regularizing policy updates, and
actively shaping the data distribution. Our analysis demystifies some myths
about the roles of importance sampling and clipping in GRPO, unifies and
reinterprets two recent algorithms -- Online Policy Mirror Descent (OPMD) and
Asymmetric REINFORCE (AsymRE) -- as regularized forms of the REINFORCE loss,
and offers theoretical justification for seemingly heuristic data-weighting
strategies. Our findings lead to actionable insights that are validated with
extensive empirical studies, and open up new opportunities for principled
algorithm design in off-policy RL for LLMs. Source code for this work is
available at
https://github.com/modelscope/Trinity-RFT/tree/main/examples/rec_gsm8k.
The Unreasonable Effectiveness of Scaling Agents for Computer Use
Authors: name: Gonzalo Gonzalez-Pumariega, name: Vincent Tu, name: Chih-Lun Lee, name: Jiachen Yang, name: Ang Li, name: Xin Eric Wang
Published: 2025-10-02 17:37:08

Computer-use agents (CUAs) hold promise for automating everyday digital
tasks, but their unreliability and high variance hinder their application to
long-horizon, complex tasks. We introduce Behavior Best-of-N (bBoN), a method
that scales over agents by generating multiple rollouts and selecting among
them using behavior narratives that describe the agents' rollouts. It enables
both wide exploration and principled trajectory selection, substantially
improving robustness and success rates. On OSWorld, our bBoN scaling method
establishes a new state of the art (SoTA) at 69.9%, significantly outperforming
prior methods and approaching human-level performance at 72%, with
comprehensive ablations validating key design choices. We further demonstrate
strong generalization results to different operating systems on
WindowsAgentArena and AndroidWorld. Crucially, our results highlight the
unreasonable effectiveness of scaling CUAs, when you do it right: effective
scaling requires structured trajectory understanding and selection, and bBoN
provides a practical framework to achieve this.
VLA-R1: Enhancing Reasoning in Vision-Language-Action Models
Authors: name: Angen Ye, name: Zeyu Zhang, name: Boyuan Wang, name: Xiaofeng Wang, name: Dapeng Zhang, name: Zheng Zhu
Published: 2025-10-02 02:54:03

Vision-Language-Action (VLA) models aim to unify perception, language
understanding, and action generation, offering strong cross-task and
cross-scene generalization with broad impact on embodied AI. However, current
VLA models often lack explicit step-by-step reasoning, instead emitting final
actions without considering affordance constraints or geometric relations.
Their post-training pipelines also rarely reinforce reasoning quality, relying
primarily on supervised fine-tuning with weak reward design. To address these
challenges, we present VLA-R1, a reasoning-enhanced VLA that integrates
Reinforcement Learning from Verifiable Rewards (RLVR) with Group Relative
Policy Optimization (GRPO) to systematically optimize both reasoning and
execution. Specifically, we design an RLVR-based post-training strategy with
verifiable rewards for region alignment, trajectory consistency, and output
formatting, thereby strengthening reasoning robustness and execution accuracy.
Moreover, we develop VLA-CoT-13K, a high-quality dataset that provides
chain-of-thought supervision explicitly aligned with affordance and trajectory
annotations. Furthermore, extensive evaluations on in-domain, out-of-domain,
simulation, and real-robot platforms demonstrate that VLA-R1 achieves superior
generalization and real-world performance compared to prior VLA methods. We
plan to release the model, code, and dataset following the publication of this
work. Code: https://github.com/GigaAI-research/VLA-R1. Website:
https://gigaai-research.github.io/VLA-R1.
RLP: Reinforcement as a Pretraining Objective
Authors: name: Ali Hatamizadeh, name: Syeda Nahida Akter, name: Shrimai Prabhumoye, name: Jan Kautz, name: Mostofa Patwary, name: Mohammad Shoeybi, name: Bryan Catanzaro, name: Yejin Choi
Published: 2025-09-26 17:53:54

The dominant paradigm for training large reasoning models starts with
pre-training using next-token prediction loss on vast amounts of data.
Reinforcement learning, while powerful in scaling reasoning, is introduced only
as the very last phase of post-training, preceded by supervised fine-tuning.
While dominant, is this an optimal way of training? In this paper, we present
RLP, an information-driven reinforcement pretraining objective, that brings the
core spirit of reinforcement learning -- exploration -- to the last phase of
pretraining. The key idea is to treat chain-of-thought as an exploratory
action, with rewards computed based on the information gain it provides for
predicting future tokens. This training objective essentially encourages the
model to think for itself before predicting what comes next, thus teaching an
independent thinking behavior earlier in the pretraining. More concretely, the
reward signal measures the increase in log-likelihood of the next token when
conditioning on both context and a sampled reasoning chain, compared to
conditioning on context alone. This approach yields a verifier-free dense
reward signal, allowing for efficient training for the full document stream
during pretraining. Specifically, RLP reframes reinforcement learning for
reasoning as a pretraining objective on ordinary text, bridging the gap between
next-token prediction and the emergence of useful chain-of-thought reasoning.
Pretraining with RLP on Qwen3-1.7B-Base lifts the overall average across an
eight-benchmark math-and-science suite by 19%. With identical post-training,
the gains compound, with the largest improvements on reasoning-heavy tasks such
as AIME25 and MMLU-Pro. Applying RLP to the hybrid Nemotron-Nano-12B-v2
increases the overall average from 42.81% to 61.32% and raises the average on
scientific reasoning by 23%, demonstrating scalability across architectures and
model sizes.
A Rigorous Benchmark with Multidimensional Evaluation for Deep Research
  Agents: From Answers to Reports
Authors: name: Yang Yao, name: Yixu Wang, name: Yuxuan Zhang, name: Yi Lu, name: Tianle Gu, name: Lingyu Li, name: Dingyi Zhao, name: Keming Wu, name: Haozhe Wang, name: Ping Nie, name: Yan Teng, name: Yingchun Wang
Published: 2025-10-02 16:40:02

Artificial intelligence is undergoing the paradigm shift from closed language
models to interconnected agent systems capable of external perception and
information integration. As a representative embodiment, Deep Research Agents
(DRAs) systematically exhibit the capabilities for task decomposition,
cross-source retrieval, multi-stage reasoning, and structured output, which
markedly enhance performance on complex and open-ended tasks. However, existing
benchmarks remain deficient in evaluation dimensions, response formatting, and
scoring mechanisms, limiting their capacity to assess such systems effectively.
This paper introduces a rigorous benchmark and a multidimensional evaluation
framework tailored to DRAs and report-style responses. The benchmark comprises
214 expert-curated challenging queries distributed across 10 broad thematic
domains, each accompanied by manually constructed reference bundles to support
composite evaluation. The framework enables comprehensive evaluation of
long-form reports generated by DRAs, incorporating integrated scoring metrics
for semantic quality, topical focus, and retrieval trustworthiness. Extensive
experimentation confirms the superior performance of mainstream DRAs over
web-search-tool-augmented reasoning models, yet reveals considerable scope for
further improvement. This study provides a robust foundation for capability
assessment, architectural refinement, and paradigm advancement in DRA systems.
FrameThinker: Learning to Think with Long Videos via Multi-Turn Frame
  Spotlighting
Authors: name: Zefeng He, name: Xiaoye Qu, name: Yafu Li, name: Siyuan Huang, name: Daizong Liu, name: Yu Cheng
Published: 2025-09-29 05:36:58

While Large Vision-Language Models (LVLMs) have achieved substantial progress
in video understanding, their application to long video reasoning is hindered
by uniform frame sampling and static textual reasoning, which are inefficient
and struggle to handle visually intensive video tasks. To overcome these
challenges, in this paper, we introduce the concept of thinking with long
videos and propose a novel framework FrameThinker. Within this framework, LVLMs
are able to iteratively interrogate video content. Developing such video
reasoning capabilities in LVLMs presents notable challenges, particularly in
adapting the model to new video actions (e.g. select frame), and designing
reward functions to guide LVLMs to adopt the newly introduced action. To solve
these challenges, we propose a two-phase training strategy, first employing
Supervised Fine-Tuning (SFT) to instill fundamental action capabilities,
followed by Reinforcement Learning (RL) to optimize a strategic decision-making
policy. Notably, in this RL phase, we conduct an in-depth and comprehensive
exploration of the reward design for each action and format reward. Extensive
experiments on reasoning benchmarks like Video-Holmes, LongVideo-Reason, and
long-video understanding benchmarks such as LongVideoBench, MLVU, VideoMME, and
LVBench, demonstrate that FrameThinker achieves a significant average
improvement of +10.4% over baselines while drastically reducing the number of
processed frames. Most notably, our 7B model, FrameThinker establishes a new
state-of-the-art on LongVideo-Reason, achieving 76.1% accuracy using an average
of only 20.6 frames. This not only outperforms the competitive LongVILA-R1
(72.0%) but does so with over 20x fewer frames (vs. 512), demonstrating
unparalleled efficiency and effectiveness.
Just Do It!? Computer-Use Agents Exhibit Blind Goal-Directedness
Authors: name: Erfan Shayegani, name: Keegan Hines, name: Yue Dong, name: Nael Abu-Ghazaleh, name: Roman Lutz, name: Spencer Whitehead, name: Vidhisha Balachandran, name: Besmira Nushi, name: Vibhav Vineet
Published: 2025-10-02 04:52:15

Computer-Use Agents (CUAs) are an increasingly deployed class of agents that
take actions on GUIs to accomplish user goals. In this paper, we show that CUAs
consistently exhibit Blind Goal-Directedness (BGD): a bias to pursue goals
regardless of feasibility, safety, reliability, or context. We characterize
three prevalent patterns of BGD: (i) lack of contextual reasoning, (ii)
assumptions and decisions under ambiguity, and (iii) contradictory or
infeasible goals. We develop BLIND-ACT, a benchmark of 90 tasks capturing these
three patterns. Built on OSWorld, BLIND-ACT provides realistic environments and
employs LLM-based judges to evaluate agent behavior, achieving 93.75% agreement
with human annotations. We use BLIND-ACT to evaluate nine frontier models,
including Claude Sonnet and Opus 4, Computer-Use-Preview, and GPT-5, observing
high average BGD rates (80.8%) across them. We show that BGD exposes subtle
risks that arise even when inputs are not directly harmful. While
prompting-based interventions lower BGD levels, substantial risk persists,
highlighting the need for stronger training- or inference-time interventions.
Qualitative analysis reveals observed failure modes: execution-first bias
(focusing on how to act over whether to act), thought-action disconnect
(execution diverging from reasoning), and request-primacy (justifying actions
due to user request). Identifying BGD and introducing BLIND-ACT establishes a
foundation for future research on studying and mitigating this fundamental risk
and ensuring safe CUA deployment.
Automated Structured Radiology Report Generation with Rich Clinical
  Context
Authors: name: Seongjae Kang, name: Dong Bok Lee, name: Juho Jung, name: Dongseop Kim, name: Won Hwa Kim, name: Sunghoon Joo
Published: 2025-10-01 02:14:23

Automated structured radiology report generation (SRRG) from chest X-ray
images offers significant potential to reduce workload of radiologists by
generating reports in structured formats that ensure clarity, consistency, and
adherence to clinical reporting standards. While radiologists effectively
utilize available clinical contexts in their diagnostic reasoning, existing
SRRG systems overlook these essential elements. This fundamental gap leads to
critical problems including temporal hallucinations when referencing
non-existent clinical contexts. To address these limitations, we propose
contextualized SRRG (C-SRRG) that comprehensively incorporates rich clinical
context for SRRG. We curate C-SRRG dataset by integrating comprehensive
clinical context encompassing 1) multi-view X-ray images, 2) clinical
indication, 3) imaging techniques, and 4) prior studies with corresponding
comparisons based on patient histories. Through extensive benchmarking with
state-of-the-art multimodal large language models, we demonstrate that
incorporating clinical context with the proposed C-SRRG significantly improves
report generation quality. We publicly release dataset, code, and checkpoints
to facilitate future research for clinically-aligned automated RRG at
https://github.com/vuno/contextualized-srrg.