Orionnnn/hub-stats huggingface dataset:
ERGO: Entropy-guided Resetting for Generation Optimization in Multi-turn
  Language Models
Authors: name: Haziq Mohammad Khalid, name: Athikash Jeyaganthan, name: Timothy Do, name: Yicheng Fu, name: Sean O'Brien, name: Vasu Sharma, name: Kevin Zhu
Published: 2025-10-15 20:33:08

Large Language Models (LLMs) suffer significant performance degradation in
multi-turn conversations when information is presented incrementally. Given
that multi-turn conversations characterize everyday interactions with LLMs,
this degradation poses a severe challenge to real world usability. We
hypothesize that abrupt increases in model uncertainty signal misalignment in
multi-turn LLM interactions, and we exploit this insight to dynamically realign
conversational context. We introduce ERGO (Entropy-guided Resetting for
Generation Optimization), which continuously quantifies internal uncertainty
via Shannon entropy over next token distributions and triggers adaptive prompt
consolidation when a sharp spike in entropy is detected. By treating
uncertainty as a first class signal rather than a nuisance to eliminate, ERGO
embraces variability in language and modeling, representing and responding to
uncertainty. In multi-turn tasks with incrementally revealed instructions, ERGO
yields a 56.6% average performance gain over standard baselines, increases
aptitude (peak performance capability) by 24.7%, and decreases unreliability
(variability in performance) by 35.3%, demonstrating that uncertainty aware
interventions can improve both accuracy and reliability in conversational AI.
Rewiring Experts on the Fly:Continuous Rerouting for Better Online
  Adaptation in Mixture-of-Expert models
Authors: name: Guinan Su, name: Yanwu Yang, name: Li Shen, name: Lu Yin, name: Shiwei Liu, name: Jonas Geiping
Published: 2025-10-16 16:24:36

Mixture-of-Experts (MoE) models achieve efficient scaling through sparse
expert activation, but often suffer from suboptimal routing decisions due to
distribution shifts in deployment. While existing test-time adaptation methods
could potentially address these issues, they primarily focus on dense models
and require access to external data, limiting their practical applicability to
MoE architectures. However, we find that, instead of relying on reference data,
we can optimize MoE expert selection on-the-fly based only on input context. As
such, we propose a data-free, online test-time framework that
continuously adapts MoE routing decisions during text generation without
external supervision or data. Our method cycles between two phases: During the
prefill stage, and later in regular intervals, we optimize the routing
decisions of the model using self-supervision based on the already generated
sequence. Then, we generate text as normal, maintaining the modified router
until the next adaption. We implement this through lightweight additive vectors
that only update router logits in selected layers, maintaining computational
efficiency while preventing over-adaptation. The experimental results show
consistent performance gains on challenging reasoning tasks while maintaining
robustness to context shifts. For example, our method achieves a 5.5\%
improvement on HumanEval with OLMoE. Furthermore, owing to its plug-and-play
property, our method naturally complements existing test-time scaling
techniques, e.g., achieving 6\% average gains when incorporated with
self-consistency on DeepSeek-V2-Lite.
Language Models Model Language
Authors: name: Łukasz Borchmann
Published: 2025-10-14 17:45:31

Linguistic commentary on LLMs, heavily influenced by the theoretical
frameworks of de Saussure and Chomsky, is often speculative and unproductive.
Critics challenge whether LLMs can legitimately model language, citing the need
for "deep structure" or "grounding" to achieve an idealized linguistic
"competence." We argue for a radical shift in perspective towards the
empiricist principles of Witold Ma\'nczak, a prominent general and historical
linguist. He defines language not as a "system of signs" or a "computational
system of the brain" but as the totality of all that is said and written. Above
all, he identifies frequency of use of particular language elements as
language's primary governing principle. Using his framework, we challenge prior
critiques of LLMs and provide a constructive guide for designing, evaluating,
and interpreting language models.
Build Your Personalized Research Group: A Multiagent Framework for
  Continual and Interactive Science Automation
Authors: name: Ed Li, name: Junyu Ren, name: Xintian Pan, name: Cat Yan, name: Chuanhao Li, name: Dirk Bergemann, name: Zhuoran Yang
Published: 2025-10-17 13:13:32

The automation of scientific discovery represents a critical milestone in
Artificial Intelligence (AI) research. However, existing agentic systems for
science suffer from two fundamental limitations: rigid, pre-programmed
workflows that cannot adapt to intermediate findings, and inadequate context
management that hinders long-horizon research. We present
freephdlabor, an open-source multiagent framework featuring
fully dynamic workflows determined by real-time agent reasoning and a
\textit{modular architecture} enabling seamless customization --
users can modify, add, or remove agents to address domain-specific
requirements. The framework provides comprehensive infrastructure including
automatic context compaction, workspace-based communication
to prevent information degradation, memory persistence across
sessions, and non-blocking human intervention mechanisms. These
features collectively transform automated research from isolated, single-run
attempts into continual research programs that build systematically on
prior explorations and incorporate human feedback. By providing both the
architectural principles and practical implementation for building customizable
co-scientist systems, this work aims to facilitate broader adoption of
automated research across scientific domains, enabling practitioners to deploy
interactive multiagent systems that autonomously conduct end-to-end research --
from ideation through experimentation to publication-ready manuscripts.
NANO3D: A Training-Free Approach for Efficient 3D Editing Without Masks
Authors: name: Junliang Ye, name: Shenghao Xie, name: Ruowen Zhao, name: Zhengyi Wang, name: Hongyu Yan, name: Wenqiang Zu, name: Lei Ma, name: Jun Zhu
Published: 2025-10-16 17:51:50

3D object editing is essential for interactive content creation in gaming,
animation, and robotics, yet current approaches remain inefficient,
inconsistent, and often fail to preserve unedited regions. Most methods rely on
editing multi-view renderings followed by reconstruction, which introduces
artifacts and limits practicality. To address these challenges, we propose
Nano3D, a training-free framework for precise and coherent 3D object editing
without masks. Nano3D integrates FlowEdit into TRELLIS to perform localized
edits guided by front-view renderings, and further introduces region-aware
merging strategies, Voxel/Slat-Merge, which adaptively preserve structural
fidelity by ensuring consistency between edited and unedited areas. Experiments
demonstrate that Nano3D achieves superior 3D consistency and visual quality
compared with existing methods. Based on this framework, we construct the first
large-scale 3D editing datasets Nano3D-Edit-100k, which contains over 100,000
high-quality 3D editing pairs. This work addresses long-standing challenges in
both algorithm design and data availability, significantly improving the
generality and reliability of 3D editing, and laying the groundwork for the
development of feed-forward 3D editing models. Project
Page:https://jamesyjl.github.io/Nano3D
Foundation Models for Scientific Discovery: From Paradigm Enhancement to
  Paradigm Transition
Authors: name: Fan Liu, name: Jindong Han, name: Tengfei Lyu, name: Weijia Zhang, name: Zhe-Rui Yang, name: Lu Dai, name: Cancheng Liu, name: Hao Liu
Published: 2025-10-17 03:40:26

Foundation models (FMs), such as GPT-4 and AlphaFold, are reshaping the
landscape of scientific research. Beyond accelerating tasks such as hypothesis
generation, experimental design, and result interpretation, they prompt a more
fundamental question: Are FMs merely enhancing existing scientific
methodologies, or are they redefining the way science is conducted? In this
paper, we argue that FMs are catalyzing a transition toward a new scientific
paradigm. We introduce a three-stage framework to describe this evolution: (1)
Meta-Scientific Integration, where FMs enhance workflows within traditional
paradigms; (2) Hybrid Human-AI Co-Creation, where FMs become active
collaborators in problem formulation, reasoning, and discovery; and (3)
Autonomous Scientific Discovery, where FMs operate as independent agents
capable of generating new scientific knowledge with minimal human intervention.
Through this lens, we review current applications and emerging capabilities of
FMs across existing scientific paradigms. We further identify risks and future
directions for FM-enabled scientific discovery. This position paper aims to
support the scientific community in understanding the transformative role of
FMs and to foster reflection on the future of scientific discovery. Our project
is available at
https://github.com/usail-hkust/Awesome-Foundation-Models-for-Scientific-Discovery.
A^2FM: An Adaptive Agent Foundation Model for Tool-Aware Hybrid
  Reasoning
Authors: name: Qianben Chen, name: Jingyi Cao, name: Jiayu Zhang, name: Tianrui Qin, name: Xiaowan Li, name: King Zhu, name: Dingfeng Shi, name: He Zhu, name: Minghao Liu, name: Xiaobo Liang, name: Xin Gui, name: Ge Zhang, name: Jian Yang, name: Yuchen Eleanor Jiang, name: Wangchunshu Zhou
Published: 2025-10-13 17:08:25

Large language models split into two families: reasoning-centric LLMs, which
strengthen internal chain-of-thought reasoning but cannot invoke external
tools, and agentic LLMs, which learn to interact with environments and leverage
tools but often lag in deep reasoning. This divide arises from fundamentally
different training objectives, leading to mismatched strengths and inefficiency
on simple queries, where both families tend to overthink or over-call tools. In
this work, we present Adaptive Agent Foundation Model (A^2FM), a unified
framework that follows a route-then-align principle: the model first learns
task-aware routing and then aligns mode-specific trajectories under a shared
backbone. To address the inefficiency gap, we introduce a third
mode-instant-that handles simple queries directly, preventing unnecessary
reasoning or tool calls while complementing the agentic and reasoning modes. To
jointly enhance accuracy and efficiency, we propose Adaptive Policy
Optimization (APO), which enforces adaptive sampling across modes and applies a
cost-regularized reward. On the 32B scale, A^2FM achieves 13.4% on
BrowseComp, 70.4% on AIME25, and 16.7% on HLE, setting new SOTA among
comparable models and performing competitively with frontier LLMs across
agentic, reasoning, and general benchmarks. Notably, the adaptive execution
achieves a cost of pass of only $0.00487 per correct answer-cutting cost by
45.2% relative to reasoning and 33.5% relative to agentic, thus delivering
substantially higher cost efficiency while maintaining comparable accuracy.
InfiMed-ORBIT: Aligning LLMs on Open-Ended Complex Tasks via
  Rubric-Based Incremental Training
Authors: name: Pengkai Wang, name: Qi Zuo, name: Pengwei Liu, name: Zhijie Sang, name: Congkai Xie, name: Hongxia Yang
Published: 2025-10-17 17:51:28

Large Language Models (LLMs) have shown substantial advances through
reinforcement learning (RL), particularly in domains where rewards can be
programmatically verified, such as mathematics and code. In these areas, models
benefit from a well-defined operational base guided by explicit rule-based
objectives. However, this progress reveals a significant limitation: in
open-ended domains where rewards are ambiguous, subjective, or
context-dependent, such as creative writing, scientific reasoning, and notably
medical consultation, robust reward functions are lacking, making these areas
challenging for current RL strategies. To bridge this gap, we introduce ORBIT,
an open-ended rubric-based incremental training framework specifically designed
for high-stakes medical dialogue. ORBIT integrates syn- thetic dialogue
generation with the dynamic creation of rubrics, employing these rubrics to
direct an incremental RL process. In particular, this approach does not depend
on external medical knowledge or manual rules, instead utilizing rubric-guided
feedback to shape learning. When implemented on the Qwen3-4B-Instruct model,
our method can greatly enhance its performance on the HealthBench-Hard
benchmark from 7.0 to 27.2 using only 2k samples, thus achieving
state-of-the-art results for models of this scale. Our analysis confirms that
rubric-driven RL fos-ters consistent performance gains across diverse
consultation scenarios, going beyond simple numerical improvements. These
findings underscore rubric-based feedback as a scalable strategy for advancing
LLMs in intricate, open-ended tasks.
Train a Unified Multimodal Data Quality Classifier with Synthetic Data
Authors: name: Weizhi Wang, name: Rongmei Lin, name: Shiyang Li, name: Colin Lockard, name: Ritesh Sarkhel, name: Sanket Lokegaonkar, name: Jingbo Shang, name: Xifeng Yan, name: Nasser Zalmout, name: Xian Li
Published: 2025-10-16 21:53:28

The Multimodal Large Language Models (MLLMs) are continually pre-trained on a
mixture of image-text caption data and interleaved document data, while the
high-quality data filtering towards image-text interleaved document data is
under-explored. We propose to train an efficient MLLM as a Unified Mulitmodal
Data Quality Classifier to Filter both high-quality image-text caption and
interleaved data (UniFilter). To address the challenge of collecting diverse
labeled multimodal data, we introduce a semi-synthetic approach that leverages
readily available raw images and generates corresponding text across four
quality levels. This method enables efficient creation of sample-score pairs
for both caption and interleaved document data to train UniFilter. We apply
UniFilter to curate high-quality caption data from DataComp caption dataset and
interleaved data from the OBELICS image-text interleaved dataset. MLLMs
pre-trained on the filtered data demonstrate significantly enhanced
capabilities compared to those trained on baseline-filtered data, achieving
stronger zero-shot reasoning and in-context learning capabilities. After visual
supervised fine-tuning, these UniFilter-induced MLLMs achieve stronger
performance on various benchmarks, highlighting the downstream benefits of
high-quality multimodal pre-training. We release the synthetic training data
used for training UniFilter, the UniFilter model checkpoints, and the
high-quality interleaved document subset OBELICS-HQ, curated by UniFilter, to
the community for reproduction and further development.
Robust Layerwise Scaling Rules by Proper Weight Decay Tuning
Authors: name: Zhiyuan Fan, name: Yifeng Liu, name: Qingyue Zhao, name: Angela Yuan, name: Quanquan Gu
Published: 2025-10-17 02:58:35

Empirical scaling laws prescribe how to allocate parameters, data, and
compute, while maximal-update parameterization (muP) enables learning-rate
transfer across widths by equalizing early-time update magnitudes. However, in
modern scale-invariant architectures, training quickly enters an
optimizer-governed steady state where normalization layers create backward
scale sensitivity and the effective learning rate becomes width dependent,
degrading muP transfer. We address this by introducing a weight-decay
scaling rule for AdamW that preserves sublayer gain across widths. Empirically,
the singular-value spectrum of each matrix parameter scales in norm as
eta/lambda with an approximately invariant shape; under width
scaling d, we observe that the top singular value scales approximately as
eta/lambdacdot d^{0.75}. Combining this observation with the muP
learning-rate rule eta_2propto d^{-1} for matrix-like parameters implies an
empirical weight-decay scaling rule lambda_2propto d that
approximately keeps sublayer gains width invariant. Together with vector-like
parameters trained at eta_1=Theta_d(1) and lambda_1=0, this yields
zero-shot transfer of both learning rate and weight decay from proxy to
target widths, removing per-width sweeps. We validate the rule on LLaMA-style
Transformers and in a minimal synthetic setting, and we provide a simple
diagnostic, matching top singular values, to check sublayer-gain invariance.
Our results extend muP beyond the near-init regime by explicitly controlling
steady-state scales set by the optimizer, offering a practical recipe for
width-robust hyperparameter transfer under AdamW.
DLER: Doing Length pEnalty Right - Incentivizing More Intelligence per
  Token via Reinforcement Learning
Authors: name: Shih-Yang Liu, name: Xin Dong, name: Ximing Lu, name: Shizhe Diao, name: Mingjie Liu, name: Min-Hung Chen, name: Hongxu Yin, name: Yu-Chiang Frank Wang, name: Kwang-Ting Cheng, name: Yejin Choi, name: Jan Kautz, name: Pavlo Molchanov
Published: 2025-10-16 20:05:57

Reasoning language models such as OpenAI-o1, DeepSeek-R1, and Qwen achieve
strong performance via extended chains of thought but often generate
unnecessarily long outputs. Maximizing intelligence per token--accuracy
relative to response length--remains an open problem. We revisit reinforcement
learning (RL) with the simplest length penalty--truncation--and show that
accuracy degradation arises not from the lack of sophisticated penalties but
from inadequate RL optimization. We identify three key challenges: (i) large
bias in advantage estimation, (ii) entropy collapse, and (iii) sparse reward
signal. We address them with Doing Length pEnalty Right (DLER), a training
recipe combining batch-wise reward normalization, higher clipping, dynamic
sampling, and a simple truncation length penalty. DLER achieves
state-of-the-art accuracy--efficiency trade-offs, cutting output length by over
70 percent while surpassing all previous baseline accuracy. It also improves
test-time scaling: compared to DeepSeek-R1-7B, DLER-7B generates multiple
concise responses in parallel with 28 percent higher accuracy and lower
latency. We further introduce Difficulty-Aware DLER, which adaptively tightens
truncation on easier questions for additional efficiency gains. We also propose
an update-selective merging method that preserves baseline accuracy while
retaining the concise reasoning ability of the DLER model, which is useful for
scenarios where RL training data is scarce.
VISTA: A Test-Time Self-Improving Video Generation Agent
Authors: name: Do Xuan Long, name: Xingchen Wan, name: Hootan Nakhost, name: Chen-Yu Lee, name: Tomas Pfister, name: Sercan Ö. Arık
Published: 2025-10-17 17:12:08

Despite rapid advances in text-to-video synthesis, generated video quality
remains critically dependent on precise user prompts. Existing test-time
optimization methods, successful in other domains, struggle with the
multi-faceted nature of video. In this work, we introduce VISTA (Video
Iterative Self-improvemenT Agent), a novel multi-agent system that autonomously
improves video generation through refining prompts in an iterative loop. VISTA
first decomposes a user idea into a structured temporal plan. After generation,
the best video is identified through a robust pairwise tournament. This winning
video is then critiqued by a trio of specialized agents focusing on visual,
audio, and contextual fidelity. Finally, a reasoning agent synthesizes this
feedback to introspectively rewrite and enhance the prompt for the next
generation cycle. Experiments on single- and multi-scene video generation
scenarios show that while prior methods yield inconsistent gains, VISTA
consistently improves video quality and alignment with user intent, achieving
up to 60% pairwise win rate against state-of-the-art baselines. Human
evaluators concur, preferring VISTA outputs in 66.4% of comparisons.
Scaling Instruction-Based Video Editing with a High-Quality Synthetic
  Dataset
Authors: name: Qingyan Bai, name: Qiuyu Wang, name: Hao Ouyang, name: Yue Yu, name: Hanlin Wang, name: Wen Wang, name: Ka Leong Cheng, name: Shuailei Ma, name: Yanhong Zeng, name: Zichen Liu, name: Yinghao Xu, name: Yujun Shen, name: Qifeng Chen
Published: 2025-10-17 15:31:40

Instruction-based video editing promises to democratize content creation, yet
its progress is severely hampered by the scarcity of large-scale, high-quality
training data. We introduce Ditto, a holistic framework designed to tackle this
fundamental challenge. At its heart, Ditto features a novel data generation
pipeline that fuses the creative diversity of a leading image editor with an
in-context video generator, overcoming the limited scope of existing models. To
make this process viable, our framework resolves the prohibitive cost-quality
trade-off by employing an efficient, distilled model architecture augmented by
a temporal enhancer, which simultaneously reduces computational overhead and
improves temporal coherence. Finally, to achieve full scalability, this entire
pipeline is driven by an intelligent agent that crafts diverse instructions and
rigorously filters the output, ensuring quality control at scale. Using this
framework, we invested over 12,000 GPU-days to build Ditto-1M, a new dataset of
one million high-fidelity video editing examples. We trained our model, Editto,
on Ditto-1M with a curriculum learning strategy. The results demonstrate
superior instruction-following ability and establish a new state-of-the-art in
instruction-based video editing.
Latent Diffusion Model without Variational Autoencoder
Authors: name: Minglei Shi, name: Haolin Wang, name: Wenzhao Zheng, name: Ziyang Yuan, name: Xiaoshi Wu, name: Xintao Wang, name: Pengfei Wan, name: Jie Zhou, name: Jiwen Lu
Published: 2025-10-17 04:17:44

Recent progress in diffusion-based visual generation has largely relied on
latent diffusion models with variational autoencoders (VAEs). While effective
for high-fidelity synthesis, this VAE+diffusion paradigm suffers from limited
training efficiency, slow inference, and poor transferability to broader vision
tasks. These issues stem from a key limitation of VAE latent spaces: the lack
of clear semantic separation and strong discriminative structure. Our analysis
confirms that these properties are crucial not only for perception and
understanding tasks, but also for the stable and efficient training of latent
diffusion models. Motivated by this insight, we introduce SVG, a novel latent
diffusion model without variational autoencoders, which leverages
self-supervised representations for visual generation. SVG constructs a feature
space with clear semantic discriminability by leveraging frozen DINO features,
while a lightweight residual branch captures fine-grained details for
high-fidelity reconstruction. Diffusion models are trained directly on this
semantically structured latent space to facilitate more efficient learning. As
a result, SVG enables accelerated diffusion training, supports few-step
sampling, and improves generative quality. Experimental results further show
that SVG preserves the semantic and discriminative capabilities of the
underlying self-supervised representations, providing a principled pathway
toward task-general, high-quality visual representations.