Orionnnn/hub-stats huggingface dataset:
When Thoughts Meet Facts: Reusable Reasoning for Long-Context LMs
Authors: name: Soyeong Jeong, name: Taehee Jung, name: Sung Ju Hwang, name: Joo-Kyung Kim, name: Dongyeop Kang
Published: 2025-10-08 19:52:35

Recent Long-Context Language Models (LCLMs) can process hundreds of thousands
of tokens in a single prompt, enabling new opportunities for
knowledge-intensive multi-hop reasoning by integrating large sets of retrieved
documents or, in some cases, directly all necessary information. However,
simply feeding more documents into the context window fails to capture how
evidence should be connected. We address this gap with thought templates, which
recast reasoning as reusable thought caches, derived from prior problem solving
traces, structuring how evidence is combined and guiding multi-hop inference
with factual documents. To keep these templates effective, we propose an update
strategy that iteratively refines templates derived from training data through
natural-language feedback. Across diverse benchmarks and LCLM families, our
approach delivers consistent gains over strong baselines in both
retrieval-based and retrieval-free settings. Furthermore, we show that
optimized templates can be distilled into smaller open-source models,
demonstrating its broad applicability and transparent reasoning reuse. We refer
to our framework as Thought Template Augmented LCLMs (ToTAL).
ARTDECO: Towards Efficient and High-Fidelity On-the-Fly 3D
  Reconstruction with Structured Scene Representation
Authors: name: Guanghao Li, name: Kerui Ren, name: Linning Xu, name: Zhewen Zheng, name: Changjian Jiang, name: Xin Gao, name: Bo Dai, name: Jian Pu, name: Mulin Yu, name: Jiangmiao Pang
Published: 2025-10-09 17:57:38

On-the-fly 3D reconstruction from monocular image sequences is a
long-standing challenge in computer vision, critical for applications such as
real-to-sim, AR/VR, and robotics. Existing methods face a major tradeoff:
per-scene optimization yields high fidelity but is computationally expensive,
whereas feed-forward foundation models enable real-time inference but struggle
with accuracy and robustness. In this work, we propose ARTDECO, a unified
framework that combines the efficiency of feed-forward models with the
reliability of SLAM-based pipelines. ARTDECO uses 3D foundation models for pose
estimation and point prediction, coupled with a Gaussian decoder that
transforms multi-scale features into structured 3D Gaussians. To sustain both
fidelity and efficiency at scale, we design a hierarchical Gaussian
representation with a LoD-aware rendering strategy, which improves rendering
fidelity while reducing redundancy. Experiments on eight diverse indoor and
outdoor benchmarks show that ARTDECO delivers interactive performance
comparable to SLAM, robustness similar to feed-forward systems, and
reconstruction quality close to per-scene optimization, providing a practical
path toward on-the-fly digitization of real-world environments with both
accurate geometry and high visual fidelity. Explore more demos on our project
page: https://city-super.github.io/artdeco/.
The Alignment Waltz: Jointly Training Agents to Collaborate for Safety
Authors: name: Jingyu Zhang, name: Haozhu Wang, name: Eric Michael Smith, name: Sid Wang, name: Amr Sharaf, name: Mahesh Pasupuleti, name: Benjamin Van Durme, name: Daniel Khashabi, name: Jason Weston, name: Hongyuan Zhan
Published: 2025-10-09 14:03:05

Harnessing the power of LLMs requires a delicate dance between being helpful
and harmless. This creates a fundamental tension between two competing
challenges: vulnerability to adversarial attacks that elicit unsafe content,
and a tendency for overrefusal on benign but sensitive prompts. Current
approaches often navigate this dance with safeguard models that completely
reject any content that contains unsafe portions. This approach cuts the music
entirely-it may exacerbate overrefusals and fails to provide nuanced guidance
for queries it refuses. To teach models a more coordinated choreography, we
propose WaltzRL, a novel multi-agent reinforcement learning framework that
formulates safety alignment as a collaborative, positive-sum game. WaltzRL
jointly trains a conversation agent and a feedback agent, where the latter is
incentivized to provide useful suggestions that improve the safety and
helpfulness of the conversation agent's responses. At the core of WaltzRL is a
Dynamic Improvement Reward (DIR) that evolves over time based on how well the
conversation agent incorporates the feedback. At inference time, unsafe or
overrefusing responses from the conversation agent are improved rather than
discarded. The feedback agent is deployed together with the conversation agent
and only engages adaptively when needed, preserving helpfulness and low latency
on safe queries. Our experiments, conducted across five diverse datasets,
demonstrate that WaltzRL significantly reduces both unsafe responses (e.g.,
from 39.0% to 4.6% on WildJailbreak) and overrefusals (from 45.3% to 9.9% on
OR-Bench) compared to various baselines. By enabling the conversation and
feedback agents to co-evolve and adaptively apply feedback, WaltzRL enhances
LLM safety without degrading general capabilities, thereby advancing the Pareto
front between helpfulness and harmlessness.
Beyond Turn Limits: Training Deep Search Agents with Dynamic Context
  Window
Authors: name: Qiaoyu Tang, name: Hao Xiang, name: Le Yu, name: Bowen Yu, name: Yaojie Lu, name: Xianpei Han, name: Le Sun, name: WenJuan Zhang, name: Pengbo Wang, name: Shixuan Liu, name: Zhenru Zhang, name: Jianhong Tu, name: Hongyu Lin, name: Junyang Lin
Published: 2025-10-09 14:31:39

While recent advances in reasoning models have demonstrated cognitive
behaviors through reinforcement learning, existing approaches struggle to
invoke deep reasoning capabilities in multi-turn agents with long-horizon
interactions. We propose DeepMiner, a novel framework that elicits such
abilities by introducing high-difficulty training tasks and dynamic context
window. DeepMiner presents a reverse construction method to generate complex
but verifiable question-answer pairs from authentic web sources, which ensures
the challenge and reliability of training data while injecting cognitive
capabilities into multi-turn reasoning scenarios. We further design an elegant
yet effective dynamic context management strategy for both training and
inference, utilizing sliding window mechanisms while eliminating the dependency
on external summarization models, thereby efficiently empowering the model to
handle continuously expanding long-horizon contexts. Through reinforcement
learning on Qwen3-32B, we develop DeepMiner-32B, which achieves substantial
performance improvements across multiple search agent benchmarks. DeepMiner
attains 33.5% accuracy on BrowseComp-en, surpassing the previous best
open-source agent by almost 20 percentage points, and demonstrates consistent
improvements on BrowseComp-zh, XBench-DeepSearch, and GAIA. Notably, our
dynamic context management enables sustained interactions of nearly 100 turns
within standard 32k context length, effectively addressing the context
limitations that constrain existing multi-turn interaction systems.
Hybrid Reinforcement: When Reward Is Sparse, It's Better to Be Dense
Authors: name: Leitian Tao, name: Ilia Kulikov, name: Swarnadeep Saha, name: Tianlu Wang, name: Jing Xu, name: Yixuan Li, name: Jason E Weston, name: Ping Yu
Published: 2025-10-08 17:09:41

Post-training for reasoning of large language models (LLMs) increasingly
relies on verifiable rewards: deterministic checkers that provide 0-1
correctness signals. While reliable, such binary feedback is brittle--many
tasks admit partially correct or alternative answers that verifiers
under-credit, and the resulting all-or-nothing supervision limits learning.
Reward models offer richer, continuous feedback, which can serve as a
complementary supervisory signal to verifiers. We introduce HERO (Hybrid
Ensemble Reward Optimization), a reinforcement learning framework that
integrates verifier signals with reward-model scores in a structured way. HERO
employs stratified normalization to bound reward-model scores within
verifier-defined groups, preserving correctness while refining quality
distinctions, and variance-aware weighting to emphasize challenging prompts
where dense signals matter most. Across diverse mathematical reasoning
benchmarks, HERO consistently outperforms RM-only and verifier-only baselines,
with strong gains on both verifiable and hard-to-verify tasks. Our results show
that hybrid reward design retains the stability of verifiers while leveraging
the nuance of reward models to advance reasoning.
DexNDM: Closing the Reality Gap for Dexterous In-Hand Rotation via
  Joint-Wise Neural Dynamics Model
Authors: name: Xueyi Liu, name: He Wang, name: Li Yi
Published: 2025-10-09 17:59:11

Achieving generalized in-hand object rotation remains a significant challenge
in robotics, largely due to the difficulty of transferring policies from
simulation to the real world. The complex, contact-rich dynamics of dexterous
manipulation create a "reality gap" that has limited prior work to constrained
scenarios involving simple geometries, limited object sizes and aspect ratios,
constrained wrist poses, or customized hands. We address this sim-to-real
challenge with a novel framework that enables a single policy, trained in
simulation, to generalize to a wide variety of objects and conditions in the
real world. The core of our method is a joint-wise dynamics model that learns
to bridge the reality gap by effectively fitting limited amount of real-world
collected data and then adapting the sim policy's actions accordingly. The
model is highly data-efficient and generalizable across different whole-hand
interaction distributions by factorizing dynamics across joints, compressing
system-wide influences into low-dimensional variables, and learning each
joint's evolution from its own dynamic profile, implicitly capturing these net
effects. We pair this with a fully autonomous data collection strategy that
gathers diverse, real-world interaction data with minimal human intervention.
Our complete pipeline demonstrates unprecedented generality: a single policy
successfully rotates challenging objects with complex shapes (e.g., animals),
high aspect ratios (up to 5.33), and small sizes, all while handling diverse
wrist orientations and rotation axes. Comprehensive real-world evaluations and
a teleoperation application for complex tasks validate the effectiveness and
robustness of our approach. Website: https://meowuu7.github.io/DexNDM/
Reinforcing Diffusion Models by Direct Group Preference Optimization
Authors: name: Yihong Luo, name: Tianyang Hu, name: Jing Tang
Published: 2025-10-09 16:40:43

While reinforcement learning methods such as Group Relative Preference
Optimization (GRPO) have significantly enhanced Large Language Models, adapting
them to diffusion models remains challenging. In particular, GRPO demands a
stochastic policy, yet the most cost-effective diffusion samplers are based on
deterministic ODEs. Recent work addresses this issue by using inefficient
SDE-based samplers to induce stochasticity, but this reliance on model-agnostic
Gaussian noise leads to slow convergence. To resolve this conflict, we propose
Direct Group Preference Optimization (DGPO), a new online RL algorithm that
dispenses with the policy-gradient framework entirely. DGPO learns directly
from group-level preferences, which utilize relative information of samples
within groups. This design eliminates the need for inefficient stochastic
policies, unlocking the use of efficient deterministic ODE samplers and faster
training. Extensive results show that DGPO trains around 20 times faster than
existing state-of-the-art methods and achieves superior performance on both
in-domain and out-of-domain reward metrics. Code is available at
https://github.com/Luo-Yihong/DGPO.
UniMMVSR: A Unified Multi-Modal Framework for Cascaded Video
  Super-Resolution
Authors: name: Shian Du, name: Menghan Xia, name: Chang Liu, name: Quande Liu, name: Xintao Wang, name: Pengfei Wan, name: Xiangyang Ji
Published: 2025-10-09 12:25:16

Cascaded video super-resolution has emerged as a promising technique for
decoupling the computational burden associated with generating high-resolution
videos using large foundation models. Existing studies, however, are largely
confined to text-to-video tasks and fail to leverage additional generative
conditions beyond text, which are crucial for ensuring fidelity in multi-modal
video generation. We address this limitation by presenting UniMMVSR, the first
unified generative video super-resolution framework to incorporate hybrid-modal
conditions, including text, images, and videos. We conduct a comprehensive
exploration of condition injection strategies, training schemes, and data
mixture techniques within a latent video diffusion model. A key challenge was
designing distinct data construction and condition utilization methods to
enable the model to precisely utilize all condition types, given their varied
correlations with the target video. Our experiments demonstrate that UniMMVSR
significantly outperforms existing methods, producing videos with superior
detail and a higher degree of conformity to multi-modal conditions. We also
validate the feasibility of combining UniMMVSR with a base model to achieve
multi-modal guided generation of 4K video, a feat previously unattainable with
existing techniques.
A^2Search: Ambiguity-Aware Question Answering with Reinforcement
  Learning
Authors: name: Fengji Zhang, name: Xinyao Niu, name: Chengyang Ying, name: Guancheng Lin, name: Zhongkai Hao, name: Zhou Fan, name: Chengen Huang, name: Jacky Keung, name: Bei Chen, name: Junyang Lin
Published: 2025-10-09 08:53:31

Recent advances in Large Language Models (LLMs) and Reinforcement Learning
(RL) have led to strong performance in open-domain question answering (QA).
However, existing models still struggle with questions that admit multiple
valid answers. Standard QA benchmarks, which typically assume a single gold
answer, overlook this reality and thus produce inappropriate training signals.
Existing attempts to handle ambiguity often rely on costly manual annotation,
which is difficult to scale to multi-hop datasets such as HotpotQA and MuSiQue.
In this paper, we present A^2Search, an annotation-free, end-to-end training
framework to recognize and handle ambiguity. At its core is an automated
pipeline that detects ambiguous questions and gathers alternative answers via
trajectory sampling and evidence verification. The model is then optimized with
RL using a carefully designed AnsF1 reward, which naturally
accommodates multiple answers. Experiments on eight open-domain QA benchmarks
demonstrate that A^2Search achieves new state-of-the-art performance. With
only a single rollout, A^2Search-7B yields an average AnsF1@1
score of 48.4% across four multi-hop benchmarks, outperforming all strong
baselines, including the substantially larger ReSearch-32B (46.2%).
Extensive analyses further show that A^2Search resolves ambiguity and
generalizes across benchmarks, highlighting that embracing ambiguity is
essential for building more reliable QA systems. Our code, data, and model
weights can be found at https://github.com/zfj1998/A2Search
Recycling Pretrained Checkpoints: Orthogonal Growth of
  Mixture-of-Experts for Efficient Large Language Model Pre-Training
Authors: name: Ruizhe Wang, name: Yucheng Ding, name: Xiao Liu, name: Yaoxiang Wang, name: Peng Cheng, name: Baining Guo, name: Zhengjun Zha, name: Yeyun Gong
Published: 2025-10-09 09:45:45

The rapidly increasing computational cost of pretraining Large Language
Models necessitates more efficient approaches. Numerous computational costs
have been invested in existing well-trained checkpoints, but many of them
remain underutilized due to engineering constraints or limited model capacity.
To efficiently reuse this "sunk" cost, we propose to recycle pretrained
checkpoints by expanding their parameter counts and continuing training. We
propose orthogonal growth method well-suited for converged Mixture-of-Experts
model: interpositional layer copying for depth growth and expert duplication
with injected noise for width growth. To determine the optimal timing for such
growth across checkpoints sequences, we perform comprehensive scaling
experiments revealing that the final accuracy has a strong positive correlation
with the amount of sunk cost, indicating that greater prior investment leads to
better performance. We scale our approach to models with 70B parameters and
over 1T training tokens, achieving 10.66% accuracy gain over training from
scratch under the same additional compute budget. Our checkpoint recycling
approach establishes a foundation for economically efficient large language
model pretraining.
Training-Free Group Relative Policy Optimization
Authors: name: Yuzheng Cai, name: Siqi Cai, name: Yuchen Shi, name: Zihan Xu, name: Lichao Chen, name: Yulei Qin, name: Xiaoyu Tan, name: Gang Li, name: Zongyi Li, name: Haojia Lin, name: Yong Mao, name: Ke Li, name: Xing Sun
Published: 2025-10-09 13:18:17

Recent advances in Large Language Model (LLM) agents have demonstrated their
promising general capabilities. However, their performance in specialized
real-world domains often degrades due to challenges in effectively integrating
external tools and specific prompting strategies. While methods like agentic
reinforcement learning have been proposed to address this, they typically rely
on costly parameter updates, for example, through a process that uses
Supervised Fine-Tuning (SFT) followed by a Reinforcement Learning (RL) phase
with Group Relative Policy Optimization (GRPO) to alter the output
distribution. However, we argue that LLMs can achieve a similar effect on the
output distribution by learning experiential knowledge as a token prior, which
is a far more lightweight approach that not only addresses practical data
scarcity but also avoids the common issue of overfitting. To this end, we
propose Training-Free Group Relative Policy Optimization (Training-Free GRPO),
a cost-effective solution that enhances LLM agent performance without any
parameter updates. Our method leverages the group relative semantic advantage
instead of numerical ones within each group of rollouts, iteratively distilling
high-quality experiential knowledge during multi-epoch learning on a minimal
ground-truth data. Such knowledge serves as the learned token prior, which is
seamlessly integrated during LLM API calls to guide model behavior. Experiments
on mathematical reasoning and web searching tasks demonstrate that
Training-Free GRPO, when applied to DeepSeek-V3.1-Terminus, significantly
improves out-of-domain performance. With just a few dozen training samples,
Training-Free GRPO outperforms fine-tuned small LLMs with marginal training
data and cost.
Low-probability Tokens Sustain Exploration in Reinforcement Learning
  with Verifiable Reward
Authors: name: Guanhua Huang, name: Tingqiang Xu, name: Mingze Wang, name: Qi Yi, name: Xue Gong, name: Siheng Li, name: Ruibin Xiong, name: Kejiao Li, name: Yuhao Jiang, name: Bo Zhou
Published: 2025-10-03 17:56:13

Reinforcement Learning with Verifiable Rewards (RLVR) has propelled Large
Language Models in complex reasoning, yet its scalability is often hindered by
a training bottleneck where performance plateaus as policy entropy collapses,
signaling a loss of exploration. Previous methods typically address this by
maintaining high policy entropy, yet the precise mechanisms that govern
meaningful exploration have remained underexplored. Our analysis suggests that
an unselective focus on entropy risks amplifying irrelevant tokens and
destabilizing training. This paper investigates the exploration dynamics within
RLVR and identifies a key issue: the gradual elimination of valuable
low-probability exploratory tokens, which we term \textit{reasoning
sparks}. We find that while abundant in pre-trained models, these sparks are
systematically extinguished during RLVR due to over-penalization, leading to a
degeneracy in exploration. To address this, we introduce Low-probability
Regularization (Lp-Reg). Its core mechanism regularizes the policy towards a
heuristic proxy distribution. This proxy is constructed by filtering out
presumed noise tokens and re-normalizing the distribution over the remaining
candidates. The result is a less-noisy proxy where the probability of
reasoning sparks is amplified, which then serves as a soft
regularization target to shield these valuable tokens from elimination via KL
divergence. Experiments show that Lp-Reg enables stable on-policy training for
around 1,000 steps, a regime where baseline entropy-control methods collapse.
This sustained exploration leads to state-of-the-art performance, achieving a
60.17% average accuracy on five math benchmarks, an improvement of 2.66%
over prior methods. Code is available at https://github.com/CarlanLark/Lp-Reg.
Entropy Regularizing Activation: Boosting Continuous Control, Large
  Language Models, and Image Classification with Activation as Entropy
  Constraints
Authors: name: Zilin Kang, name: Chonghua Liao, name: Tingqiang Xu, name: Huazhe Xu
Published: 2025-10-09 17:56:17

We propose ERA, a new paradigm that constrains the sampling entropy above
given thresholds by applying specially designed activations to the outputs of
models. Our approach demonstrates broad effectiveness across different domains:
1) for large language models(LLMs), boosting the AIME 2025 score for
Qwen2.5-Math-7B by 37.4%; 2) for continuous control reinforcement learning
agents, improving performance by more than 30% over strong baselines such as
SAC on the challenging HumanoidBench; 3) for image classification, enhancing
ImageNet top-1 accuracy by 0.69% for ResNet-50. These gains are achieved with a
computational overhead of less than 7%. Our work validates output activation as
a powerful tool for entropy control, opening a new direction for designing
simpler and more robust algorithms.
UniVideo: Unified Understanding, Generation, and Editing for Videos
Authors: name: Cong Wei, name: Quande Liu, name: Zixuan Ye, name: Qiulin Wang, name: Xintao Wang, name: Pengfei Wan, name: Kun Gai, name: Wenhu Chen
Published: 2025-10-09 16:01:30

Unified multimodal models have shown promising results in multimodal content
generation and editing but remain largely limited to the image domain. In this
work, we present UniVideo, a versatile framework that extends unified modeling
to the video domain. UniVideo adopts a dual-stream design, combining a
Multimodal Large Language Model (MLLM) for instruction understanding with a
Multimodal DiT (MMDiT) for video generation. This design enables accurate
interpretation of complex multimodal instructions while preserving visual
consistency. Built on this architecture, UniVideo unifies diverse video
generation and editing tasks under a single multimodal instruction paradigm and
is jointly trained across them. Extensive experiments demonstrate that UniVideo
matches or surpasses state-of-the-art task-specific baselines in
text/image-to-video generation, in-context video generation and in-context
video editing. Notably, the unified design of UniVideo enables two forms of
generalization. First, UniVideo supports task composition, such as combining
editing with style transfer, by integrating multiple capabilities within a
single instruction. Second, even without explicit training on free-form video
editing, UniVideo transfers its editing capability from large-scale image
editing data to this setting, handling unseen instructions such as
green-screening characters or changing materials within a video. Beyond these
core capabilities, UniVideo also supports visual-prompt-based video generation,
where the MLLM interprets visual prompts and guides the MMDiT during synthesis.
To foster future research, we will release our model and code.