Orionnnn/hub-stats huggingface dataset:
Think-on-Graph 3.0: Efficient and Adaptive LLM Reasoning on
  Heterogeneous Graphs via Multi-Agent Dual-Evolving Context Retrieval
Authors: name: Xiaojun Wu, name: Cehao Yang, name: Xueyuan Lin, name: Chengjin Xu, name: Xuhui Jiang, name: Yuanliang Sun, name: Hui Xiong, name: Jia Li, name: Jian Guo
Published: 2025-09-26 00:13:10

Retrieval-Augmented Generation (RAG) and Graph-based RAG has become the
important paradigm for enhancing Large Language Models (LLMs) with external
knowledge. However, existing approaches face a fundamental trade-off. While
graph-based methods are inherently dependent on high-quality graph structures,
they face significant practical constraints: manually constructed knowledge
graphs are prohibitively expensive to scale, while automatically extracted
graphs from corpora are limited by the performance of the underlying LLM
extractors, especially when using smaller, local-deployed models. This paper
presents Think-on-Graph 3.0 (ToG-3), a novel framework that introduces
Multi-Agent Context Evolution and Retrieval (MACER) mechanism to overcome these
limitations. Our core innovation is the dynamic construction and refinement of
a Chunk-Triplets-Community heterogeneous graph index, which pioneeringly
incorporates a dual-evolution mechanism of Evolving Query and Evolving
Sub-Graph for precise evidence retrieval. This approach addresses a critical
limitation of prior Graph-based RAG methods, which typically construct a static
graph index in a single pass without adapting to the actual query. A
multi-agent system, comprising Constructor, Retriever, Reflector, and Responser
agents, collaboratively engages in an iterative process of evidence retrieval,
answer generation, sufficiency reflection, and, crucially, evolving query and
subgraph. This dual-evolving multi-agent system allows ToG-3 to adaptively
build a targeted graph index during reasoning, mitigating the inherent
drawbacks of static, one-time graph construction and enabling deep, precise
reasoning even with lightweight LLMs. Extensive experiments demonstrate that
ToG-3 outperforms compared baselines on both deep and broad reasoning
benchmarks, and ablation studies confirm the efficacy of the components of
MACER framework.
D-Artemis: A Deliberative Cognitive Framework for Mobile GUI
  Multi-Agents
Authors: name: Hongze Mi, name: Yibo Feng, name: Wenjie Lu, name: Yuqi Wang, name: Jinyuan Li, name: Song Cao, name: He Cui, name: Tengfei Tian, name: Xuelin Zhang, name: Haotian Luo, name: Di Sun, name: Naiqiang Tan, name: Gang Pan
Published: 2025-09-26 02:56:19

Graphical User Interface (GUI) agents aim to automate a wide spectrum of
human tasks by emulating user interaction. Despite rapid advancements, current
approaches are hindered by several critical challenges: data bottleneck in
end-to-end training, high cost of delayed error detection, and risk of
contradictory guidance. Inspired by the human cognitive loop of Thinking,
Alignment, and Reflection, we present D-Artemis -- a novel deliberative
framework in this paper. D-Artemis leverages a fine-grained, app-specific tip
retrieval mechanism to inform its decision-making process. It also employs a
proactive Pre-execution Alignment stage, where Thought-Action Consistency (TAC)
Check module and Action Correction Agent (ACA) work in concert to mitigate the
risk of execution failures. A post-execution Status Reflection Agent (SRA)
completes the cognitive loop, enabling strategic learning from experience.
Crucially, D-Artemis enhances the capabilities of general-purpose Multimodal
large language models (MLLMs) for GUI tasks without the need for training on
complex trajectory datasets, demonstrating strong generalization. D-Artemis
establishes new state-of-the-art (SOTA) results across both major benchmarks,
achieving a 75.8% success rate on AndroidWorld and 96.8% on ScreenSpot-V2.
Extensive ablation studies further demonstrate the significant contribution of
each component to the framework.
WebGen-Agent: Enhancing Interactive Website Generation with Multi-Level
  Feedback and Step-Level Reinforcement Learning
Authors: name: Zimu Lu, name: Houxing Ren, name: Yunqiao Yang, name: Ke Wang, name: Zhuofan Zong, name: Junting Pan, name: Mingjie Zhan, name: Hongsheng Li
Published: 2025-09-26 17:59:51

Agent systems powered by large language models (LLMs) have demonstrated
impressive performance on repository-level code-generation tasks. However, for
tasks such as website codebase generation, which depend heavily on visual
effects and user-interaction feedback, current code agents rely only on simple
code execution for feedback and verification. This approach fails to capture
the actual quality of the generated code. In this paper, we propose
WebGen-Agent, a novel website-generation agent that leverages comprehensive and
multi-level visual feedback to iteratively generate and refine the website
codebase. Detailed and expressive text descriptions and suggestions regarding
the screenshots and GUI-agent testing of the websites are generated by a visual
language model (VLM), together with scores that quantify their quality. The
screenshot and GUI-agent scores are further integrated with a backtracking and
select-best mechanism, enhancing the performance of the agent. Utilizing the
accurate visual scores inherent in the WebGen-Agent workflow, we further
introduce Step-GRPO with Screenshot and GUI-agent Feedback to improve
the ability of LLMs to act as the reasoning engine of WebGen-Agent. By using
the screenshot and GUI-agent scores at each step as the reward in Step-GRPO, we
provide a dense and reliable process supervision signal, which effectively
improves the model's website-generation ability. On the WebGen-Bench dataset,
WebGen-Agent increases the accuracy of Claude-3.5-Sonnet from 26.4% to 51.9%
and its appearance score from 3.0 to 3.9, outperforming the previous
state-of-the-art agent system. Additionally, our Step-GRPO training approach
increases the accuracy of Qwen2.5-Coder-7B-Instruct from 38.9% to 45.4% and
raises the appearance score from 3.4 to 3.7.
CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement
  Learning
Authors: name: Long Xing, name: Xiaoyi Dong, name: Yuhang Zang, name: Yuhang Cao, name: Jianze Liang, name: Qidong Huang, name: Jiaqi Wang, name: Feng Wu, name: Dahua Lin
Published: 2025-09-26 17:59:55

Image captioning is a fundamental task that bridges the visual and linguistic
domains, playing a critical role in pre-training Large Vision-Language Models
(LVLMs). Current state-of-the-art captioning models are typically trained with
Supervised Fine-Tuning (SFT), a paradigm that relies on expensive, non-scalable
data annotated by humans or proprietary models. This approach often leads to
models that memorize specific ground-truth answers, limiting their generality
and ability to generate diverse, creative descriptions. To overcome the
limitation of SFT, we propose applying the Reinforcement Learning with
Verifiable Rewards (RLVR) paradigm to the open-ended task of image captioning.
A primary challenge, however, is designing an objective reward function for the
inherently subjective nature of what constitutes a "good" caption. We introduce
Captioning Reinforcement Learning (CapRL), a novel training framework that
redefines caption quality through its utility: a high-quality caption should
enable a non-visual language model to accurately answer questions about the
corresponding image. CapRL employs a decoupled two-stage pipeline where an LVLM
generates a caption, and the objective reward is derived from the accuracy of a
separate, vision-free LLM answering Multiple-Choice Questions based solely on
that caption. As the first study to apply RLVR to the subjective image
captioning task, we demonstrate that CapRL significantly enhances multiple
settings. Pretraining on the CapRL-5M caption dataset annotated by CapRL-3B
results in substantial gains across 12 benchmarks. Moreover, within the Prism
Framework for caption quality evaluation, CapRL achieves performance comparable
to Qwen2.5-VL-72B, while exceeding the baseline by an average margin of 8.4%.
Code is available here: https://github.com/InternLM/CapRL.
UltraHorizon: Benchmarking Agent Capabilities in Ultra Long-Horizon
  Scenarios
Authors: name: Haotian Luo, name: Huaisong Zhang, name: Xuelin Zhang, name: Haoyu Wang, name: Zeyu Qin, name: Wenjie Lu, name: Guozheng Ma, name: Haiying He, name: Yingsha Xie, name: Qiyang Zhou, name: Zixuan Hu, name: Hongze Mi, name: Yibo Wang, name: Naiqiang Tan, name: Hong Chen, name: Yi R. Fung, name: Chun Yuan, name: Li Shen
Published: 2025-09-26 02:04:00

Autonomous agents have recently achieved remarkable progress across diverse
domains, yet most evaluations focus on short-horizon, fully observable tasks.
In contrast, many critical real-world tasks, such as large-scale software
development, commercial investment, and scientific discovery, unfold in
long-horizon and partially observable scenarios where success hinges on
sustained reasoning, planning, memory management, and tool use. Existing
benchmarks rarely capture these long-horizon challenges, leaving a gap in
systematic evaluation. To bridge this gap, we introduce UltraHorizon a
novel benchmark that measures the foundational capabilities essential for
complex real-world challenges. We use exploration as a unifying task across
three distinct environments to validate these core competencies. Agents are
designed in long-horizon discovery tasks where they must iteratively uncover
hidden rules through sustained reasoning, planning, memory and tools
management, and interaction with environments. Under the heaviest scale
setting, trajectories average 200k+ tokens and 400+ tool
calls, whereas in standard configurations they still exceed 35k tokens
and involve more than 60 tool calls on average. Our extensive
experiments reveal that LLM-agents consistently underperform in these settings,
whereas human participants achieve higher scores, underscoring a persistent gap
in agents' long-horizon abilities. We also observe that simple scaling fails in
our task. To better illustrate the failure of agents, we conduct an in-depth
analysis of collected trajectories. We identify eight types of errors and
attribute them to two primary causes: in-context locking and functional
fundamental capability gaps.
https://github.com/StarDewXXX/UltraHorizon{Our code will be available
here.}
Variational Reasoning for Language Models
Authors: name: Xiangxin Zhou, name: Zichen Liu, name: Haonan Wang, name: Chao Du, name: Min Lin, name: Chongxuan Li, name: Liang Wang, name: Tianyu Pang
Published: 2025-09-26 17:58:10

We introduce a variational reasoning framework for language models that
treats thinking traces as latent variables and optimizes them through
variational inference. Starting from the evidence lower bound (ELBO), we extend
it to a multi-trace objective for tighter bounds and propose a forward-KL
formulation that stabilizes the training of the variational posterior. We
further show that rejection sampling finetuning and binary-reward RL, including
GRPO, can be interpreted as local forward-KL objectives, where an implicit
weighting by model accuracy naturally arises from the derivation and reveals a
previously unnoticed bias toward easier questions. We empirically validate our
method on the Qwen 2.5 and Qwen 3 model families across a wide range of
reasoning tasks. Overall, our work provides a principled probabilistic
perspective that unifies variational inference with RL-style methods and yields
stable objectives for improving the reasoning ability of language models. Our
code is available at https://github.com/sail-sg/variational-reasoning.
Language Models Can Learn from Verbal Feedback Without Scalar Rewards
Authors: name: Renjie Luo, name: Zichen Liu, name: Xiangyan Liu, name: Chao Du, name: Min Lin, name: Wenhu Chen, name: Wei Lu, name: Tianyu Pang
Published: 2025-09-26 17:58:27

LLMs are often trained with RL from human or AI feedback, yet such methods
typically compress nuanced feedback into scalar rewards, discarding much of
their richness and inducing scale imbalance. We propose treating verbal
feedback as a conditioning signal. Inspired by language priors in text-to-image
generation, which enable novel outputs from unseen prompts, we introduce the
feedback-conditional policy (FCP). FCP learns directly from response-feedback
pairs, approximating the feedback-conditional posterior through maximum
likelihood training on offline data. We further develop an online bootstrapping
stage where the policy generates under positive conditions and receives fresh
feedback to refine itself. This reframes feedback-driven learning as
conditional generation rather than reward optimization, offering a more
expressive way for LLMs to directly learn from verbal feedback. Our code is
available at https://github.com/sail-sg/feedback-conditional-policy.
Learn the Ropes, Then Trust the Wins: Self-imitation with Progressive
  Exploration for Agentic Reinforcement Learning
Authors: name: Yulei Qin, name: Xiaoyu Tan, name: Zhengbao He, name: Gang Li, name: Haojia Lin, name: Zongyi Li, name: Zihan Xu, name: Yuchen Shi, name: Siqi Cai, name: Renting Rui, name: Shaofei Cai, name: Yuzheng Cai, name: Xuan Zhang, name: Sheng Ye, name: Ke Li, name: Xing Sun
Published: 2025-09-26 17:20:38

Reinforcement learning (RL) is the dominant paradigm for sharpening strategic
tool use capabilities of LLMs on long-horizon, sparsely-rewarded agent tasks,
yet it faces a fundamental challenge of exploration-exploitation trade-off.
Existing studies stimulate exploration through the lens of policy entropy, but
such mechanical entropy maximization is prone to RL training instability due to
the multi-turn distribution shifting. In this paper, we target the progressive
exploration-exploitation balance under the guidance of the agent own
experiences without succumbing to either entropy collapsing or runaway
divergence. We propose SPEAR, a curriculum-based self-imitation learning (SIL)
recipe for training agentic LLMs. It extends the vanilla SIL framework, where a
replay buffer stores self-generated promising trajectories for off-policy
update, by gradually steering the policy evolution within a well-balanced range
of entropy across stages. Specifically, our approach incorporates a curriculum
to manage the exploration process, utilizing intrinsic rewards to foster
skill-level exploration and facilitating action-level exploration through SIL.
At first, the auxiliary tool call reward plays a critical role in the
accumulation of tool-use skills, enabling broad exposure to the unfamiliar
distributions of the environment feedback with an upward entropy trend. As
training progresses, self-imitation gets strengthened to exploit existing
successful patterns from replayed experiences for comparative action-level
exploration, accelerating solution iteration without unbounded entropy growth.
To further stabilize training, we recalibrate the advantages of experiences in
the replay buffer to address the potential policy drift. Reugularizations such
as the clipping of tokens with high covariance between probability and
advantage are introduced to the trajectory-level entropy control to curb
over-confidence.
VoiceAssistant-Eval: Benchmarking AI Assistants across Listening,
  Speaking, and Viewing
Authors: name: Ke Wang, name: Houxing Ren, name: Zimu Lu, name: Mingjie Zhan, name: Hongsheng Li
Published: 2025-09-26 17:59:59

The growing capabilities of large language models and multimodal systems have
spurred interest in voice-first AI assistants, yet existing benchmarks are
inadequate for evaluating the full range of these systems' capabilities. We
introduce VoiceAssistant-Eval, a comprehensive benchmark designed to assess AI
assistants across listening, speaking, and viewing. VoiceAssistant-Eval
comprises 10,497 curated examples spanning 13 task categories. These tasks
include natural sounds, music, and spoken dialogue for listening; multi-turn
dialogue, role-play imitation, and various scenarios for speaking; and highly
heterogeneous images for viewing. To demonstrate its utility, we evaluate 21
open-source models and GPT-4o-Audio, measuring the quality of the response
content and speech, as well as their consistency. The results reveal three key
findings: (1) proprietary models do not universally outperform open-source
models; (2) most models excel at speaking tasks but lag in audio understanding;
and (3) well-designed smaller models can rival much larger ones. Notably, the
mid-sized Step-Audio-2-mini (7B) achieves more than double the listening
accuracy of LLaMA-Omni2-32B-Bilingual. However, challenges remain: multimodal
(audio plus visual) input and role-play voice imitation tasks are difficult for
current models, and significant gaps persist in robustness and safety
alignment. VoiceAssistant-Eval identifies these gaps and establishes a rigorous
framework for evaluating and guiding the development of next-generation AI
assistants. Code and data will be released at
https://mathllm.github.io/VoiceAssistantEval/ .
Quantile Advantage Estimation for Entropy-Safe Reasoning
Authors: name: Junkang Wu, name: Kexin Huang, name: Jiancan Wu, name: An Zhang, name: Xiang Wang, name: Xiangnan He
Published: 2025-09-26 17:37:52

Reinforcement Learning with Verifiable Rewards (RLVR) strengthens LLM
reasoning, but training often oscillates between {entropy collapse} and
{entropy explosion}. We trace both hazards to the mean baseline used in
value-free RL (e.g., GRPO and DAPO), which improperly penalizes
negative-advantage samples under reward outliers. We propose {Quantile
Advantage Estimation} (QAE), replacing the mean with a group-wise K-quantile
baseline. QAE induces a response-level, two-regime gate: on hard queries (p <=
1 - K) it reinforces rare successes, while on easy queries (p > 1 - K) it
targets remaining failures. Under first-order softmax updates, we prove
{two-sided entropy safety}, giving lower and upper bounds on one-step entropy
change that curb explosion and prevent collapse. Empirically, this minimal
modification stabilizes entropy, sparsifies credit assignment (with tuned K,
roughly 80% of responses receive zero advantage), and yields sustained pass@1
gains on Qwen3-8B/14B-Base across AIME 2024/2025 and AMC 2023. These results
identify {baseline design} -- rather than token-level heuristics -- as the
primary mechanism for scaling RLVR.
LongLive: Real-time Interactive Long Video Generation
Authors: name: Shuai Yang, name: Wei Huang, name: Ruihang Chu, name: Yicheng Xiao, name: Yuyang Zhao, name: Xianbang Wang, name: Muyang Li, name: Enze Xie, name: Yingcong Chen, name: Yao Lu, name: Song Han, name: Yukang Chen
Published: 2025-09-26 17:48:24

We present LongLive, a frame-level autoregressive (AR) framework for
real-time and interactive long video generation. Long video generation presents
challenges in both efficiency and quality. Diffusion and Diffusion-Forcing
models can produce high-quality videos but suffer from low efficiency due to
bidirectional attention. Causal attention AR models support KV caching for
faster inference, but often degrade in quality on long videos due to memory
challenges during long-video training. In addition, beyond static prompt-based
generation, interactive capabilities, such as streaming prompt inputs, are
critical for dynamic content creation, enabling users to guide narratives in
real time. This interactive requirement significantly increases complexity,
especially in ensuring visual consistency and semantic coherence during prompt
transitions. To address these challenges, LongLive adopts a causal, frame-level
AR design that integrates a KV-recache mechanism that refreshes cached states
with new prompts for smooth, adherent switches; streaming long tuning to enable
long video training and to align training and inference (train-long-test-long);
and short window attention paired with a frame-level attention sink, shorten as
frame sink, preserving long-range consistency while enabling faster generation.
With these key designs, LongLive fine-tunes a 1.3B-parameter short-clip model
to minute-long generation in just 32 GPU-days. At inference, LongLive sustains
20.7 FPS on a single NVIDIA H100, achieves strong performance on VBench in both
short and long videos. LongLive supports up to 240-second videos on a single
H100 GPU. LongLive further supports INT8-quantized inference with only marginal
quality loss.
EPO: Entropy-regularized Policy Optimization for LLM Agents
  Reinforcement Learning
Authors: name: Xu Wujiang, name: Wentian Zhao, name: Zhenting Wang, name: Li Yu-Jhe, name: Jin Can, name: Jin Mingyu, name: Mei Kai, name: Wan Kun, name: Metaxas Dimitris
Published: 2025-09-26 16:51:44

Training LLM agents in multi-turn environments with sparse rewards, where
completing a single task requires 30+ turns of interaction within an episode,
presents a fundamental challenge for reinforcement learning. We identify a
critical failure mode unique to this setting: the exploration-exploitation
cascade failure. This cascade begins with early-stage policy premature
convergence, where sparse feedback causes agents to commit to flawed,
low-entropy strategies. Subsequently, agents enter late-stage policy collapse,
where conventional entropy regularization becomes counterproductive, promoting
chaotic exploration that destabilizes training. We propose Entropy-regularized
Policy Optimization (EPO), a general framework that breaks this failure cycle
through three synergistic mechanisms: (1) adopting entropy regularization in
multi-turn settings to enhance exploration, (2) an entropy smoothing
regularizer that bounds policy entropy within historical averages to prevent
abrupt fluctuations, and (3) adaptive phase-based weighting that balances
exploration and exploitation across training. Our analysis justifies that EPO
guarantees monotonically decreasing entropy variance while maintaining
convergence. EPO achieves up to 152% performance improvement on ScienceWorld
and up to 19.8% on ALFWorld. Our work demonstrates that multi-turn
sparse-reward settings require fundamentally different entropy control than
traditional RL, with broad implications for LLM agent training.
X-Streamer: Unified Human World Modeling with Audiovisual Interaction
Authors: name: You Xie, name: Tianpei Gu, name: Zenan Li, name: Chenxu Zhang, name: Guoxian Song, name: Xiaochen Zhao, name: Chao Liang, name: Jianwen Jiang, name: Hongyi Xu, name: Linjie Luo
Published: 2025-09-25 20:53:27

We introduce X-Streamer, an end-to-end multimodal human world modeling
framework for building digital human agents capable of infinite interactions
across text, speech, and video within a single unified architecture. Starting
from a single portrait, X-Streamer enables real-time, open-ended video calls
driven by streaming multimodal inputs. At its core is a Thinker-Actor
dual-transformer architecture that unifies multimodal understanding and
generation, turning a static portrait into persistent and intelligent
audiovisual interactions. The Thinker module perceives and reasons over
streaming user inputs, while its hidden states are translated by the Actor into
synchronized multimodal streams in real time. Concretely, the Thinker leverages
a pretrained large language-speech model, while the Actor employs a chunk-wise
autoregressive diffusion model that cross-attends to the Thinker's hidden
states to produce time-aligned multimodal responses with interleaved discrete
text and audio tokens and continuous video latents. To ensure long-horizon
stability, we design inter- and intra-chunk attentions with time-aligned
multimodal positional embeddings for fine-grained cross-modality alignment and
context retention, further reinforced by chunk-wise diffusion forcing and
global identity referencing. X-Streamer runs in real time on two A100 GPUs,
sustaining hours-long consistent video chat experiences from arbitrary
portraits and paving the way toward unified world modeling of interactive
digital humans.