Orionnnn/hub-stats huggingface dataset:
FREESON: Retriever-Free Retrieval-Augmented Reasoning via
  Corpus-Traversing MCTS
Authors: name: Chaeeun Kim, name: Seungone Kim
Published: 2025-05-22 09:00:08

Large Reasoning Models (LRMs) have demonstrated remarkable capabilities in
multi-step reasoning and calling search engines at appropriate steps. However,
existing retrieval-augmented reasoning approaches rely on separate retrieval
models, limiting the LRM's role in retrieval to deciding when to retrieve and
how to query. This separation not only increases hardware and operational costs
but also leads to errors in the retrieval process due to the representation
bottleneck, a phenomenon where the retriever's embedding space is not
expressive enough to meet the generator's requirements. To address this, we
shift our perspective from sequence-to-sequence matching to locating the
answer-containing paths within the corpus, and propose a novel framework called
FREESON (Retriever-FREE Retrieval-Augmented ReaSONing). This framework enables
LRMs to retrieve relevant knowledge on their own by acting as both a generator
and retriever. To achieve this, we introduce a variant of the MCTS algorithm
specialized for the retrieval task, which we call CT-MCTS (Corpus-Traversing
Monte Carlo Tree Search). In this algorithm, LRMs traverse through the corpus
toward answer-containing regions. Our results on five open-domain QA
benchmarks, including single-hop and multi-hop questions, show that FREESON
achieves an average improvement of 14.4% in EM and F1 over four multi-step
reasoning models with a separate retriever, and it also performs comparably to
the strongest baseline, surpassing it by 3% on PopQA and 2WikiMultihopQA.
DanceTogether! Identity-Preserving Multi-Person Interactive Video
  Generation
Authors: name: Junhao Chen, name: Mingjin Chen, name: Jianjin Xu, name: Xiang Li, name: Junting Dong, name: Mingze Sun, name: Puhua Jiang, name: Hongxiang Li, name: Yuhang Yang, name: Hao Zhao, name: Xiaoxiao Long, name: Ruqi Huang
Published: 2025-05-23 16:37:14

Controllable video generation (CVG) has advanced rapidly, yet current systems
falter when more than one actor must move, interact, and exchange positions
under noisy control signals. We address this gap with DanceTogether, the first
end-to-end diffusion framework that turns a single reference image plus
independent pose-mask streams into long, photorealistic videos while strictly
preserving every identity. A novel MaskPoseAdapter binds "who" and "how" at
every denoising step by fusing robust tracking masks with semantically rich-but
noisy-pose heat-maps, eliminating the identity drift and appearance bleeding
that plague frame-wise pipelines. To train and evaluate at scale, we introduce
(i) PairFS-4K, 26 hours of dual-skater footage with 7,000+ distinct IDs, (ii)
HumanRob-300, a one-hour humanoid-robot interaction set for rapid cross-domain
transfer, and (iii) TogetherVideoBench, a three-track benchmark centered on the
DanceTogEval-100 test suite covering dance, boxing, wrestling, yoga, and figure
skating. On TogetherVideoBench, DanceTogether outperforms the prior arts by a
significant margin. Moreover, we show that a one-hour fine-tune yields
convincing human-robot videos, underscoring broad generalization to embodied-AI
and HRI tasks. Extensive ablations confirm that persistent identity-action
binding is critical to these gains. Together, our model, datasets, and
benchmark lift CVG from single-subject choreography to compositionally
controllable, multi-actor interaction, opening new avenues for digital
production, simulation, and embodied intelligence. Our video demos and code are
available at https://DanceTog.github.io/.
ReflAct: World-Grounded Decision Making in LLM Agents via Goal-State
  Reflection
Authors: name: Jeonghye Kim, name: Sojeong Rhee, name: Minbeom Kim, name: Dohyung Kim, name: Sangmook Lee, name: Youngchul Sung, name: Kyomin Jung
Published: 2025-05-21 06:57:39

Recent advances in LLM agents have largely built on reasoning backbones like
ReAct, which interleave thought and action in complex environments. However,
ReAct often produces ungrounded or incoherent reasoning steps, leading to
misalignment between the agent's actual state and goal. Our analysis finds that
this stems from ReAct's inability to maintain consistent internal beliefs and
goal alignment, causing compounding errors and hallucinations. To address this,
we introduce ReflAct, a novel backbone that shifts reasoning from merely
planning next actions to continuously reflecting on the agent's state relative
to its goal. By explicitly grounding decisions in states and enforcing ongoing
goal alignment, ReflAct dramatically improves strategic reliability. This
design delivers substantial empirical gains: ReflAct surpasses ReAct by 27.7%
on average, achieving a 93.3% success rate in ALFWorld. Notably, ReflAct even
outperforms ReAct with added enhancement modules (e.g., Reflexion, WKM),
showing that strengthening the core reasoning backbone is key to reliable agent
performance.
Diffusion Classifiers Understand Compositionality, but Conditions Apply
Authors: name: Yujin Jeong, name: Arnas Uselis, name: Seong Joon Oh, name: Anna Rohrbach
Published: 2025-05-23 14:29:52

Understanding visual scenes is fundamental to human intelligence. While
discriminative models have significantly advanced computer vision, they often
struggle with compositional understanding. In contrast, recent generative
text-to-image diffusion models excel at synthesizing complex scenes, suggesting
inherent compositional capabilities. Building on this, zero-shot diffusion
classifiers have been proposed to repurpose diffusion models for discriminative
tasks. While prior work offered promising results in discriminative
compositional scenarios, these results remain preliminary due to a small number
of benchmarks and a relatively shallow analysis of conditions under which the
models succeed. To address this, we present a comprehensive study of the
discriminative capabilities of diffusion classifiers on a wide range of
compositional tasks. Specifically, our study covers three diffusion models (SD
1.5, 2.0, and, for the first time, 3-m) spanning 10 datasets and over 30 tasks.
Further, we shed light on the role that target dataset domains play in
respective performance; to isolate the domain effects, we introduce a new
diagnostic benchmark Self-Bench comprised of images created by diffusion models
themselves. Finally, we explore the importance of timestep weighting and
uncover a relationship between domain gap and timestep sensitivity,
particularly for SD3-m. To sum up, diffusion classifiers understand
compositionality, but conditions apply! Code and dataset are available at
https://github.com/eugene6923/Diffusion-Classifiers-Compositionality.
s3: You Don't Need That Much Data to Train a Search Agent via RL
Authors: name: Pengcheng Jiang, name: Xueqiang Xu, name: Jiacheng Lin, name: Jinfeng Xiao, name: Zifeng Wang, name: Jimeng Sun, name: Jiawei Han
Published: 2025-05-20 09:53:56

Retrieval-augmented generation (RAG) systems empower large language models
(LLMs) to access external knowledge during inference. Recent advances have
enabled LLMs to act as search agents via reinforcement learning (RL), improving
information acquisition through multi-turn interactions with retrieval engines.
However, existing approaches either optimize retrieval using search-only
metrics (e.g., NDCG) that ignore downstream utility or fine-tune the entire LLM
to jointly reason and retrieve-entangling retrieval with generation and
limiting the real search utility and compatibility with frozen or proprietary
models. In this work, we propose s3, a lightweight, model-agnostic framework
that decouples the searcher from the generator and trains the searcher using a
Gain Beyond RAG reward: the improvement in generation accuracy over naive RAG.
s3 requires only 2.4k training samples to outperform baselines trained on over
70x more data, consistently delivering stronger downstream performance across
six general QA and five medical QA benchmarks.
Quartet: Native FP4 Training Can Be Optimal for Large Language Models
Authors: name: Roberto L. Castro, name: Andrei Panferov, name: Soroush Tabesh, name: Oliver Sieberling, name: Jiale Chen, name: Mahdi Nikdan, name: Saleh Ashkboos, name: Dan Alistarh
Published: 2025-05-20 17:55:50

The rapid advancement of large language models (LLMs) has been paralleled by
unprecedented increases in computational demands, with training costs for
state-of-the-art models doubling every few months. Training models directly in
low-precision arithmetic offers a solution, by improving both computational
throughput and energy efficiency. Specifically, NVIDIA's recent Blackwell
architecture facilitates extremely low-precision operations, specifically FP4
variants, promising substantial efficiency gains. Yet, current algorithms for
training LLMs in FP4 precision face significant accuracy degradation and often
rely on mixed-precision fallbacks. In this paper, we systematically investigate
hardware-supported FP4 training and introduce Quartet, a new approach enabling
accurate, end-to-end FP4 training with all the major computations (in e.g.
linear layers) being performed in low precision. Through extensive evaluations
on Llama-type models, we reveal a new low-precision scaling law that quantifies
performance trade-offs across varying bit-widths and allows us to identify a
"near-optimal" low-precision training technique in terms of
accuracy-vs-computation, called Quartet. We implement Quartet using optimized
CUDA kernels tailored for NVIDIA Blackwell GPUs, and show that it can achieve
state-of-the-art accuracy for FP4 precision, successfully training
billion-scale models. Our method demonstrates that fully FP4-based training is
a competitive alternative to standard-precision and FP8 training. Our code is
available at https://github.com/IST-DASLab/Quartet.
FullFront: Benchmarking MLLMs Across the Full Front-End Engineering
  Workflow
Authors: name: Haoyu Sun, name: Huichen Will Wang, name: Jiawei Gu, name: Linjie Li, name: Yu Cheng
Published: 2025-05-23 02:16:11

Front-end engineering involves a complex workflow where engineers
conceptualize designs, translate them into code, and iteratively refine the
implementation. While recent benchmarks primarily focus on converting visual
designs to code, we present FullFront, a benchmark designed to evaluate
Multimodal Large Language Models (MLLMs) across the full front-end
development pipeline. FullFront assesses three fundamental tasks that map
directly to the front-end engineering pipeline: Webpage Design
(conceptualization phase), Webpage Perception QA (comprehension of visual
organization and elements), and Webpage Code Generation (implementation phase).
Unlike existing benchmarks that use either scraped websites with bloated code
or oversimplified LLM-generated HTML, FullFront employs a novel, two-stage
process to transform real-world webpages into clean, standardized HTML while
maintaining diverse visual designs and avoiding copyright issues. Extensive
testing of state-of-the-art MLLMs reveals significant limitations in page
perception, code generation (particularly for image handling and layout), and
interaction implementation. Our results quantitatively demonstrate performance
disparities across models and tasks, and highlight a substantial gap between
current MLLM capabilities and human expert performance in front-end
engineering. The FullFront benchmark and code are available in
https://github.com/Mikivishy/FullFront.
Not All Models Suit Expert Offloading: On Local Routing Consistency of
  Mixture-of-Expert Models
Authors: name: Jingcong Liang, name: Siyuan Wang, name: Miren Tian, name: Yitong Li, name: Duyu Tang, name: Zhongyu Wei
Published: 2025-05-21 22:13:09

Mixture-of-Experts (MoE) enables efficient scaling of large language models
(LLMs) with sparsely activated experts during inference. To effectively deploy
large MoE models on memory-constrained devices, many systems introduce *expert
offloading* that caches a subset of experts in fast memory, leaving others on
slow memory to run on CPU or load on demand. While some research has exploited
the locality of expert activations, where consecutive tokens activate similar
experts, the degree of this **local routing consistency** varies across models
and remains understudied. In this paper, we propose two metrics to measure
local routing consistency of MoE models: (1) **Segment Routing Best Performance
(SRP)**, which evaluates how well a fixed group of experts can cover the needs
of a segment of tokens, and (2) **Segment Cache Best Hit Rate (SCH)**, which
measures the optimal segment-level cache hit rate under a given cache size
limit. We analyzed 20 MoE LLMs with diverse sizes and architectures and found
that models that apply MoE on every layer and do not use shared experts exhibit
the highest local routing consistency. We further showed that
domain-specialized experts contribute more to routing consistency than
vocabulary-specialized ones, and that most models can balance between cache
effectiveness and efficiency with cache sizes approximately 2x the active
experts. These findings pave the way for memory-efficient MoE design and
deployment without compromising inference speed. We publish the code for
replicating experiments at https://github.com/ljcleo/moe-lrc .
NOVER: Incentive Training for Language Models via Verifier-Free
  Reinforcement Learning
Authors: name: Wei Liu, name: Siya Qi, name: Xinyu Wang, name: Chen Qian, name: Yali Du, name: Yulan He
Published: 2025-05-21 21:12:35

Recent advances such as DeepSeek R1-Zero highlight the effectiveness of
incentive training, a reinforcement learning paradigm that computes rewards
solely based on the final answer part of a language model's output, thereby
encouraging the generation of intermediate reasoning steps. However, these
methods fundamentally rely on external verifiers, which limits their
applicability to domains like mathematics and coding where such verifiers are
readily available. Although reward models can serve as verifiers, they require
high-quality annotated data and are costly to train. In this work, we propose
NOVER, NO-VERifier Reinforcement Learning, a general reinforcement learning
framework that requires only standard supervised fine-tuning data with no need
for an external verifier. NOVER enables incentive training across a wide range
of text-to-text tasks and outperforms the model of the same size distilled from
large reasoning models such as DeepSeek R1 671B by 7.7 percent. Moreover, the
flexibility of NOVER enables new possibilities for optimizing large language
models, such as inverse incentive training.
Keep Security! Benchmarking Security Policy Preservation in Large
  Language Model Contexts Against Indirect Attacks in Question Answering
Authors: name: Hwan Chang, name: Yumin Kim, name: Yonghyun Jun, name: Hwanhee Lee
Published: 2025-05-21 17:58:11

As Large Language Models (LLMs) are increasingly deployed in sensitive
domains such as enterprise and government, ensuring that they adhere to
user-defined security policies within context is critical-especially with
respect to information non-disclosure. While prior LLM studies have focused on
general safety and socially sensitive data, large-scale benchmarks for
contextual security preservation against attacks remain lacking. To address
this, we introduce a novel large-scale benchmark dataset, CoPriva, evaluating
LLM adherence to contextual non-disclosure policies in question answering.
Derived from realistic contexts, our dataset includes explicit policies and
queries designed as direct and challenging indirect attacks seeking prohibited
information. We evaluate 10 LLMs on our benchmark and reveal a significant
vulnerability: many models violate user-defined policies and leak sensitive
information. This failure is particularly severe against indirect attacks,
highlighting a critical gap in current LLM safety alignment for sensitive
applications. Our analysis reveals that while models can often identify the
correct answer to a query, they struggle to incorporate policy constraints
during generation. In contrast, they exhibit a partial ability to revise
outputs when explicitly prompted. Our findings underscore the urgent need for
more robust methods to guarantee contextual security.
RBench-V: A Primary Assessment for Visual Reasoning Models with
  Multi-modal Outputs
Authors: name: Meng-Hao Guo, name: Xuanyu Chu, name: Qianrui Yang, name: Zhe-Han Mo, name: Yiqing Shen, name: Pei-lin Li, name: Xinjie Lin, name: Jinnian Zhang, name: Xin-Sheng Chen, name: Yi Zhang, name: Kiyohiro Nakayama, name: Zhengyang Geng, name: Houwen Peng, name: Han Hu, name: Shi-Nin Hu
Published: 2025-05-22 15:11:57

The rapid advancement of native multi-modal models and omni-models,
exemplified by GPT-4o, Gemini, and o3, with their capability to process and
generate content across modalities such as text and images, marks a significant
milestone in the evolution of intelligence. Systematic evaluation of their
multi-modal output capabilities in visual thinking processes (also known as
multi-modal chain of thought, M-CoT) becomes critically important. However,
existing benchmarks for evaluating multi-modal models primarily focus on
assessing multi-modal inputs and text-only reasoning while neglecting the
importance of reasoning through multi-modal outputs. In this paper, we present
a benchmark, dubbed RBench-V, designed to assess models' vision-indispensable
reasoning abilities. To construct RBench-V, we carefully hand-pick 803
questions covering math, physics, counting, and games. Unlike previous
benchmarks that typically specify certain input modalities, RBench-V presents
problems centered on multi-modal outputs, which require image manipulation such
as generating novel images and constructing auxiliary lines to support the
reasoning process. We evaluate numerous open- and closed-source models on
RBench-V, including o3, Gemini 2.5 Pro, Qwen2.5-VL, etc. Even the
best-performing model, o3, achieves only 25.8% accuracy on RBench-V, far below
the human score of 82.3%, highlighting that current models struggle to leverage
multi-modal reasoning. Data and code are available at
https://evalmodels.github.io/rbenchv
Time-R1: Towards Comprehensive Temporal Reasoning in LLMs
Authors: name: Zijia Liu, name: Peixuan Han, name: Haofei Yu, name: Haoru Li, name: Jiaxuan You
Published: 2025-05-16 13:46:28

Large Language Models (LLMs) demonstrate impressive capabilities but lack
robust temporal intelligence, struggling to integrate reasoning about the past
with predictions and plausible generations of the future. Meanwhile, existing
methods typically target isolated temporal skills, such as question answering
about past events or basic forecasting, and exhibit poor generalization,
particularly when dealing with events beyond their knowledge cutoff or
requiring creative foresight. To address these limitations, we introduce
Time-R1, the first framework to endow a moderate-sized (3B-parameter)
LLM with comprehensive temporal abilities: understanding, prediction, and
creative generation. Our approach features a novel three-stage development
path; the first two constitute a reinforcement learning (RL)
curriculum driven by a meticulously designed dynamic rule-based reward system.
This framework progressively builds (1) foundational temporal understanding and
logical event-time mappings from historical data, (2) future event prediction
skills for events beyond its knowledge cutoff, and finally (3) enables
remarkable generalization to creative future scenario generation without any
fine-tuning. Strikingly, experiments demonstrate that Time-R1 outperforms
models over 200 times larger, including the state-of-the-art 671B DeepSeek-R1,
on highly challenging future event prediction and creative scenario generation
benchmarks. This work provides strong evidence that thoughtfully engineered,
progressive RL fine-tuning allows smaller, efficient models to achieve superior
temporal performance, offering a practical and scalable path towards truly
time-aware AI. To foster further research, we also release Time-Bench,
a large-scale multi-task temporal reasoning dataset derived from 10 years of
news data, and our series of Time-R1 checkpoints.
MOOSE-Chem3: Toward Experiment-Guided Hypothesis Ranking via Simulated
  Experimental Feedback
Authors: name: Wanhao Liu, name: Zonglin Yang, name: Jue Wang, name: Lidong Bing, name: Di Zhang, name: Dongzhan Zhou, name: Yuqiang Li, name: Houqiang Li, name: Erik Cambria, name: Wanli Ouyang
Published: 2025-05-23 13:24:50

Hypothesis ranking is a crucial component of automated scientific discovery,
particularly in natural sciences where wet-lab experiments are costly and
throughput-limited. Existing approaches focus on pre-experiment ranking,
relying solely on large language model's internal reasoning without
incorporating empirical outcomes from experiments. We introduce the task of
experiment-guided ranking, which aims to prioritize candidate hypotheses based
on the results of previously tested ones. However, developing such strategies
is challenging due to the impracticality of repeatedly conducting real
experiments in natural science domains. To address this, we propose a simulator
grounded in three domain-informed assumptions, modeling hypothesis performance
as a function of similarity to a known ground truth hypothesis, perturbed by
noise. We curate a dataset of 124 chemistry hypotheses with experimentally
reported outcomes to validate the simulator. Building on this simulator, we
develop a pseudo experiment-guided ranking method that clusters hypotheses by
shared functional characteristics and prioritizes candidates based on insights
derived from simulated experimental feedback. Experiments show that our method
outperforms pre-experiment baselines and strong ablations.
PhyX: Does Your Model Have the "Wits" for Physical Reasoning?
Authors: name: Hui Shen, name: Taiqiang Wu, name: Qi Han, name: Yunta Hsieh, name: Jizhou Wang, name: Yuyue Zhang, name: Yuxin Cheng, name: Zijian Hao, name: Yuansheng Ni, name: Xin Wang, name: Zhongwei Wan, name: Kai Zhang, name: Wendong Xu, name: Jing Xiong, name: Ping Luo, name: Wenhu Chen, name: Chaofan Tao, name: Zhuoqing Mao, name: Ngai Wong
Published: 2025-05-21 18:33:50

Existing benchmarks fail to capture a crucial aspect of intelligence:
physical reasoning, the integrated ability to combine domain knowledge,
symbolic reasoning, and understanding of real-world constraints. To address
this gap, we introduce PhyX: the first large-scale benchmark designed to assess
models capacity for physics-grounded reasoning in visual scenarios. PhyX
includes 3K meticulously curated multimodal questions spanning 6 reasoning
types across 25 sub-domains and 6 core physics domains: thermodynamics,
electromagnetism, mechanics, modern physics, optics, and wave\&acoustics. In
our comprehensive evaluation, even state-of-the-art models struggle
significantly with physical reasoning. GPT-4o, Claude3.7-Sonnet, and
GPT-o4-mini achieve only 32.5\%, 42.2\%, and 45.8\% accuracy
respectively-performance gaps exceeding 29\% compared to human experts. Our
analysis exposes critical limitations in current models: over-reliance on
memorized disciplinary knowledge, excessive dependence on mathematical
formulations, and surface-level visual pattern matching rather than genuine
physical understanding. We provide in-depth analysis through fine-grained
statistics, detailed case studies, and multiple evaluation paradigms to
thoroughly examine physical reasoning capabilities. To ensure reproducibility,
we implement a compatible evaluation protocol based on widely-used toolkits
such as VLMEvalKit, enabling one-click evaluation.