Orionnnn/hub-stats huggingface dataset:
NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged
  Information Guidance
Authors: name: Wenzhe Cai, name: Jiaqi Peng, name: Yuqiang Yang, name: Yujian Zhang, name: Meng Wei, name: Hanqing Wang, name: Yilun Chen, name: Tai Wang, name: Jiangmiao Pang
Published: 2025-05-13 16:20:28

Learning navigation in dynamic open-world environments is an important yet
challenging skill for robots. Most previous methods rely on precise
localization and mapping or learn from expensive real-world demonstrations. In
this paper, we propose the Navigation Diffusion Policy (NavDP), an end-to-end
framework trained solely in simulation and can zero-shot transfer to different
embodiments in diverse real-world environments. The key ingredient of NavDP's
network is the combination of diffusion-based trajectory generation and a
critic function for trajectory selection, which are conditioned on only local
observation tokens encoded from a shared policy transformer. Given the
privileged information of the global environment in simulation, we scale up the
demonstrations of good quality to train the diffusion policy and formulate the
critic value function targets with contrastive negative samples. Our
demonstration generation approach achieves about 2,500 trajectories/GPU per
day, 20times more efficient than real-world data collection, and results in
a large-scale navigation dataset with 363.2km trajectories across 1244 scenes.
Trained with this simulation dataset, NavDP achieves state-of-the-art
performance and consistently outstanding generalization capability on
quadruped, wheeled, and humanoid robots in diverse indoor and outdoor
environments. In addition, we present a preliminary attempt at using Gaussian
Splatting to make in-domain real-to-sim fine-tuning to further bridge the
sim-to-real gap. Experiments show that adding such real-to-sim data can improve
the success rate by 30\% without hurting its generalization capability.
Measuring General Intelligence with Generated Games
Authors: name: Vivek Verma, name: David Huang, name: William Chen, name: Dan Klein, name: Nicholas Tomlin
Published: 2025-05-12 04:01:03

We present gg-bench, a collection of game environments designed to evaluate
general reasoning capabilities in language models. Unlike most static
benchmarks, gg-bench is a data generating process where new evaluation
instances can be generated at will. In particular, gg-bench is synthetically
generated by (1) using a large language model (LLM) to generate natural
language descriptions of novel games, (2) using the LLM to implement each game
in code as a Gym environment, and (3) training reinforcement learning (RL)
agents via self-play on the generated games. We evaluate language models by
their winrate against these RL agents by prompting models with the game
description, current board state, and a list of valid moves, after which models
output the moves they wish to take. gg-bench is challenging: state-of-the-art
LLMs such as GPT-4o and Claude 3.7 Sonnet achieve winrates of 7-9% on gg-bench
using in-context learning, while reasoning models such as o1, o3-mini and
DeepSeek-R1 achieve average winrates of 31-36%. We release the generated games,
data generation process, and evaluation code in order to support future
modeling work and expansion of our benchmark.
ViMRHP: A Vietnamese Benchmark Dataset for Multimodal Review Helpfulness
  Prediction via Human-AI Collaborative Annotation
Authors: name: Truc Mai-Thanh Nguyen, name: Dat Minh Nguyen, name: Son T. Luu, name: Kiet Van Nguyen
Published: 2025-05-12 10:11:28

Multimodal Review Helpfulness Prediction (MRHP) is an essential task in
recommender systems, particularly in E-commerce platforms. Determining the
helpfulness of user-generated reviews enhances user experience and improves
consumer decision-making. However, existing datasets focus predominantly on
English and Indonesian, resulting in a lack of linguistic diversity, especially
for low-resource languages such as Vietnamese. In this paper, we introduce
ViMRHP (Vietnamese Multimodal Review Helpfulness Prediction), a large-scale
benchmark dataset for MRHP task in Vietnamese. This dataset covers four
domains, including 2K products with 46K reviews. Meanwhile, a large-scale
dataset requires considerable time and cost. To optimize the annotation
process, we leverage AI to assist annotators in constructing the ViMRHP
dataset. With AI assistance, annotation time is reduced (90 to 120 seconds per
task down to 20 to 40 seconds per task) while maintaining data quality and
lowering overall costs by approximately 65%. However, AI-generated annotations
still have limitations in complex annotation tasks, which we further examine
through a detailed performance analysis. In our experiment on ViMRHP, we
evaluate baseline models on human-verified and AI-generated annotations to
assess their quality differences. The ViMRHP dataset is publicly available at
https://github.com/trng28/ViMRHP
A Multi-Dimensional Constraint Framework for Evaluating and Improving
  Instruction Following in Large Language Models
Authors: name: Junjie Ye, name: Caishuang Huang, name: Zhuohan Chen, name: Wenjie Fu, name: Chenyuan Yang, name: Leyi Yang, name: Yilong Wu, name: Peng Wang, name: Meng Zhou, name: Xiaolong Yang, name: Tao Gui, name: Qi Zhang, name: Zhongchao Shi, name: Jianping Fan, name: Xuanjing Huang
Published: 2025-05-12 14:16:55

Instruction following evaluates large language models (LLMs) on their ability
to generate outputs that adhere to user-defined constraints. However, existing
benchmarks often rely on templated constraint prompts, which lack the diversity
of real-world usage and limit fine-grained performance assessment. To fill this
gap, we propose a multi-dimensional constraint framework encompassing three
constraint patterns, four constraint categories, and four difficulty levels.
Building on this framework, we develop an automated instruction generation
pipeline that performs constraint expansion, conflict detection, and
instruction rewriting, yielding 1,200 code-verifiable instruction-following
test samples. We evaluate 19 LLMs across seven model families and uncover
substantial variation in performance across constraint forms. For instance,
average performance drops from 77.67% at Level I to 32.96% at Level IV.
Furthermore, we demonstrate the utility of our approach by using it to generate
data for reinforcement learning, achieving substantial gains in instruction
following without degrading general performance. In-depth analysis indicates
that these gains stem primarily from modifications in the model's attention
modules parameters, which enhance constraint recognition and adherence. Code
and data are available in https://github.com/Junjie-Ye/MulDimIF.
SkillFormer: Unified Multi-View Video Understanding for Proficiency
  Estimation
Authors: name: Edoardo Bianchi, name: Antonio Liotta
Published: 2025-05-13 15:27:24

Assessing human skill levels in complex activities is a challenging problem
with applications in sports, rehabilitation, and training. In this work, we
present SkillFormer, a parameter-efficient architecture for unified multi-view
proficiency estimation from egocentric and exocentric videos. Building on the
TimeSformer backbone, SkillFormer introduces a CrossViewFusion module that
fuses view-specific features using multi-head cross-attention, learnable
gating, and adaptive self-calibration. We leverage Low-Rank Adaptation to
fine-tune only a small subset of parameters, significantly reducing training
costs. In fact, when evaluated on the EgoExo4D dataset, SkillFormer achieves
state-of-the-art accuracy in multi-view settings while demonstrating remarkable
computational efficiency, using 4.5x fewer parameters and requiring 3.75x fewer
training epochs than prior baselines. It excels in multiple structured tasks,
confirming the value of multi-view integration for fine-grained skill
assessment.
INTELLECT-2: A Reasoning Model Trained Through Globally Decentralized
  Reinforcement Learning
Authors: name: Prime Intellect Team, name: Sami Jaghouar, name: Justus Mattern, name: Jack Min Ong, name: Jannik Straube, name: Manveer Basra, name: Aaron Pazdera, name: Kushal Thaman, name: Matthew Di Ferrante, name: Felix Gabriel, name: Fares Obeid, name: Kemal Erdem, name: Michael Keiblinger, name: Johannes Hagemann
Published: 2025-05-12 07:24:33

We introduce INTELLECT-2, the first globally distributed reinforcement
learning (RL) training run of a 32 billion parameter language model. Unlike
traditional centralized training efforts, INTELLECT-2 trains a reasoning model
using fully asynchronous RL across a dynamic, heterogeneous swarm of
permissionless compute contributors.
  To enable a training run with this unique infrastructure, we built various
components from scratch: we introduce PRIME-RL, our training framework
purpose-built for distributed asynchronous reinforcement learning, based on top
of novel components such as TOPLOC, which verifies rollouts from untrusted
inference workers, and SHARDCAST, which efficiently broadcasts policy weights
from training nodes to inference workers.
  Beyond infrastructure components, we propose modifications to the standard
GRPO training recipe and data filtering techniques that were crucial to achieve
training stability and ensure that our model successfully learned its training
objective, thus improving upon QwQ-32B, the state of the art reasoning model in
the 32B parameter range.
  We open-source INTELLECT-2 along with all of our code and data, hoping to
encourage and enable more open research in the field of decentralized training.
LLAMAPIE: Proactive In-Ear Conversation Assistants
Authors: name: Tuochao Chen, name: Nicholas Batchelder, name: Alisa Liu, name: Noah Smith, name: Shyamnath Gollakota
Published: 2025-05-07 02:08:56

We introduce LlamaPIE, the first real-time proactive assistant designed to
enhance human conversations through discreet, concise guidance delivered via
hearable devices. Unlike traditional language models that require explicit user
invocation, this assistant operates in the background, anticipating user needs
without interrupting conversations. We address several challenges, including
determining when to respond, crafting concise responses that enhance
conversations, leveraging knowledge of the user for context-aware assistance,
and real-time, on-device processing. To achieve this, we construct a
semi-synthetic dialogue dataset and propose a two-model pipeline: a small model
that decides when to respond and a larger model that generates the response. We
evaluate our approach on real-world datasets, demonstrating its effectiveness
in providing helpful, unobtrusive assistance. User studies with our assistant,
implemented on Apple Silicon M2 hardware, show a strong preference for the
proactive assistant over both a baseline with no assistance and a reactive
model, highlighting the potential of LlamaPie to enhance live conversations.
DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for
  Dynamic Reranking in Retrieval-Augmented Generation
Authors: name: Jiashuo Sun, name: Xianrui Zhong, name: Sizhe Zhou, name: Jiawei Han
Published: 2025-05-12 05:19:01

Retrieval-augmented generation (RAG) systems combine large language models
(LLMs) with external knowledge retrieval, making them highly effective for
knowledge-intensive tasks. A crucial but often under-explored component of
these systems is the reranker, which refines retrieved documents to enhance
generation quality and explainability. The challenge of selecting the optimal
number of documents (k) remains unsolved: too few may omit critical
information, while too many introduce noise and inefficiencies. Although recent
studies have explored LLM-based rerankers, they primarily leverage internal
model knowledge and overlook the rich supervisory signals that LLMs can
provide, such as using response quality as feedback for optimizing reranking
decisions. In this paper, we propose DynamicRAG, a novel RAG framework where
the reranker dynamically adjusts both the order and number of retrieved
documents based on the query. We model the reranker as an agent optimized
through reinforcement learning (RL), using rewards derived from LLM output
quality. Across seven knowledge-intensive datasets, DynamicRAG demonstrates
superior performance, achieving state-of-the-art results. The model, data and
code are available at https://github.com/GasolSun36/DynamicRAG
Position: AI Competitions Provide the Gold Standard for Empirical Rigor
  in GenAI Evaluation
Authors: name: D. Sculley, name: Will Cukierski, name: Phil Culliton, name: Sohier Dane, name: Maggie Demkin, name: Ryan Holbrook, name: Addison Howard, name: Paul Mooney, name: Walter Reade, name: Megan Risdal, name: Nate Keating
Published: 2025-05-01 15:43:51

In this position paper, we observe that empirical evaluation in Generative AI
is at a crisis point since traditional ML evaluation and benchmarking
strategies are insufficient to meet the needs of evaluating modern GenAI models
and systems. There are many reasons for this, including the fact that these
models typically have nearly unbounded input and output spaces, typically do
not have a well defined ground truth target, and typically exhibit strong
feedback loops and prediction dependence based on context of previous model
outputs. On top of these critical issues, we argue that the problems of {\em
leakage} and {\em contamination} are in fact the most important and difficult
issues to address for GenAI evaluations. Interestingly, the field of AI
Competitions has developed effective measures and practices to combat leakage
for the purpose of counteracting cheating by bad actors within a competition
setting. This makes AI Competitions an especially valuable (but underutilized)
resource. Now is time for the field to view AI Competitions as the gold
standard for empirical rigor in GenAI evaluation, and to harness and harvest
their results with according value.
MonetGPT: Solving Puzzles Enhances MLLMs' Image Retouching Skills
Authors: name: Niladri Shekhar Dutt, name: Duygu Ceylan, name: Niloy J. Mitra
Published: 2025-05-09 16:38:27

Retouching is an essential task in post-manipulation of raw photographs.
Generative editing, guided by text or strokes, provides a new tool accessible
to users but can easily change the identity of the original objects in
unacceptable and unpredictable ways. In contrast, although traditional
procedural edits, as commonly supported by photoediting tools (e.g., Gimp,
Lightroom), are conservative, they are still preferred by professionals.
Unfortunately, professional quality retouching involves many individual
procedural editing operations that is challenging to plan for most novices. In
this paper, we ask if a multimodal large language model (MLLM) can be taught to
critique raw photographs, suggest suitable remedies, and finally realize them
with a given set of pre-authored procedural image operations. We demonstrate
that MLLMs can be first made aware of the underlying image processing
operations, by training them to solve specially designed visual puzzles.
Subsequently, such an operation-aware MLLM can both plan and propose edit
sequences. To facilitate training, given a set of expert-edited photos, we
synthesize a reasoning dataset by procedurally manipulating the expert edits
and then grounding a pretrained LLM on the visual adjustments, to synthesize
reasoning for finetuning. The proposed retouching operations are, by
construction, understandable by the users, preserve object details and
resolution, and can be optionally overridden. We evaluate our setup on a
variety of test examples and show advantages, in terms of explainability and
identity preservation, over existing generative and other procedural
alternatives. Code, data, models, and supplementary results can be found via
our project website at https://monetgpt.github.io.
Skywork-VL Reward: An Effective Reward Model for Multimodal
  Understanding and Reasoning
Authors: name: Xiaokun Wang, name: Chris, name: Jiangbo Pei, name: Wei Shen, name: Yi Peng, name: Yunzhuo Hao, name: Weijie Qiu, name: Ai Jian, name: Tianyidan Xie, name: Xuchen Song, name: Yang Liu, name: Yahui Zhou
Published: 2025-05-12 06:23:08

We propose Skywork-VL Reward, a multimodal reward model that provides reward
signals for both multimodal understanding and reasoning tasks. Our technical
approach comprises two key components: First, we construct a large-scale
multimodal preference dataset that covers a wide range of tasks and scenarios,
with responses collected from both standard vision-language models (VLMs) and
advanced VLM reasoners. Second, we design a reward model architecture based on
Qwen2.5-VL-7B-Instruct, integrating a reward head and applying multi-stage
fine-tuning using pairwise ranking loss on pairwise preference data.
Experimental evaluations show that Skywork-VL Reward achieves state-of-the-art
results on multimodal VL-RewardBench and exhibits competitive performance on
the text-only RewardBench benchmark. Furthermore, preference data constructed
based on our Skywork-VL Reward proves highly effective for training Mixed
Preference Optimization (MPO), leading to significant improvements in
multimodal reasoning capabilities. Our results underscore Skywork-VL Reward as
a significant advancement toward general-purpose, reliable reward models for
multimodal alignment. Our model has been publicly released to promote
transparency and reproducibility.
Continuous Visual Autoregressive Generation via Score Maximization
Authors: name: Chenze Shao, name: Fandong Meng, name: Jie Zhou
Published: 2025-05-12 17:58:14

Conventional wisdom suggests that autoregressive models are used to process
discrete data. When applied to continuous modalities such as visual data,
Visual AutoRegressive modeling (VAR) typically resorts to quantization-based
approaches to cast the data into a discrete space, which can introduce
significant information loss. To tackle this issue, we introduce a Continuous
VAR framework that enables direct visual autoregressive generation without
vector quantization. The underlying theoretical foundation is strictly proper
scoring rules, which provide powerful statistical tools capable of evaluating
how well a generative model approximates the true distribution. Within this
framework, all we need is to select a strictly proper score and set it as the
training objective to optimize. We primarily explore a class of training
objectives based on the energy score, which is likelihood-free and thus
overcomes the difficulty of making probabilistic predictions in the continuous
space. Previous efforts on continuous autoregressive generation, such as GIVT
and diffusion loss, can also be derived from our framework using other strictly
proper scores. Source code: https://github.com/shaochenze/EAR.
UMoE: Unifying Attention and FFN with Shared Experts
Authors: name: Yuanhang Yang, name: Chaozheng Wang, name: Jing Li
Published: 2025-05-12 06:21:44

Sparse Mixture of Experts (MoE) architectures have emerged as a promising
approach for scaling Transformer models. While initial works primarily
incorporated MoE into feed-forward network (FFN) layers, recent studies have
explored extending the MoE paradigm to attention layers to enhance model
performance. However, existing attention-based MoE layers require specialized
implementations and demonstrate suboptimal performance compared to their
FFN-based counterparts. In this paper, we aim to unify the MoE designs in
attention and FFN layers by introducing a novel reformulation of the attention
mechanism, revealing an underlying FFN-like structure within attention modules.
Our proposed architecture, UMoE, achieves superior performance through
attention-based MoE layers while enabling efficient parameter sharing between
FFN and attention components.
H^{3}DP: Triply-Hierarchical Diffusion Policy for Visuomotor
  Learning
Authors: name: Yiyang Lu, name: Yufeng Tian, name: Zhecheng Yuan, name: Xianbang Wang, name: Pu Hua, name: Zhengrong Xue, name: Huazhe Xu
Published: 2025-05-12 17:59:43

Visuomotor policy learning has witnessed substantial progress in robotic
manipulation, with recent approaches predominantly relying on generative models
to model the action distribution. However, these methods often overlook the
critical coupling between visual perception and action prediction. In this
work, we introduce Triply-Hierarchical Diffusion
Policy~(H^{\mathbf{3}DP}), a novel visuomotor learning framework
that explicitly incorporates hierarchical structures to strengthen the
integration between visual features and action generation. H^{3}DP contains
3 levels of hierarchy: (1) depth-aware input layering that organizes
RGB-D observations based on depth information; (2) multi-scale visual
representations that encode semantic features at varying levels of granularity;
and (3) a hierarchically conditioned diffusion process that aligns the
generation of coarse-to-fine actions with corresponding visual features.
Extensive experiments demonstrate that H^{3}DP yields a +27.5%
average relative improvement over baselines across 44 simulation
tasks and achieves superior performance in 4 challenging bimanual
real-world manipulation tasks. Project Page: https://lyy-iiis.github.io/h3dp/.
Physics-Assisted and Topology-Informed Deep Learning for Weather
  Prediction
Authors: name: Jiaqi Zheng, name: Qing Ling, name: Yerong Feng
Published: 2025-05-08 03:25:55

Although deep learning models have demonstrated remarkable potential in
weather prediction, most of them overlook either the physics of the
underlying weather evolution or the topology of the Earth's surface.
In light of these disadvantages, we develop PASSAT, a novel Physics-ASSisted
And Topology-informed deep learning model for weather prediction. PASSAT
attributes the weather evolution to two key factors: (i) the advection process
that can be characterized by the advection equation and the Navier-Stokes
equation; (ii) the Earth-atmosphere interaction that is difficult to both model
and calculate. PASSAT also takes the topology of the Earth's surface into
consideration, other than simply treating it as a plane. With these
considerations, PASSAT numerically solves the advection equation and the
Navier-Stokes equation on the spherical manifold, utilizes a spherical graph
neural network to capture the Earth-atmosphere interaction, and generates the
initial velocity fields that are critical to solving the advection equation
from the same spherical graph neural network. In the 5.625^circ-resolution
ERA5 data set, PASSAT outperforms both the state-of-the-art deep learning-based
weather prediction models and the operational numerical weather prediction
model IFS T42. Code and checkpoint are available at
https://github.com/Yumenomae/PASSAT_5p625.
Overflow Prevention Enhances Long-Context Recurrent LLMs
Authors: name: Assaf Ben-Kish, name: Itamar Zimerman, name: M. Jehanzeb Mirza, name: James Glass, name: Leonid Karlinsky, name: Raja Giryes
Published: 2025-05-12 17:45:05

A recent trend in LLMs is developing recurrent sub-quadratic models that
improve long-context processing efficiency. We investigate leading large
long-context models, focusing on how their fixed-size recurrent memory affects
their performance. Our experiments reveal that, even when these models are
trained for extended contexts, their use of long contexts remains
underutilized. Specifically, we demonstrate that a chunk-based inference
procedure, which identifies and processes only the most relevant portion of the
input can mitigate recurrent memory failures and be effective for many
long-context tasks: On LongBench, our method improves the overall performance
of Falcon3-Mamba-Inst-7B by 14%, Falcon-Mamba-Inst-7B by 28%,
RecurrentGemma-IT-9B by 50%, and RWKV6-Finch-7B by 51%. Surprisingly, this
simple approach also leads to state-of-the-art results in the challenging
LongBench v2 benchmark, showing competitive performance with equivalent size
Transformers. Furthermore, our findings raise questions about whether recurrent
models genuinely exploit long-range dependencies, as our single-chunk strategy
delivers stronger performance - even in tasks that presumably require
cross-context relations.