Orionnnn/hub-stats huggingface dataset:
ZeroSearch: Incentivize the Search Capability of LLMs without Searching
Authors: name: Hao Sun, name: Zile Qiao, name: Jiayan Guo, name: Xuanbo Fan, name: Yingyan Hou, name: Yong Jiang, name: Pengjun Xie, name: Fei Huang, name: Yan Zhang
Published: 2025-05-07 17:30:22

Effective information searching is essential for enhancing the reasoning and
generation capabilities of large language models (LLMs). Recent research has
explored using reinforcement learning (RL) to improve LLMs' search capabilities
by interacting with live search engines in real-world environments. While these
approaches show promising results, they face two major challenges: (1)
Uncontrolled Document Quality: The quality of documents returned by search
engines is often unpredictable, introducing noise and instability into the
training process. (2) Prohibitively High API Costs: RL training requires
frequent rollouts, potentially involving hundreds of thousands of search
requests, which incur substantial API expenses and severely constrain
scalability. To address these challenges, we introduce ZeroSearch, a
reinforcement learning framework that incentivizes the search capabilities of
LLMs without interacting with real search engines. Our approach begins with
lightweight supervised fine-tuning to transform the LLM into a retrieval module
capable of generating both relevant and noisy documents in response to a query.
During RL training, we employ a curriculum-based rollout strategy that
incrementally degrades the quality of generated documents, progressively
eliciting the model's reasoning ability by exposing it to increasingly
challenging retrieval scenarios. Extensive experiments demonstrate that
ZeroSearch effectively incentivizes the search capabilities of LLMs using a 3B
LLM as the retrieval module. Remarkably, a 7B retrieval module achieves
comparable performance to the real search engine, while a 14B retrieval module
even surpasses it. Furthermore, it generalizes well across both base and
instruction-tuned models of various parameter sizes and is compatible with a
wide range of RL algorithms.
Alpha Excel Benchmark
Authors: name: David Noever, name: Forrest McKee
Published: 2025-05-07 03:56:26

This study presents a novel benchmark for evaluating Large Language Models
(LLMs) using challenges derived from the Financial Modeling World Cup (FMWC)
Excel competitions. We introduce a methodology for converting 113 existing FMWC
challenges into programmatically evaluable JSON formats and use this dataset to
compare the performance of several leading LLMs. Our findings demonstrate
significant variations in performance across different challenge categories,
with models showing specific strengths in pattern recognition tasks but
struggling with complex numerical reasoning. The benchmark provides a
standardized framework for assessing LLM capabilities in realistic
business-oriented tasks rather than abstract academic problems. This research
contributes to the growing field of AI benchmarking by establishing proficiency
among the 1.5 billion people who daily use Microsoft Excel as a meaningful
evaluation metric that bridges the gap between academic AI benchmarks and
practical business applications.
Scenethesis: A Language and Vision Agentic Framework for 3D Scene
  Generation
Authors: name: Lu Ling, name: Chen-Hsuan Lin, name: Tsung-Yi Lin, name: Yifan Ding, name: Yu Zeng, name: Yichen Sheng, name: Yunhao Ge, name: Ming-Yu Liu, name: Aniket Bera, name: Zhaoshuo Li
Published: 2025-05-05 17:59:58

Synthesizing interactive 3D scenes from text is essential for gaming, virtual
reality, and embodied AI. However, existing methods face several challenges.
Learning-based approaches depend on small-scale indoor datasets, limiting the
scene diversity and layout complexity. While large language models (LLMs) can
leverage diverse text-domain knowledge, they struggle with spatial realism,
often producing unnatural object placements that fail to respect common sense.
Our key insight is that vision perception can bridge this gap by providing
realistic spatial guidance that LLMs lack. To this end, we introduce
Scenethesis, a training-free agentic framework that integrates LLM-based scene
planning with vision-guided layout refinement. Given a text prompt, Scenethesis
first employs an LLM to draft a coarse layout. A vision module then refines it
by generating an image guidance and extracting scene structure to capture
inter-object relations. Next, an optimization module iteratively enforces
accurate pose alignment and physical plausibility, preventing artifacts like
object penetration and instability. Finally, a judge module verifies spatial
coherence. Comprehensive experiments show that Scenethesis generates diverse,
realistic, and physically plausible 3D interactive scenes, making it valuable
for virtual content creation, simulation environments, and embodied AI
research.
Teaching Models to Understand (but not Generate) High-risk Data
Authors: name: Ryan Wang, name: Matthew Finlayson, name: Luca Soldaini, name: Swabha Swayamdipta, name: Robin Jia
Published: 2025-05-05 22:24:06

Language model developers typically filter out high-risk content -- such as
toxic or copyrighted text -- from their pre-training data to prevent models
from generating similar outputs. However, removing such data altogether limits
models' ability to recognize and appropriately respond to harmful or sensitive
content. In this paper, we introduce Selective Loss to Understand but Not
Generate (SLUNG), a pre-training paradigm through which models learn to
understand high-risk data without learning to generate it. Instead of uniformly
applying the next-token prediction loss, SLUNG selectively avoids incentivizing
the generation of high-risk tokens while ensuring they remain within the
model's context window. As the model learns to predict low-risk tokens that
follow high-risk ones, it is forced to understand the high-risk content.
Through our experiments, we show that SLUNG consistently improves models'
understanding of high-risk data (e.g., ability to recognize toxic content)
without increasing its generation (e.g., toxicity of model responses). Overall,
our SLUNG paradigm enables models to benefit from high-risk text that would
otherwise be filtered out.
Which Agent Causes Task Failures and When? On Automated Failure
  Attribution of LLM Multi-Agent Systems
Authors: name: Shaokun Zhang, name: Ming Yin, name: Jieyu Zhang, name: Jiale Liu, name: Zhiguang Han, name: Jingyang Zhang, name: Beibin Li, name: Chi Wang, name: Huazheng Wang, name: Yiran Chen, name: Qingyun Wu
Published: 2025-04-30 23:09:44

Failure attribution in LLM multi-agent systems-identifying the agent and step
responsible for task failures-provides crucial clues for systems debugging but
remains underexplored and labor-intensive. In this paper, we propose and
formulate a new research area: automated failure attribution for LLM
multi-agent systems. To support this initiative, we introduce the Who&When
dataset, comprising extensive failure logs from 127 LLM multi-agent systems
with fine-grained annotations linking failures to specific agents and decisive
error steps. Using the Who&When, we develop and evaluate three automated
failure attribution methods, summarizing their corresponding pros and cons. The
best method achieves 53.5% accuracy in identifying failure-responsible agents
but only 14.2% in pinpointing failure steps, with some methods performing below
random. Even SOTA reasoning models, such as OpenAI o1 and DeepSeek R1, fail to
achieve practical usability. These results highlight the task's complexity and
the need for further research in this area. Code and dataset are available at
https://github.com/mingyin1/Agents_Failure_Attribution
SWE-smith: Scaling Data for Software Engineering Agents
Authors: name: John Yang, name: Kilian Leret, name: Carlos E. Jimenez, name: Alexander Wettig, name: Kabir Khandpur, name: Yanzhe Zhang, name: Binyuan Hui, name: Ofir Press, name: Ludwig Schmidt, name: Diyi Yang
Published: 2025-04-30 16:56:06

Despite recent progress in Language Models (LMs) for software engineering,
collecting training data remains a significant pain point. Existing datasets
are small, with at most 1,000s of training instances from 11 or fewer GitHub
repositories. The procedures to curate such datasets are often complex,
necessitating hundreds of hours of human labor; companion execution
environments also take up several terabytes of storage, severely limiting their
scalability and usability. To address this pain point, we introduce SWE-smith,
a novel pipeline for generating software engineering training data at scale.
Given any Python codebase, SWE-smith constructs a corresponding execution
environment, then automatically synthesizes 100s to 1,000s of task instances
that break existing test(s) in the codebase. Using SWE-smith, we create a
dataset of 50k instances sourced from 128 GitHub repositories, an order of
magnitude larger than all previous works. We train SWE-agent-LM-32B, achieving
40.2% Pass@1 resolve rate on the SWE-bench Verified benchmark, state of the art
among open source models. We open source SWE-smith (collection procedure, task
instances, trajectories, models) to lower the barrier of entry for research in
LM systems for automated software engineering. All assets available at
https://swesmith.com.
VITA-Audio: Fast Interleaved Cross-Modal Token Generation for Efficient
  Large Speech-Language Model
Authors: name: Zuwei Long, name: Yunhang Shen, name: Chaoyou Fu, name: Heting Gao, name: Lijiang Li, name: Peixian Chen, name: Mengdan Zhang, name: Hang Shao, name: Jian Li, name: Jinlong Peng, name: Haoyu Cao, name: Ke Li, name: Rongrong Ji, name: Xing Sun
Published: 2025-05-06 17:59:53

With the growing requirement for natural human-computer interaction,
speech-based systems receive increasing attention as speech is one of the most
common forms of daily communication. However, the existing speech models still
experience high latency when generating the first audio token during streaming,
which poses a significant bottleneck for deployment. To address this issue, we
propose VITA-Audio, an end-to-end large speech model with fast audio-text token
generation. Specifically, we introduce a lightweight Multiple Cross-modal Token
Prediction (MCTP) module that efficiently generates multiple audio tokens
within a single model forward pass, which not only accelerates the inference
but also significantly reduces the latency for generating the first audio in
streaming scenarios. In addition, a four-stage progressive training strategy is
explored to achieve model acceleration with minimal loss of speech quality. To
our knowledge, VITA-Audio is the first multi-modal large language model capable
of generating audio output during the first forward pass, enabling real-time
conversational capabilities with minimal latency. VITA-Audio is fully
reproducible and is trained on open-source data only. Experimental results
demonstrate that our model achieves an inference speedup of 3~5x at the 7B
parameter scale, but also significantly outperforms open-source models of
similar model size on multiple benchmarks for automatic speech recognition
(ASR), text-to-speech (TTS), and spoken question answering (SQA) tasks.
HoloTime: Taming Video Diffusion Models for Panoramic 4D Scene
  Generation
Authors: name: Haiyang Zhou, name: Wangbo Yu, name: Jiawen Guan, name: Xinhua Cheng, name: Yonghong Tian, name: Li Yuan
Published: 2025-04-30 13:55:28

The rapid advancement of diffusion models holds the promise of
revolutionizing the application of VR and AR technologies, which typically
require scene-level 4D assets for user experience. Nonetheless, existing
diffusion models predominantly concentrate on modeling static 3D scenes or
object-level dynamics, constraining their capacity to provide truly immersive
experiences. To address this issue, we propose HoloTime, a framework that
integrates video diffusion models to generate panoramic videos from a single
prompt or reference image, along with a 360-degree 4D scene reconstruction
method that seamlessly transforms the generated panoramic video into 4D assets,
enabling a fully immersive 4D experience for users. Specifically, to tame video
diffusion models for generating high-fidelity panoramic videos, we introduce
the 360World dataset, the first comprehensive collection of panoramic videos
suitable for downstream 4D scene reconstruction tasks. With this curated
dataset, we propose Panoramic Animator, a two-stage image-to-video diffusion
model that can convert panoramic images into high-quality panoramic videos.
Following this, we present Panoramic Space-Time Reconstruction, which leverages
a space-time depth estimation method to transform the generated panoramic
videos into 4D point clouds, enabling the optimization of a holistic 4D
Gaussian Splatting representation to reconstruct spatially and temporally
consistent 4D scenes. To validate the efficacy of our method, we conducted a
comparative analysis with existing approaches, revealing its superiority in
both panoramic video generation and 4D scene reconstruction. This demonstrates
our method's capability to create more engaging and realistic immersive
environments, thereby enhancing user experiences in VR and AR applications.
Geospatial Mechanistic Interpretability of Large Language Models
Authors: name: Stef De Sabbata, name: Stefano Mizzaro, name: Kevin Roitero
Published: 2025-05-06 09:40:06

Large Language Models (LLMs) have demonstrated unprecedented capabilities
across various natural language processing tasks. Their ability to process and
generate viable text and code has made them ubiquitous in many fields, while
their deployment as knowledge bases and "reasoning" tools remains an area of
ongoing research. In geography, a growing body of literature has been focusing
on evaluating LLMs' geographical knowledge and their ability to perform spatial
reasoning. However, very little is still known about the internal functioning
of these models, especially about how they process geographical information.
  In this chapter, we establish a novel framework for the study of geospatial
mechanistic interpretability - using spatial analysis to reverse engineer how
LLMs handle geographical information. Our aim is to advance our understanding
of the internal representations that these complex models generate while
processing geographical information - what one might call "how LLMs think about
geographic information" if such phrasing was not an undue anthropomorphism.
  We first outline the use of probing in revealing internal structures within
LLMs. We then introduce the field of mechanistic interpretability, discussing
the superposition hypothesis and the role of sparse autoencoders in
disentangling polysemantic internal representations of LLMs into more
interpretable, monosemantic features. In our experiments, we use spatial
autocorrelation to show how features obtained for placenames display spatial
patterns related to their geographic location and can thus be interpreted
geospatially, providing insights into how these models process geographical
information. We conclude by discussing how our framework can help shape the
study and use of foundation models in geography.
Decoding Open-Ended Information Seeking Goals from Eye Movements in
  Reading
Authors: name: Cfir Avraham Hadar, name: Omer Shubi, name: Yoav Meiri, name: Yevgeni Berzak
Published: 2025-05-04 13:23:48

When reading, we often have specific information that interests us in a text.
For example, you might be reading this paper because you are curious about LLMs
for eye movements in reading, the experimental design, or perhaps you only care
about the question ``but does it work?''. More broadly, in daily life, people
approach texts with any number of text-specific goals that guide their reading
behavior. In this work, we ask, for the first time, whether open-ended reading
goals can be automatically decoded from eye movements in reading. To address
this question, we introduce goal classification and goal reconstruction tasks
and evaluation frameworks, and use large-scale eye tracking for reading data in
English with hundreds of text-specific information seeking tasks. We develop
and compare several discriminative and generative multimodal LLMs that combine
eye movements and text for goal classification and goal reconstruction. Our
experiments show considerable success on both tasks, suggesting that LLMs can
extract valuable information about the readers' text-specific goals from eye
movements.
Auto-SLURP: A Benchmark Dataset for Evaluating Multi-Agent Frameworks in
  Smart Personal Assistant
Authors: name: Lei Shen, name: Xiaoyu Shen
Published: 2025-04-25 14:17:47

In recent years, multi-agent frameworks powered by large language models
(LLMs) have advanced rapidly. Despite this progress, there is still a notable
absence of benchmark datasets specifically tailored to evaluate their
performance. To bridge this gap, we introduce Auto-SLURP, a benchmark dataset
aimed at evaluating LLM-based multi-agent frameworks in the context of
intelligent personal assistants. Auto-SLURP extends the original SLURP dataset
-- initially developed for natural language understanding tasks -- by
relabeling the data and integrating simulated servers and external services.
This enhancement enables a comprehensive end-to-end evaluation pipeline,
covering language understanding, task execution, and response generation. Our
experiments demonstrate that Auto-SLURP presents a significant challenge for
current state-of-the-art frameworks, highlighting that truly reliable and
intelligent multi-agent personal assistants remain a work in progress. The
dataset and related code are available at
https://github.com/lorashen/Auto-SLURP/.
An Empirical Study of Qwen3 Quantization
Authors: name: Xingyu Zheng, name: Yuye Li, name: Haoran Chu, name: Yue Feng, name: Xudong Ma, name: Jie Luo, name: Jinyang Guo, name: Haotong Qin, name: Michele Magno, name: Xianglong Liu
Published: 2025-05-04 18:43:44

The Qwen series has emerged as a leading family of open-source Large Language
Models (LLMs), demonstrating remarkable capabilities in natural language
understanding tasks. With the recent release of Qwen3, which exhibits superior
performance across diverse benchmarks, there is growing interest in deploying
these models efficiently in resource-constrained environments. Low-bit
quantization presents a promising solution, yet its impact on Qwen3's
performance remains underexplored. This study conducts a systematic evaluation
of Qwen3's robustness under various quantization settings, aiming to uncover
both opportunities and challenges in compressing this state-of-the-art model.
We rigorously assess 5 existing classic post-training quantization techniques
applied to Qwen3, spanning bit-widths from 1 to 8 bits, and evaluate their
effectiveness across multiple datasets. Our findings reveal that while Qwen3
maintains competitive performance at moderate bit-widths, it experiences
notable degradation in linguistic tasks under ultra-low precision, underscoring
the persistent hurdles in LLM compression. These results emphasize the need for
further research to mitigate performance loss in extreme quantization
scenarios. We anticipate that this empirical analysis will provide actionable
insights for advancing quantization methods tailored to Qwen3 and future LLMs,
ultimately enhancing their practicality without compromising accuracy. Our
project is released on https://github.com/Efficient-ML/Qwen3-Quantization and
https://huggingface.co/collections/Efficient-ML/qwen3-quantization-68164450decb1c868788cb2b.
Multi-Agent System for Comprehensive Soccer Understanding
Authors: name: Jiayuan Rao, name: Zifeng Li, name: Haoning Wu, name: Ya Zhang, name: Yanfeng Wang, name: Weidi Xie
Published: 2025-05-06 17:59:31

Recent advancements in AI-driven soccer understanding have demonstrated rapid
progress, yet existing research predominantly focuses on isolated or narrow
tasks. To bridge this gap, we propose a comprehensive framework for holistic
soccer understanding. Specifically, we make the following contributions in this
paper: (i) we construct SoccerWiki, the first large-scale multimodal soccer
knowledge base, integrating rich domain knowledge about players, teams,
referees, and venues to enable knowledge-driven reasoning; (ii) we present
SoccerBench, the largest and most comprehensive soccer-specific benchmark,
featuring around 10K standardized multimodal (text, image, video) multi-choice
QA pairs across 13 distinct understanding tasks, curated through automated
pipelines and manual verification; (iii) we introduce SoccerAgent, a novel
multi-agent system that decomposes complex soccer questions via collaborative
reasoning, leveraging domain expertise from SoccerWiki and achieving robust
performance; (iv) extensive evaluations and ablations that benchmark
state-of-the-art MLLMs on SoccerBench, highlighting the superiority of our
proposed agentic system. All data and code are publicly available at:
https://jyrao.github.io/SoccerAgent/.
RADLADS: Rapid Attention Distillation to Linear Attention Decoders at
  Scale
Authors: name: Daniel Goldstein, name: Eric Alcaide, name: Janna Lu, name: Eugene Cheah
Published: 2025-05-05 20:03:28

We present Rapid Attention Distillation to Linear Attention Decoders at Scale
(RADLADS), a protocol for rapidly converting softmax attention transformers
into linear attention decoder models, along with two new RWKV-variant
architectures, and models converted from popular Qwen2.5 open source models in
7B, 32B, and 72B sizes. Our conversion process requires only 350-700M tokens,
less than 0.005% of the token count used to train the original teacher models.
Converting to our 72B linear attention model costs less than \$2,000 USD at
today's prices, yet quality at inference remains close to the original
transformer. These models achieve state-of-the-art downstream performance
across a set of standard benchmarks for linear attention models of their size.
We release all our models on HuggingFace under the Apache 2.0 license, with the
exception of our 72B models which are also governed by the Qwen License
Agreement.
  Models at
https://huggingface.co/collections/recursal/radlads-6818ee69e99e729ba8a87102
Training Code at https://github.com/recursal/RADLADS-paper
InfoVids: Reimagining the Viewer Experience with Alternative
  Visualization-Presenter Relationships
Authors: name: Ji Won Chung, name: Tongyu Zhou, name: Ivy Chen, name: Kevin Hsu, name: Ryan A. Rossi, name: Alexa Siu, name: Shunan Guo, name: Franck Dernoncourt, name: James Tompkin, name: Jeff Huang
Published: 2025-05-06 04:18:42

Traditional data presentations typically separate the presenter and
visualization into two separate spaces--the 3D world and a 2D screen--enforcing
visualization-centric stories. To create a more human-centric viewing
experience, we establish a more equitable relationship between the
visualization and the presenter through our InfoVids. These
infographics-inspired informational videos are crafted to redefine
relationships between the presenter and visualizations. As we design InfoVids,
we explore how the use of layout, form, and interactions affects the viewer
experience. We compare InfoVids against their baseline 2D `slides' equivalents
across 9 metrics with 30 participants and provide practical, long-term insights
from an autobiographical perspective. Our mixed methods analyses reveal that
this paradigm reduced viewer attention splitting, shifted the focus from the
visualization to the presenter, and led to more interactive, natural, and
engaging full-body data performances for viewers. Ultimately, InfoVids helped
viewers re-imagine traditional dynamics between the presenter and
visualizations.