Orionnnn/hub-stats huggingface dataset:
ReTool: Reinforcement Learning for Strategic Tool Use in LLMs
Authors: name: Jiazhan Feng, name: Shijue Huang, name: Xingwei Qu, name: Ge Zhang, name: Yujia Qin, name: Baoquan Zhong, name: Chengquan Jiang, name: Jinxin Chi, name: Wanjun Zhong
Published: 2025-04-15 18:10:22

While reasoning models (e.g., DeepSeek R1) trained with reinforcement
learning (RL), excel in textual reasoning, they struggle in scenarios requiring
structured problem-solving, such as geometric reasoning, concise computation,
or complex equation solving-areas where computational tools like code
interpreters (CI) demonstrate distinct advantages. To bridge this gap, we
propose ReTool, which enhances long-form reasoning with tool-integrated
learning, including two key features: (1) dynamic interleaving of real-time
code execution within natural language reasoning processes, and (2) an
automated RL paradigm that allows policy rollouts with multi-turn real-time
code execution and teaches the model in learning when and how to invoke tools
based on outcome feedback. ReTool employs a systematic training framework,
beginning with synthetic cold-start data generation to produce code-augmented
long-form reasoning traces for fine-tuning base models. Subsequent RL training
leverages task outcomes as rewards to iteratively refine the model's tool use
strategy, enabling autonomous discovery of optimal tool invocation patterns
without human priors. Experiments on the challenging MATH Olympiad benchmark
AIME demonstrate ReTool's superiority: Our 32B model achieves 67% accuracy with
400 training steps, outperforming text-based RL baseline (40% accuracy, 1080
steps) in efficiency and performance. Remarkably, ReTool-32B attains 72.5%
accuracy in extended settings, surpassing OpenAI's o1-preview by 27.9%. Further
analysis reveals emergent behaviors such as code self-correction, signaling an
''aha moment'' in which the model autonomously masters adaptive tool use. These
findings highlight the promise of outcome-driven tool integration for advancing
complex mathematical reasoning and offer new insights into hybrid
neuro-symbolic systems.
SIFT-50M: A Large-Scale Multilingual Dataset for Speech Instruction
  Fine-Tuning
Authors: name: Prabhat Pandey, name: Rupak Vignesh Swaminathan, name: K V Vijay Girish, name: Arunasish Sen, name: Jian Xie, name: Grant P. Strimel, name: Andreas Schwarz
Published: 2025-04-12 04:45:48

We introduce SIFT (Speech Instruction Fine-Tuning), a 50M-example dataset
designed for instruction fine-tuning and pre-training of speech-text large
language models (LLMs). SIFT-50M is built from publicly available speech
corpora, which collectively contain 14K hours of speech, and leverages LLMs
along with off-the-shelf expert models. The dataset spans five languages,
encompassing a diverse range of speech understanding as well as controllable
speech generation instructions. Using SIFT-50M, we train SIFT-LLM, which
outperforms existing speech-text LLMs on instruction-following benchmarks while
achieving competitive performance on foundational speech tasks. To support
further research, we also introduce EvalSIFT, a benchmark dataset specifically
designed to evaluate the instruction-following capabilities of speech-text
LLMs.
Vivid4D: Improving 4D Reconstruction from Monocular Video by Video
  Inpainting
Authors: name: Jiaxin Huang, name: Sheng Miao, name: BangBnag Yang, name: Yuewen Ma, name: Yiyi Liao
Published: 2025-04-15 11:38:14

Reconstructing 4D dynamic scenes from casually captured monocular videos is
valuable but highly challenging, as each timestamp is observed from a single
viewpoint. We introduce Vivid4D, a novel approach that enhances 4D monocular
video synthesis by augmenting observation views - synthesizing multi-view
videos from a monocular input. Unlike existing methods that either solely
leverage geometric priors for supervision or use generative priors while
overlooking geometry, we integrate both. This reformulates view augmentation as
a video inpainting task, where observed views are warped into new viewpoints
based on monocular depth priors. To achieve this, we train a video inpainting
model on unposed web videos with synthetically generated masks that mimic
warping occlusions, ensuring spatially and temporally consistent completion of
missing regions. To further mitigate inaccuracies in monocular depth priors, we
introduce an iterative view augmentation strategy and a robust reconstruction
loss. Experiments demonstrate that our method effectively improves monocular 4D
scene reconstruction and completion.
AlayaDB: The Data Foundation for Efficient and Effective Long-context
  LLM Inference
Authors: name: Yangshen Deng, name: Zhengxin You, name: Long Xiang, name: Qilong Li, name: Peiqi Yuan, name: Zhaoyang Hong, name: Yitao Zheng, name: Wanting Li, name: Runzhong Li, name: Haotian Liu, name: Kyriakos Mouratidis, name: Man Lung Yiu, name: Huan Li, name: Qiaomu Shen, name: Rui Mao, name: Bo Tang
Published: 2025-04-14 15:34:26

AlayaDB is a cutting-edge vector database system natively architected for
efficient and effective long-context inference for Large Language Models (LLMs)
at AlayaDB AI. Specifically, it decouples the KV cache and attention
computation from the LLM inference systems, and encapsulates them into a novel
vector database system. For the Model as a Service providers (MaaS), AlayaDB
consumes fewer hardware resources and offers higher generation quality for
various workloads with different kinds of Service Level Objectives (SLOs), when
comparing with the existing alternative solutions (e.g., KV cache
disaggregation, retrieval-based sparse attention). The crux of AlayaDB is that
it abstracts the attention computation and cache management for LLM inference
into a query processing procedure, and optimizes the performance via a native
query optimizer. In this work, we demonstrate the effectiveness of AlayaDB via
(i) three use cases from our industry partners, and (ii) extensive experimental
results on LLM inference benchmarks.
REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion
  Transformers
Authors: name: Xingjian Leng, name: Jaskirat Singh, name: Yunzhong Hou, name: Zhenchang Xing, name: Saining Xie, name: Liang Zheng
Published: 2025-04-14 17:59:53

In this paper we tackle a fundamental question: "Can we train latent
diffusion models together with the variational auto-encoder (VAE) tokenizer in
an end-to-end manner?" Traditional deep-learning wisdom dictates that
end-to-end training is often preferable when possible. However, for latent
diffusion transformers, it is observed that end-to-end training both VAE and
diffusion-model using standard diffusion-loss is ineffective, even causing a
degradation in final performance. We show that while diffusion loss is
ineffective, end-to-end training can be unlocked through the
representation-alignment (REPA) loss -- allowing both VAE and diffusion model
to be jointly tuned during the training process. Despite its simplicity, the
proposed training recipe (REPA-E) shows remarkable performance; speeding up
diffusion model training by over 17x and 45x over REPA and vanilla training
recipes, respectively. Interestingly, we observe that end-to-end tuning with
REPA-E also improves the VAE itself; leading to improved latent space structure
and downstream generation performance. In terms of final performance, our
approach sets a new state-of-the-art; achieving FID of 1.26 and 1.83 with and
without classifier-free guidance on ImageNet 256 x 256. Code is available at
https://end2end-diffusion.github.io.
BitNet b1.58 2B4T Technical Report
Authors: name: Shuming Ma, name: Hongyu Wang, name: Shaohan Huang, name: Xingxing Zhang, name: Ying Hu, name: Ting Song, name: Yan Xia, name: Furu Wei
Published: 2025-04-16 17:51:43

We introduce BitNet b1.58 2B4T, the first open-source, native 1-bit Large
Language Model (LLM) at the 2-billion parameter scale. Trained on a corpus of 4
trillion tokens, the model has been rigorously evaluated across benchmarks
covering language understanding, mathematical reasoning, coding proficiency,
and conversational ability. Our results demonstrate that BitNet b1.58 2B4T
achieves performance on par with leading open-weight, full-precision LLMs of
similar size, while offering significant advantages in computational
efficiency, including substantially reduced memory footprint, energy
consumption, and decoding latency. To facilitate further research and adoption,
the model weights are released via Hugging Face along with open-source
inference implementations for both GPU and CPU architectures.
ColorBench: Can VLMs See and Understand the Colorful World? A
  Comprehensive Benchmark for Color Perception, Reasoning, and Robustness
Authors: name: Yijun Liang, name: Ming Li, name: Chenrui Fan, name: Ziyue Li, name: Dang Nguyen, name: Kwesi Cobbina, name: Shweta Bhardwaj, name: Jiuhai Chen, name: Fuxiao Liu, name: Tianyi Zhou
Published: 2025-04-10 16:36:26

Color plays an important role in human perception and usually provides
critical clues in visual reasoning. However, it is unclear whether and how
vision-language models (VLMs) can perceive, understand, and leverage color as
humans. This paper introduces ColorBench, an innovative benchmark meticulously
crafted to assess the capabilities of VLMs in color understanding, including
color perception, reasoning, and robustness. By curating a suite of diverse
test scenarios, with grounding in real applications, ColorBench evaluates how
these models perceive colors, infer meanings from color-based cues, and
maintain consistent performance under varying color transformations. Through an
extensive evaluation of 32 VLMs with varying language models and vision
encoders, our paper reveals some undiscovered findings: (i) The scaling law
(larger models are better) still holds on ColorBench, while the language model
plays a more important role than the vision encoder. (ii) However, the
performance gaps across models are relatively small, indicating that color
understanding has been largely neglected by existing VLMs. (iii) CoT reasoning
improves color understanding accuracies and robustness, though they are
vision-centric tasks. (iv) Color clues are indeed leveraged by VLMs on
ColorBench but they can also mislead models in some tasks. These findings
highlight the critical limitations of current VLMs and underscore the need to
enhance color comprehension. Our ColorBenchcan serve as a foundational tool for
advancing the study of human-level color understanding of multimodal AI.
Robust and Fine-Grained Detection of AI Generated Texts
Authors: name: Ram Mohan Rao Kadiyala, name: Siddartha Pullakhandam, name: Kanwal Mehreen, name: Drishti Sharma, name: Siddhant Gupta, name: Jebish Purbey, name: Ashay Srivastava, name: Subhasya TippaReddy, name: Arvind Reddy Bobbili, name: Suraj Telugara Chandrashekhar, name: Modabbir Adeeb, name: Srinadh Vura, name: Hamza Farooq
Published: 2025-04-16 10:29:30

An ideal detection system for machine generated content is supposed to work
well on any generator as many more advanced LLMs come into existence day by
day. Existing systems often struggle with accurately identifying AI-generated
content over shorter texts. Further, not all texts might be entirely authored
by a human or LLM, hence we focused more over partial cases i.e human-LLM
co-authored texts. Our paper introduces a set of models built for the task of
token classification which are trained on an extensive collection of
human-machine co-authored texts, which performed well over texts of unseen
domains, unseen generators, texts by non-native speakers and those with
adversarial inputs. We also introduce a new dataset of over 2.4M such texts
mostly co-authored by several popular proprietary LLMs over 23 languages. We
also present findings of our models' performance over each texts of each domain
and generator. Additional findings include comparison of performance against
each adversarial method, length of input texts and characteristics of generated
texts compared to the original human authored texts.
Cobra: Efficient Line Art COlorization with BRoAder References
Authors: name: Junhao Zhuang, name: Lingen Li, name: Xuan Ju, name: Zhaoyang Zhang, name: Chun Yuan, name: Ying Shan
Published: 2025-04-16 16:45:19

The comic production industry requires reference-based line art colorization
with high accuracy, efficiency, contextual consistency, and flexible control. A
comic page often involves diverse characters, objects, and backgrounds, which
complicates the coloring process. Despite advancements in diffusion models for
image generation, their application in line art colorization remains limited,
facing challenges related to handling extensive reference images,
time-consuming inference, and flexible control. We investigate the necessity of
extensive contextual image guidance on the quality of line art colorization. To
address these challenges, we introduce Cobra, an efficient and versatile method
that supports color hints and utilizes over 200 reference images while
maintaining low latency. Central to Cobra is a Causal Sparse DiT architecture,
which leverages specially designed positional encodings, causal sparse
attention, and Key-Value Cache to effectively manage long-context references
and ensure color identity consistency. Results demonstrate that Cobra achieves
accurate line art colorization through extensive contextual reference,
significantly enhancing inference speed and interactivity, thereby meeting
critical industrial demands. We release our codes and models on our project
page: https://zhuang2002.github.io/Cobra/.
DataDecide: How to Predict Best Pretraining Data with Small Experiments
Authors: name: Ian Magnusson, name: Nguyen Tai, name: Ben Bogin, name: David Heineman, name: Jena D. Hwang, name: Luca Soldaini, name: Akshita Bhagia, name: Jiacheng Liu, name: Dirk Groeneveld, name: Oyvind Tafjord, name: Noah A. Smith, name: Pang Wei Koh, name: Jesse Dodge
Published: 2025-04-15 17:02:15

Because large language models are expensive to pretrain on different
datasets, using smaller-scale experiments to decide on data is crucial for
reducing costs. Which benchmarks and methods of making decisions from observed
performance at small scale most accurately predict the datasets that yield the
best large models? To empower open exploration of this question, we release
models, data, and evaluations in DataDecide -- the most extensive open suite of
models over differences in data and scale. We conduct controlled pretraining
experiments across 25 corpora with differing sources, deduplication, and
filtering up to 100B tokens, model sizes up to 1B parameters, and 3 random
seeds. We find that the ranking of models at a single, small size (e.g., 150M
parameters) is a strong baseline for predicting best models at our larger
target scale (1B) (~80% of com parisons correct). No scaling law methods among
8 baselines exceed the compute-decision frontier of single-scale predictions,
but DataDecide can measure improvement in future scaling laws. We also identify
that using continuous likelihood metrics as proxies in small experiments makes
benchmarks including MMLU, ARC, HellaSwag, MBPP, and HumanEval >80% predictable
at the target 1B scale with just 0.01% of the compute.
Efficient Hybrid Language Model Compression through Group-Aware SSM
  Pruning
Authors: name: Ali Taghibakhshi, name: Sharath Turuvekere Sreenivas, name: Saurav Muralidharan, name: Marcin Chochowski, name: Yashaswi Karnati, name: Raviraj Joshi, name: Ameya Sunil Mahabaleshwarkar, name: Zijia Chen, name: Yoshi Suhara, name: Oluwatobi Olabiyi, name: Daniel Korzekwa, name: Mostofa Patwary, name: Mohammad Shoeybi, name: Jan Kautz, name: Bryan Catanzaro, name: Ashwath Aithal, name: Nima Tajbakhsh, name: Pavlo Molchanov
Published: 2025-04-15 17:26:29

Hybrid LLM architectures that combine Attention and State Space Models (SSMs)
achieve state-of-the-art accuracy and runtime performance. Recent work has
demonstrated that applying compression and distillation to Attention-only
models yields smaller, more accurate models at a fraction of the training cost.
In this work, we explore the effectiveness of compressing Hybrid architectures.
We introduce a novel group-aware pruning strategy that preserves the structural
integrity of SSM blocks and their sequence modeling capabilities. Furthermore,
we demonstrate the necessity of such SSM pruning to achieve improved accuracy
and inference speed compared to traditional approaches. Our compression recipe
combines SSM, FFN, embedding dimension, and layer pruning, followed by
knowledge distillation-based retraining, similar to the MINITRON technique.
Using this approach, we compress the Nemotron-H 8B Hybrid model down to 4B
parameters with up to 40x fewer training tokens. The resulting model surpasses
the accuracy of similarly-sized models while achieving 2x faster inference,
significantly advancing the Pareto frontier.
RealHarm: A Collection of Real-World Language Model Application Failures
Authors: name: Pierre Le Jeune, name: Jiaen Liu, name: Luca Rossi, name: Matteo Dora
Published: 2025-04-14 14:44:41

Language model deployments in consumer-facing applications introduce numerous
risks. While existing research on harms and hazards of such applications
follows top-down approaches derived from regulatory frameworks and theoretical
analyses, empirical evidence of real-world failure modes remains underexplored.
In this work, we introduce RealHarm, a dataset of annotated problematic
interactions with AI agents built from a systematic review of publicly reported
incidents. Analyzing harms, causes, and hazards specifically from the
deployer's perspective, we find that reputational damage constitutes the
predominant organizational harm, while misinformation emerges as the most
common hazard category. We empirically evaluate state-of-the-art guardrails and
content moderation systems to probe whether such systems would have prevented
the incidents, revealing a significant gap in the protection of AI
applications.
Aligning Generative Denoising with Discriminative Objectives Unleashes
  Diffusion for Visual Perception
Authors: name: Ziqi Pang, name: Xin Xu, name: Yu-Xiong Wang
Published: 2025-04-15 17:59:54

With the success of image generation, generative diffusion models are
increasingly adopted for discriminative tasks, as pixel generation provides a
unified perception interface. However, directly repurposing the generative
denoising process for discriminative objectives reveals critical gaps rarely
addressed previously. Generative models tolerate intermediate sampling errors
if the final distribution remains plausible, but discriminative tasks require
rigorous accuracy throughout, as evidenced in challenging multi-modal tasks
like referring image segmentation. Motivated by this gap, we analyze and
enhance alignment between generative diffusion processes and perception tasks,
focusing on how perception quality evolves during denoising. We find: (1)
earlier denoising steps contribute disproportionately to perception quality,
prompting us to propose tailored learning objectives reflecting varying
timestep contributions; (2) later denoising steps show unexpected perception
degradation, highlighting sensitivity to training-denoising distribution
shifts, addressed by our diffusion-tailored data augmentation; and (3)
generative processes uniquely enable interactivity, serving as controllable
user interfaces adaptable to correctional prompts in multi-round interactions.
Our insights significantly improve diffusion-based perception models without
architectural changes, achieving state-of-the-art performance on depth
estimation, referring image segmentation, and generalist perception tasks. Code
available at https://github.com/ziqipang/ADDP.
VisualPuzzles: Decoupling Multimodal Reasoning Evaluation from Domain
  Knowledge
Authors: name: Yueqi Song, name: Tianyue Ou, name: Yibo Kong, name: Zecheng Li, name: Graham Neubig, name: Xiang Yue
Published: 2025-04-14 15:50:39

Current multimodal benchmarks often conflate reasoning with domain-specific
knowledge, making it difficult to isolate and evaluate general reasoning
abilities in non-expert settings. To address this, we introduce VisualPuzzles,
a benchmark that targets visual reasoning while deliberately minimizing
reliance on specialized knowledge. VisualPuzzles consists of diverse questions
spanning five categories: algorithmic, analogical, deductive, inductive, and
spatial reasoning. One major source of our questions is manually translated
logical reasoning questions from the Chinese Civil Service Examination.
Experiments show that VisualPuzzles requires significantly less intensive
domain-specific knowledge and more complex reasoning compared to benchmarks
like MMMU, enabling us to better evaluate genuine multimodal reasoning.
Evaluations show that state-of-the-art multimodal large language models
consistently lag behind human performance on VisualPuzzles, and that strong
performance on knowledge-intensive benchmarks does not necessarily translate to
success on reasoning-focused, knowledge-light tasks. Additionally, reasoning
enhancements such as scaling up inference compute (with "thinking" modes) yield
inconsistent gains across models and task types, and we observe no clear
correlation between model size and performance. We also found that models
exhibit different reasoning and answering patterns on VisualPuzzles compared to
benchmarks with heavier emphasis on knowledge. VisualPuzzles offers a clearer
lens through which to evaluate reasoning capabilities beyond factual recall and
domain knowledge.
SimpleAR: Pushing the Frontier of Autoregressive Visual Generation
  through Pretraining, SFT, and RL
Authors: name: Junke Wang, name: Zhi Tian, name: Xun Wang, name: Xinyu Zhang, name: Weilin Huang, name: Zuxuan Wu, name: Yu-Gang Jiang
Published: 2025-04-15 17:59:46

This work presents SimpleAR, a vanilla autoregressive visual generation
framework without complex architecure modifications. Through careful
exploration of training and inference optimization, we demonstrate that: 1)
with only 0.5B parameters, our model can generate 1024x1024 resolution images
with high fidelity, and achieve competitive results on challenging
text-to-image benchmarks, e.g., 0.59 on GenEval and 79.66 on DPG; 2) both
supervised fine-tuning (SFT) and Group Relative Policy Optimization (GRPO)
training could lead to significant improvements on generation aesthectics and
prompt alignment; and 3) when optimized with inference acceleraton techniques
like vLLM, the time for SimpleAR to generate an 1024x1024 image could be
reduced to around 14 seconds. By sharing these findings and open-sourcing the
code, we hope to reveal the potential of autoregressive visual generation and
encourage more participation in this research field. Code is available at
https://github.com/wdrink/SimpleAR.
Efficient Reasoning Models: A Survey
Authors: name: Sicheng Feng, name: Gongfan Fang, name: Xinyin Ma, name: Xinchao Wang
Published: 2025-04-15 06:28:00

Reasoning models have demonstrated remarkable progress in solving complex and
logic-intensive tasks by generating extended Chain-of-Thoughts (CoTs) prior to
arriving at a final answer. Yet, the emergence of this "slow-thinking"
paradigm, with numerous tokens generated in sequence, inevitably introduces
substantial computational overhead. To this end, it highlights an urgent need
for effective acceleration. This survey aims to provide a comprehensive
overview of recent advances in efficient reasoning. It categorizes existing
works into three key directions: (1) shorter - compressing lengthy CoTs into
concise yet effective reasoning chains; (2) smaller - developing compact
language models with strong reasoning capabilities through techniques such as
knowledge distillation, other model compression techniques, and reinforcement
learning; and (3) faster - designing efficient decoding strategies to
accelerate inference. A curated collection of papers discussed in this survey
is available in our GitHub repository.