Orionnnn/hub-stats huggingface dataset:
VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model
Authors: name: Haozhan Shen, name: Peng Liu, name: Jingcheng Li, name: Chunxin Fang, name: Yibo Ma, name: Jiajia Liao, name: Qiaoli Shen, name: Zilun Zhang, name: Kangjia Zhao, name: Qianqian Zhang, name: Ruochen Xu, name: Tiancheng Zhao
Published: 2025-04-10 10:05:15

Recently DeepSeek R1 has shown that reinforcement learning (RL) can
substantially improve the reasoning capabilities of Large Language Models
(LLMs) through a simple yet effective design. The core of R1 lies in its
rule-based reward formulation, which leverages tasks with deterministic
ground-truth answers to enable precise and stable reward computation. In the
visual domain, we similarly observe that a wide range of visual understanding
tasks are inherently equipped with well-defined ground-truth annotations. This
property makes them naturally compatible with rule-based reward mechanisms.
Motivated by this observation, we investigate the extension of R1-style
reinforcement learning to Vision-Language Models (VLMs), aiming to enhance
their visual reasoning capabilities. To this end, we develop VLM-R1, a
dedicated framework designed to harness RL for improving VLMs' performance on
general vision-language tasks. Using this framework, we further explore the
feasibility of applying RL to visual domain. Experimental results indicate that
the RL-based model not only delivers competitive performance on visual
understanding tasks but also surpasses Supervised Fine-Tuning (SFT) in
generalization ability. Furthermore, we conduct comprehensive ablation studies
that uncover a series of noteworthy insights, including the presence of reward
hacking in object detection, the emergence of the "OD aha moment", the impact
of training data quality, and the scaling behavior of RL across different model
sizes. Through these analyses, we aim to deepen the understanding of how
reinforcement learning enhances the capabilities of vision-language models, and
we hope our findings and open-source contributions will support continued
progress in the vision-language RL community. Our code and model are available
at https://github.com/om-ai-lab/VLM-R1
Visual Chronicles: Using Multimodal LLMs to Analyze Massive Collections
  of Images
Authors: name: Boyang Deng, name: Songyou Peng, name: Kyle Genova, name: Gordon Wetzstein, name: Noah Snavely, name: Leonidas Guibas, name: Thomas Funkhouser
Published: 2025-04-11 17:55:45

We present a system using Multimodal LLMs (MLLMs) to analyze a large database
with tens of millions of images captured at different times, with the aim of
discovering patterns in temporal changes. Specifically, we aim to capture
frequent co-occurring changes ("trends") across a city over a certain period.
Unlike previous visual analyses, our analysis answers open-ended queries (e.g.,
"what are the frequent types of changes in the city?") without any
predetermined target subjects or training labels. These properties cast prior
learning-based or unsupervised visual analysis tools unsuitable. We identify
MLLMs as a novel tool for their open-ended semantic understanding capabilities.
Yet, our datasets are four orders of magnitude too large for an MLLM to ingest
as context. So we introduce a bottom-up procedure that decomposes the massive
visual analysis problem into more tractable sub-problems. We carefully design
MLLM-based solutions to each sub-problem. During experiments and ablation
studies with our system, we find it significantly outperforms baselines and is
able to discover interesting trends from images captured in large cities (e.g.,
"addition of outdoor dining,", "overpass was painted blue," etc.). See more
results and interactive demos at https://boyangdeng.com/visual-chronicles.
Latent Diffusion Autoencoders: Toward Efficient and Meaningful
  Unsupervised Representation Learning in Medical Imaging
Authors: name: Gabriele Lozupone, name: Alessandro Bria, name: Francesco Fontanella, name: Frederick J. A. Meijer, name: Claudio De Stefano, name: Henkjan Huisman
Published: 2025-04-11 15:37:46

This study presents Latent Diffusion Autoencoder (LDAE), a novel
encoder-decoder diffusion-based framework for efficient and meaningful
unsupervised learning in medical imaging, focusing on Alzheimer disease (AD)
using brain MR from the ADNI database as a case study. Unlike conventional
diffusion autoencoders operating in image space, LDAE applies the diffusion
process in a compressed latent representation, improving computational
efficiency and making 3D medical imaging representation learning tractable. To
validate the proposed approach, we explore two key hypotheses: (i) LDAE
effectively captures meaningful semantic representations on 3D brain MR
associated with AD and ageing, and (ii) LDAE achieves high-quality image
generation and reconstruction while being computationally efficient.
Experimental results support both hypotheses: (i) linear-probe evaluations
demonstrate promising diagnostic performance for AD (ROC-AUC: 90%, ACC: 84%)
and age prediction (MAE: 4.1 years, RMSE: 5.2 years); (ii) the learned semantic
representations enable attribute manipulation, yielding anatomically plausible
modifications; (iii) semantic interpolation experiments show strong
reconstruction of missing scans, with SSIM of 0.969 (MSE: 0.0019) for a 6-month
gap. Even for longer gaps (24 months), the model maintains robust performance
(SSIM > 0.93, MSE < 0.004), indicating an ability to capture temporal
progression trends; (iv) compared to conventional diffusion autoencoders, LDAE
significantly increases inference throughput (20x faster) while also enhancing
reconstruction quality. These findings position LDAE as a promising framework
for scalable medical imaging applications, with the potential to serve as a
foundation model for medical image analysis. Code available at
https://github.com/GabrieleLozupone/LDAE
Pangu Ultra: Pushing the Limits of Dense Large Language Models on Ascend
  NPUs
Authors: name: Yichun Yin, name: Wenyong Huang, name: Kaikai Song, name: Yehui Tang, name: Xueyu Wu, name: Wei Guo, name: Peng Guo, name: Yaoyuan Wang, name: Xiaojun Meng, name: Yasheng Wang, name: Dong Li, name: Can Chen, name: Dandan Tu, name: Yin Li, name: Fisher Yu, name: Ruiming Tang, name: Yunhe Wang, name: Baojun Wang, name: Bin Wang, name: Bo Wang, name: Boxiao Liu, name: Changzheng Zhang, name: Duyu Tang, name: Fei Mi, name: Hui Jin, name: Jiansheng Wei, name: Jiarui Qin, name: Jinpeng Li, name: Jun Zhao, name: Liqun Deng, name: Lin Li, name: Minghui Xu, name: Naifu Zhang, name: Nianzu Zheng, name: Qiang Li, name: Rongju Ruan, name: Shengjun Cheng, name: Tianyu Guo, name: Wei He, name: Wei Li, name: Weiwen Liu, name: Wulong Liu, name: Xinyi Dai, name: Yonghan Dong, name: Yu Pan, name: Yue Li, name: Yufei Wang, name: Yujun Li, name: Yunsheng Ni, name: Zhe Liu, name: Zhenhe Zhang, name: Zhicheng Liu
Published: 2025-04-10 15:41:51

We present Pangu Ultra, a Large Language Model (LLM) with 135 billion
parameters and dense Transformer modules trained on Ascend Neural Processing
Units (NPUs). Although the field of LLM has been witnessing unprecedented
advances in pushing the scale and capability of LLM in recent years, training
such a large-scale model still involves significant optimization and system
challenges. To stabilize the training process, we propose depth-scaled sandwich
normalization, which effectively eliminates loss spikes during the training
process of deep models. We pre-train our model on 13.2 trillion diverse and
high-quality tokens and further enhance its reasoning capabilities during
post-training. To perform such large-scale training efficiently, we utilize
8,192 Ascend NPUs with a series of system optimizations. Evaluations on
multiple diverse benchmarks indicate that Pangu Ultra significantly advances
the state-of-the-art capabilities of dense LLMs such as Llama 405B and Mistral
Large 2, and even achieves competitive results with DeepSeek-R1, whose sparse
model structure contains much more parameters. Our exploration demonstrates
that Ascend NPUs are capable of efficiently and effectively training dense
models with more than 100 billion parameters. Our model and system will be
available for our commercial customers.
Do PhD-level LLMs Truly Grasp Elementary Addition? Probing Rule Learning
  vs. Memorization in Large Language Models
Authors: name: Yang Yan, name: Yu Lu, name: Renjun Xu, name: Zhenzhong Lan
Published: 2025-04-07 16:57:10

Despite high benchmark scores, Large Language Models (LLMs) often fail simple
problem, raising a critical question: Do LLMs learn mathematical principles or
merely memorize patterns? Rather than designing increasingly complex benchmarks
like recent works, we investigate this using elementary two-integer addition
(0 to 2^{64}), probing two core properties: commutativity (A+B=B+A) and
compositional generalization (via isomorphic symbolic mappings, e.g., 7
rightarrow y). While state-of-the-art LLMs achieve 73.8-99.8\% accuracy on
numerical addition, performance collapses to leq7.5\% under symbolic
mapping, indicating failure to generalize learned rules. Non-monotonic
performance scaling with digit count and frequent commutativity violations
(over 1,700 cases of A+B neq B+A) further support this. Explicitly providing
addition rules degrades performance by 81.2\% on average, while
self-explanation maintains baseline accuracy, suggesting LLM arithmetic
processing is misaligned with human-defined principles. Our findings indicate
current LLMs rely on memory pattern over genuine rule learning, highlighting
architectural limitations and the need for new approaches to achieve true
mathematical reasoning.
InteractVLM: 3D Interaction Reasoning from 2D Foundational Models
Authors: name: Sai Kumar Dwivedi, name: Dimitrije Antić, name: Shashank Tripathi, name: Omid Taheri, name: Cordelia Schmid, name: Michael J. Black, name: Dimitrios Tzionas
Published: 2025-04-07 17:59:33

We introduce InteractVLM, a novel method to estimate 3D contact points on
human bodies and objects from single in-the-wild images, enabling accurate
human-object joint reconstruction in 3D. This is challenging due to occlusions,
depth ambiguities, and widely varying object shapes. Existing methods rely on
3D contact annotations collected via expensive motion-capture systems or
tedious manual labeling, limiting scalability and generalization. To overcome
this, InteractVLM harnesses the broad visual knowledge of large Vision-Language
Models (VLMs), fine-tuned with limited 3D contact data. However, directly
applying these models is non-trivial, as they reason only in 2D, while
human-object contact is inherently 3D. Thus we introduce a novel
Render-Localize-Lift module that: (1) embeds 3D body and object surfaces in 2D
space via multi-view rendering, (2) trains a novel multi-view localization
model (MV-Loc) to infer contacts in 2D, and (3) lifts these to 3D.
Additionally, we propose a new task called Semantic Human Contact estimation,
where human contact predictions are conditioned explicitly on object semantics,
enabling richer interaction modeling. InteractVLM outperforms existing work on
contact estimation and also facilitates 3D reconstruction from an in-the wild
image. Code and models are available at https://interactvlm.is.tue.mpg.de.
CoRAG: Collaborative Retrieval-Augmented Generation
Authors: name: Aashiq Muhamed, name: Mona Diab, name: Virginia Smith
Published: 2025-04-02 16:40:43

Retrieval-Augmented Generation (RAG) models excel in knowledge-intensive
tasks, especially under few-shot learning constraints. We introduce CoRAG, a
framework extending RAG to collaborative settings, where clients jointly train
a shared model using a collaborative passage store. To evaluate CoRAG, we
introduce CRAB, a benchmark for collaborative homogeneous open-domain question
answering. Our experiments demonstrate that CoRAG consistently outperforms both
parametric collaborative learning methods and locally trained RAG models in
low-resource scenarios. Further analysis reveals the critical importance of
relevant passages within the shared store, the surprising benefits of
incorporating irrelevant passages, and the potential for hard negatives to
negatively impact performance. This introduces a novel consideration in
collaborative RAG: the trade-off between leveraging a collectively enriched
knowledge base and the potential risk of incorporating detrimental passages
from other clients. Our findings underscore the viability of CoRAG, while also
highlighting key design challenges and promising avenues for future research.
SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder
  Guardrails for Precision Unlearning in LLMs
Authors: name: Aashiq Muhamed, name: Jacopo Bonato, name: Mona Diab, name: Virginia Smith
Published: 2025-04-11 01:24:03

Machine unlearning is a promising approach to improve LLM safety by removing
unwanted knowledge from the model. However, prevailing gradient-based
unlearning methods suffer from issues such as high computational costs,
hyperparameter instability, poor sequential unlearning capability,
vulnerability to relearning attacks, low data efficiency, and lack of
interpretability. While Sparse Autoencoders are well-suited to improve these
aspects by enabling targeted activation-based unlearning, prior approaches
underperform gradient-based methods. This work demonstrates that, contrary to
these earlier findings, SAEs can significantly improve unlearning when employed
dynamically. We introduce Dynamic DAE Guardrails (DSG), a novel
method for precision unlearning that leverages principled feature selection and
a dynamic classifier. Our experiments show DSG substantially outperforms
leading unlearning methods, achieving superior forget-utility trade-offs. DSG
addresses key drawbacks of gradient-based approaches for unlearning -- offering
enhanced computational efficiency and stability, robust performance in
sequential unlearning, stronger resistance to relearning attacks, better data
efficiency including zero-shot settings, and more interpretable unlearning.
ModernBERT or DeBERTaV3? Examining Architecture and Data Influence on
  Transformer Encoder Models Performance
Authors: name: Wissam Antoun, name: Benoît Sagot, name: Djamé Seddah
Published: 2025-04-11 17:29:35

Pretrained transformer-encoder models like DeBERTaV3 and ModernBERT introduce
architectural advancements aimed at improving efficiency and performance.
Although the authors of ModernBERT report improved performance over DeBERTaV3
on several benchmarks, the lack of disclosed training data and the absence of
comparisons using a shared dataset make it difficult to determine whether these
gains are due to architectural improvements or differences in training data. In
this work, we conduct a controlled study by pretraining ModernBERT on the same
dataset as CamemBERTaV2, a DeBERTaV3 French model, isolating the effect of
model design. Our results show that the previous model generation remains
superior in sample efficiency and overall benchmark performance, with
ModernBERT's primary advantage being faster training and inference speed.
However, the new proposed model still provides meaningful architectural
improvements compared to earlier models such as BERT and RoBERTa. Additionally,
we observe that high-quality pre-training data accelerates convergence but does
not significantly improve final performance, suggesting potential benchmark
saturation. These findings show the importance of disentangling pretraining
data from architectural innovations when evaluating transformer models.
SQL-R1: Training Natural Language to SQL Reasoning Model By
  Reinforcement Learning
Authors: name: Peixian Ma, name: Xialie Zhuang, name: Chengjin Xu, name: Xuhui Jiang, name: Ran Chen, name: Jian Guo
Published: 2025-04-11 15:01:30

Natural Language to SQL (NL2SQL) enables intuitive interactions with
databases by transforming natural language queries into structured SQL
statements. Despite recent advancements in enhancing human-computer interaction
within database applications, significant challenges persist, particularly
regarding the inference performance in complex scenarios involving multi-table
joins and nested queries. Current methodologies primarily utilize supervised
fine-tuning (SFT) to train the NL2SQL model, which may limit adaptability and
interpretability in new environments (e.g., finance and healthcare). In order
to enhance the reasoning performance of the NL2SQL model in the above complex
situations, we introduce SQL-R1, a novel NL2SQL reasoning model trained by the
reinforcement learning (RL) algorithms. We design a specialized RL-based reward
function tailored for NL2SQL tasks and discussed the impact of cold start on
the effectiveness of intensive training. In addition, we achieve competitive
accuracy using only a tiny amount of synthetic NL2SQL data for augmented
training and further explore data engineering for RL. In existing experiments,
SQL-R1 achieves execution accuracy of 88.6% and 66.6% on the benchmark Spider
and BIRD, respectively, only using the 7B base model.
PixelFlow: Pixel-Space Generative Models with Flow
Authors: name: Shoufa Chen, name: Chongjian Ge, name: Shilong Zhang, name: Peize Sun, name: Ping Luo
Published: 2025-04-10 17:59:56

We present PixelFlow, a family of image generation models that operate
directly in the raw pixel space, in contrast to the predominant latent-space
models. This approach simplifies the image generation process by eliminating
the need for a pre-trained Variational Autoencoder (VAE) and enabling the whole
model end-to-end trainable. Through efficient cascade flow modeling, PixelFlow
achieves affordable computation cost in pixel space. It achieves an FID of 1.98
on 256times256 ImageNet class-conditional image generation benchmark. The
qualitative text-to-image results demonstrate that PixelFlow excels in image
quality, artistry, and semantic control. We hope this new paradigm will inspire
and open up new opportunities for next-generation visual generation models.
Code and models are available at https://github.com/ShoufaChen/PixelFlow.
GigaTok: Scaling Visual Tokenizers to 3 Billion Parameters for
  Autoregressive Image Generation
Authors: name: Tianwei Xiong, name: Jun Hao Liew, name: Zilong Huang, name: Jiashi Feng, name: Xihui Liu
Published: 2025-04-11 17:59:58

In autoregressive (AR) image generation, visual tokenizers compress images
into compact discrete latent tokens, enabling efficient training of downstream
autoregressive models for visual generation via next-token prediction. While
scaling visual tokenizers improves image reconstruction quality, it often
degrades downstream generation quality -- a challenge not adequately addressed
in existing literature. To address this, we introduce GigaTok, the first
approach to simultaneously improve image reconstruction, generation, and
representation learning when scaling visual tokenizers. We identify the growing
complexity of latent space as the key factor behind the reconstruction vs.
generation dilemma. To mitigate this, we propose semantic regularization, which
aligns tokenizer features with semantically consistent features from a
pre-trained visual encoder. This constraint prevents excessive latent space
complexity during scaling, yielding consistent improvements in both
reconstruction and downstream autoregressive generation. Building on semantic
regularization, we explore three key practices for scaling tokenizers:(1) using
1D tokenizers for better scalability, (2) prioritizing decoder scaling when
expanding both encoder and decoder, and (3) employing entropy loss to stabilize
training for billion-scale tokenizers. By scaling to 3 space billion
parameters, GigaTok achieves state-of-the-art performance in reconstruction,
downstream AR generation, and downstream AR representation quality.
MineWorld: a Real-Time and Open-Source Interactive World Model on
  Minecraft
Authors: name: Junliang Guo, name: Yang Ye, name: Tianyu He, name: Haoyu Wu, name: Yushu Jiang, name: Tim Pearce, name: Jiang Bian
Published: 2025-04-11 09:41:04

World modeling is a crucial task for enabling intelligent agents to
effectively interact with humans and operate in dynamic environments. In this
work, we propose MineWorld, a real-time interactive world model on Minecraft,
an open-ended sandbox game which has been utilized as a common testbed for
world modeling. MineWorld is driven by a visual-action autoregressive
Transformer, which takes paired game scenes and corresponding actions as input,
and generates consequent new scenes following the actions. Specifically, by
transforming visual game scenes and actions into discrete token ids with an
image tokenizer and an action tokenizer correspondingly, we consist the model
input with the concatenation of the two kinds of ids interleaved. The model is
then trained with next token prediction to learn rich representations of game
states as well as the conditions between states and actions simultaneously. In
inference, we develop a novel parallel decoding algorithm that predicts the
spatial redundant tokens in each frame at the same time, letting models in
different scales generate 4 to 7 frames per second and enabling real-time
interactions with game players. In evaluation, we propose new metrics to assess
not only visual quality but also the action following capacity when generating
new scenes, which is crucial for a world model. Our comprehensive evaluation
shows the efficacy of MineWorld, outperforming SoTA open-sourced diffusion
based world models significantly. The code and model have been released.
Seaweed-7B: Cost-Effective Training of Video Generation Foundation Model
Authors: name: Team Seawead, name: Ceyuan Yang, name: Zhijie Lin, name: Yang Zhao, name: Shanchuan Lin, name: Zhibei Ma, name: Haoyuan Guo, name: Hao Chen, name: Lu Qi, name: Sen Wang, name: Feng Cheng, name: Feilong Zuo Xuejiao Zeng, name: Ziyan Yang, name: Fangyuan Kong, name: Zhiwu Qing, name: Fei Xiao, name: Meng Wei, name: Tuyen Hoang, name: Siyu Zhang, name: Peihao Zhu, name: Qi Zhao, name: Jiangqiao Yan, name: Liangke Gui, name: Sheng Bi, name: Jiashi Li, name: Yuxi Ren, name: Rui Wang, name: Huixia Li, name: Xuefeng Xiao, name: Shu Liu, name: Feng Ling, name: Heng Zhang, name: Houmin Wei, name: Huafeng Kuang, name: Jerry Duncan, name: Junda Zhang, name: Junru Zheng, name: Li Sun, name: Manlin Zhang, name: Renfei Sun, name: Xiaobin Zhuang, name: Xiaojie Li, name: Xin Xia, name: Xuyan Chi, name: Yanghua Peng, name: Yuping Wang, name: Yuxuan Wang, name: Zhongkai Zhao, name: Zhuo Chen, name: Zuquan Song, name: Zhenheng Yang, name: Jiashi Feng, name: Jianchao Yang, name: Lu Jiang
Published: 2025-04-11 16:46:20

This technical report presents a cost-efficient strategy for training a video
generation foundation model. We present a mid-sized research model with
approximately 7 billion parameters (7B) called Seaweed-7B trained from scratch
using 665,000 H100 GPU hours. Despite being trained with moderate computational
resources, Seaweed-7B demonstrates highly competitive performance compared to
contemporary video generation models of much larger size. Design choices are
especially crucial in a resource-constrained setting. This technical report
highlights the key design decisions that enhance the performance of the
medium-sized diffusion model. Empirically, we make two observations: (1)
Seaweed-7B achieves performance comparable to, or even surpasses, larger models
trained on substantially greater GPU resources, and (2) our model, which
exhibits strong generalization ability, can be effectively adapted across a
wide range of downstream applications either by lightweight fine-tuning or
continue training. See the project page at https://seaweed.video/