Orionnnn/hub-stats huggingface dataset:
Do I look like a `cat.n.01` to you? A Taxonomy Image Generation
  Benchmark
Authors: name: Viktor Moskvoretskii, name: Alina Lobanova, name: Ekaterina Neminova, name: Chris Biemann, name: Alexander Panchenko, name: Irina Nikishina
Published: 2025-03-13 13:37:54

This paper explores the feasibility of using text-to-image models in a
zero-shot setup to generate images for taxonomy concepts. While text-based
methods for taxonomy enrichment are well-established, the potential of the
visual dimension remains unexplored. To address this, we propose a
comprehensive benchmark for Taxonomy Image Generation that assesses models'
abilities to understand taxonomy concepts and generate relevant, high-quality
images. The benchmark includes common-sense and randomly sampled WordNet
concepts, alongside the LLM generated predictions. The 12 models are evaluated
using 9 novel taxonomy-related text-to-image metrics and human feedback.
Moreover, we pioneer the use of pairwise evaluation with GPT-4 feedback for
image generation. Experimental results show that the ranking of models differs
significantly from standard T2I tasks. Playground-v2 and FLUX consistently
outperform across metrics and subsets and the retrieval-based approach performs
poorly. These findings highlight the potential for automating the curation of
structured data resources.
DiT-Air: Revisiting the Efficiency of Diffusion Model Architecture
  Design in Text to Image Generation
Authors: name: Chen Chen, name: Rui Qian, name: Wenze Hu, name: Tsu-Jui Fu, name: Lezhi Li, name: Bowen Zhang, name: Alex Schwing, name: Wei Liu, name: Yinfei Yang
Published: 2025-03-13 17:57:25

In this work, we empirically study Diffusion Transformers (DiTs) for
text-to-image generation, focusing on architectural choices, text-conditioning
strategies, and training protocols. We evaluate a range of DiT-based
architectures--including PixArt-style and MMDiT variants--and compare them with
a standard DiT variant which directly processes concatenated text and noise
inputs. Surprisingly, our findings reveal that the performance of standard DiT
is comparable with those specialized models, while demonstrating superior
parameter-efficiency, especially when scaled up. Leveraging the layer-wise
parameter sharing strategy, we achieve a further reduction of 66% in model size
compared to an MMDiT architecture, with minimal performance impact. Building on
an in-depth analysis of critical components such as text encoders and
Variational Auto-Encoders (VAEs), we introduce DiT-Air and DiT-Air-Lite. With
supervised and reward fine-tuning, DiT-Air achieves state-of-the-art
performance on GenEval and T2I CompBench, while DiT-Air-Lite remains highly
competitive, surpassing most existing models despite its compact size.
Charting and Navigating Hugging Face's Model Atlas
Authors: name: Eliahu Horwitz, name: Nitzan Kurer, name: Jonathan Kahana, name: Liel Amar, name: Yedid Hoshen
Published: 2025-03-13 17:59:53

As there are now millions of publicly available neural networks, searching
and analyzing large model repositories becomes increasingly important.
Navigating so many models requires an atlas, but as most models are poorly
documented charting such an atlas is challenging. To explore the hidden
potential of model repositories, we chart a preliminary atlas representing the
documented fraction of Hugging Face. It provides stunning visualizations of the
model landscape and evolution. We demonstrate several applications of this
atlas including predicting model attributes (e.g., accuracy), and analyzing
trends in computer vision models. However, as the current atlas remains
incomplete, we propose a method for charting undocumented regions.
Specifically, we identify high-confidence structural priors based on dominant
real-world model training practices. Leveraging these priors, our approach
enables accurate mapping of previously undocumented areas of the atlas. We
publicly release our datasets, code, and interactive atlas.
Transformers without Normalization
Authors: name: Jiachen Zhu, name: Xinlei Chen, name: Kaiming He, name: Yann LeCun, name: Zhuang Liu
Published: 2025-03-13 17:59:06

Normalization layers are ubiquitous in modern neural networks and have long
been considered essential. This work demonstrates that Transformers without
normalization can achieve the same or better performance using a remarkably
simple technique. We introduce Dynamic Tanh (DyT), an element-wise operation
DyT(x) = tanh(alpha x), as a drop-in replacement for normalization
layers in Transformers. DyT is inspired by the observation that layer
normalization in Transformers often produces tanh-like, S-shaped input-output
mappings. By incorporating DyT, Transformers without normalization can match or
exceed the performance of their normalized counterparts, mostly without
hyperparameter tuning. We validate the effectiveness of Transformers with DyT
across diverse settings, ranging from recognition to generation, supervised to
self-supervised learning, and computer vision to language models. These
findings challenge the conventional understanding that normalization layers are
indispensable in modern neural networks, and offer new insights into their role
in deep networks.
SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency
  Distillation
Authors: name: Junsong Chen, name: Shuchen Xue, name: Yuyang Zhao, name: Jincheng Yu, name: Sayak Paul, name: Junyu Chen, name: Han Cai, name: Enze Xie, name: Song Han
Published: 2025-03-12 04:53:07

This paper presents SANA-Sprint, an efficient diffusion model for ultra-fast
text-to-image (T2I) generation. SANA-Sprint is built on a pre-trained
foundation model and augmented with hybrid distillation, dramatically reducing
inference steps from 20 to 1-4. We introduce three key innovations: (1) We
propose a training-free approach that transforms a pre-trained flow-matching
model for continuous-time consistency distillation (sCM), eliminating costly
training from scratch and achieving high training efficiency. Our hybrid
distillation strategy combines sCM with latent adversarial distillation (LADD):
sCM ensures alignment with the teacher model, while LADD enhances single-step
generation fidelity. (2) SANA-Sprint is a unified step-adaptive model that
achieves high-quality generation in 1-4 steps, eliminating step-specific
training and improving efficiency. (3) We integrate ControlNet with SANA-Sprint
for real-time interactive image generation, enabling instant visual feedback
for user interaction. SANA-Sprint establishes a new Pareto frontier in
speed-quality tradeoffs, achieving state-of-the-art performance with 7.59 FID
and 0.74 GenEval in only 1 step - outperforming FLUX-schnell (7.94 FID / 0.71
GenEval) while being 10x faster (0.1s vs 1.1s on H100). It also achieves 0.1s
(T2I) and 0.25s (ControlNet) latency for 1024 x 1024 images on H100, and 0.31s
(T2I) on an RTX 4090, showcasing its exceptional efficiency and potential for
AI-powered consumer applications (AIPC). Code and pre-trained models will be
open-sourced.
4D LangSplat: 4D Language Gaussian Splatting via Multimodal Large
  Language Models
Authors: name: Wanhua Li, name: Renping Zhou, name: Jiawei Zhou, name: Yingwei Song, name: Johannes Herter, name: Minghan Qin, name: Gao Huang, name: Hanspeter Pfister
Published: 2025-03-13 14:58:22

Learning 4D language fields to enable time-sensitive, open-ended language
queries in dynamic scenes is essential for many real-world applications. While
LangSplat successfully grounds CLIP features into 3D Gaussian representations,
achieving precision and efficiency in 3D static scenes, it lacks the ability to
handle dynamic 4D fields as CLIP, designed for static image-text tasks, cannot
capture temporal dynamics in videos. Real-world environments are inherently
dynamic, with object semantics evolving over time. Building a precise 4D
language field necessitates obtaining pixel-aligned, object-wise video
features, which current vision models struggle to achieve. To address these
challenges, we propose 4D LangSplat, which learns 4D language fields to handle
time-agnostic or time-sensitive open-vocabulary queries in dynamic scenes
efficiently. 4D LangSplat bypasses learning the language field from vision
features and instead learns directly from text generated from object-wise video
captions via Multimodal Large Language Models (MLLMs). Specifically, we propose
a multimodal object-wise video prompting method, consisting of visual and text
prompts that guide MLLMs to generate detailed, temporally consistent,
high-quality captions for objects throughout a video. These captions are
encoded using a Large Language Model into high-quality sentence embeddings,
which then serve as pixel-aligned, object-specific feature supervision,
facilitating open-vocabulary text queries through shared embedding spaces.
Recognizing that objects in 4D scenes exhibit smooth transitions across states,
we further propose a status deformable network to model these continuous
changes over time effectively. Our results across multiple benchmarks
demonstrate that 4D LangSplat attains precise and efficient results for both
time-sensitive and time-agnostic open-vocabulary queries.
Long Context Tuning for Video Generation
Authors: name: Yuwei Guo, name: Ceyuan Yang, name: Ziyan Yang, name: Zhibei Ma, name: Zhijie Lin, name: Zhenheng Yang, name: Dahua Lin, name: Lu Jiang
Published: 2025-03-13 17:40:07

Recent advances in video generation can produce realistic, minute-long
single-shot videos with scalable diffusion transformers. However, real-world
narrative videos require multi-shot scenes with visual and dynamic consistency
across shots. In this work, we introduce Long Context Tuning (LCT), a training
paradigm that expands the context window of pre-trained single-shot video
diffusion models to learn scene-level consistency directly from data. Our
method expands full attention mechanisms from individual shots to encompass all
shots within a scene, incorporating interleaved 3D position embedding and an
asynchronous noise strategy, enabling both joint and auto-regressive shot
generation without additional parameters. Models with bidirectional attention
after LCT can further be fine-tuned with context-causal attention, facilitating
auto-regressive generation with efficient KV-cache. Experiments demonstrate
single-shot models after LCT can produce coherent multi-shot scenes and exhibit
emerging capabilities, including compositional generation and interactive shot
extension, paving the way for more practical visual content creation. See
https://guoyww.github.io/projects/long-context-video/ for more details.
Open-Sora 2.0: Training a Commercial-Level Video Generation Model in
  $200k
Authors: name: Xiangyu Peng, name: Zangwei Zheng, name: Chenhui Shen, name: Tom Young, name: Xinying Guo, name: Binluo Wang, name: Hang Xu, name: Hongxin Liu, name: Mingyan Jiang, name: Wenjun Li, name: Yuhui Wang, name: Anbang Ye, name: Gang Ren, name: Qianran Ma, name: Wanying Liang, name: Xiang Lian, name: Xiwen Wu, name: Yuting Zhong, name: Zhuangyan Li, name: Chaoyu Gong, name: Guojun Lei, name: Leijun Cheng, name: Limin Zhang, name: Minghao Li, name: Ruijie Zhang, name: Silan Hu, name: Shijie Huang, name: Xiaokang Wang, name: Yuanheng Zhao, name: Yuqi Wang, name: Ziang Wei, name: Yang You
Published: 2025-03-12 05:00:07

Video generation models have achieved remarkable progress in the past year.
The quality of AI video continues to improve, but at the cost of larger model
size, increased data quantity, and greater demand for training compute. In this
report, we present Open-Sora 2.0, a commercial-level video generation model
trained for only $200k. With this model, we demonstrate that the cost of
training a top-performing video generation model is highly controllable. We
detail all techniques that contribute to this efficiency breakthrough,
including data curation, model architecture, training strategy, and system
optimization. According to human evaluation results and VBench scores,
Open-Sora 2.0 is comparable to global leading video generation models including
the open-source HunyuanVideo and the closed-source Runway Gen-3 Alpha. By
making Open-Sora 2.0 fully open-source, we aim to democratize access to
advanced video generation technology, fostering broader innovation and
creativity in content creation. All resources are publicly available at:
https://github.com/hpcaitech/Open-Sora.
R1-Onevision: Advancing Generalized Multimodal Reasoning through
  Cross-Modal Formalization
Authors: name: Yi Yang, name: Xiaoxuan He, name: Hongkun Pan, name: Xiyan Jiang, name: Yan Deng, name: Xingtao Yang, name: Haoyu Lu, name: Dacheng Yin, name: Fengyun Rao, name: Minfeng Zhu, name: Bo Zhang, name: Wei Chen
Published: 2025-03-13 17:56:05

Large Language Models have demonstrated remarkable reasoning capability in
complex textual tasks. However, multimodal reasoning, which requires
integrating visual and textual information, remains a significant challenge.
Existing visual-language models often struggle to effectively analyze and
reason visual content, resulting in suboptimal performance on complex reasoning
tasks. Moreover, the absence of comprehensive benchmarks hinders the accurate
assessment of multimodal reasoning capabilities. In this paper, we introduce
R1-Onevision, a multimodal reasoning model designed to bridge the gap between
visual perception and deep reasoning. To achieve this, we propose a cross-modal
reasoning pipeline that transforms images into formal textural representations,
enabling precise language-based reasoning. Leveraging this pipeline, we
construct the R1-Onevision dataset which provides detailed, step-by-step
multimodal reasoning annotations across diverse domains. We further develop the
R1-Onevision model through supervised fine-tuning and reinforcement learning to
cultivate advanced reasoning and robust generalization abilities. To
comprehensively evaluate multimodal reasoning performance across different
grades, we introduce R1-Onevision-Bench, a benchmark aligned with human
educational stages, covering exams from junior high school to university and
beyond. Experimental results show that R1-Onevision achieves state-of-the-art
performance, outperforming models such as GPT-4o and Qwen2.5-VL on multiple
challenging multimodal reasoning benchmarks.
Autoregressive Image Generation with Randomized Parallel Decoding
Authors: name: Haopeng Li, name: Jinyue Yang, name: Guoqi Li, name: Huan Wang
Published: 2025-03-13 17:19:51

We introduce ARPG, a novel visual autoregressive model that enables
randomized parallel generation, addressing the inherent limitations of
conventional raster-order approaches, which hinder inference efficiency and
zero-shot generalization due to their sequential, predefined token generation
order. Our key insight is that effective random-order modeling necessitates
explicit guidance for determining the position of the next predicted token. To
this end, we propose a novel guided decoding framework that decouples
positional guidance from content representation, encoding them separately as
queries and key-value pairs. By directly incorporating this guidance into the
causal attention mechanism, our approach enables fully random-order training
and generation, eliminating the need for bidirectional attention. Consequently,
ARPG readily generalizes to zero-shot tasks such as image inpainting,
outpainting, and resolution expansion. Furthermore, it supports parallel
inference by concurrently processing multiple queries using a shared KV cache.
On the ImageNet-1K 256 benchmark, our approach attains an FID of 1.94 with only
64 sampling steps, achieving over a 20-fold increase in throughput while
reducing memory consumption by over 75% compared to representative recent
autoregressive models at a similar scale.
GroundingSuite: Measuring Complex Multi-Granular Pixel Grounding
Authors: name: Rui Hu, name: Lianghui Zhu, name: Yuxuan Zhang, name: Tianheng Cheng, name: Lei Liu, name: Heng Liu, name: Longjin Ran, name: Xiaoxin Chen, name: Wenyu Liu, name: Xinggang Wang
Published: 2025-03-13 17:43:10

Pixel grounding, encompassing tasks such as Referring Expression Segmentation
(RES), has garnered considerable attention due to its immense potential for
bridging the gap between vision and language modalities. However, advancements
in this domain are currently constrained by limitations inherent in existing
datasets, including limited object categories, insufficient textual diversity,
and a scarcity of high-quality annotations. To mitigate these limitations, we
introduce GroundingSuite, which comprises: (1) an automated data annotation
framework leveraging multiple Vision-Language Model (VLM) agents; (2) a
large-scale training dataset encompassing 9.56 million diverse referring
expressions and their corresponding segmentations; and (3) a meticulously
curated evaluation benchmark consisting of 3,800 images. The GroundingSuite
training dataset facilitates substantial performance improvements, enabling
models trained on it to achieve state-of-the-art results. Specifically, a cIoU
of 68.9 on gRefCOCO and a gIoU of 55.3 on RefCOCOm. Moreover, the
GroundingSuite annotation framework demonstrates superior efficiency compared
to the current leading data annotation method, i.e., 4.5 times faster than
the GLaMM.
Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and
  Beyond
Authors: name: Liang Wen, name: Yunke Cai, name: Fenrui Xiao, name: Xin He, name: Qi An, name: Zhenyu Duan, name: Yimin Du, name: Junchen Liu, name: Lifu Tang, name: Xiaowei Lv, name: Haosheng Zou, name: Yongchao Deng, name: Shousheng Jia, name: Xiangzheng Zhang
Published: 2025-03-13 15:29:22

This paper presents our work on the Light-R1 series, with models, data, and
code all released.
  We first focus on training long COT models from scratch, specifically
starting from models initially lacking long COT capabilities. Using a
curriculum training recipe consisting of two-stage SFT and semi-on-policy DPO,
we train our model Light-R1-32B from Qwen2.5-32B-Instruct, resulting in
superior math performance compared to DeepSeek-R1-Distill-Qwen-32B. Despite
being trained exclusively on math data, Light-R1-32B shows strong
generalization across other domains. In the subsequent phase of this work, we
highlight the significant benefit of the 3k dataset constructed for the second
SFT stage on enhancing other models. By fine-tuning DeepSeek-R1-Distilled
models using this dataset, we obtain new SOTA models in 7B and 14B, while the
32B model, Light-R1-32B-DS performed comparably to QwQ-32B and DeepSeek-R1.
  Furthermore, we extend our work by applying reinforcement learning,
specifically GRPO, on long-COT models to further improve reasoning performance.
We successfully train our final Light-R1-14B-DS with RL, achieving SOTA
performance among 14B parameter models in math. With AIME24 & 25 scores of 74.0
and 60.2 respectively, Light-R1-14B-DS surpasses even many 32B models and
DeepSeek-R1-Distill-Llama-70B. Its RL training also exhibits well expected
behavior, showing simultaneous increase in response length and reward score.
  The Light-R1 series of work validates training long-COT models from scratch,
showcases the art in SFT data and releases SOTA models from RL.
GoT: Unleashing Reasoning Capability of Multimodal Large Language Model
  for Visual Generation and Editing
Authors: name: Rongyao Fang, name: Chengqi Duan, name: Kun Wang, name: Linjiang Huang, name: Hao Li, name: Shilin Yan, name: Hao Tian, name: Xingyu Zeng, name: Rui Zhao, name: Jifeng Dai, name: Xihui Liu, name: Hongsheng Li
Published: 2025-03-13 17:59:59

Current image generation and editing methods primarily process textual
prompts as direct inputs without reasoning about visual composition and
explicit operations. We present Generation Chain-of-Thought (GoT), a novel
paradigm that enables generation and editing through an explicit language
reasoning process before outputting images. This approach transforms
conventional text-to-image generation and editing into a reasoning-guided
framework that analyzes semantic relationships and spatial arrangements. We
define the formulation of GoT and construct large-scale GoT datasets containing
over 9M samples with detailed reasoning chains capturing semantic-spatial
relationships. To leverage the advantages of GoT, we implement a unified
framework that integrates Qwen2.5-VL for reasoning chain generation with an
end-to-end diffusion model enhanced by our novel Semantic-Spatial Guidance
Module. Experiments show our GoT framework achieves excellent performance on
both generation and editing tasks, with significant improvements over
baselines. Additionally, our approach enables interactive visual generation,
allowing users to explicitly modify reasoning steps for precise image
adjustments. GoT pioneers a new direction for reasoning-driven visual
generation and editing, producing images that better align with human intent.
To facilitate future research, we make our datasets, code, and pretrained
models publicly available at https://github.com/rongyaofang/GoT.
Silent Branding Attack: Trigger-free Data Poisoning Attack on
  Text-to-Image Diffusion Models
Authors: name: Sangwon Jang, name: June Suk Choi, name: Jaehyeong Jo, name: Kimin Lee, name: Sung Ju Hwang
Published: 2025-03-12 17:21:57

Text-to-image diffusion models have achieved remarkable success in generating
high-quality contents from text prompts. However, their reliance on publicly
available data and the growing trend of data sharing for fine-tuning make these
models particularly vulnerable to data poisoning attacks. In this work, we
introduce the Silent Branding Attack, a novel data poisoning method that
manipulates text-to-image diffusion models to generate images containing
specific brand logos or symbols without any text triggers. We find that when
certain visual patterns are repeatedly in the training data, the model learns
to reproduce them naturally in its outputs, even without prompt mentions.
Leveraging this, we develop an automated data poisoning algorithm that
unobtrusively injects logos into original images, ensuring they blend naturally
and remain undetected. Models trained on this poisoned dataset generate images
containing logos without degrading image quality or text alignment. We
experimentally validate our silent branding attack across two realistic
settings on large-scale high-quality image datasets and style personalization
datasets, achieving high success rates even without a specific text trigger.
Human evaluation and quantitative metrics including logo detection show that
our method can stealthily embed logos.