Orionnnn/hub-stats huggingface dataset:
Next Visual Granularity Generation
Authors: name: Yikai Wang, name: Zhouxia Wang, name: Zhonghua Wu, name: Qingyi Tao, name: Kang Liao, name: Chen Change Loy
Published: 2025-08-18 10:47:37

We propose a novel approach to image generation by decomposing an image into
a structured sequence, where each element in the sequence shares the same
spatial resolution but differs in the number of unique tokens used, capturing
different level of visual granularity. Image generation is carried out through
our newly introduced Next Visual Granularity (NVG) generation framework, which
generates a visual granularity sequence beginning from an empty image and
progressively refines it, from global layout to fine details, in a structured
manner. This iterative process encodes a hierarchical, layered representation
that offers fine-grained control over the generation process across multiple
granularity levels. We train a series of NVG models for class-conditional image
generation on the ImageNet dataset and observe clear scaling behavior. Compared
to the VAR series, NVG consistently outperforms it in terms of FID scores (3.30
-> 3.03, 2.57 ->2.44, 2.09 -> 2.06). We also conduct extensive analysis to
showcase the capability and potential of the NVG framework. Our code and models
will be released.
S^2-Guidance: Stochastic Self Guidance for Training-Free Enhancement of
  Diffusion Models
Authors: name: Chubin Chen, name: Jiashu Zhu, name: Xiaokun Feng, name: Nisha Huang, name: Meiqi Wu, name: Fangyuan Mao, name: Jiahong Wu, name: Xiangxiang Chu, name: Xiu Li
Published: 2025-08-18 12:31:20

Classifier-free Guidance (CFG) is a widely used technique in modern diffusion
models for enhancing sample quality and prompt adherence. However, through an
empirical analysis on Gaussian mixture modeling with a closed-form solution, we
observe a discrepancy between the suboptimal results produced by CFG and the
ground truth. The model's excessive reliance on these suboptimal predictions
often leads to semantic incoherence and low-quality outputs. To address this
issue, we first empirically demonstrate that the model's suboptimal predictions
can be effectively refined using sub-networks of the model itself. Building on
this insight, we propose S^2-Guidance, a novel method that leverages stochastic
block-dropping during the forward process to construct stochastic sub-networks,
effectively guiding the model away from potential low-quality predictions and
toward high-quality outputs. Extensive qualitative and quantitative experiments
on text-to-image and text-to-video generation tasks demonstrate that
S^2-Guidance delivers superior performance, consistently surpassing CFG and
other advanced guidance strategies. Our code will be released.
Lumen: Consistent Video Relighting and Harmonious Background Replacement
  with Video Generative Models
Authors: name: Jianshu Zeng, name: Yuxuan Liu, name: Yutong Feng, name: Chenxuan Miao, name: Zixiang Gao, name: Jiwang Qu, name: Jianzhang Zhang, name: Bin Wang, name: Kun Yuan
Published: 2025-08-18 14:21:22

Video relighting is a challenging yet valuable task, aiming to replace the
background in videos while correspondingly adjusting the lighting in the
foreground with harmonious blending. During translation, it is essential to
preserve the original properties of the foreground, e.g., albedo, and propagate
consistent relighting among temporal frames. In this paper, we propose Lumen,
an end-to-end video relighting framework developed on large-scale video
generative models, receiving flexible textual description for instructing the
control of lighting and background. Considering the scarcity of high-qualified
paired videos with the same foreground in various lighting conditions, we
construct a large-scale dataset with a mixture of realistic and synthetic
videos. For the synthetic domain, benefiting from the abundant 3D assets in the
community, we leverage advanced 3D rendering engine to curate video pairs in
diverse environments. For the realistic domain, we adapt a HDR-based lighting
simulation to complement the lack of paired in-the-wild videos. Powered by the
aforementioned dataset, we design a joint training curriculum to effectively
unleash the strengths of each domain, i.e., the physical consistency in
synthetic videos, and the generalized domain distribution in realistic videos.
To implement this, we inject a domain-aware adapter into the model to decouple
the learning of relighting and domain appearance distribution. We construct a
comprehensive benchmark to evaluate Lumen together with existing methods, from
the perspectives of foreground preservation and video consistency assessment.
Experimental results demonstrate that Lumen effectively edit the input into
cinematic relighted videos with consistent lighting and strict foreground
preservation. Our project page: https://lumen-relight.github.io/
Matrix-Game 2.0: An Open-Source, Real-Time, and Streaming Interactive
  World Model
Authors: name: Xianglong He, name: Chunli Peng, name: Zexiang Liu, name: Boyang Wang, name: Yifan Zhang, name: Qi Cui, name: Fei Kang, name: Biao Jiang, name: Mengyin An, name: Yangyang Ren, name: Baixin Xu, name: Hao-Xiang Guo, name: Kaixiong Gong, name: Cyrus Wu, name: Wei Li, name: Xuchen Song, name: Yang Liu, name: Eric Li, name: Yahui Zhou
Published: 2025-08-18 15:28:53

Recent advances in interactive video generations have demonstrated diffusion
model's potential as world models by capturing complex physical dynamics and
interactive behaviors. However, existing interactive world models depend on
bidirectional attention and lengthy inference steps, severely limiting
real-time performance. Consequently, they are hard to simulate real-world
dynamics, where outcomes must update instantaneously based on historical
context and current actions. To address this, we present Matrix-Game 2.0, an
interactive world model generates long videos on-the-fly via few-step
auto-regressive diffusion. Our framework consists of three key components: (1)
A scalable data production pipeline for Unreal Engine and GTA5 environments to
effectively produce massive amounts (about 1200 hours) of video data with
diverse interaction annotations; (2) An action injection module that enables
frame-level mouse and keyboard inputs as interactive conditions; (3) A few-step
distillation based on the casual architecture for real-time and streaming video
generation. Matrix Game 2.0 can generate high-quality minute-level videos
across diverse scenes at an ultra-fast speed of 25 FPS. We open-source our
model weights and codebase to advance research in interactive world modeling.
Representing Speech Through Autoregressive Prediction of Cochlear Tokens
Authors: name: Greta Tuckute, name: Klemen Kotar, name: Evelina Fedorenko, name: Daniel L. K. Yamins
Published: 2025-08-15 17:06:04

We introduce AuriStream, a biologically inspired model for encoding speech
via a two-stage framework inspired by the human auditory processing hierarchy.
The first stage transforms raw audio into a time-frequency representation based
on the human cochlea, from which we extract discrete cochlear tokens.
The second stage applies an autoregressive sequence model over the cochlear
tokens. AuriStream learns meaningful phoneme and word representations, and
state-of-the-art lexical semantics. AuriStream shows competitive performance on
diverse downstream SUPERB speech tasks. Complementing AuriStream's strong
representational capabilities, it generates continuations of audio which can be
visualized in a spectrogram space and decoded back into audio, providing
insights into the model's predictions. In summary, we present a two-stage
framework for speech representation learning to advance the development of more
human-like models that efficiently handle a range of speech-based tasks.
ComoRAG: A Cognitive-Inspired Memory-Organized RAG for Stateful Long
  Narrative Reasoning
Authors: name: Juyuan Wang, name: Rongchen Zhao, name: Wei Wei, name: Yufeng Wang, name: Mo Yu, name: Jie Zhou, name: Jin Xu, name: Liyan Xu
Published: 2025-08-14 07:52:09

Narrative comprehension on long stories and novels has been a challenging
domain attributed to their intricate plotlines and entangled, often evolving
relations among characters and entities. Given the LLM's diminished reasoning
over extended context and high computational cost, retrieval-based approaches
remain a pivotal role in practice. However, traditional RAG methods can fall
short due to their stateless, single-step retrieval process, which often
overlooks the dynamic nature of capturing interconnected relations within
long-range context. In this work, we propose ComoRAG, holding the principle
that narrative reasoning is not a one-shot process, but a dynamic, evolving
interplay between new evidence acquisition and past knowledge consolidation,
analogous to human cognition when reasoning with memory-related signals in the
brain. Specifically, when encountering a reasoning impasse, ComoRAG undergoes
iterative reasoning cycles while interacting with a dynamic memory workspace.
In each cycle, it generates probing queries to devise new exploratory paths,
then integrates the retrieved evidence of new aspects into a global memory
pool, thereby supporting the emergence of a coherent context for the query
resolution. Across four challenging long-context narrative benchmarks (200K+
tokens), ComoRAG outperforms strong RAG baselines with consistent relative
gains up to 11% compared to the strongest baseline. Further analysis reveals
that ComoRAG is particularly advantageous for complex queries requiring global
comprehension, offering a principled, cognitively motivated paradigm for
retrieval-based long context comprehension towards stateful reasoning. Our code
is publicly released at https://github.com/EternityJune25/ComoRAG
Ovis2.5 Technical Report
Authors: name: Shiyin Lu, name: Yang Li, name: Yu Xia, name: Yuwei Hu, name: Shanshan Zhao, name: Yanqing Ma, name: Zhichao Wei, name: Yinglun Li, name: Lunhao Duan, name: Jianshan Zhao, name: Yuxuan Han, name: Haijun Li, name: Wanying Chen, name: Junke Tang, name: Chengkun Hou, name: Zhixing Du, name: Tianli Zhou, name: Wenjie Zhang, name: Huping Ding, name: Jiahe Li, name: Wen Li, name: Gui Hu, name: Yiliang Gu, name: Siran Yang, name: Jiamang Wang, name: Hailong Sun, name: Yibo Wang, name: Hui Sun, name: Jinlong Huang, name: Yuping He, name: Shengze Shi, name: Weihong Zhang, name: Guodong Zheng, name: Junpeng Jiang, name: Sensen Gao, name: Yi-Feng Wu, name: Sijia Chen, name: Yuhui Chen, name: Qing-Guo Chen, name: Zhao Xu, name: Weihua Luo, name: Kaifu Zhang
Published: 2025-08-15 17:01:08

We present Ovis2.5, a successor to Ovis2 designed for native-resolution
visual perception and strong multimodal reasoning. Ovis2.5 integrates a
native-resolution vision transformer that processes images at their native,
variable resolutions, avoiding the degradation from fixed-resolution tiling and
preserving both fine detail and global layout -- crucial for visually dense
content like complex charts. To strengthen reasoning, we train the model to
move beyond linear chain-of-thought and perform reflection -- including
self-checking and revision. This advanced capability is exposed as an optional
"thinking mode" at inference time, allowing users to trade latency for enhanced
accuracy on difficult inputs. The model is trained via a comprehensive
five-phase curriculum that progressively builds its skills. The process begins
with foundational visual and multimodal pretraining, advances through
large-scale instruction tuning, and culminates in alignment and reasoning
enhancement using DPO and GRPO. To scale these upgrades efficiently, we employ
multimodal data packing and hybrid parallelism, yielding a significant
end-to-end speedup. We release two open-source models: Ovis2.5-9B and
Ovis2.5-2B. The latter continues the "small model, big performance" philosophy
of Ovis2, making it ideal for resource-constrained, on-device scenarios. On the
OpenCompass multimodal leaderboard, Ovis2.5-9B averages 78.3, marking a
substantial improvement over its predecessor, Ovis2-8B, and achieving
state-of-the-art results among open-source MLLMs in the sub-40B parameter
range; Ovis2.5-2B scores 73.9, establishing SOTA for its size. Beyond aggregate
scores, Ovis2.5 achieves leading results on STEM benchmarks, exhibits strong
capabilities on grounding and video tasks, and achieves open-source SOTA at its
scale for complex chart analysis.
Speed Always Wins: A Survey on Efficient Architectures for Large
  Language Models
Authors: name: Weigao Sun, name: Jiaxi Hu, name: Yucheng Zhou, name: Jusen Du, name: Disen Lan, name: Kexin Wang, name: Tong Zhu, name: Xiaoye Qu, name: Yu Zhang, name: Xiaoyu Mo, name: Daizong Liu, name: Yuxuan Liang, name: Wenliang Chen, name: Guoqi Li, name: Yu Cheng
Published: 2025-08-13 14:13:46

Large Language Models (LLMs) have delivered impressive results in language
understanding, generation, reasoning, and pushes the ability boundary of
multimodal models. Transformer models, as the foundation of modern LLMs, offer
a strong baseline with excellent scaling properties. However, the traditional
transformer architecture requires substantial computations and poses
significant obstacles for large-scale training and practical deployment. In
this survey, we offer a systematic examination of innovative LLM architectures
that address the inherent limitations of transformers and boost the efficiency.
Starting from language modeling, this survey covers the background and
technical details of linear and sparse sequence modeling methods, efficient
full attention variants, sparse mixture-of-experts, hybrid model architectures
incorporating the above techniques, and emerging diffusion LLMs. Additionally,
we discuss applications of these techniques to other modalities and consider
their wider implications for developing scalable, resource-aware foundation
models. By grouping recent studies into the above category, this survey
presents a blueprint of modern efficient LLM architectures, and we hope this
could help motivate future research toward more efficient, versatile AI
systems.
4DNeX: Feed-Forward 4D Generative Modeling Made Easy
Authors: name: Zhaoxi Chen, name: Tianqi Liu, name: Long Zhuo, name: Jiawei Ren, name: Zeng Tao, name: He Zhu, name: Fangzhou Hong, name: Liang Pan, name: Ziwei Liu
Published: 2025-08-18 17:59:55

We present 4DNeX, the first feed-forward framework for generating 4D (i.e.,
dynamic 3D) scene representations from a single image. In contrast to existing
methods that rely on computationally intensive optimization or require
multi-frame video inputs, 4DNeX enables efficient, end-to-end image-to-4D
generation by fine-tuning a pretrained video diffusion model. Specifically, 1)
to alleviate the scarcity of 4D data, we construct 4DNeX-10M, a large-scale
dataset with high-quality 4D annotations generated using advanced
reconstruction approaches. 2) we introduce a unified 6D video representation
that jointly models RGB and XYZ sequences, facilitating structured learning of
both appearance and geometry. 3) we propose a set of simple yet effective
adaptation strategies to repurpose pretrained video diffusion models for 4D
modeling. 4DNeX produces high-quality dynamic point clouds that enable
novel-view video synthesis. Extensive experiments demonstrate that 4DNeX
outperforms existing 4D generation methods in efficiency and generalizability,
offering a scalable solution for image-to-4D modeling and laying the foundation
for generative 4D world models that simulate dynamic scene evolution.
Inverse-LLaVA: Eliminating Alignment Pre-training Through Text-to-Vision
  Mapping
Authors: name: Xuhui Zhan, name: Tyler Derr
Published: 2025-08-17 18:36:04

Traditional multimodal learning approaches require expensive alignment
pre-training to bridge vision and language modalities, typically projecting
visual features into discrete text token spaces. We challenge both fundamental
assumptions underlying this paradigm by proposing Inverse-LLaVA, a novel
approach that eliminates alignment pre-training entirely while inverting the
conventional mapping direction. Rather than projecting visual features to text
space, our method maps text embeddings into continuous visual representation
space and performs fusion within transformer intermediate layers. Through
selective additive components in attention mechanisms, we enable dynamic
integration of visual and textual representations without requiring massive
image-text alignment datasets. Comprehensive experiments across nine multimodal
benchmarks demonstrate nuanced performance trade-offs: Inverse-LLaVA achieves
notable improvements on reasoning-intensive and cognitive tasks (MM-VET: +0.2%,
VizWiz: +1.8%, ScienceQA: +0.2%, cognitive reasoning: +27.2%), while showing
expected decreases in perception tasks requiring memorized visual-text
associations (celebrity recognition: -49.5%, OCR: -21.3%). These results
provide the first empirical evidence that alignment pre-training is not
necessary for effective multimodal learning, particularly for complex reasoning
tasks. Our work establishes the feasibility of a new paradigm that reduces
computational requirements by 45%, challenges conventional wisdom about
modality fusion, and opens new research directions for efficient multimodal
architectures that preserve modality-specific characteristics. Our project
website with code and additional resources is available at
https://inverse-llava.github.io.
BeyondWeb: Lessons from Scaling Synthetic Data for Trillion-scale
  Pretraining
Authors: name: Pratyush Maini, name: Vineeth Dorna, name: Parth Doshi, name: Aldo Carranza, name: Fan Pan, name: Jack Urbanek, name: Paul Burstein, name: Alex Fang, name: Alvin Deng, name: Amro Abbas, name: Brett Larsen, name: Cody Blakeney, name: Charvi Bannur, name: Christina Baek, name: Darren Teh, name: David Schwab, name: Haakon Mongstad, name: Haoli Yin, name: Josh Wills, name: Kaleigh Mentzer, name: Luke Merrick, name: Ricardo Monti, name: Rishabh Adiga, name: Siddharth Joshi, name: Spandan Das, name: Zhengping Wang, name: Bogdan Gaza, name: Ari Morcos, name: Matthew Leavitt
Published: 2025-08-14 17:55:47

Recent advances in large language model (LLM) pretraining have shown that
simply scaling data quantity eventually leads to diminishing returns, hitting a
data wall. In response, the use of synthetic data for pretraining has emerged
as a promising paradigm for pushing the frontier of performance. Despite this,
the factors affecting synthetic data quality remain poorly understood. In this
work, we introduce BeyondWeb, a synthetic data generation framework that
produces high-quality synthetic data for pretraining. BeyondWeb significantly
extends the capabilities of traditional web-scale datasets, outperforming
state-of-the-art synthetic pretraining datasets such as Cosmopedia and
Nemotron-CC's high-quality synthetic subset (Nemotron-Synth) by up to 5.1
percentage points (pp) and 2.6pp, respectively, when averaged across a suite of
14 benchmark evaluations. It delivers up to 7.7x faster training than open web
data and 2.7x faster than Nemotron-Synth. Remarkably, a 3B model trained for
180B tokens on BeyondWeb outperforms an 8B model trained for the same token
budget on Cosmopedia. We also present several insights from BeyondWeb on
synthetic data for pretraining: what drives its benefits, which data to
rephrase and how, and the impact of model size and family on data quality.
Overall, our work shows that there's no silver bullet for generating
high-quality synthetic pretraining data. The best outcomes require jointly
optimizing many factors, a challenging task that requires rigorous science and
practical expertise. Naive approaches can yield modest improvements,
potentially at great cost, while well-executed methods can yield transformative
improvements, as exemplified by BeyondWeb.
XQuant: Breaking the Memory Wall for LLM Inference with KV Cache
  Rematerialization
Authors: name: Aditya Tomar, name: Coleman Hooper, name: Minjae Lee, name: Haocheng Xi, name: Rishabh Tiwari, name: Wonjun Kang, name: Luca Manolache, name: Michael W. Mahoney, name: Kurt Keutzer, name: Amir Gholami
Published: 2025-08-14 06:52:38

Although LLM inference has emerged as a critical workload for many downstream
applications, efficiently inferring LLMs is challenging due to the substantial
memory footprint and bandwidth requirements. In parallel, compute capabilities
have steadily outpaced both memory capacity and bandwidth over the last few
decades, a trend that remains evident in modern GPU hardware and exacerbates
the challenge of LLM inference. As such, new algorithms are emerging that trade
increased computation for reduced memory operations. To that end, we present
XQuant, which takes advantage of this trend, enabling an order-of-magnitude
reduction in memory consumption through low-bit quantization with substantial
accuracy benefits relative to state-of-the-art KV cache quantization methods.
We accomplish this by quantizing and caching the layer input activations X,
instead of using standard KV caching, and then rematerializing the Keys and
Values on-the-fly during inference. This results in an immediate 2times
memory savings compared to KV caching. By applying XQuant, we achieve up to
sim 7.7times memory savings with <0.1 perplexity degradation compared to
the FP16 baseline. Furthermore, our approach leverages the fact that X values
are similar across layers. Building on this observation, we introduce
XQuant-CL, which exploits the cross-layer similarity in the X embeddings for
extreme compression. Across different models, XQuant-CL attains up to
10times memory savings relative to the FP16 baseline with only 0.01
perplexity degradation, and 12.5times memory savings with only 0.1
perplexity degradation. XQuant exploits the rapidly increasing compute
capabilities of hardware platforms to eliminate the memory bottleneck, while
surpassing state-of-the-art KV cache quantization methods and achieving
near-FP16 accuracy across a wide range of models.
SSRL: Self-Search Reinforcement Learning
Authors: name: Yuchen Fan, name: Kaiyan Zhang, name: Heng Zhou, name: Yuxin Zuo, name: Yanxu Chen, name: Yu Fu, name: Xinwei Long, name: Xuekai Zhu, name: Che Jiang, name: Yuchen Zhang, name: Li Kang, name: Gang Chen, name: Cheng Huang, name: Zhizhou He, name: Bingning Wang, name: Lei Bai, name: Ning Ding, name: Bowen Zhou
Published: 2025-08-14 17:46:01

We investigate the potential of large language models (LLMs) to serve as
efficient simulators for agentic search tasks in reinforcement learning (RL),
thereby reducing dependence on costly interactions with external search
engines. To this end, we first quantify the intrinsic search capability of LLMs
via structured prompting and repeated sampling, which we term Self-Search. Our
results reveal that LLMs exhibit strong scaling behavior with respect to the
inference budget, achieving high pass@k on question-answering benchmarks,
including the challenging BrowseComp task. Building on these observations, we
introduce Self-Search RL (SSRL), which enhances LLMs' Self-Search capability
through format-based and rule-based rewards. SSRL enables models to iteratively
refine their knowledge utilization internally, without requiring access to
external tools. Empirical evaluations demonstrate that SSRL-trained policy
models provide a cost-effective and stable environment for search-driven RL
training, reducing reliance on external search engines and facilitating robust
sim-to-real transfer. We draw the following conclusions: 1) LLMs possess world
knowledge that can be effectively elicited to achieve high performance; 2) SSRL
demonstrates the potential of leveraging internal knowledge to reduce
hallucination; 3) SSRL-trained models integrate seamlessly with external search
engines without additional effort. Our findings highlight the potential of LLMs
to support more scalable RL agent training.