Orionnnn/hub-stats huggingface dataset:
World-in-World: World Models in a Closed-Loop World
Authors: name: Jiahan Zhang, name: Muqing Jiang, name: Nanru Dai, name: Taiming Lu, name: Arda Uzunoglu, name: Shunchi Zhang, name: Yana Wei, name: Jiahao Wang, name: Vishal M. Patel, name: Paul Pu Liang, name: Daniel Khashabi, name: Cheng Peng, name: Rama Chellappa, name: Tianmin Shu, name: Alan Yuille, name: Yilun Du, name: Jieneng Chen
Published: 2025-10-20 22:09:15

Generative world models (WMs) can now simulate worlds with striking visual
realism, which naturally raises the question of whether they can endow embodied
agents with predictive perception for decision making. Progress on this
question has been limited by fragmented evaluation: most existing benchmarks
adopt open-loop protocols that emphasize visual quality in isolation, leaving
the core issue of embodied utility unresolved, i.e., do WMs actually help
agents succeed at embodied tasks? To address this gap, we introduce
World-in-World, the first open platform that benchmarks WMs in a closed-loop
world that mirrors real agent-environment interactions. World-in-World provides
a unified online planning strategy and a standardized action API, enabling
heterogeneous WMs for decision making. We curate four closed-loop environments
that rigorously evaluate diverse WMs, prioritize task success as the primary
metric, and move beyond the common focus on visual quality; we also present the
first data scaling law for world models in embodied settings. Our study
uncovers three surprises: (1) visual quality alone does not guarantee task
success, controllability matters more; (2) scaling post-training with
action-observation data is more effective than upgrading the pretrained video
generators; and (3) allocating more inference-time compute allows WMs to
substantially improve closed-loop performance.
Towards Faithful and Controllable Personalization via Critique-Post-Edit
  Reinforcement Learning
Authors: name: Chenghao Zhu, name: Meiling Tao, name: Tiannan Wang, name: Dongyi Ding, name: Yuchen Eleanor Jiang, name: Wangchunshu Zhou
Published: 2025-10-21 17:40:03

Faithfully personalizing large language models (LLMs) to align with
individual user preferences is a critical but challenging task. While
supervised fine-tuning (SFT) quickly reaches a performance plateau, standard
reinforcement learning from human feedback (RLHF) also struggles with the
nuances of personalization. Scalar-based reward models are prone to reward
hacking which leads to verbose and superficially personalized responses. To
address these limitations, we propose Critique-Post-Edit, a robust
reinforcement learning framework that enables more faithful and controllable
personalization. Our framework integrates two key components: (1) a
Personalized Generative Reward Model (GRM) that provides multi-dimensional
scores and textual critiques to resist reward hacking, and (2) a
Critique-Post-Edit mechanism where the policy model revises its own outputs
based on these critiques for more targeted and efficient learning. Under a
rigorous length-controlled evaluation, our method substantially outperforms
standard PPO on personalization benchmarks. Personalized Qwen2.5-7B achieves an
average 11\% win-rate improvement, and personalized Qwen2.5-14B model surpasses
the performance of GPT-4.1. These results demonstrate a practical path to
faithful, efficient, and controllable personalization.
MUG-V 10B: High-efficiency Training Pipeline for Large Video Generation
  Models
Authors: name: Yongshun Zhang, name: Zhongyi Fan, name: Yonghang Zhang, name: Zhangzikang Li, name: Weifeng Chen, name: Zhongwei Feng, name: Chaoyue Wang, name: Peng Hou, name: Anxiang Zeng
Published: 2025-10-20 13:20:37

In recent years, large-scale generative models for visual content
(e.g., images, videos, and 3D objects/scenes) have made remarkable
progress. However, training large-scale video generation models remains
particularly challenging and resource-intensive due to cross-modal text-video
alignment, the long sequences involved, and the complex spatiotemporal
dependencies. To address these challenges, we present a training framework that
optimizes four pillars: (i) data processing, (ii) model architecture, (iii)
training strategy, and (iv) infrastructure for large-scale video generation
models. These optimizations delivered significant efficiency gains and
performance improvements across all stages of data preprocessing, video
compression, parameter scaling, curriculum-based pretraining, and
alignment-focused post-training. Our resulting model, MUG-V 10B, matches recent
state-of-the-art video generators overall and, on e-commerce-oriented video
generation tasks, surpasses leading open-source baselines in human evaluations.
More importantly, we open-source the complete stack, including model weights,
Megatron-Core-based large-scale training code, and inference pipelines for
video generation and enhancement. To our knowledge, this is the first public
release of large-scale video generation training code that exploits
Megatron-Core to achieve high training efficiency and near-linear multi-node
scaling, details are available in
https://github.com/Shopee-MUG/MUG-V{our webpage}.
Video Reasoning without Training
Authors: name: Deepak Sridhar, name: Kartikeya Bhardwaj, name: Jeya Pradha Jeyaraj, name: Nuno Vasconcelos, name: Ankita Nayak, name: Harris Teague
Published: 2025-10-19 23:17:13

Video reasoning using Large Multimodal Models (LMMs) relies on costly
reinforcement learning (RL) and verbose chain-of-thought, resulting in
substantial computational overhead during both training and inference.
Moreover, the mechanisms that control the thinking process in these reasoning
models are very limited. In this paper, using entropy of the model's output as
a signal, we discover that the high-quality models go through a series of
micro-explorations and micro-exploitations which keep the reasoning process
grounded (i.e., avoid excessive randomness while the model is exploring or
thinking through an answer). We further observe that once this "thinking"
process is over, more accurate models demonstrate a better convergence by
reducing the entropy significantly via a final exploitation phase (i.e., a more
certain convergence towards a solution trajectory). We then use these novel,
theoretically-grounded insights to tune the model's behavior directly at
inference, without using any RL or supervised fine-tuning. Specifically, during
inference, our proposed approach called V-Reason (Video-Reason) adapts the
value cache of the LMM via a few optimization steps on a small, trainable
controller using an entropy-based objective, i.e., no supervision from any
dataset or RL is necessary. This tuning improves the model's micro-exploration
and exploitation behavior during inference. Our experiments show that our
proposed method achieves significant improvements over the base
instruction-tuned models across several video reasoning datasets, narrowing the
gap with RL-trained models to within 0.6% average accuracy without any
training, while offering massive efficiency benefits: output tokens are reduced
by 58.6% compared to the RL model.
ProCLIP: Progressive Vision-Language Alignment via LLM-based Embedder
Authors: name: Xiaoxing Hu, name: Kaicheng Yang, name: Ziyong Feng, name: Qi Ming, name: Zonghao Guo, name: Xiang An, name: Ziyong Feng, name: Junchi Yan, name: Xue Yang
Published: 2025-10-21 16:48:49

The original CLIP text encoder is limited by a maximum input length of 77
tokens, which hampers its ability to effectively process long texts and perform
fine-grained semantic understanding. In addition, the CLIP text encoder lacks
support for multilingual inputs. All these limitations significantly restrict
its applicability across a broader range of tasks. Recent studies have
attempted to replace the CLIP text encoder with an LLM-based embedder to
enhance its ability in processing long texts, multilingual understanding, and
fine-grained semantic comprehension. However, because the representation spaces
of LLMs and the vision-language space of CLIP are pretrained independently
without alignment priors, direct alignment using contrastive learning can
disrupt the intrinsic vision-language alignment in the CLIP image encoder,
leading to an underutilization of the knowledge acquired during pre-training.
To address this challenge, we propose ProCLIP, a curriculum learning-based
progressive vision-language alignment framework to effectively align the CLIP
image encoder with an LLM-based embedder. Specifically, ProCLIP first distills
knowledge from CLIP's text encoder into the LLM-based embedder to leverage
CLIP's rich pretrained knowledge while establishing initial alignment between
the LLM embedder and CLIP image encoder. Subsequently, ProCLIP further aligns
the CLIP image encoder with the LLM-based embedder through image-text
contrastive tuning, employing self-distillation regularization to avoid
overfitting. To achieve a more effective alignment, instance semantic alignment
loss and embedding structure alignment loss are employed during representation
inheritance and contrastive tuning. The Code is available at
https://github.com/VisionXLab/ProCLIP
LightMem: Lightweight and Efficient Memory-Augmented Generation
Authors: name: Jizhan Fang, name: Xinle Deng, name: Haoming Xu, name: Ziyan Jiang, name: Yuqi Tang, name: Ziwen Xu, name: Shumin Deng, name: Yunzhi Yao, name: Mengru Wang, name: Shuofei Qiao, name: Huajun Chen, name: Ningyu Zhang
Published: 2025-10-21 17:58:17

Despite their remarkable capabilities, Large Language Models (LLMs) struggle
to effectively leverage historical interaction information in dynamic and
complex environments. Memory systems enable LLMs to move beyond stateless
interactions by introducing persistent information storage, retrieval, and
utilization mechanisms. However, existing memory systems often introduce
substantial time and computational overhead. To this end, we introduce a new
memory system called LightMem, which strikes a balance between the performance
and efficiency of memory systems. Inspired by the Atkinson-Shiffrin model of
human memory, LightMem organizes memory into three complementary stages. First,
cognition-inspired sensory memory rapidly filters irrelevant information
through lightweight compression and groups information according to their
topics. Next, topic-aware short-term memory consolidates these topic-based
groups, organizing and summarizing content for more structured access. Finally,
long-term memory with sleep-time update employs an offline procedure that
decouples consolidation from online inference. Experiments on LongMemEval with
GPT and Qwen backbones show that LightMem outperforms strong baselines in
accuracy (up to 10.9% gains) while reducing token usage by up to 117x, API
calls by up to 159x, and runtime by over 12x. The code is available at
https://github.com/zjunlp/LightMem.
Chem-R: Learning to Reason as a Chemist
Authors: name: Weida Wang, name: Benteng Chen, name: Di Zhang, name: Wanhao Liu, name: Shuchen Pu, name: Ben Gao, name: Jin Zeng, name: Lei Bai, name: Wanli Ouyang, name: Xiaoyong Wei, name: Tianshu Yu, name: Tianfan Fu, name: Shuzhou Sun, name: Jiatong Li, name: Zifu Wang, name: Yuqiang Li, name: Shufei Zhang
Published: 2025-10-19 15:27:13

Although large language models (LLMs) have significant potential to advance
chemical discovery, current LLMs lack core chemical knowledge, produce
unreliable reasoning trajectories, and exhibit suboptimal performance across
diverse chemical tasks. To address these challenges, we propose Chem-R, a
generalizable Chemical Reasoning model designed to emulate the deliberative
processes of chemists. Chem-R is trained through a three-phase framework that
progressively builds advanced reasoning capabilities, including: 1) Chemical
Foundation Training, which establishes core chemical knowledge. 2) Chemical
Reasoning Protocol Distillation, incorporating structured, expert-like
reasoning traces to guide systematic and reliable problem solving. 3)
Multi-task Group Relative Policy Optimization that optimizes the model for
balanced performance across diverse molecular- and reaction-level tasks. This
structured pipeline enables Chem-R to achieve state-of-the-art performance on
comprehensive benchmarks, surpassing leading large language models, including
Gemini-2.5-Pro and DeepSeek-R1, by up to 46% on molecular tasks and 66% on
reaction tasks. Meanwhile, Chem-R also consistently outperforms the existing
chemical foundation models across both molecular and reaction level tasks.
These results highlight Chem-R's robust generalization, interpretability, and
potential as a foundation for next-generation AI-driven chemical discovery.
MoGA: Mixture-of-Groups Attention for End-to-End Long Video Generation
Authors: name: Weinan Jia, name: Yuning Lu, name: Mengqi Huang, name: Hualiang Wang, name: Binyuan Huang, name: Nan Chen, name: Mu Liu, name: Jidong Jiang, name: Zhendong Mao
Published: 2025-10-21 14:50:42

Long video generation with Diffusion Transformers (DiTs) is bottlenecked by
the quadratic scaling of full attention with sequence length. Since attention
is highly redundant, outputs are dominated by a small subset of query-key
pairs. Existing sparse methods rely on blockwise coarse estimation, whose
accuracy-efficiency trade-offs are constrained by block size. This paper
introduces Mixture-of-Groups Attention (MoGA), an efficient sparse attention
that uses a lightweight, learnable token router to precisely match tokens
without blockwise estimation. Through semantic-aware routing, MoGA enables
effective long-range interactions. As a kernel-free method, MoGA integrates
seamlessly with modern attention stacks, including FlashAttention and sequence
parallelism. Building on MoGA, we develop an efficient long video generation
model that end-to-end produces minute-level, multi-shot, 480p videos at 24 fps,
with a context length of approximately 580k. Comprehensive experiments on
various video generation tasks validate the effectiveness of our approach.
UniGenBench++: A Unified Semantic Evaluation Benchmark for Text-to-Image
  Generation
Authors: name: Yibin Wang, name: Zhimin Li, name: Yuhang Zang, name: Jiazi Bu, name: Yujie Zhou, name: Yi Xin, name: Junjun He, name: Chunyu Wang, name: Qinglin Lu, name: Cheng Jin, name: Jiaqi Wang
Published: 2025-10-21 14:56:46

Recent progress in text-to-image (T2I) generation underscores the importance
of reliable benchmarks in evaluating how accurately generated images reflect
the semantics of their textual prompt. However, (1) existing benchmarks lack
the diversity of prompt scenarios and multilingual support, both essential for
real-world applicability; (2) they offer only coarse evaluations across primary
dimensions, covering a narrow range of sub-dimensions, and fall short in
fine-grained sub-dimension assessment. To address these limitations, we
introduce UniGenBench++, a unified semantic assessment benchmark for T2I
generation. Specifically, it comprises 600 prompts organized hierarchically to
ensure both coverage and efficiency: (1) spans across diverse real-world
scenarios, i.e., 5 main prompt themes and 20 subthemes; (2) comprehensively
probes T2I models' semantic consistency over 10 primary and 27 sub evaluation
criteria, with each prompt assessing multiple testpoints. To rigorously assess
model robustness to variations in language and prompt length, we provide both
English and Chinese versions of each prompt in short and long forms. Leveraging
the general world knowledge and fine-grained image understanding capabilities
of a closed-source Multi-modal Large Language Model (MLLM), i.e.,
Gemini-2.5-Pro, an effective pipeline is developed for reliable benchmark
construction and streamlined model assessment. Moreover, to further facilitate
community use, we train a robust evaluation model that enables offline
assessment of T2I model outputs. Through comprehensive benchmarking of both
open- and closed-sourced T2I models, we systematically reveal their strengths
and weaknesses across various aspects.
MT-Video-Bench: A Holistic Video Understanding Benchmark for Evaluating
  Multimodal LLMs in Multi-Turn Dialogues
Authors: name: Yaning Pan, name: Zekun Wang, name: Qianqian Xie, name: Yongqian Wen, name: Yuanxing Zhang, name: Guohui Zhang, name: Haoxuan Hu, name: Zhiyu Pan, name: Yibing Huang, name: Zhidong Gan, name: Yonghong Lin, name: An Ping, name: Tianhao Peng, name: Jiaheng Liu
Published: 2025-10-20 16:38:40

The recent development of Multimodal Large Language Models (MLLMs) has
significantly advanced AI's ability to understand visual modalities. However,
existing evaluation benchmarks remain limited to single-turn question
answering, overlooking the complexity of multi-turn dialogues in real-world
scenarios. To bridge this gap, we introduce MT-Video-Bench, a holistic video
understanding benchmark for evaluating MLLMs in multi-turn dialogues.
Specifically, our MT-Video-Bench mainly assesses six core competencies that
focus on perceptivity and interactivity, encompassing 987 meticulously curated
multi-turn dialogues from diverse domains. These capabilities are rigorously
aligned with real-world applications, such as interactive sports analysis and
multi-turn video-based intelligent tutoring. With MT-Video-Bench, we
extensively evaluate various state-of-the-art open-source and closed-source
MLLMs, revealing their significant performance discrepancies and limitations in
handling multi-turn video dialogues. The benchmark will be publicly available
to foster future research.
UltraGen: High-Resolution Video Generation with Hierarchical Attention
Authors: name: Teng Hu, name: Jiangning Zhang, name: Zihan Su, name: Ran Yi
Published: 2025-10-21 16:23:21

Recent advances in video generation have made it possible to produce visually
compelling videos, with wide-ranging applications in content creation,
entertainment, and virtual reality. However, most existing diffusion
transformer based video generation models are limited to low-resolution outputs
(<=720P) due to the quadratic computational complexity of the attention
mechanism with respect to the output width and height. This computational
bottleneck makes native high-resolution video generation (1080P/2K/4K)
impractical for both training and inference. To address this challenge, we
present UltraGen, a novel video generation framework that enables i) efficient
and ii) end-to-end native high-resolution video synthesis. Specifically,
UltraGen features a hierarchical dual-branch attention architecture based on
global-local attention decomposition, which decouples full attention into a
local attention branch for high-fidelity regional content and a global
attention branch for overall semantic consistency. We further propose a
spatially compressed global modeling strategy to efficiently learn global
dependencies, and a hierarchical cross-window local attention mechanism to
reduce computational costs while enhancing information flow across different
local windows. Extensive experiments demonstrate that UltraGen can effectively
scale pre-trained low-resolution video models to 1080P and even 4K resolution
for the first time, outperforming existing state-of-the-art methods and
super-resolution based two-stage pipelines in both qualitative and quantitative
evaluations.
IF-VidCap: Can Video Caption Models Follow Instructions?
Authors: name: Shihao Li, name: Yuanxing Zhang, name: Jiangtao Wu, name: Zhide Lei, name: Yiwen He, name: Runzhe Wen, name: Chenxi Liao, name: Chengkang Jiang, name: An Ping, name: Shuo Gao, name: Suhan Wang, name: Zhaozhou Bian, name: Zijun Zhou, name: Jingyi Xie, name: Jiayi Zhou, name: Jing Wang, name: Yifan Yao, name: Weihao Xie, name: Yingshui Tan, name: Yanghai Wang, name: Qianqian Xie, name: Zhaoxiang Zhang, name: Jiaheng Liu
Published: 2025-10-21 15:25:08

Although Multimodal Large Language Models (MLLMs) have demonstrated
proficiency in video captioning, practical applications require captions that
follow specific user instructions rather than generating exhaustive,
unconstrained descriptions. Current benchmarks, however, primarily assess
descriptive comprehensiveness while largely overlooking instruction-following
capabilities. To address this gap, we introduce IF-VidCap, a new benchmark for
evaluating controllable video captioning, which contains 1,400 high-quality
samples. Distinct from existing video captioning or general
instruction-following benchmarks, IF-VidCap incorporates a systematic framework
that assesses captions on two dimensions: format correctness and content
correctness. Our comprehensive evaluation of over 20 prominent models reveals a
nuanced landscape: despite the continued dominance of proprietary models, the
performance gap is closing, with top-tier open-source solutions now achieving
near-parity. Furthermore, we find that models specialized for dense captioning
underperform general-purpose MLLMs on complex instructions, indicating that
future work should simultaneously advance both descriptive richness and
instruction-following fidelity.
Grasp Any Region: Towards Precise, Contextual Pixel Understanding for
  Multimodal LLMs
Authors: name: Haochen Wang, name: Yuhao Wang, name: Tao Zhang, name: Yikang Zhou, name: Yanwei Li, name: Jiacong Wang, name: Ye Tian, name: Jiahao Meng, name: Zilong Huang, name: Guangcan Mai, name: Anran Wang, name: Yunhai Tong, name: Zhuochen Wang, name: Xiangtai Li, name: Zhaoxiang Zhang
Published: 2025-10-21 17:59:59

While Multimodal Large Language Models (MLLMs) excel at holistic
understanding, they struggle in capturing the dense world with complex scenes,
requiring fine-grained analysis of intricate details and object
inter-relationships. Region-level MLLMs have been a promising step. However,
previous attempts are generally optimized to understand given regions in
isolation, neglecting crucial global contexts. To address this, we introduce
Grasp Any Region (GAR) for comprehen- sive region-level visual understanding.
Empowered by an effective RoI-aligned feature replay technique, GAR supports
(1) precise perception by leveraging necessary global contexts, and (2)
modeling interactions between multiple prompts. Together, it then naturally
achieves (3) advanced compositional reasoning to answer specific free-form
questions about any region, shifting the paradigm from passive description to
active dialogue. Moreover, we construct GAR-Bench, which not only provides a
more accurate evaluation of single-region comprehension, but also, more
importantly, measures interactions and complex reasoning across multiple
regions. Extensive experiments have demonstrated that GAR-1B not only maintains
the state-of-the-art captioning capabilities, e.g., outperforming DAM-3B +4.5
on DLC-Bench, but also excels at modeling relationships between multiple
prompts with advanced comprehension capabilities, even surpassing InternVL3-78B
on GAR-Bench-VQA. More importantly, our zero-shot GAR-8B even outperforms
in-domain VideoRefer-7B on VideoRefer-BenchQ, indicating its strong
capabilities can be easily transferred to videos.