Orionnnn/hub-stats huggingface dataset:
Knocking-Heads Attention
Authors: name: Zhanchao Zhou, name: Xiaodong Chen, name: Haoxing Chen, name: Zhenzhong Lan, name: Jianguo Li
Published: 2025-10-27 06:28:58

Multi-head attention (MHA) has become the cornerstone of modern large
language models, enhancing representational capacity through parallel attention
heads. However, increasing the number of heads inherently weakens individual
head capacity, and existing attention mechanisms - whether standard MHA or its
variants like grouped-query attention (GQA) and grouped-tied attention (GTA) -
simply concatenate outputs from isolated heads without strong interaction. To
address this limitation, we propose knocking-heads attention (KHA), which
enables attention heads to "knock" on each other - facilitating cross-head
feature-level interactions before the scaled dot-product attention. This is
achieved by applying a shared, diagonally-initialized projection matrix across
all heads. The diagonal initialization preserves head-specific specialization
at the start of training while allowing the model to progressively learn
integrated cross-head representations. KHA adds only minimal parameters and
FLOPs and can be seamlessly integrated into MHA, GQA, GTA, and other attention
variants. We validate KHA by training a 6.1B parameter MoE model (1.01B
activated) on 1T high-quality tokens. Compared to baseline attention
mechanisms, KHA brings superior and more stable training dynamics, achieving
better performance across downstream tasks.
FARMER: Flow AutoRegressive Transformer over Pixels
Authors: name: Guangting Zheng, name: Qinyu Zhao, name: Tao Yang, name: Fei Xiao, name: Zhijie Lin, name: Jie Wu, name: Jiajun Deng, name: Yanyong Zhang, name: Rui Zhu
Published: 2025-10-27 17:54:08

Directly modeling the explicit likelihood of the raw data distribution is key
topic in the machine learning area, which achieves the scaling successes in
Large Language Models by autoregressive modeling. However, continuous AR
modeling over visual pixel data suffer from extremely long sequences and
high-dimensional spaces. In this paper, we present FARMER, a novel end-to-end
generative framework that unifies Normalizing Flows (NF) and Autoregressive
(AR) models for tractable likelihood estimation and high-quality image
synthesis directly from raw pixels. FARMER employs an invertible autoregressive
flow to transform images into latent sequences, whose distribution is modeled
implicitly by an autoregressive model. To address the redundancy and complexity
in pixel-level modeling, we propose a self-supervised dimension reduction
scheme that partitions NF latent channels into informative and redundant
groups, enabling more effective and efficient AR modeling. Furthermore, we
design a one-step distillation scheme to significantly accelerate inference
speed and introduce a resampling-based classifier-free guidance algorithm to
boost image generation quality. Extensive experiments demonstrate that FARMER
achieves competitive performance compared to existing pixel-based generative
models while providing exact likelihoods and scalable training.
RobotArena infty: Scalable Robot Benchmarking via Real-to-Sim
  Translation
Authors: name: Yash Jangir, name: Yidi Zhang, name: Kashu Yamazaki, name: Chenyu Zhang, name: Kuan-Hsun Tu, name: Tsung-Wei Ke, name: Lei Ke, name: Yonatan Bisk, name: Katerina Fragkiadaki
Published: 2025-10-27 17:41:38

The pursuit of robot generalists - instructable agents capable of performing
diverse tasks across diverse environments - demands rigorous and scalable
evaluation. Yet real-world testing of robot policies remains fundamentally
constrained: it is labor-intensive, slow, unsafe at scale, and difficult to
reproduce. Existing simulation benchmarks are similarly limited, as they train
and test policies within the same synthetic domains and cannot assess models
trained from real-world demonstrations or alternative simulation environments.
As policies expand in scope and complexity, these barriers only intensify,
since defining "success" in robotics often hinges on nuanced human judgments of
execution quality. In this paper, we introduce a new benchmarking framework
that overcomes these challenges by shifting VLA evaluation into large-scale
simulated environments augmented with online human feedback. Leveraging
advances in vision-language models, 2D-to-3D generative modeling, and
differentiable rendering, our approach automatically converts video
demonstrations from widely used robot datasets into simulated counterparts.
Within these digital twins, we assess VLA policies using both automated
VLM-guided scoring and scalable human preference judgments collected from
crowdworkers, transforming human involvement from tedious scene setup,
resetting, and safety supervision into lightweight preference comparisons. To
measure robustness, we systematically perturb simulated environments along
multiple axes, such as textures and object placements, stress-testing policy
generalization under controlled variation. The result is a continuously
evolving, reproducible, and scalable benchmark for real-world trained robot
manipulation policies, addressing a critical missing capability in today's
robotics landscape.
Distilled Decoding 2: One-step Sampling of Image Auto-regressive Models
  with Conditional Score Distillation
Authors: name: Enshu Liu, name: Qian Chen, name: Xuefei Ning, name: Shengen Yan, name: Guohao Dai, name: Zinan Lin, name: Yu Wang
Published: 2025-10-23 21:21:38

Image Auto-regressive (AR) models have emerged as a powerful paradigm of
visual generative models. Despite their promising performance, they suffer from
slow generation speed due to the large number of sampling steps required.
Although Distilled Decoding 1 (DD1) was recently proposed to enable few-step
sampling for image AR models, it still incurs significant performance
degradation in the one-step setting, and relies on a pre-defined mapping that
limits its flexibility. In this work, we propose a new method, Distilled
Decoding 2 (DD2), to further advances the feasibility of one-step sampling for
image AR models. Unlike DD1, DD2 does not without rely on a pre-defined
mapping. We view the original AR model as a teacher model which provides the
ground truth conditional score in the latent embedding space at each token
position. Based on this, we propose a novel conditional score
distillation loss to train a one-step generator. Specifically, we train a
separate network to predict the conditional score of the generated distribution
and apply score distillation at every token position conditioned on previous
tokens. Experimental results show that DD2 enables one-step sampling for image
AR models with an minimal FID increase from 3.40 to 5.43 on ImageNet-256.
Compared to the strongest baseline DD1, DD2 reduces the gap between the
one-step sampling and original AR model by 67%, with up to 12.3times
training speed-up simultaneously. DD2 takes a significant step toward the goal
of one-step AR generation, opening up new possibilities for fast and
high-quality AR modeling. Code is available at
https://github.com/imagination-research/Distilled-Decoding-2.
LongCat-Video Technical Report
Authors: name: Meituan LongCat Team, name: Xunliang Cai, name: Qilong Huang, name: Zhuoliang Kang, name: Hongyu Li, name: Shijun Liang, name: Liya Ma, name: Siyu Ren, name: Xiaoming Wei, name: Rixu Xie, name: Tong Zhang
Published: 2025-10-25 07:41:02

Video generation is a critical pathway toward world models, with efficient
long video inference as a key capability. Toward this end, we introduce
LongCat-Video, a foundational video generation model with 13.6B parameters,
delivering strong performance across multiple video generation tasks. It
particularly excels in efficient and high-quality long video generation,
representing our first step toward world models. Key features include: Unified
architecture for multiple tasks: Built on the Diffusion Transformer (DiT)
framework, LongCat-Video supports Text-to-Video, Image-to-Video, and
Video-Continuation tasks with a single model; Long video generation:
Pretraining on Video-Continuation tasks enables LongCat-Video to maintain high
quality and temporal coherence in the generation of minutes-long videos;
Efficient inference: LongCat-Video generates 720p, 30fps videos within minutes
by employing a coarse-to-fine generation strategy along both the temporal and
spatial axes. Block Sparse Attention further enhances efficiency, particularly
at high resolutions; Strong performance with multi-reward RLHF: Multi-reward
RLHF training enables LongCat-Video to achieve performance on par with the
latest closed-source and leading open-source models. Code and model weights are
publicly available to accelerate progress in the field.
Concerto: Joint 2D-3D Self-Supervised Learning Emerges Spatial
  Representations
Authors: name: Yujia Zhang, name: Xiaoyang Wu, name: Yixing Lao, name: Chengyao Wang, name: Zhuotao Tian, name: Naiyan Wang, name: Hengshuang Zhao
Published: 2025-10-27 17:59:59

Humans learn abstract concepts through multisensory synergy, and once formed,
such representations can often be recalled from a single modality. Inspired by
this principle, we introduce Concerto, a minimalist simulation of human concept
learning for spatial cognition, combining 3D intra-modal self-distillation with
2D-3D cross-modal joint embedding. Despite its simplicity, Concerto learns more
coherent and informative spatial features, as demonstrated by zero-shot
visualizations. It outperforms both standalone SOTA 2D and 3D self-supervised
models by 14.2% and 4.8%, respectively, as well as their feature concatenation,
in linear probing for 3D scene perception. With full fine-tuning, Concerto sets
new SOTA results across multiple scene understanding benchmarks (e.g., 80.7%
mIoU on ScanNet). We further present a variant of Concerto tailored for
video-lifted point cloud spatial understanding, and a translator that linearly
projects Concerto representations into CLIP's language space, enabling
open-world perception. These results highlight that Concerto emerges spatial
representations with superior fine-grained geometric and semantic consistency.
LimRank: Less is More for Reasoning-Intensive Information Reranking
Authors: name: Tingyu Song, name: Yilun Zhao, name: Siyue Zhang, name: Chen Zhao, name: Arman Cohan
Published: 2025-10-27 17:19:37

Existing approaches typically rely on large-scale fine-tuning to adapt LLMs
for information reranking tasks, which is computationally expensive. In this
work, we demonstrate that modern LLMs can be effectively adapted using only
minimal, high-quality supervision. To enable this, we design
LIMRANK-SYNTHESIZER, a reusable and open-source pipeline for generating
diverse, challenging, and realistic reranking examples. Using this synthetic
data, we fine-tune our reranker model, LIMRANK. We evaluate LIMRANK on two
challenging benchmarks, i.e., BRIGHT for reasoning-intensive retrieval and
FollowIR for instruction-following retrieval. Our experiments demonstrate that
LIMRANK achieves competitive performance, while being trained on less than 5%
of the data typically used in prior work. Further ablation studies demonstrate
the effectiveness of LIMRANK-SYNTHESIZER and the strong generalization
capabilities of LIMRANK across downstream tasks, including scientific
literature search and retrieval-augmented generation for knowledge-intensive
problem solving.
E^2Rank: Your Text Embedding can Also be an Effective
  and Efficient Listwise Reranker
Authors: name: Qi Liu, name: Yanzhao Zhang, name: Mingxin Li, name: Dingkun Long, name: Pengjun Xie, name: Jiaxin Mao
Published: 2025-10-26 16:04:48

Text embedding models serve as a fundamental component in real-world search
applications. By mapping queries and documents into a shared embedding space,
they deliver competitive retrieval performance with high efficiency. However,
their ranking fidelity remains limited compared to dedicated rerankers,
especially recent LLM-based listwise rerankers, which capture fine-grained
query-document and document-document interactions. In this paper, we propose a
simple yet effective unified framework E^2Rank, means Efficient
Embedding-based Ranking (also means Embedding-to-Rank), which extends a single
text embedding model to perform both high-quality retrieval and listwise
reranking through continued training under a listwise ranking objective,
thereby achieving strong effectiveness with remarkable efficiency. By applying
cosine similarity between the query and document embeddings as a unified
ranking function, the listwise ranking prompt, which is constructed from the
original query and its candidate documents, serves as an enhanced query
enriched with signals from the top-K documents, akin to pseudo-relevance
feedback (PRF) in traditional retrieval models. This design preserves the
efficiency and representational quality of the base embedding model while
significantly improving its reranking performance. Empirically,
E^2Rank achieves state-of-the-art results on the BEIR
reranking benchmark and demonstrates competitive performance on the
reasoning-intensive BRIGHT benchmark, with very low reranking latency. We also
show that the ranking training process improves embedding performance on the
MTEB benchmark. Our findings indicate that a single embedding model can
effectively unify retrieval and reranking, offering both computational
efficiency and competitive ranking accuracy.
LightBagel: A Light-weighted, Double Fusion Framework for Unified
  Multimodal Understanding and Generation
Authors: name: Zeyu Wang, name: Zilong Chen, name: Chenhui Gou, name: Feng Li, name: Chaorui Deng, name: Deyao Zhu, name: Kunchang Li, name: Weihao Yu, name: Haoqin Tu, name: Haoqi Fan, name: Cihang Xie
Published: 2025-10-27 02:59:57

Unified multimodal models have recently shown remarkable gains in both
capability and versatility, yet most leading systems are still trained from
scratch and require substantial computational resources. In this paper, we show
that competitive performance can be obtained far more efficiently by
strategically fusing publicly available models specialized for either
generation or understanding. Our key design is to retain the original blocks
while additionally interleaving multimodal self-attention blocks throughout the
networks. This double fusion mechanism (1) effectively enables rich multi-modal
fusion while largely preserving the original strengths of the base models, and
(2) catalyzes synergistic fusion of high-level semantic representations from
the understanding encoder with low-level spatial signals from the generation
encoder. By training with only ~ 35B tokens, this approach achieves strong
results across multiple benchmarks: 0.91 on GenEval for compositional
text-to-image generation, 82.16 on DPG-Bench for complex text-to-image
generation, 6.06 on GEditBench, and 3.77 on ImgEdit-Bench for image editing. By
fully releasing the entire suite of code, model weights, and datasets, we hope
to support future research on unified multimodal modeling.
PRISM-Bench: A Benchmark of Puzzle-Based Visual Tasks with CoT Error
  Detection
Authors: name: Yusu Qian, name: Cheng Wan, name: Chao Jia, name: Yinfei Yang, name: Qingyu Zhao, name: Zhe Gan
Published: 2025-10-27 17:57:52

We introduce PRISM-Bench, a benchmark of puzzle-based visual
challenges designed to evaluate not only whether models can solve problems, but
how their reasoning unfolds. Unlike prior evaluations that measure only
final-answer accuracy, PRISM-Bench introduces a diagnostic task: given a visual
puzzle and a step-by-step chain-of-thought (CoT) containing exactly one error,
models must identify the first incorrect step. This setting enables
fine-grained assessment of logical consistency, error detection, and visual
reasoning. The puzzles in PRISM-Bench require multi-step symbolic, geometric,
and analogical reasoning, resisting shortcuts based on superficial pattern
matching. Evaluations across state-of-the-art MLLMs reveal a persistent gap
between fluent generation and faithful reasoning: models that produce plausible
CoTs often fail to locate simple logical faults. By disentangling answer
generation from reasoning verification, PRISM-Bench offers a sharper lens on
multimodal reasoning competence and underscores the need for diagnostic
evaluation protocols in the development of trustworthy MLLMs.
PixelRefer: A Unified Framework for Spatio-Temporal Object Referring
  with Arbitrary Granularity
Authors: name: Yuqian Yuan, name: Wenqiao Zhang, name: Xin Li, name: Shihao Wang, name: Kehan Li, name: Wentong Li, name: Jun Xiao, name: Lei Zhang, name: Beng Chin Ooi
Published: 2025-10-27 17:59:32

Multimodal large language models (MLLMs) have demonstrated strong
general-purpose capabilities in open-world visual comprehension. However, most
existing MLLMs primarily focus on holistic, scene-level understanding, often
overlooking the need for fine-grained, object-centric reasoning. In this paper,
we present PixelRefer, a unified region-level MLLM framework that enables
advanced fine-grained understanding over user-specified regions across both
images and videos. Motivated by the observation that LLM attention
predominantly focuses on object-level tokens, we propose a Scale-Adaptive
Object Tokenizer (SAOT) to generate compact and semantically rich object
representations from free-form regions. Our analysis reveals that global visual
tokens contribute mainly in early LLM layers, inspiring the design of
PixelRefer-Lite, an efficient variant that employs an Object-Centric Infusion
module to pre-fuse global context into object tokens. This yields a lightweight
Object-Only Framework that substantially reduces computational cost while
maintaining high semantic fidelity. To facilitate fine-grained instruction
tuning, we curate PixelRefer-2.2M, a high-quality object-centric instruction
dataset. Extensive experiments across a range of benchmarks validate that
PixelRefer achieves leading performance with fewer training samples, while
PixelRefer-Lite offers competitive accuracy with notable gains in efficiency.
Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with
  Free-Form Preferences
Authors: name: Zhuoran Jin, name: Hongbang Yuan, name: Kejian Zhu, name: Jiachun Li, name: Pengfei Cao, name: Yubo Chen, name: Kang Liu, name: Jun Zhao
Published: 2025-10-27 15:53:20

Reward models (RMs) play a critical role in aligning AI behaviors with human
preferences, yet they face two fundamental challenges: (1) Modality Imbalance,
where most RMs are mainly focused on text and image modalities, offering
limited support for video, audio, and other modalities; and (2) Preference
Rigidity, where training on fixed binary preference pairs fails to capture the
complexity and diversity of personalized preferences. To address the above
challenges, we propose Omni-Reward, a step toward generalist omni-modal reward
modeling with support for free-form preferences, consisting of: (1) Evaluation:
We introduce Omni-RewardBench, the first omni-modal RM benchmark with free-form
preferences, covering nine tasks across five modalities including text, image,
video, audio, and 3D; (2) Data: We construct Omni-RewardData, a multimodal
preference dataset comprising 248K general preference pairs and 69K
instruction-tuning pairs for training generalist omni-modal RMs; (3) Model: We
propose Omni-RewardModel, which includes both discriminative and generative
RMs, and achieves strong performance on Omni-RewardBench as well as other
widely used reward modeling benchmarks.
Language Server CLI Empowers Language Agents with Process Rewards
Authors: name: Yifan Zhang, name: Lanser Contributors
Published: 2025-10-27 01:25:20

Large language models routinely hallucinate APIs and mislocalize edits, while
language servers compute verified, IDE-grade facts about real code. We present
Lanser-CLI, a CLI-first orchestration layer that pins and mediates a Language
Server Protocol (LSP) server for coding agents and CI, exposing deterministic,
replayable workflows. Our position is that language servers provide not only
structural information (definitions, references, types, diagnostics) but also
an actionable process reward: machine-checked, step-wise signals that align an
agent's planning loop with program reality. In this work, Lanser-CLI
contributes: (i) a robust addressing scheme beyond brittle "file:line:col" via
a Selector DSL (symbolic, AST-path, and content-anchored selectors) with a
principled relocation algorithm; (ii) deterministic Analysis Bundles that
normalize Language Server responses and capture environment/capability metadata
with stable content hashes; (iii) a safety envelope for mutating operations
(rename, code actions) with preview, workspace jails, and Git-aware,
transactional apply; and (iv) a process-reward functional derived from Language
Server facts (diagnostic deltas, disambiguation confidence, and safe-apply
checks) that is computable online and replayable offline. We formalize
determinism under frozen snapshots and establish a monotonicity property for
the process reward, making it suitable for process supervision and
counterfactual analysis. Project Page:
https://github.com/yifanzhang-pro/lanser-cli
VITA-E: Natural Embodied Interaction with Concurrent Seeing, Hearing,
  Speaking, and Acting
Authors: name: Xiaoyu Liu, name: Chaoyou Fu, name: Chi Yan, name: Chu Wu, name: Haihan Gao, name: Yi-Fan Zhang, name: Shaoqi Dong, name: Cheng Qian, name: Bin Luo, name: Xiuyong Yang, name: Guanwu Li, name: Yusheng Cai, name: Yunhang Shen, name: Deqiang Jiang, name: Haoyu Cao, name: Xing Sun, name: Caifeng Shan, name: Ran He
Published: 2025-10-21 17:59:56

Current Vision-Language-Action (VLA) models are often constrained by a rigid,
static interaction paradigm, which lacks the ability to see, hear, speak, and
act concurrently as well as handle real-time user interruptions dynamically.
This hinders seamless embodied collaboration, resulting in an inflexible and
unresponsive user experience. To address these limitations, we introduce
VITA-E, a novel embodied interaction framework designed for both behavioral
concurrency and nearly real-time interruption. The core of our approach is a
dual-model architecture where two parallel VLA instances operate as an ``Active
Model'' and a ``Standby Model'', allowing the embodied agent to observe its
environment, listen to user speech, provide verbal responses, and execute
actions, all concurrently and interruptibly, mimicking human-like multitasking
capabilities. We further propose a ``model-as-controller'' paradigm, where we
fine-tune the VLM to generate special tokens that serve as direct system-level
commands, coupling the model's reasoning with the system's behavior.
Experiments conducted on a physical humanoid platform demonstrate that VITA-E
can reliably handle complex interactive scenarios. Our framework is compatible
with various dual-system VLA models, achieving an extremely high success rate
on emergency stops and speech interruptions while also successfully performing
concurrent speech and action. This represents a significant step towards more
natural and capable embodied assistants.