Orionnnn/hub-stats huggingface dataset:
Value Drifts: Tracing Value Alignment During LLM Post-Training
Authors: name: Mehar Bhatia, name: Shravan Nayak, name: Gaurav Kamath, name: Marius Mosbach, name: Karolina Stańczak, name: Vered Shwartz, name: Siva Reddy
Published: 2025-10-30 17:09:09

As LLMs occupy an increasingly important role in society, they are more and
more confronted with questions that require them not only to draw on their
general knowledge but also to align with certain human value systems.
Therefore, studying the alignment of LLMs with human values has become a
crucial field of inquiry. Prior work, however, mostly focuses on evaluating the
alignment of fully trained models, overlooking the training dynamics by which
models learn to express human values. In this work, we investigate how and at
which stage value alignment arises during the course of a model's
post-training. Our analysis disentangles the effects of post-training
algorithms and datasets, measuring both the magnitude and time of value drifts
during training. Experimenting with Llama-3 and Qwen-3 models of different
sizes and popular supervised fine-tuning (SFT) and preference optimization
datasets and algorithms, we find that the SFT phase generally establishes a
model's values, and subsequent preference optimization rarely re-aligns these
values. Furthermore, using a synthetic preference dataset that enables
controlled manipulation of values, we find that different preference
optimization algorithms lead to different value alignment outcomes, even when
preference data is held constant. Our findings provide actionable insights into
how values are learned during post-training and help to inform data curation,
as well as the selection of models and algorithms for preference optimization
to improve model alignment to human values.
A Survey on Efficient Vision-Language-Action Models
Authors: name: Zhaoshu Yu, name: Bo Wang, name: Pengpeng Zeng, name: Haonan Zhang, name: Ji Zhang, name: Lianli Gao, name: Jingkuan Song, name: Nicu Sebe, name: Heng Tao Shen
Published: 2025-10-27 17:57:33

Vision-Language-Action models (VLAs) represent a significant frontier in
embodied intelligence, aiming to bridge digital knowledge with physical-world
interaction. While these models have demonstrated remarkable generalist
capabilities, their deployment is severely hampered by the substantial
computational and data requirements inherent to their underlying large-scale
foundation models. Motivated by the urgent need to address these challenges,
this survey presents the first comprehensive review of Efficient
Vision-Language-Action models (Efficient VLAs) across the entire
data-model-training process. Specifically, we introduce a unified taxonomy to
systematically organize the disparate efforts in this domain, categorizing
current techniques into three core pillars: (1) Efficient Model Design,
focusing on efficient architectures and model compression; (2) Efficient
Training, which reduces computational burdens during model learning; and (3)
Efficient Data Collection, which addresses the bottlenecks in acquiring and
utilizing robotic data. Through a critical review of state-of-the-art methods
within this framework, this survey not only establishes a foundational
reference for the community but also summarizes representative applications,
delineates key challenges, and charts a roadmap for future research. We
maintain a continuously updated project page to track our latest developments:
https://evla-survey.github.io/
Limits of Generalization in RLVR: Two Case Studies in Mathematical
  Reasoning
Authors: name: Md Tanvirul Alam, name: Nidhi Rastogi
Published: 2025-10-30 23:16:02

Mathematical reasoning is a central challenge for large language models
(LLMs), requiring not only correct answers but also faithful reasoning
processes. Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as
a promising approach for enhancing such capabilities; however, its ability to
foster genuine reasoning remains unclear. We investigate RLVR on two
combinatorial problems with fully verifiable solutions: Activity
Scheduling and the Longest Increasing Subsequence, using carefully
curated datasets with unique optima. Across multiple reward designs, we find
that RLVR improves evaluation metrics but often by reinforcing superficial
heuristics rather than acquiring new reasoning strategies. These findings
highlight the limits of RLVR generalization, emphasizing the importance of
benchmarks that disentangle genuine mathematical reasoning from shortcut
exploitation and provide faithful measures of progress. Code available at
https://github.com/xashru/rlvr-seq-generalization.
Revisiting Multimodal Positional Encoding in Vision-Language Models
Authors: name: Jie Huang, name: Xuejing Liu, name: Sibo Song, name: Ruibing Hou, name: Hong Chang, name: Junyang Lin, name: Shuai Bai
Published: 2025-10-27 08:00:46

Multimodal position encoding is essential for vision-language models, yet
there has been little systematic investigation into multimodal position
encoding. We conduct a comprehensive analysis of multimodal Rotary Positional
Embedding (RoPE) by examining its two core components: position design and
frequency allocation. Through extensive experiments, we identify three key
guidelines: positional coherence, full frequency utilization, and preservation
of textual priors-ensuring unambiguous layout, rich representation, and
faithful transfer from the pre-trained LLM. Based on these insights, we propose
Multi-Head RoPE (MHRoPE) and MRoPE-Interleave (MRoPE-I), two simple and
plug-and-play variants that require no architectural changes. Our methods
consistently outperform existing approaches across diverse benchmarks, with
significant improvements in both general and fine-grained multimodal
understanding. Code will be avaliable at
https://github.com/JJJYmmm/Multimodal-RoPEs.
π_RL: Online RL Fine-tuning for Flow-based
  Vision-Language-Action Models
Authors: name: Kang Chen, name: Zhihao Liu, name: Tonghe Zhang, name: Zhen Guo, name: Si Xu, name: Hao Lin, name: Hongzhi Zang, name: Quanlu Zhang, name: Zhaofei Yu, name: Guoliang Fan, name: Tiejun Huang, name: Yu Wang, name: Chao Yu
Published: 2025-10-29 18:37:39

Vision-Language-Action (VLA) models enable robots to understand and perform
complex tasks from multimodal input. Although recent work explores using
reinforcement learning (RL) to automate the laborious data collection process
in scaling supervised fine-tuning (SFT), applying large-scale RL to flow-based
VLAs (e.g., pi_0, pi_{0.5}) remains challenging due to intractable action
log-likelihoods from iterative denoising.
  We address this challenge with pi_{RL}, an open-source framework
for training flow-based VLAs in parallel simulation. pi_{RL}
implements two RL algorithms: (1) {Flow-Noise} models the denoising process as
a discrete-time MDP with a learnable noise network for exact log-likelihood
computation. (2) {Flow-SDE} integrates denoising with agent-environment
interaction, formulating a two-layer MDP that employs ODE-to-SDE conversion for
efficient RL exploration.
  We evaluate pi_{RL} on LIBERO and ManiSkill benchmarks. On LIBERO,
pi_{RL} boosts few-shot SFT models pi_0 and pi_{0.5} from 57.6%
to 97.6% and from 77.1% to 98.3%, respectively. In ManiSkill, we train
pi_{RL} in 320 parallel environments, improving pi_0 from 41.6% to
85.7% and pi_{0.5} from 40.0% to 84.8% across 4352 pick-and-place tasks,
demonstrating scalable multitask RL under heterogeneous simulation.
  Overall, pi_{RL} achieves significant performance gains and
stronger generalization over SFT-models, validating the effectiveness of online
RL for flow-based VLAs.
Mask-to-Height: A YOLOv11-Based Architecture for Joint Building Instance
  Segmentation and Height Classification from Satellite Imagery
Authors: name: Mahmoud El Hussieni, name: Bahadır K. Güntürk, name: Hasan F. Ateş, name: Oğuz Hanoğlu
Published: 2025-10-31 06:37:08

Accurate building instance segmentation and height classification are
critical for urban planning, 3D city modeling, and infrastructure monitoring.
This paper presents a detailed analysis of YOLOv11, the recent advancement in
the YOLO series of deep learning models, focusing on its application to joint
building extraction and discrete height classification from satellite imagery.
YOLOv11 builds on the strengths of earlier YOLO models by introducing a more
efficient architecture that better combines features at different scales,
improves object localization accuracy, and enhances performance in complex
urban scenes. Using the DFC2023 Track 2 dataset -- which includes over 125,000
annotated buildings across 12 cities -- we evaluate YOLOv11's performance using
metrics such as precision, recall, F1 score, and mean average precision (mAP).
Our findings demonstrate that YOLOv11 achieves strong instance segmentation
performance with 60.4\% mAP@50 and 38.3\% mAP@50--95 while maintaining robust
classification accuracy across five predefined height tiers. The model excels
in handling occlusions, complex building shapes, and class imbalance,
particularly for rare high-rise structures. Comparative analysis confirms that
YOLOv11 outperforms earlier multitask frameworks in both detection accuracy and
inference speed, making it well-suited for real-time, large-scale urban
mapping. This research highlights YOLOv11's potential to advance semantic urban
reconstruction through streamlined categorical height modeling, offering
actionable insights for future developments in remote sensing and geospatial
intelligence.
Spatial-SSRL: Enhancing Spatial Understanding via Self-Supervised
  Reinforcement Learning
Authors: name: Yuhong Liu, name: Beichen Zhang, name: Yuhang Zang, name: Yuhang Cao, name: Long Xing, name: Xiaoyi Dong, name: Haodong Duan, name: Dahua Lin, name: Jiaqi Wang
Published: 2025-10-31 16:30:08

Spatial understanding remains a weakness of Large Vision-Language Models
(LVLMs). Existing supervised fine-tuning (SFT) and recent reinforcement
learning with verifiable rewards (RLVR) pipelines depend on costly supervision,
specialized tools, or constrained environments that limit scale. We introduce
Spatial-SSRL, a self-supervised RL paradigm that derives verifiable signals
directly from ordinary RGB or RGB-D images. Spatial-SSRL automatically
formulates five pretext tasks that capture 2D and 3D spatial structure:
shuffled patch reordering, flipped patch recognition, cropped patch inpainting,
regional depth ordering, and relative 3D position prediction. These tasks
provide ground-truth answers that are easy to verify and require no human or
LVLM annotation. Training on our tasks substantially improves spatial reasoning
while preserving general visual capabilities. On seven spatial understanding
benchmarks in both image and video settings, Spatial-SSRL delivers average
accuracy gains of 4.63% (3B) and 3.89% (7B) over the Qwen2.5-VL baselines. Our
results show that simple, intrinsic supervision enables RLVR at scale and
provides a practical route to stronger spatial intelligence in LVLMs.
Defeating the Training-Inference Mismatch via FP16
Authors: name: Penghui Qi, name: Zichen Liu, name: Xiangxin Zhou, name: Tianyu Pang, name: Chao Du, name: Wee Sun Lee, name: Min Lin
Published: 2025-10-30 17:58:11

Reinforcement learning (RL) fine-tuning of large language models (LLMs) often
suffers from instability due to the numerical mismatch between the training and
inference policies. While prior work has attempted to mitigate this issue
through algorithmic corrections or engineering alignments, we show that its
root cause lies in the floating point precision itself. The widely adopted
BF16, despite its large dynamic range, introduces large rounding errors that
breaks the consistency between training and inference. In this work, we
demonstrate that simply reverting to FP16 effectively eliminates this
mismatch. The change is simple, fully supported by modern frameworks with only
a few lines of code change, and requires no modification to the model
architecture or learning algorithm. Our results suggest that using FP16
uniformly yields more stable optimization, faster convergence, and stronger
performance across diverse tasks, algorithms and frameworks. We hope these
findings motivate a broader reconsideration of precision trade-offs in RL
fine-tuning.
INT v.s. FP: A Comprehensive Study of Fine-Grained Low-bit Quantization
  Formats
Authors: name: Mengzhao Chen, name: Meng Wu, name: Hui Jin, name: Zhihang Yuan, name: Jing Liu, name: Chaoyi Zhang, name: Yunshui Li, name: Jie Huang, name: Jin Ma, name: Zeyue Xue, name: Zhiheng Liu, name: Xingyan Bin, name: Ping Luo
Published: 2025-10-29 15:11:53

Modern AI hardware, such as Nvidia's Blackwell architecture, is increasingly
embracing low-precision floating-point (FP) formats to handle the pervasive
activation outliers in Large Language Models (LLMs). Despite this industry
trend, a unified comparison of FP and integer (INT) quantization across varying
granularities has been missing, leaving algorithm and hardware co-design
without clear guidance. This paper fills that gap by systematically
investigating the trade-offs between FP and INT formats. We reveal a critical
performance crossover: while FP excels in coarse-grained quantization, the
comparison at fine-grained (block-wise) levels is more nuanced. Our
comprehensive comparison demonstrates that for popular 8-bit fine-grained
formats (e.g., MX with block size 32), MXINT8 is superior to its FP counterpart
in both algorithmic accuracy and hardware efficiency. However, for 4-bit
formats, FP (e.g., MXFP4, NVFP4) often holds an accuracy advantage , though we
show that NVINT4 can surpass NVFP4 when outlier-mitigation techniques like
Hadamard rotation are applied. We also introduce a symmetric clipping method
that resolves gradient bias in fine-grained low-bit INT training, enabling
nearly lossless performance for MXINT8 training. These findings challenge the
current hardware trajectory, demonstrating that a one-size-fits-all FP approach
is suboptimal and advocating that fine-grained INT formats, particularly
MXINT8, offer a better balance of accuracy, power, and efficiency for future AI
accelerators.
OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid
  Validation in Realistic Workflows
Authors: name: Qiushi Sun, name: Mukai Li, name: Zhoumianze Liu, name: Zhihui Xie, name: Fangzhi Xu, name: Zhangyue Yin, name: Kanzhi Cheng, name: Zehao Li, name: Zichen Ding, name: Qi Liu, name: Zhiyong Wu, name: Zhuosheng Zhang, name: Ben Kao, name: Lingpeng Kong
Published: 2025-10-28 13:22:39

Computer-using agents powered by Vision-Language Models (VLMs) have
demonstrated human-like capabilities in operating digital environments like
mobile platforms. While these agents hold great promise for advancing digital
automation, their potential for unsafe operations, such as system compromise
and privacy leakage, is raising significant concerns. Detecting these safety
concerns across the vast and complex operational space of mobile environments
presents a formidable challenge that remains critically underexplored. To
establish a foundation for mobile agent safety research, we introduce
MobileRisk-Live, a dynamic sandbox environment accompanied by a safety
detection benchmark comprising realistic trajectories with fine-grained
annotations. Built upon this, we propose OS-Sentinel, a novel hybrid safety
detection framework that synergistically combines a Formal Verifier for
detecting explicit system-level violations with a VLM-based Contextual Judge
for assessing contextual risks and agent actions. Experiments show that
OS-Sentinel achieves 10%-30% improvements over existing approaches across
multiple metrics. Further analysis provides critical insights that foster the
development of safer and more reliable autonomous mobile agents.
Higher-order Linear Attention
Authors: name: Yifan Zhang, name: Zhen Qin, name: Quanquan Gu
Published: 2025-10-31 07:54:37

The quadratic cost of scaled dot-product attention is a central obstacle to
scaling autoregressive language models to long contexts. Linear-time attention
and State Space Models (SSMs) provide scalable alternatives but are typically
restricted to first-order or kernel-based approximations, which can limit
expressivity. We introduce Higher-order Linear Attention (HLA), a causal,
streaming mechanism that realizes higher interactions via compact prefix
sufficient statistics. In the second-order case, HLA maintains a constant-size
state and computes per-token outputs in linear time without materializing any
n times n matrices. We give closed-form streaming identities, a strictly
causal masked variant using two additional summaries, and a chunk-parallel
training scheme based on associative scans that reproduces the activations of a
serial recurrence exactly. We further outline extensions to third and higher
orders. Collectively, these results position HLA as a principled, scalable
building block that combines attention-like, data-dependent mixing with the
efficiency of modern recurrent architectures. Project Page:
https://github.com/yifanzhang-pro/HLA.
Phased DMD: Few-step Distribution Matching Distillation via Score
  Matching within Subintervals
Authors: name: Xiangyu Fan, name: Zesong Qiu, name: Zhuguanyu Wu, name: Fanzhou Wang, name: Zhiqian Lin, name: Tianxiang Ren, name: Dahua Lin, name: Ruihao Gong, name: Lei Yang
Published: 2025-10-31 17:55:10

Distribution Matching Distillation (DMD) distills score-based generative
models into efficient one-step generators, without requiring a one-to-one
correspondence with the sampling trajectories of their teachers. However,
limited model capacity causes one-step distilled models underperform on complex
generative tasks, e.g., synthesizing intricate object motions in text-to-video
generation. Directly extending DMD to multi-step distillation increases memory
usage and computational depth, leading to instability and reduced efficiency.
While prior works propose stochastic gradient truncation as a potential
solution, we observe that it substantially reduces the generation diversity of
multi-step distilled models, bringing it down to the level of their one-step
counterparts. To address these limitations, we propose Phased DMD, a multi-step
distillation framework that bridges the idea of phase-wise distillation with
Mixture-of-Experts (MoE), reducing learning difficulty while enhancing model
capacity. Phased DMD is built upon two key ideas: progressive distribution
matching and score matching within subintervals. First, our model divides the
SNR range into subintervals, progressively refining the model to higher SNR
levels, to better capture complex distributions. Next, to ensure the training
objective within each subinterval is accurate, we have conducted rigorous
mathematical derivations. We validate Phased DMD by distilling state-of-the-art
image and video generation models, including Qwen-Image (20B parameters) and
Wan2.2 (28B parameters). Experimental results demonstrate that Phased DMD
preserves output diversity better than DMD while retaining key generative
capabilities. We will release our code and models.
HyperClick: Advancing Reliable GUI Grounding via Uncertainty Calibration
Authors: name: Shaojie Zhang, name: Pei Fu, name: Ruoceng Zhang, name: Jiahui Yang, name: Anan Du, name: Xiuwen Xi, name: Shaokang Wang, name: Ying Huang, name: Bin Qin, name: Zhenbo Luo, name: Jian Luan
Published: 2025-10-31 08:07:02

Autonomous Graphical User Interface (GUI) agents rely on accurate GUI
grounding, which maps language instructions to on-screen coordinates, to
execute user commands. However, current models, whether trained via supervised
fine-tuning (SFT) or reinforcement fine-tuning (RFT), lack self-awareness of
their capability boundaries, leading to overconfidence and unreliable
predictions. We first systematically evaluate probabilistic and verbalized
confidence in general and GUI-specific models, revealing a misalignment between
confidence and actual accuracy, which is particularly critical in dynamic GUI
automation tasks, where single errors can cause task failure. To address this,
we propose HyperClick, a novel framework that enhances reliable GUI grounding
through uncertainty calibration. HyperClick introduces a dual reward mechanism,
combining a binary reward for correct actions with a truncated Gaussian-based
spatial confidence modeling, calibrated using the Brier score. This approach
jointly optimizes grounding accuracy and confidence reliability, fostering
introspective self-criticism. Extensive experiments on seven challenge
benchmarks show that HyperClick achieves state-of-the-art performance while
providing well-calibrated confidence. By enabling explicit confidence
calibration and introspective self-criticism, HyperClick reduces overconfidence
and supports more reliable GUI automation.
ThinkMorph: Emergent Properties in Multimodal Interleaved
  Chain-of-Thought Reasoning
Authors: name: Jiawei Gu, name: Yunzhuo Hao, name: Huichen Will Wang, name: Linjie Li, name: Michael Qizhe Shieh, name: Yejin Choi, name: Ranjay Krishna, name: Yu Cheng
Published: 2025-10-30 17:51:38

Multimodal reasoning requires iterative coordination between language and
vision, yet it remains unclear what constitutes a meaningful interleaved chain
of thought. We posit that text and image thoughts should function as
complementary, rather than isomorphic, modalities that mutually advance
reasoning. Guided by this principle, we build ThinkMorph, a unified model
fine-tuned on 24K high-quality interleaved reasoning traces spanning tasks with
varying visual engagement. ThinkMorph learns to generate progressive text-image
reasoning steps that concretely manipulate visual content while maintaining
coherent verbal logic. It delivers large gains on vision-centric benchmarks
(averaging 34.7% over the base model) and generalizes to out-of-domain tasks,
matching or surpassing larger and proprietary VLMs. Beyond performance,
ThinkMorph exhibits emergent multimodal intelligence, including unseen visual
manipulation skills, adaptive switching between reasoning modes, and better
test-time scaling through diversified multimodal thoughts.These findings
suggest promising directions for characterizing the emergent capabilities of
unified models for multimodal reasoning.
Visual Backdoor Attacks on MLLM Embodied Decision Making via Contrastive
  Trigger Learning
Authors: name: Qiusi Zhan, name: Hyeonjeong Ha, name: Rui Yang, name: Sirui Xu, name: Hanyang Chen, name: Liang-Yan Gui, name: Yu-Xiong Wang, name: Huan Zhang, name: Heng Ji, name: Daniel Kang
Published: 2025-10-31 16:50:49

Multimodal large language models (MLLMs) have advanced embodied agents by
enabling direct perception, reasoning, and planning task-oriented actions from
visual inputs. However, such vision driven embodied agents open a new attack
surface: visual backdoor attacks, where the agent behaves normally until a
visual trigger appears in the scene, then persistently executes an
attacker-specified multi-step policy. We introduce BEAT, the first framework to
inject such visual backdoors into MLLM-based embodied agents using objects in
the environments as triggers. Unlike textual triggers, object triggers exhibit
wide variation across viewpoints and lighting, making them difficult to implant
reliably. BEAT addresses this challenge by (1) constructing a training set that
spans diverse scenes, tasks, and trigger placements to expose agents to trigger
variability, and (2) introducing a two-stage training scheme that first applies
supervised fine-tuning (SFT) and then our novel Contrastive Trigger Learning
(CTL). CTL formulates trigger discrimination as preference learning between
trigger-present and trigger-free inputs, explicitly sharpening the decision
boundaries to ensure precise backdoor activation. Across various embodied agent
benchmarks and MLLMs, BEAT achieves attack success rates up to 80%, while
maintaining strong benign task performance, and generalizes reliably to
out-of-distribution trigger placements. Notably, compared to naive SFT, CTL
boosts backdoor activation accuracy up to 39% under limited backdoor data.
These findings expose a critical yet unexplored security risk in MLLM-based
embodied agents, underscoring the need for robust defenses before real-world
deployment.