Orionnnn/hub-stats huggingface dataset:
SAVANT: Semantic Analysis with Vision-Augmented Anomaly deTection
Authors: name: Roberto Brusnicki, name: David Pop, name: Yuan Gao, name: Mattia Piccinini, name: Johannes Betz
Published: 2025-10-20 19:14:29

Autonomous driving systems remain critically vulnerable to the long-tail of
rare, out-of-distribution scenarios with semantic anomalies. While Vision
Language Models (VLMs) offer promising reasoning capabilities, naive prompting
approaches yield unreliable performance and depend on expensive proprietary
models, limiting practical deployment. We introduce SAVANT (Semantic Analysis
with Vision-Augmented Anomaly deTection), a structured reasoning framework that
achieves high accuracy and recall in detecting anomalous driving scenarios from
input images through layered scene analysis and a two-phase pipeline:
structured scene description extraction followed by multi-modal evaluation. Our
approach transforms VLM reasoning from ad-hoc prompting to systematic analysis
across four semantic layers: Street, Infrastructure, Movable Objects, and
Environment. SAVANT achieves 89.6% recall and 88.0% accuracy on real-world
driving scenarios, significantly outperforming unstructured baselines. More
importantly, we demonstrate that our structured framework enables a fine-tuned
7B parameter open-source model (Qwen2.5VL) to achieve 90.8% recall and 93.8%
accuracy - surpassing all models evaluated while enabling local deployment at
near-zero cost. By automatically labeling over 9,640 real-world images with
high accuracy, SAVANT addresses the critical data scarcity problem in anomaly
detection and provides a practical path toward reliable, accessible semantic
monitoring for autonomous systems.
Steering Autoregressive Music Generation with Recursive Feature Machines
Authors: name: Daniel Zhao, name: Daniel Beaglehole, name: Taylor Berg-Kirkpatrick, name: Julian McAuley, name: Zachary Novack
Published: 2025-10-21 23:23:14

Controllable music generation remains a significant challenge, with existing
methods often requiring model retraining or introducing audible artifacts. We
introduce MusicRFM, a framework that adapts Recursive Feature Machines (RFMs)
to enable fine-grained, interpretable control over frozen, pre-trained music
models by directly steering their internal activations. RFMs analyze a model's
internal gradients to produce interpretable "concept directions", or specific
axes in the activation space that correspond to musical attributes like notes
or chords. We first train lightweight RFM probes to discover these directions
within MusicGen's hidden states; then, during inference, we inject them back
into the model to guide the generation process in real-time without per-step
optimization. We present advanced mechanisms for this control, including
dynamic, time-varying schedules and methods for the simultaneous enforcement of
multiple musical properties. Our method successfully navigates the trade-off
between control and generation quality: we can increase the accuracy of
generating a target musical note from 0.23 to 0.82, while text prompt adherence
remains within approximately 0.02 of the unsteered baseline, demonstrating
effective control with minimal impact on prompt fidelity. We release code to
encourage further exploration on RFMs in the music domain.
What Questions Should Robots Be Able to Answer? A Dataset of User
  Questions for Explainable Robotics
Authors: name: Lennart Wachowiak, name: Andrew Coles, name: Gerard Canal, name: Oya Celiktutan
Published: 2025-10-18 10:16:45

With the growing use of large language models and conversational interfaces
in human-robot interaction, robots' ability to answer user questions is more
important than ever. We therefore introduce a dataset of 1,893 user questions
for household robots, collected from 100 participants and organized into 12
categories and 70 subcategories. Most work in explainable robotics focuses on
why-questions. In contrast, our dataset provides a wide variety of questions,
from questions about simple execution details to questions about how the robot
would act in hypothetical scenarios -- thus giving roboticists valuable
insights into what questions their robot needs to be able to answer. To collect
the dataset, we created 15 video stimuli and 7 text stimuli, depicting robots
performing varied household tasks. We then asked participants on Prolific what
questions they would want to ask the robot in each portrayed situation. In the
final dataset, the most frequent categories are questions about task execution
details (22.5%), the robot's capabilities (12.7%), and performance assessments
(11.3%). Although questions about how robots would handle potentially difficult
scenarios and ensure correct behavior are less frequent, users rank them as the
most important for robots to be able to answer. Moreover, we find that users
who identify as novices in robotics ask different questions than more
experienced users. Novices are more likely to inquire about simple facts, such
as what the robot did or the current state of the environment. As robots enter
environments shared with humans and language becomes central to giving
instructions and interaction, this dataset provides a valuable foundation for
(i) identifying the information robots need to log and expose to conversational
interfaces, (ii) benchmarking question-answering modules, and (iii) designing
explanation strategies that align with user expectations.
Decomposed Attention Fusion in MLLMs for Training-Free Video Reasoning
  Segmentation
Authors: name: Su Ho Han, name: Jeongseok Hyun, name: Pilhyeon Lee, name: Minho Shim, name: Dongyoon Wee, name: Seon Joo Kim
Published: 2025-10-22 13:42:59

Multimodal large language models (MLLMs) demonstrate strong video
understanding by attending to visual tokens relevant to textual queries. To
directly adapt this for localization in a training-free manner, we cast video
reasoning segmentation as a video QA task and extract attention maps via
rollout mechanism. However, raw attention maps are noisy and poorly aligned
with object regions. We propose Decomposed Attention Fusion (DecAF), which
refines these maps through two mechanisms: (1) contrastive object-background
fusion and (2) complementary video-frame fusion. This method suppresses
irrelevant activations and enhances object-focused cues, enabling direct
conversion of attention maps into coarse segmentation masks. In addition, we
introduce attention-guided SAM2 prompting for obtaining fine-grained masks.
Unlike existing methods that jointly train MLLMs with SAM, our method operates
entirely without retraining. DecAF outperforms training-free methods and
achieves performance comparable to training-based methods on both referring and
reasoning VOS benchmarks. The code will be available at
https://github.com/HYUNJS/DecAF.
Language Models are Injective and Hence Invertible
Authors: name: Giorgos Nikolaou, name: Tommaso Mencattini, name: Donato Crisostomi, name: Andrea Santilli, name: Yannis Panagakis, name: Emanuele Rodola'
Published: 2025-10-17 10:25:30

Transformer components such as non-linear activations and normalization are
inherently non-injective, suggesting that different inputs could map to the
same output and prevent exact recovery of the input from a model's
representations. In this paper, we challenge this view. First, we prove
mathematically that transformer language models mapping discrete input
sequences to their corresponding sequence of continuous representations are
injective and therefore lossless, a property established at initialization and
preserved during training. Second, we confirm this result empirically through
billions of collision tests on six state-of-the-art language models, and
observe no collisions. Third, we operationalize injectivity: we introduce
SipIt, the first algorithm that provably and efficiently reconstructs the exact
input text from hidden activations, establishing linear-time guarantees and
demonstrating exact invertibility in practice. Overall, our work establishes
injectivity as a fundamental and exploitable property of language models, with
direct implications for transparency, interpretability, and safe deployment.
DeLeaker: Dynamic Inference-Time Reweighting For Semantic Leakage
  Mitigation in Text-to-Image Models
Authors: name: Mor Ventura, name: Michael Toker, name: Or Patashnik, name: Yonatan Belinkov, name: Roi Reichart
Published: 2025-10-16 17:39:21

Text-to-Image (T2I) models have advanced rapidly, yet they remain vulnerable
to semantic leakage, the unintended transfer of semantically related features
between distinct entities. Existing mitigation strategies are often
optimization-based or dependent on external inputs. We introduce DeLeaker, a
lightweight, optimization-free inference-time approach that mitigates leakage
by directly intervening on the model's attention maps. Throughout the diffusion
process, DeLeaker dynamically reweights attention maps to suppress excessive
cross-entity interactions while strengthening the identity of each entity. To
support systematic evaluation, we introduce SLIM (Semantic Leakage in IMages),
the first dataset dedicated to semantic leakage, comprising 1,130
human-verified samples spanning diverse scenarios, together with a novel
automatic evaluation framework. Experiments demonstrate that DeLeaker
consistently outperforms all baselines, even when they are provided with
external information, achieving effective leakage mitigation without
compromising fidelity or quality. These results underscore the value of
attention control and pave the way for more semantically precise T2I models.
See the Text: From Tokenization to Visual Reading
Authors: name: Ling Xing, name: Alex Jinpeng Wang, name: Rui Yan, name: Hongyu Qu, name: Zechao Li, name: Jinhui Tang
Published: 2025-10-21 17:34:48

People see text. Humans read by recognizing words as visual objects,
including their shapes, layouts, and patterns, before connecting them to
meaning, which enables us to handle typos, distorted fonts, and various scripts
effectively. Modern large language models (LLMs), however, rely on subword
tokenization, fragmenting text into pieces from a fixed vocabulary. While
effective for high-resource languages, this approach over-segments low-resource
languages, yielding long, linguistically meaningless sequences and inflating
computation. In this work, we challenge this entrenched paradigm and move
toward a vision-centric alternative. Our method, SeeTok, renders text as images
(visual-text) and leverages pretrained multimodal LLMs to interpret them,
reusing strong OCR and text-vision alignment abilities learned from large-scale
multimodal training. Across three different language tasks, SeeTok matches or
surpasses subword tokenizers while requiring 4.43 times fewer tokens and
reducing FLOPs by 70.5%, with additional gains in cross-lingual generalization,
robustness to typographic noise, and linguistic hierarchy. SeeTok signals a
shift from symbolic tokenization to human-like visual reading, and takes a step
toward more natural and cognitively inspired language models.
Directional Reasoning Injection for Fine-Tuning MLLMs
Authors: name: Chao Huang, name: Zeliang Zhang, name: Jiang Liu, name: Ximeng Sun, name: Jialian Wu, name: Xiaodong Yu, name: Ze Wang, name: Chenliang Xu, name: Emad Barsoum, name: Zicheng Liu
Published: 2025-10-16 18:06:46

Multimodal large language models (MLLMs) are rapidly advancing, yet their
reasoning ability often lags behind that of strong text-only counterparts.
Existing methods to bridge this gap rely on supervised fine-tuning over
large-scale multimodal reasoning data or reinforcement learning, both of which
are resource-intensive. A promising alternative is model merging, which
interpolates parameters between reasoning-enhanced LLMs and multimodal
variants. However, our analysis shows that naive merging is not always a "free
lunch": its effectiveness varies drastically across model families, with some
(e.g., LLaVA, Idefics) benefiting while others (e.g., Qwen) suffer performance
degradation. To address this, we propose Directional Reasoning Injection for
Fine-Tuning (DRIFT) MLLMs, a lightweight method that transfers reasoning
knowledge in the gradient space, without destabilizing multimodal alignment.
DRIFT precomputes a reasoning prior as the parameter-space difference between
reasoning and multimodal variants, then uses it to bias gradients during
multimodal fine-tuning. This approach preserves the simplicity of standard
supervised fine-tuning pipelines while enabling efficient reasoning transfer.
Extensive experiments on multimodal reasoning benchmarks, including MathVista
and MathVerse, demonstrate that DRIFT consistently improves reasoning
performance over naive merging and supervised fine-tuning, while matching or
surpassing training-heavy methods at a fraction of the cost.
Attention Sinks in Diffusion Language Models
Authors: name: Maximo Eduardo Rulli, name: Simone Petruzzi, name: Edoardo Michielon, name: Fabrizio Silvestri, name: Simone Scardapane, name: Alessio Devoto
Published: 2025-10-17 15:23:58

Masked Diffusion Language Models (DLMs) have recently emerged as a promising
alternative to traditional Autoregressive Models (ARMs). DLMs employ
transformer encoders with bidirectional attention, enabling parallel token
generation while maintaining competitive performance. Although their efficiency
and effectiveness have been extensively studied, the internal mechanisms that
govern DLMs remain largely unexplored. In this work, we conduct an empirical
analysis of DLM attention patterns, focusing on the attention sinking
phenomenon, an effect previously observed in various transformer-based
architectures. Our findings reveal that DLMs also exhibit attention sinks, but
with distinct characteristics. First, unlike in ARMs, the sink positions in
DLMs tend to shift throughout the generation process, displaying a dynamic
behaviour. Second, while ARMs are highly sensitive to the removal of attention
sinks, DLMs remain robust: masking sinks leads to only a minor degradation in
performance. These results provide new insights into the inner workings of
diffusion-based language models and highlight fundamental differences in how
they allocate and utilize attention compared to autoregressive models.
From Charts to Code: A Hierarchical Benchmark for Multimodal Models
Authors: name: Jiahao Tang, name: Henry Hengyuan Zhao, name: Lijian Wu, name: Yifei Tao, name: Dongxing Mao, name: Yang Wan, name: Jingru Tan, name: Min Zeng, name: Min Li, name: Alex Jinpeng Wang
Published: 2025-10-20 15:11:56

We introduce Chart2Code, a new benchmark for evaluating the chart
understanding and code generation capabilities of large multimodal models
(LMMs). Chart2Code is explicitly designed from a user-driven perspective,
capturing diverse real-world scenarios and progressively increasing task
difficulty. It consists of three levels: Level 1 (Chart Reproduction)
reproduces charts from a reference figure and user query; Level 2 (Chart
Editing) involves complex modifications such as changing chart types or adding
elements; and Level 3 (Long-Table to Chart Generation) requires models to
transform long, information-dense tables into faithful charts following user
instructions. To our knowledge, this is the first hierarchical benchmark that
reflects practical chart2code usage while systematically scaling task
complexity. In total, Chart2Code contains 2,023 tasks across 22 chart types,
paired with multi-level evaluation metrics that assess both code correctness
and the visual fidelity of rendered charts. We benchmark 25 state-of-the-art
(SoTA) LMMs, including both proprietary and the latest open-source models such
as GPT-5, Qwen2.5-VL, InternVL3/3.5, MiMo-VL, and Seed-1.6-VL. Experimental
results demonstrate that even the SoTA model GPT-5 averages only 0.57 on
code-based evaluation and 0.22 on chart-quality assessment across the editing
tasks, underscoring the difficulty of Chart2Code. We anticipate this benchmark
will drive advances in multimodal reasoning and foster the development of more
robust and general-purpose LMMs. Our code and data are available on Chart2Code.
When Do Transformers Learn Heuristics for Graph Connectivity?
Authors: name: Qilin Ye, name: Deqing Fu, name: Robin Jia, name: Vatsal Sharan
Published: 2025-10-22 16:43:32

Transformers often fail to learn generalizable algorithms, instead relying on
brittle heuristics. Using graph connectivity as a testbed, we explain this
phenomenon both theoretically and empirically. We consider a simplified
Transformer architecture, the disentangled Transformer, and prove that an
L-layer model has capacity to solve for graphs with diameters up to exactly
3^L, implementing an algorithm equivalent to computing powers of the
adjacency matrix. We analyze the training-dynamics, and show that the learned
strategy hinges on whether most training instances are within this model
capacity. Within-capacity graphs (diameter leq 3^L) drive the learning of a
correct algorithmic solution while beyond-capacity graphs drive the learning of
a simple heuristic based on node degrees. Finally, we empirically demonstrate
that restricting training data within a model's capacity leads to both standard
and disentangled transformers learning the exact algorithm rather than the
degree-based heuristic.
GigaBrain-0: A World Model-Powered Vision-Language-Action Model
Authors: name: GigaBrain Team, name: Angen Ye, name: Boyuan Wang, name: Chaojun Ni, name: Guan Huang, name: Guosheng Zhao, name: Haoyun Li, name: Jie Li, name: Jiagang Zhu, name: Lv Feng, name: Peng Li, name: Qiuping Deng, name: Runqi Ouyang, name: Wenkang Qin, name: Xinze Chen, name: Xiaofeng Wang, name: Yang Wang, name: Yifan Li, name: Yilong Li, name: Yiran Ding, name: Yuan Xu, name: Yun Ye, name: Yukun Zhou, name: Zhehao Dong, name: Zhenan Wang, name: Zhichao Liu, name: Zheng Zhu
Published: 2025-10-22 09:57:13

Training Vision-Language-Action (VLA) models for generalist robots typically
requires large-scale real-world robot data, which is expensive and
time-consuming to collect. The inefficiency of physical data collection
severely limits the scalability, and generalization capacity of current VLA
systems. To address this challenge, we introduce GigaBrain-0, a novel VLA
foundation model empowered by world model-generated data (e.g., video
generation, real2real transfer, human transfer, view transfer, sim2real
transfer data). By leveraging world models to generate diverse data at scale,
GigaBrain-0 significantly reduces reliance on real robot data while improving
cross-task generalization. Our approach further improves policy robustness
through RGBD input modeling and embodied Chain-of-Thought (CoT) supervision,
enabling the model to reason about spatial geometry, object states, and
long-horizon dependencies during task execution. This leads to substantial
gains in real-world performance on dexterous, long-horizon, and mobile
manipulation tasks. Extensive experiments demonstrate that GigaBrain-0 achieves
superior generalization across variations in appearances (e.g., textures,
colors), object placements, and camera viewpoints. Additionally, we present
GigaBrain-0-Small, an optimized lightweight variant designed to run efficiently
on devices such as the NVIDIA Jetson AGX Orin.
Learning from the Best, Differently: A Diversity-Driven Rethinking on
  Data Selection
Authors: name: Hongyi He, name: Xiao Liu, name: Zhenghao Lin, name: Mingni Tang, name: Yi Cheng, name: Jintao Wang, name: Wenjie Li, name: Peng Cheng, name: Yeyun Gong
Published: 2025-10-21 03:37:31

High-quality pre-training data is crutial for large language models, where
quality captures factual reliability and semantic value, and diversity ensures
broad coverage and distributional heterogeneity. Existing approaches typically
rely on single or multiple-dimensional score-based selection. However, directly
selecting top-scored data often degrades performance, and sampling from a
broader range is required to recover results. The above non-monotonicity
between dataset scores and downstream benchmark results reveals a fundamental
bias: score-based methods collapse correlated dimensions, causing top-scored
data to appear high-quality while systematically overlooking diversity. We
argue that ensuring diversity requires decomposing correlated metrics into
orthogonal feature dimensions, from which the top-scored data can be directly
selected. Therefore, we proposed the Orthogonal Diversity-Aware Selection
(ODiS) algorithm, which preserves both quality and diversity during data
selection. First, ODiS evaluates data from multiple dimensions, covering
language quality, knowledge quality, and comprehension difficulty. The
multi-dimensional scores are then decorrelated via Principal Component Analysis
(PCA), yielding orthogonal evaluation dimensions. For each dimension, a
Roberta-based scorer is trained to regress the data onto PCA-projected scores,
enabling scalable inference on large corpora. Finally, ODiS constructs the
training dataset by selecting top-scored data within each orthogonal dimension,
thereby ensuring both quality and diversity. Empirical results show that
ODiS-selected data exhibit less than 2\% inter-dimension overlap, confirming
orthogonality between dimensions. More importantly, models trained with
ODiS-selected data significantly outperform other baselines on downstream
benchmarks, highlighting the necessity of orthogonal, diversity-aware data
selection for LLMs.
Are they lovers or friends? Evaluating LLMs' Social Reasoning in English
  and Korean Dialogues
Authors: name: Eunsu Kim, name: Junyeong Park, name: Juhyun Oh, name: Kiwoong Park, name: Seyoung Song, name: A. Seza Dogruoz, name: Najoung Kim, name: Alice Oh
Published: 2025-10-21 19:12:47

As large language models (LLMs) are increasingly used in human-AI
interactions, their social reasoning capabilities in interpersonal contexts are
critical. We introduce SCRIPTS, a 1k-dialogue dataset in English and Korean,
sourced from movie scripts. The task involves evaluating models' social
reasoning capability to infer the interpersonal relationships (e.g., friends,
sisters, lovers) between speakers in each dialogue. Each dialogue is annotated
with probabilistic relational labels (Highly Likely, Less Likely, Unlikely) by
native (or equivalent) Korean and English speakers from Korea and the U.S.
Evaluating nine models on our task, current proprietary LLMs achieve around
75-80% on the English dataset, whereas their performance on Korean drops to
58-69%. More strikingly, models select Unlikely relationships in 10-25% of
their responses. Furthermore, we find that thinking models and chain-of-thought
prompting, effective for general reasoning, provide minimal benefits for social
reasoning and occasionally amplify social biases. Our findings reveal
significant limitations in current LLMs' social reasoning capabilities,
highlighting the need for efforts to develop socially-aware language models.