Orionnnn/hub-stats huggingface dataset:
How to Teach Large Multimodal Models New Skills
Authors: name: Zhen Zhu, name: Yiming Gong, name: Yao Xiao, name: Yaoyao Liu, name: Derek Hoiem
Published: 2025-10-09 17:59:37

How can we teach large multimodal models (LMMs) new skills without erasing
prior abilities? We study sequential fine-tuning on five target skills while
monitoring general ability on eight held-out benchmarks across three model
families. We observe that apparent "forgetting" on held-out tasks after narrow
fine-tuning can partly recover at later stages. We trace this behavior to a
measurable shift in the output token distribution, manifested through a simple
counting-bias probe that co-varies with forgetting. Guided by this picture, we
identify two simple, robust tuning recipes that learn strongly while limiting
drift: (i) updating only the self-attention projection layers, and (ii)
updating only the MLP Gate&Up while freezing the Down projection. Across models
and tasks, these choices deliver strong target gains while largely preserving
held-out performance. Code is available at
https://github.com/jessemelpolio/LMM_CL
ARMOR: High-Performance Semi-Structured Pruning via Adaptive Matrix
  Factorization
Authors: name: Lawrence Liu, name: Alexander Liu, name: Mengdi Wang, name: Tuo Zhao, name: Lin F. Yang
Published: 2025-10-07 02:39:20

Large language models (LLMs) present significant deployment challenges due to
their immense computational and memory requirements. While semi-structured
pruning, particularly 2:4 sparsity, offers a path to practical hardware
acceleration, existing methods often incur substantial performance degradation.
To bridge this gap, we introduce ARMOR: (Adaptive Representation with
Matrix-factORization), a novel one-shot post-training pruning algorithm.
Instead of directly pruning weights, ARMOR factorizes each weight matrix into a
2:4 sparse core wrapped by two low-overhead, block diagonal matrices. These
wrappers act as efficient pre and post-transformation error correctors,
offering greater flexibility to preserve model quality compared to conventional
2:4 pruning techniques. The sparse core and block diagonal wrappers are chosen
through a block coordinate descent algorithm that minimizes a layer-wise proxy
loss. We theoretically prove this optimization is guaranteed to converge to a
solution with a proxy loss less than or equal to state-of-the-art pruning
algorithms. Experiments on Llama (Touvron et al., 2023; Dubey et al., 2024) and
Qwen (Yang et al., 2025) model families demonstrate that ARMOR consistently and
significantly outperforms state-of-the-art 2:4 pruning methods across a wide
range of downstream tasks and perplexity evaluations. ARMOR achieves this
superior performance while retaining the inference speedups and substantial
memory usage reductions of 2:4 pruning, establishing a more effective trade-off
between model compression and task accuracy
BEAR: Benchmarking and Enhancing Multimodal Language Models for Atomic
  Embodied Capabilities
Authors: name: Yu Qi, name: Haibo Zhao, name: Ziyu Guo, name: Siyuan Ma, name: Ziyan Chen, name: Yaokun Han, name: Renrui Zhang, name: Zitiantao Lin, name: Shiji Xin, name: Yijian Huang, name: Kai Cheng, name: Peiheng Wang, name: Jiazheng Liu, name: Jiayi Zhang, name: Yizhe Zhu, name: Wenqing Wang, name: Yiran Qin, name: Xupeng Zhu, name: Haojie Huang, name: Lawson L. S. Wong
Published: 2025-10-09 19:18:36

Embodied capabilities refer to a suite of fundamental abilities for an agent
to perceive, comprehend, and interact with the physical world. While multimodal
large language models (MLLMs) show promise as embodied agents, a thorough and
systematic evaluation of their embodied capabilities remains underexplored, as
existing benchmarks primarily focus on specific domains such as planning or
spatial understanding. To bridge this gap, we introduce BEAR, a comprehensive
and fine-grained benchmark that evaluates MLLMs on atomic embodied
capabilities. BEAR comprises 4,469 interleaved image-video-text entries across
14 domains in 6 categories, including tasks from low-level pointing, trajectory
understanding, spatial reasoning, to high-level planning. Extensive evaluation
results of 20 representative MLLMs reveal their persistent limitations across
all domains of embodied capabilities. To tackle the shortfall, we propose
BEAR-Agent, a multimodal conversable agent that integrates pretrained vision
models to strengthen MLLM perception, 3D understanding, and planning
capabilities. It substantially enhances MLLM performance across diverse
embodied capabilities on BEAR, yielding a 9.12% absolute gain and a relative
improvement of 17.5% on GPT-5. Furthermore, our experiments indicate that
improving MLLM embodied capabilities can benefit embodied tasks in simulated
environments. Project website: https://bear-official66.github.io/
MONKEY: Masking ON KEY-Value Activation Adapter for Personalization
Authors: name: James Baker
Published: 2025-10-09 01:20:06

Personalizing diffusion models allows users to generate new images that
incorporate a given subject, allowing more control than a text prompt. These
models often suffer somewhat when they end up just recreating the subject
image, and ignoring the text prompt. We observe that one popular method for
personalization, the IP-Adapter automatically generates masks that we
definitively segment the subject from the background during inference. We
propose to use this automatically generated mask on a second pass to mask the
image tokens, thus restricting them to the subject, not the background,
allowing the text prompt to attend to the rest of the image. For text prompts
describing locations and places, this produces images that accurately depict
the subject while definitively matching the prompt. We compare our method to a
few other test time personalization methods, and find our method displays high
prompt and source image alignment.
LightReasoner: Can Small Language Models Teach Large Language Models
  Reasoning?
Authors: name: Jingyuan Wang, name: Yankai Chen, name: Zhonghang Li, name: Chao Huang
Published: 2025-10-09 08:55:12

Large language models (LLMs) have demonstrated remarkable progress in
reasoning, often through supervised fine-tuning (SFT). However, SFT is
resource-intensive, relying on large curated datasets, rejection-sampled
demonstrations, and uniform optimization across all tokens, even though only a
fraction carry meaningful learning value. In this work, we explore a
counterintuitive idea: can smaller language models (SLMs) teach larger language
models (LLMs) by revealing high-value reasoning moments that reflect the
latter's unique strength? We propose LightReasoner, a novel framework that
leverages the behavioral divergence between a stronger expert model (LLM) and a
weaker amateur model (SLM). LightReasoner operates in two stages: (1) a
sampling stage that pinpoints critical reasoning moments and constructs
supervision examples capturing the expert's advantage through expert-amateur
contrast, and (2) a fine-tuning stage that aligns the expert model with these
distilled examples, amplifying its reasoning strengths. Across seven
mathematical benchmarks, LightReasoner improves accuracy by up to 28.1%, while
reducing time consumption by 90%, sampled problems by 80%, and tuned token
usage by 99%, all without relying on ground-truth labels. By turning weaker
SLMs into effective teaching signals, LightReasoner offers a scalable and
resource-efficient approach for advancing LLM reasoning. Code is available at:
https://github.com/HKUDS/LightReasoner
LLM4Cell: A Survey of Large Language and Agentic Models for Single-Cell
  Biology
Authors: name: Sajib Acharjee Dip, name: Adrika Zafor, name: Bikash Kumar Paul, name: Uddip Acharjee Shuvo, name: Muhit Islam Emon, name: Xuan Wang, name: Liqing Zhang
Published: 2025-10-09 05:12:09

Large language models (LLMs) and emerging agentic frameworks are beginning to
transform single-cell biology by enabling natural-language reasoning,
generative annotation, and multimodal data integration. However, progress
remains fragmented across data modalities, architectures, and evaluation
standards. LLM4Cell presents the first unified survey of 58 foundation and
agentic models developed for single-cell research, spanning RNA, ATAC,
multi-omic, and spatial modalities. We categorize these methods into five
families-foundation, text-bridge, spatial, multimodal, epigenomic, and
agentic-and map them to eight key analytical tasks including annotation,
trajectory and perturbation modeling, and drug-response prediction. Drawing on
over 40 public datasets, we analyze benchmark suitability, data diversity, and
ethical or scalability constraints, and evaluate models across 10 domain
dimensions covering biological grounding, multi-omics alignment, fairness,
privacy, and explainability. By linking datasets, models, and evaluation
domains, LLM4Cell provides the first integrated view of language-driven
single-cell intelligence and outlines open challenges in interpretability,
standardization, and trustworthy model development.
ELMUR: External Layer Memory with Update/Rewrite for Long-Horizon RL
Authors: name: Egor Cherepanov, name: Alexey K. Kovalev, name: Aleksandr I. Panov
Published: 2025-10-08 15:50:34

Real-world robotic agents must act under partial observability and long
horizons, where key cues may appear long before they affect decision making.
However, most modern approaches rely solely on instantaneous information,
without incorporating insights from the past. Standard recurrent or transformer
models struggle with retaining and leveraging long-term dependencies: context
windows truncate history, while naive memory extensions fail under scale and
sparsity. We propose ELMUR (External Layer Memory with Update/Rewrite), a
transformer architecture with structured external memory. Each layer maintains
memory embeddings, interacts with them via bidirectional cross-attention, and
updates them through an Least Recently Used (LRU) memory module using
replacement or convex blending. ELMUR extends effective horizons up to 100,000
times beyond the attention window and achieves a 100% success rate on a
synthetic T-Maze task with corridors up to one million steps. In POPGym, it
outperforms baselines on more than half of the tasks. On MIKASA-Robo
sparse-reward manipulation tasks with visual observations, it nearly doubles
the performance of strong baselines. These results demonstrate that structured,
layer-local external memory offers a simple and scalable approach to decision
making under partial observability.
Formalizing Style in Personal Narratives
Authors: name: Gustave Cortal, name: Alain Finkel
Published: 2025-10-09 06:48:06

Personal narratives are stories authors construct to make meaning of their
experiences. Style, the distinctive way authors use language to express
themselves, is fundamental to how these narratives convey subjective
experiences. Yet there is a lack of a formal framework for systematically
analyzing these stylistic choices. We present a novel approach that formalizes
style in personal narratives as patterns in the linguistic choices authors make
when communicating subjective experiences. Our framework integrates three
domains: functional linguistics establishes language as a system of meaningful
choices, computer science provides methods for automatically extracting and
analyzing sequential patterns, and these patterns are linked to psychological
observations. Using language models, we automatically extract linguistic
features such as processes, participants, and circumstances. We apply our
framework to hundreds of dream narratives, including a case study on a war
veteran with post-traumatic stress disorder. Analysis of his narratives
uncovers distinctive patterns, particularly how verbal processes dominate over
mental ones, illustrating the relationship between linguistic choices and
psychological states.
Hybrid-grained Feature Aggregation with Coarse-to-fine Language Guidance
  for Self-supervised Monocular Depth Estimation
Authors: name: Wenyao Zhang, name: Hongsi Liu, name: Bohan Li, name: Jiawei He, name: Zekun Qi, name: Yunnan Wang, name: Shengyang Zhao, name: Xinqiang Yu, name: Wenjun Zeng, name: Xin Jin
Published: 2025-10-10 12:20:19

Current self-supervised monocular depth estimation (MDE) approaches encounter
performance limitations due to insufficient semantic-spatial knowledge
extraction. To address this challenge, we propose Hybrid-depth, a novel
framework that systematically integrates foundation models (e.g., CLIP and
DINO) to extract visual priors and acquire sufficient contextual information
for MDE. Our approach introduces a coarse-to-fine progressive learning
framework: 1) Firstly, we aggregate multi-grained features from CLIP (global
semantics) and DINO (local spatial details) under contrastive language
guidance. A proxy task comparing close-distant image patches is designed to
enforce depth-aware feature alignment using text prompts; 2) Next, building on
the coarse features, we integrate camera pose information and pixel-wise
language alignment to refine depth predictions. This module seamlessly
integrates with existing self-supervised MDE pipelines (e.g., Monodepth2,
ManyDepth) as a plug-and-play depth encoder, enhancing continuous depth
estimation. By aggregating CLIP's semantic context and DINO's spatial details
through language guidance, our method effectively addresses feature granularity
mismatches. Extensive experiments on the KITTI benchmark demonstrate that our
method significantly outperforms SOTA methods across all metrics, which also
indeed benefits downstream tasks like BEV perception. Code is available at
https://github.com/Zhangwenyao1/Hybrid-depth.
Which Heads Matter for Reasoning? RL-Guided KV Cache Compression
Authors: name: Wenjie Du, name: Li Jiang, name: Keda Tao, name: Xue Liu, name: Huan Wang
Published: 2025-10-09 17:50:00

Reasoning large language models exhibit complex reasoning behaviors through
the extended chain-of-thought generation, creating unprecedented Key-Value (KV)
cache overhead during the decoding phase. Existing KV cache compression methods
underperform on reasoning models: token-dropping methods break reasoning
integrity by discarding critical information, while head-reallocating methods
mistakenly compress reasoning-critical heads since they are designed for
retrieval tasks, resulting in significant performance degradation as
compression rates increase. We hypothesize that KV heads exhibit functional
heterogeneity in reasoning models-some heads are critical for chain-of-thought
consistency while others are compressible. To validate and exploit this
insight, we propose RLKV, a novel reasoning-critical head identification
framework, which uses reinforcement learning to directly optimize the
relationship between each head's cache usage and reasoning quality. As RLKV
produces rewards from actual generated samples during training, it naturally
identifies heads relevant to reasoning behaviors. We then allocate full KV
cache to these heads while applying compressed constant KV cache to others for
efficient inference. Our experiments reveal that only a small fraction of
attention heads is essential for reasoning, enabling our KV compression
approach to outperform baseline methods while achieving 20-50% cache reduction
with near lossless performance compared to uncompressed results.
KORMo: Korean Open Reasoning Model for Everyone
Authors: name: Minjun Kim, name: Hyeonseok Lim, name: Hangyeol Yoo, name: Inho Won, name: Seungwoo Song, name: Minkyung Cho, name: Junhun Yuk, name: Changsu Choi, name: Dongjae Shin, name: Huige Lee, name: Hoyun Song, name: Alice Oh, name: Kyungtae Lim
Published: 2025-10-10 14:31:25

This work presents the first large-scale investigation into constructing a
fully open bilingual large language model (LLM) for a non-English language,
specifically Korean, trained predominantly on synthetic data. We introduce
KORMo-10B, a 10.8B-parameter model trained from scratch on a Korean-English
corpus in which 68.74% of the Korean portion is synthetic. Through systematic
experimentation, we demonstrate that synthetic data, when carefully curated
with balanced linguistic coverage and diverse instruction styles, does not
cause instability or degradation during large-scale pretraining. Furthermore,
the model achieves performance comparable to that of contemporary open-weight
multilingual baselines across a wide range of reasoning, knowledge, and
instruction-following benchmarks. Our experiments reveal two key findings: (1)
synthetic data can reliably sustain long-horizon pretraining without model
collapse, and (2) bilingual instruction tuning enables near-native reasoning
and discourse coherence in Korean. By fully releasing all components including
data, code, training recipes, and logs, this work establishes a transparent
framework for developing synthetic data-driven fully open models (FOMs) in
low-resource settings and sets a reproducible precedent for future multilingual
LLM research.
Bridging Reasoning to Learning: Unmasking Illusions using Complexity Out
  of Distribution Generalization
Authors: name: Mohammad Mahdi Samiei Paqaleh, name: Arash Marioriyad, name: Arman Tahmasebi-Zadeh, name: Mohamadreza Fereydooni, name: Mahdi Ghaznavai, name: Mahdieh Soleymani Baghshah
Published: 2025-10-06 13:08:31

Recent progress has pushed AI frontiers from pattern recognition tasks toward
problems that require step by step, System2 style reasoning, especially with
large language models. Yet, unlike learning, where generalization and out of
distribution (OoD) evaluation concepts are well formalized, there is no clear,
consistent definition or metric for reasoning ability. We propose Complexity
Out of Distribution (Complexity OoD) generalization as a framework and problem
setting to define and measure reasoning. A model exhibits Complexity OoD
generalization when it maintains performance on test instances whose minimal
required solution complexity, either representational (richer solution
structure) or computational (more reasoning steps/program length), exceeds that
of all training examples. We formalize complexity via solution description
Kolmogorov complexity and operational proxies (e.g., object/relation counts;
reasoning step counts), clarifying how Complexity OoD differs from length and
compositional OoD. This lens unifies learning and reasoning: many cases
solvable with System1 like processing at low complexity become System2 like
under complexity pressure, while System2 can be viewed as generalization over
solution structures. We translate this perspective into practice with
recommendations for operationalizing Complexity OoD across the stack:
incorporating complexity into benchmark and evaluation metric design,
rethinking supervision to target solution traces, seeking and designing
inductive biases for Complexity OoD generalization, addressing learning to
reason spillovers such as spurious shortcuts, semantic robustness, catastrophic
forgetting, and step wise calibration. Because Complexity OoD cannot be solved
by scaling data alone, progress toward robust reasoning will require
architectures and training regimes that explicitly model and allocate
computation with respect to complexity.
Adaptive Attacks on Trusted Monitors Subvert AI Control Protocols
Authors: name: Mikhail Terekhov, name: Alexander Panfilov, name: Daniil Dzenhaliou, name: Caglar Gulcehre, name: Maksym Andriushchenko, name: Ameya Prabhu, name: Jonas Geiping
Published: 2025-10-10 15:12:44

AI control protocols serve as a defense mechanism to stop untrusted LLM
agents from causing harm in autonomous settings. Prior work treats this as a
security problem, stress testing with exploits that use the deployment context
to subtly complete harmful side tasks, such as backdoor insertion. In practice,
most AI control protocols are fundamentally based on LLM monitors, which can
become a central point of failure. We study adaptive attacks by an untrusted
model that knows the protocol and the monitor model, which is plausible if the
untrusted model was trained with a later knowledge cutoff or can search for
this information autonomously. We instantiate a simple adaptive attack vector
by which the attacker embeds publicly known or zero-shot prompt injections in
the model outputs. Using this tactic, frontier models consistently evade
diverse monitors and complete malicious tasks on two main AI control
benchmarks. The attack works universally against current protocols that rely on
a monitor. Furthermore, the recent Defer-to-Resample protocol even backfires,
as its resampling amplifies the prompt injection and effectively reframes it as
a best-of-n attack. In general, adaptive attacks on monitor models represent
a major blind spot in current control protocols and should become a standard
component of evaluations for future AI control mechanisms.
D2E: Scaling Vision-Action Pretraining on Desktop Data for Transfer to
  Embodied AI
Authors: name: Suwhan Choi, name: Jaeyoon Jung, name: Haebin Seong, name: Minchan Kim, name: Minyeong Kim, name: Yongjun Cho, name: Yoonshik Kim, name: Yubeen Park, name: Youngjae Yu, name: Yunsung Lee
Published: 2025-10-07 08:40:33

Large language models leverage internet-scale text data, yet embodied AI
remains constrained by the prohibitive costs of physical trajectory collection.
Desktop environments -- particularly gaming -- offer a compelling alternative:
they provide rich sensorimotor interactions at scale while maintaining the
structured observation-action coupling essential for embodied learning. We
present D2E (Desktop to Embodied AI), a framework that demonstrates desktop
interactions can serve as an effective pretraining substrate for robotics
embodied AI tasks. Unlike prior work that remained domain-specific (e.g., VPT
for Minecraft) or kept data proprietary (e.g., SIMA), D2E establishes a
complete pipeline from scalable desktop data collection to verified transfer in
embodied domains. Our framework comprises three components: (1) the OWA Toolkit
that unifies diverse desktop interactions into a standardized format with 152x
compression, (2) the Generalist-IDM that achieves strong zero-shot
generalization across unseen games through timestamp-based event prediction,
enabling internet-scale pseudo-labeling, and (3) VAPT that transfers
desktop-pretrained representations to physical manipulation and navigation.
Using 1.3K+ hours of data (259 hours of human demonstrations, and 1K+ hours of
pseudo-labeled gameplay), we achieve a total of 96.6% success rate on LIBERO
manipulation and 83.3% on CANVAS navigation benchmarks. This validates that
sensorimotor primitives in digital interactions exhibit sufficient invariance
to transfer meaningfully to physical embodied tasks, establishing desktop
pretraining as a practical paradigm for robotics. We will make all our work
public, including the OWA toolkit, datasets of human-collected and
pseudo-labeled, and VAPT-trained models available at
https://worv-ai.github.io/d2e/