Orionnnn/hub-stats huggingface dataset:
DialectGen: Benchmarking and Improving Dialect Robustness in Multimodal
  Generation
Authors: name: Yu Zhou, name: Sohyun An, name: Haikang Deng, name: Da Yin, name: Clark Peng, name: Cho-Jui Hsieh, name: Kai-Wei Chang, name: Nanyun Peng
Published: 2025-10-16 17:56:55

Contact languages like English exhibit rich regional variations in the form
of dialects, which are often used by dialect speakers interacting with
generative models. However, can multimodal generative models effectively
produce content given dialectal textual input? In this work, we study this
question by constructing a new large-scale benchmark spanning six common
English dialects. We work with dialect speakers to collect and verify over 4200
unique prompts and evaluate on 17 image and video generative models. Our
automatic and human evaluation results show that current state-of-the-art
multimodal generative models exhibit 32.26% to 48.17% performance degradation
when a single dialect word is used in the prompt. Common mitigation methods
such as fine-tuning and prompt rewriting can only improve dialect performance
by small margins (< 7%), while potentially incurring significant performance
degradation in Standard American English (SAE). To this end, we design a
general encoder-based mitigation strategy for multimodal generative models. Our
method teaches the model to recognize new dialect features while preserving SAE
performance. Experiments on models such as Stable Diffusion 1.5 show that our
method is able to simultaneously raise performance on five dialects to be on
par with SAE (+34.4%), while incurring near zero cost to SAE performance.
The German Commons - 154 Billion Tokens of Openly Licensed Text for
  German Language Models
Authors: name: Lukas Gienapp, name: Christopher Schröder, name: Stefan Schweter, name: Christopher Akiki, name: Ferdinand Schlatt, name: Arden Zimmermann, name: Phillipe Genêt, name: Martin Potthast
Published: 2025-10-15 18:24:26

Large language model development relies on large-scale training corpora, yet
most contain data of unclear licensing status, limiting the development of
truly open models. This problem is exacerbated for non-English languages, where
openly licensed text remains critically scarce. We introduce the German
Commons, the largest collection of openly licensed German text to date. It
compiles data from 41 sources across seven domains, encompassing legal,
scientific, cultural, political, news, economic, and web text. Through
systematic sourcing from established data providers with verifiable licensing,
it yields 154.56 billion tokens of high-quality text for language model
training. Our processing pipeline implements comprehensive quality filtering,
deduplication, and text formatting fixes, ensuring consistent quality across
heterogeneous text sources. All domain subsets feature licenses of at least
CC-BY-SA 4.0 or equivalent, ensuring legal compliance for model training and
redistribution. The German Commons therefore addresses the critical gap in
openly licensed German pretraining data, and enables the development of truly
open German language models. We also release code for corpus construction and
data filtering tailored to German language text, rendering the German Commons
fully reproducible and extensible.
Beyond Correctness: Evaluating Subjective Writing Preferences Across
  Cultures
Authors: name: Shuangshuang Ying, name: Yunwen Li, name: Xingwei Qu, name: Xin Li, name: Sheng Jin, name: Minghao Liu, name: Zhoufutu Wen, name: Xeron Du, name: Tianyu Zheng, name: Yichi Zhang, name: Letian Ni, name: Yuyang Cheng, name: Qiguang Chen, name: Jingzhe Ding, name: Shengda Long, name: Wangchunshu Zhou, name: Jiazhan Feng, name: Wanjun Zhong, name: Libo Qin, name: Ge Zhang, name: Wenhao Huang, name: Wanxiang Che, name: Chenghua Lin
Published: 2025-10-16 12:23:13

Current preference learning methods achieve high accuracy on standard
benchmarks but exhibit significant performance degradation when objective
quality signals are removed. We introduce WritingPreferenceBench, a dataset of
1,800 human-annotated preference pairs (1,200 English, 600 Chinese) across 8
creative writing genres, where responses are matched for objective correctness,
factual accuracy, and length. On this benchmark, sequence-based reward
models--the standard architecture for RLHF--achieve only 52.7% mean accuracy,
while zero-shot language model judges perform at 53.9%. In contrast, generative
reward models that produce explicit reasoning chains achieve 81.8% accuracy. We
observe high within-model variance across genres: individual models range from
18.2% to 81.8% accuracy across different writing categories, with standard
deviations averaging 10.1%. This variance persists regardless of model scale,
with 27B parameter models showing no consistent improvement over 8B variants.
Our results suggest that current RLHF methods primarily learn to detect
objective errors rather than capture subjective quality preferences (e.g.,
creativity, stylistic flair, and emotional resonance), and that successful
preference modeling may require intermediate reasoning representations rather
than direct classification.
COIG-Writer: A High-Quality Dataset for Chinese Creative Writing with
  Thought Processes
Authors: name: Yunwen Li, name: Shuangshuang Ying, name: Xingwei Qu, name: Xin Li, name: Sheng Jin, name: Minghao Liu, name: Zhoufutu Wen, name: Tianyu Zheng, name: Xeron Du, name: Qiguang Chen, name: Jiajun Shi, name: Wangchunshu Zhou, name: Jiazhan Feng, name: Wanjun Zhong, name: Libo Qin, name: Stephen Huang, name: Wanxiang Che, name: Chenghua Lin, name: Eli Zhang
Published: 2025-10-16 15:01:19

Large language models exhibit systematic deficiencies in creative writing,
particularly in non-English contexts where training data is scarce and lacks
process-level supervision. We present COIG-Writer, a novel Chinese creative
writing dataset that captures both diverse outputs and their underlying thought
processes through systematic reverse-engineering of high-quality texts. Unlike
existing datasets that provide only input-output pairs, COIG-Writer comprises
1,665 meticulously curated triplets spanning 51 genres, each containing: (1) a
reverse-engineered prompt, (2) detailed creative reasoning documenting
decision-making processes, and (3) the final text. Through comprehensive
experiments, we identify a two-component model of creative writing: narrative
logic (provided by process supervision) and linguistic expression (maintained
by general-purpose data). Our findings reveal three critical insights: (1)
Process supervision is highly effective but requires stabilization with general
data. A ratio of at least one creative sample to twelve general samples is
needed to achieve optimal performance; below this threshold, the win rate
progressively degrades (from 62.75% down to 35.78%)., (2) creative capabilities
are culturally-bound with no cross-lingual transfer (89.26pp gap between
Chinese and English performance), and (3) lexical diversity inversely
correlates with creative quality (TTR paradox), suggesting high diversity
signals compensatory behavior for logical deficiencies. These findings
establish that creative excellence emerges from the interaction between logical
scaffolding and linguistic grounding, analogous to how mathematical reasoning
enhances but cannot replace linguistic competence in foundation models.
LiteStage: Latency-aware Layer Skipping for Multi-stage Reasoning
Authors: name: Beomseok Kang, name: Jiwon Song, name: Jae-Joon Kim
Published: 2025-10-16 01:37:39

Multi-stage reasoning has emerged as an effective strategy for enhancing the
reasoning capability of small language models by decomposing complex problems
into sequential sub-stages. However, this comes at the cost of increased
latency. We observe that existing adaptive acceleration techniques, such as
layer skipping, struggle to balance efficiency and accuracy in this setting due
to two key challenges: (1) stage-wise variation in skip sensitivity, and (2)
the generation of redundant output tokens. To address these, we propose
LiteStage, a latency-aware layer skipping framework for multi-stage reasoning.
LiteStage combines a stage-wise offline search that allocates optimal layer
budgets with an online confidence-based generation early exit to suppress
unnecessary decoding. Experiments on three benchmarks, e.g., OBQA, CSQA, and
StrategyQA, show that LiteStage achieves up to 1.70x speedup with less than
4.0% accuracy loss, outperforming prior training-free layer skipping methods.
Expertise need not monopolize: Action-Specialized Mixture of Experts for
  Vision-Language-Action Learning
Authors: name: Weijie Shen, name: Yitian Liu, name: Yuhao Wu, name: Zhixuan Liang, name: Sijia Gu, name: Dehui Wang, name: Tian Nian, name: Lei Xu, name: Yusen Qin, name: Jiangmiao Pang, name: Xinping Guan, name: Xiaokang Yang, name: Yao Mu
Published: 2025-10-16 04:52:57

Vision-Language-Action (VLA) models are experiencing rapid development and
demonstrating promising capabilities in robotic manipulation tasks. However,
scaling up VLA models presents several critical challenges: (1) Training new
VLA models from scratch demands substantial computational resources and
extensive datasets. Given the current scarcity of robot data, it becomes
particularly valuable to fully leverage well-pretrained VLA model weights
during the scaling process. (2) Real-time control requires carefully balancing
model capacity with computational efficiency. To address these challenges, We
propose AdaMoE, a Mixture-of-Experts (MoE) architecture that inherits
pretrained weights from dense VLA models, and scales up the action expert by
substituting the feedforward layers into sparsely activated MoE layers. AdaMoE
employs a decoupling technique that decouples expert selection from expert
weighting through an independent scale adapter working alongside the
traditional router. This enables experts to be selected based on task relevance
while contributing with independently controlled weights, allowing
collaborative expert utilization rather than winner-takes-all dynamics. Our
approach demonstrates that expertise need not monopolize. Instead, through
collaborative expert utilization, we can achieve superior performance while
maintaining computational efficiency. AdaMoE consistently outperforms the
baseline model across key benchmarks, delivering performance gains of 1.8% on
LIBERO and 9.3% on RoboTwin. Most importantly, a substantial 21.5% improvement
in real-world experiments validates its practical effectiveness for robotic
manipulation tasks.
Ponimator: Unfolding Interactive Pose for Versatile Human-human
  Interaction Animation
Authors: name: Shaowei Liu, name: Chuan Guo, name: Bing Zhou, name: Jian Wang
Published: 2025-10-16 17:59:56

Close-proximity human-human interactive poses convey rich contextual
information about interaction dynamics. Given such poses, humans can
intuitively infer the context and anticipate possible past and future dynamics,
drawing on strong priors of human behavior. Inspired by this observation, we
propose Ponimator, a simple framework anchored on proximal interactive poses
for versatile interaction animation. Our training data consists of
close-contact two-person poses and their surrounding temporal context from
motion-capture interaction datasets. Leveraging interactive pose priors,
Ponimator employs two conditional diffusion models: (1) a pose animator that
uses the temporal prior to generate dynamic motion sequences from interactive
poses, and (2) a pose generator that applies the spatial prior to synthesize
interactive poses from a single pose, text, or both when interactive poses are
unavailable. Collectively, Ponimator supports diverse tasks, including
image-based interaction animation, reaction animation, and text-to-interaction
synthesis, facilitating the transfer of interaction knowledge from high-quality
mocap data to open-world scenarios. Empirical experiments across diverse
datasets and applications demonstrate the universality of the pose prior and
the effectiveness and robustness of our framework.
RefusalBench: Generative Evaluation of Selective Refusal in Grounded
  Language Models
Authors: name: Aashiq Muhamed, name: Leonardo F. R. Ribeiro, name: Markus Dreyer, name: Virginia Smith, name: Mona T. Diab
Published: 2025-10-12 00:53:42

The ability of language models in RAG systems to selectively refuse to answer
based on flawed context is critical for safety, yet remains a significant
failure point. Our large-scale study reveals that even frontier models struggle
in this setting, with refusal accuracy dropping below 50% on multi-document
tasks, while exhibiting either dangerous overconfidence or overcaution. Static
benchmarks fail to reliably evaluate this capability, as models exploit
dataset-specific artifacts and memorize test instances. We introduce
RefusalBench, a generative methodology that programmatically creates diagnostic
test cases through controlled linguistic perturbation. Our framework employs
176 distinct perturbation strategies across six categories of informational
uncertainty and three intensity levels. Evaluation of over 30 models uncovers
systematic failure patterns: refusal comprises separable detection and
categorization skills, and neither scale nor extended reasoning improves
performance. We find that selective refusal is a trainable, alignment-sensitive
capability, offering a clear path for improvement. We release two benchmarks --
RefusalBench-NQ (single document) and RefusalBench-GaRAGe (multi-document) --
and our complete generation framework to enable continued, dynamic evaluation
of this critical capability.
Fantastic (small) Retrievers and How to Train Them:
  mxbai-edge-colbert-v0 Tech Report
Authors: name: Rikiya Takehi, name: Benjamin Clavié, name: Sean Lee, name: Aamir Shakir
Published: 2025-10-16 17:00:35

In this work, we introduce mxbai-edge-colbert-v0 models, at two different
parameter counts: 17M and 32M. As part of our research, we conduct numerous
experiments to improve retrieval and late-interaction models, which we intend
to distill into smaller models as proof-of-concepts. Our ultimate aim is to
support retrieval at all scales, from large-scale retrieval which lives in the
cloud to models that can run locally, on any device. mxbai-edge-colbert-v0 is a
model that we hope will serve as a solid foundation backbone for all future
experiments, representing the first version of a long series of small
proof-of-concepts. As part of the development of mxbai-edge-colbert-v0, we
conducted multiple ablation studies, of which we report the results. In terms
of downstream performance, mxbai-edge-colbert-v0 is a particularly capable
small model, outperforming ColBERTv2 on common short-text benchmarks (BEIR) and
representing a large step forward in long-context tasks, with unprecedented
efficiency.
TokDrift: When LLM Speaks in Subwords but Code Speaks in Grammar
Authors: name: Yinxi Li, name: Yuntian Deng, name: Pengyu Nie
Published: 2025-10-16 17:59:45

Large language models (LLMs) for code rely on subword tokenizers, such as
byte-pair encoding (BPE), learned from mixed natural language text and
programming language code but driven by statistics rather than grammar. As a
result, semantically identical code snippets can be tokenized differently
depending on superficial factors such as whitespace or identifier naming. To
measure the impact of this misalignment, we introduce TokDrift, a framework
that applies semantic-preserving rewrite rules to create code variants
differing only in tokenization. Across nine code LLMs, including large ones
with over 30B parameters, even minor formatting changes can cause substantial
shifts in model behavior. Layer-wise analysis shows that the issue originates
in early embeddings, where subword segmentation fails to capture grammar token
boundaries. Our findings identify misaligned tokenization as a hidden obstacle
to reliable code understanding and generation, highlighting the need for
grammar-aware tokenization for future code LLMs.
BitNet Distillation
Authors: name: Xun Wu, name: Shaohan Huang, name: Wenhui Wang, name: Ting Song, name: Li Dong, name: Yan Xia, name: Furu Wei
Published: 2025-10-15 18:28:12

In this paper, we present BitNet Distillation (BitDistill), a lightweight
pipeline that fine-tunes off-the-shelf full-precision LLMs (e.g., Qwen) into
1.58-bit precision (i.e., ternary weights {-1, 0, 1}) for specific downstream
tasks, achieving strong task-specific performance with minimal computational
cost. Specifically, BitDistill incorporates three key techniques: the SubLN
module, as introduced in BitNet; multi-head attention distillation, based on
MiniLM; and continual pre-training, which serves as a crucial warm-up step to
mitigate the scalability issue of the performance gap between finetuned
full-precision and 1.58-bit LLMs on specific tasks. Experimental results show
that BitDistill achieves performance comparable to the full-precision
counterpart models across model size, while enabling up to 10x memory savings
and 2.65x faster inference on CPUs. Code is available at
https://github.com/microsoft/BitNet.
Attention Is All You Need for KV Cache in Diffusion LLMs
Authors: name: Quan Nguyen-Tri, name: Mukul Ranjan, name: Zhiqiang Shen
Published: 2025-10-16 17:59:48

This work studies how to adaptively recompute key-value (KV) caches for
diffusion large language models (DLMs) to maximize prediction accuracy while
minimizing decoding latency. Prior methods' decoders recompute QKV for all
tokens at every denoising step and layer, despite KV states changing little
across most steps, especially in shallow layers, leading to substantial
redundancy. We make three observations: (1) distant {bf MASK} tokens
primarily act as a length-bias and can be cached block-wise beyond the active
prediction window; (2) KV dynamics increase with depth, suggesting that
selective refresh starting from deeper layers is sufficient; and (3) the
most-attended token exhibits the smallest KV drift, providing a conservative
lower bound on cache change for other tokens. Building on these, we propose
{bf Elastic-Cache}, a training-free, architecture-agnostic strategy that
jointly decides {when} to refresh (via an attention-aware drift test on the
most-attended token) and {where} to refresh (via a depth-aware schedule that
recomputes from a chosen layer onward while reusing shallow-layer caches and
off-window MASK caches). Unlike fixed-period schemes, Elastic-Cache performs
adaptive, layer-aware cache updates for diffusion LLMs, reducing redundant
computation and accelerating decoding with negligible loss in generation
quality. Experiments on LLaDA-Instruct, LLaDA-1.5, and LLaDA-V across
mathematical reasoning and code generation tasks demonstrate consistent
speedups: 8.7times on GSM8K (256 tokens), 45.1times on longer sequences,
and 4.8times on HumanEval, while consistently maintaining higher accuracy
than the baseline. Our method achieves significantly higher throughput
(6.8times on GSM8K) than existing confidence-based approaches while
preserving generation quality, enabling practical deployment of diffusion LLMs.
VLA-0: Building State-of-the-Art VLAs with Zero Modification
Authors: name: Ankit Goyal, name: Hugo Hadfield, name: Xuning Yang, name: Valts Blukis, name: Fabio Ramos
Published: 2025-10-15 00:31:10

Vision-Language-Action models (VLAs) hold immense promise for enabling
generalist robot manipulation. However, the best way to build them remains an
open question. Current approaches often add complexity, such as modifying the
existing vocabulary of a Vision-Language Model (VLM) with action tokens or
introducing special action heads. Curiously, the simplest strategy of
representing actions directly as text has remained largely unexplored. This
work introduces VLA-0 to investigate this idea. We find that VLA-0 is not only
effective; it is surprisingly powerful. With the right design, VLA-0
outperforms more involved models. On LIBERO, a popular benchmark for evaluating
VLAs, VLA-0 outperforms all existing methods trained on the same robotic data,
including pi_0.5-KI, OpenVLA-OFT and SmolVLA. Furthermore, without
large-scale robotics-specific training, it outperforms methods trained on
large-scale robotic data, like pi_0.5-KI, pi_0, GR00T-N1 and MolmoAct.
These findings also translate to the real world, where VLA-0 outperforms
SmolVLA, a VLA model pre-trained on large-scale real data. This paper
summarizes our unexpected findings and spells out the specific techniques
required to unlock the high performance of this simple yet potent VLA design.
Visual results, code, and trained models are provided here:
https://vla0.github.io/.
Large Language Models Do NOT Really Know What They Don't Know
Authors: name: Chi Seng Cheang, name: Hou Pong Chan, name: Wenxuan Zhang, name: Yang Deng
Published: 2025-10-10 06:09:04

Recent work suggests that large language models (LLMs) encode factuality
signals in their internal representations, such as hidden states, attention
weights, or token probabilities, implying that LLMs may "know what they don't
know". However, LLMs can also produce factual errors by relying on shortcuts or
spurious associations. These error are driven by the same training objective
that encourage correct predictions, raising the question of whether internal
computations can reliably distinguish between factual and hallucinated outputs.
In this work, we conduct a mechanistic analysis of how LLMs internally process
factual queries by comparing two types of hallucinations based on their
reliance on subject information. We find that when hallucinations are
associated with subject knowledge, LLMs employ the same internal recall process
as for correct responses, leading to overlapping and indistinguishable
hidden-state geometries. In contrast, hallucinations detached from subject
knowledge produce distinct, clustered representations that make them
detectable. These findings reveal a fundamental limitation: LLMs do not encode
truthfulness in their internal states but only patterns of knowledge recall,
demonstrating that "LLMs don't really know what they don't know".
VR-Thinker: Boosting Video Reward Models through Thinking-with-Image
  Reasoning
Authors: name: Qunzhong Wang, name: Jie Liu, name: Jiajun Liang, name: Yilei Jiang, name: Yuanxing Zhang, name: Jinyuan Chen, name: Yaozhi Zheng, name: Xintao Wang, name: Pengfei Wan, name: Xiangyu Yue, name: Jiaheng Liu
Published: 2025-10-12 09:29:50

Recent advancements in multimodal reward models (RMs) have substantially
improved post-training for visual generative models. However, current RMs face
inherent limitations: (1) visual inputs consume large context budgets, forcing
fewer frames and causing loss of fine-grained details; and (2) all visual
information is packed into the initial prompt, exacerbating hallucination and
forgetting during chain-of-thought reasoning. To overcome these issues, we
introduce VideoReward Thinker (VR-Thinker), a thinking-with-image framework
that equips the RM with visual reasoning operations (e.g., select frame) and a
configurable visual memory window. This allows the RM to actively acquire and
update visual evidence within context limits, improving reasoning fidelity and
reliability. We activate visual reasoning via a reinforcement fine-tuning
pipeline: (i) Cold Start with curated visual chain-of-thought data to distill
basic reasoning skills and operation formatting; (ii) select samples whose
per-dimension and overall judgments are all correct, then conduct Rejection
sampling Fine-Tuning on these high-quality traces to further enhance reasoning;
and (iii) apply Group Relative Policy Optimization (GRPO) to strengthen
reasoning. Our approach delivers state-of-the-art accuracy among open-source
models on video preference benchmarks, especially for longer videos: a 7B
VR-Thinker achieves 80.5% on VideoGen Reward, 82.3% on GenAI-Bench, and 75.6%
on MJ-Bench-Video. These results validate the effectiveness and promise of
thinking-with-image multimodal reward modeling.