Orionnnn/hub-stats huggingface dataset:
Through the Looking Glass: Common Sense Consistency Evaluation of Weird
  Images
Authors: name: Elisei Rykov, name: Kseniia Petrushina, name: Kseniia Titova, name: Anton Razzhigaev, name: Alexander Panchenko, name: Vasily Konovalov
Published: 2025-05-12 16:12:11

Measuring how real images look is a complex task in artificial intelligence
research. For example, an image of a boy with a vacuum cleaner in a desert
violates common sense. We introduce a novel method, which we call Through the
Looking Glass (TLG), to assess image common sense consistency using Large
Vision-Language Models (LVLMs) and Transformer-based encoder. By leveraging
LVLMs to extract atomic facts from these images, we obtain a mix of accurate
facts. We proceed by fine-tuning a compact attention-pooling classifier over
encoded atomic facts. Our TLG has achieved a new state-of-the-art performance
on the WHOOPS! and WEIRD datasets while leveraging a compact fine-tuning
component.
R3: Robust Rubric-Agnostic Reward Models
Authors: name: David Anugraha, name: Zilu Tang, name: Lester James V. Miranda, name: Hanyang Zhao, name: Mohammad Rifqi Farhansyah, name: Garry Kuwanto, name: Derry Wijaya, name: Genta Indra Winata
Published: 2025-05-19 17:29:03

Reward models are essential for aligning language model outputs with human
preferences, yet existing approaches often lack both controllability and
interpretability. These models are typically optimized for narrow objectives,
limiting their generalizability to broader downstream tasks. Moreover, their
scalar outputs are difficult to interpret without contextual reasoning. To
address these limitations, we introduce R3, a novel reward modeling framework
that is rubric-agnostic, generalizable across evaluation dimensions, and
provides interpretable, reasoned score assignments. R3 enables more transparent
and flexible evaluation of language models, supporting robust alignment with
diverse human values and use cases. Our models, data, and code are available as
open source at https://github.com/rubricreward/r3
A Token is Worth over 1,000 Tokens: Efficient Knowledge Distillation
  through Low-Rank Clone
Authors: name: Jitai Hao, name: Qiang Huang, name: Hao Liu, name: Xinyan Xiao, name: Zhaochun Ren, name: Jun Yu
Published: 2025-05-19 07:10:42

Training high-performing Small Language Models (SLMs) remains costly, even
with knowledge distillation and pruning from larger teacher models. Existing
work often faces three key challenges: (1) information loss from hard pruning,
(2) inefficient alignment of representations, and (3) underutilization of
informative activations, particularly from Feed-Forward Networks (FFNs). To
address these challenges, we introduce Low-Rank Clone (LRC), an efficient
pre-training method that constructs SLMs aspiring to behavioral equivalence
with strong teacher models. LRC trains a set of low-rank projection matrices
that jointly enable soft pruning by compressing teacher weights, and activation
clone by aligning student activations, including FFN signals, with those of the
teacher. This unified design maximizes knowledge transfer while removing the
need for explicit alignment modules. Extensive experiments with open-source
teachers (e.g., Llama-3.2-3B-Instruct, Qwen2.5-3B/7B-Instruct) show that LRC
matches or surpasses state-of-the-art models trained on trillions of
tokens--while using only 20B tokens, achieving over 1,000x training efficiency.
Our codes and model checkpoints are available at
https://github.com/CURRENTF/LowRankClone and
https://huggingface.co/collections/JitaiHao/low-rank-clone-lrc-6828389e96a93f1d4219dfaf.
Creating General User Models from Computer Use
Authors: name: Omar Shaikh, name: Shardul Sapkota, name: Shan Rizvi, name: Eric Horvitz, name: Joon Sung Park, name: Diyi Yang, name: Michael S. Bernstein
Published: 2025-05-16 04:00:31

Human-computer interaction has long imagined technology that understands
us-from our preferences and habits, to the timing and purpose of our everyday
actions. Yet current user models remain fragmented, narrowly tailored to
specific apps, and incapable of the flexible reasoning required to fulfill
these visions. This paper presents an architecture for a general user model
(GUM) that learns about you by observing any interaction you have with your
computer. The GUM takes as input any unstructured observation of a user (e.g.,
device screenshots) and constructs confidence-weighted propositions that
capture that user knowledge and preferences. GUMs can infer that a user is
preparing for a wedding they're attending from messages with a friend. Or
recognize that a user is struggling with a collaborator's feedback on a draft
by observing multiple stalled edits and a switch to reading related work. GUMs
introduce an architecture that infers new propositions about a user from
multimodal observations, retrieves related propositions for context, and
continuously revises existing propositions. To illustrate the breadth of
applications that GUMs enable, we demonstrate how they augment chat-based
assistants with context, manage OS notifications to selectively surface
important information, and enable interactive agents that adapt to preferences
across apps. We also instantiate proactive assistants (GUMBOs) that discover
and execute useful suggestions on a user's behalf using their GUM. In our
evaluations, we find that GUMs make calibrated and accurate inferences about
users, and that assistants built on GUMs proactively identify and perform
actions that users wouldn't think to request explicitly. Altogether, GUMs
introduce methods that leverage multimodal models to understand unstructured
context, enabling long-standing visions of HCI and entirely new interactive
systems that anticipate user needs.
SEED-GRPO: Semantic Entropy Enhanced GRPO for Uncertainty-Aware Policy
  Optimization
Authors: name: Minghan Chen, name: Guikun Chen, name: Wenguan Wang, name: Yi Yang
Published: 2025-05-18 10:20:59

Large language models (LLMs) exhibit varying levels of confidence across
input prompts (questions): some lead to consistent, semantically similar
answers, while others yield diverse or contradictory outputs. This variation
reflects LLM's uncertainty about the input prompt, a signal of how confidently
the model understands a given problem. However, vanilla Group Relative Policy
Optimization (GRPO) treats all prompts equally during policy updates, ignoring
this important information about the model's knowledge boundaries. To address
this limitation, we propose SEED-GRPO (Semantic Entropy EnhanceD GRPO), which
explicitly measures LLMs' uncertainty of the input prompts semantic entropy.
Semantic entropy measures the diversity of meaning in multiple generated
answers given a prompt and uses this to modulate the magnitude of policy
updates. This uncertainty-aware training mechanism enables dynamic adjustment
of policy update magnitudes based on question uncertainty. It allows more
conservative updates on high-uncertainty questions while maintaining the
original learning signal on confident ones. Experimental results on five
mathematical reasoning benchmarks (AIME24 56.7, AMC 68.7, MATH 83.4, Minerva
34.2, and OlympiadBench 48.0) demonstrate that SEED-GRPO achieves new
state-of-the-art performance in average accuracy, validating the effectiveness
of uncertainty-aware policy optimization.
HISTAI: An Open-Source, Large-Scale Whole Slide Image Dataset for
  Computational Pathology
Authors: name: Dmitry Nechaev, name: Alexey Pchelnikov, name: Ekaterina Ivanova
Published: 2025-05-17 18:59:32

Recent advancements in Digital Pathology (DP), particularly through
artificial intelligence and Foundation Models, have underscored the importance
of large-scale, diverse, and richly annotated datasets. Despite their critical
role, publicly available Whole Slide Image (WSI) datasets often lack sufficient
scale, tissue diversity, and comprehensive clinical metadata, limiting the
robustness and generalizability of AI models. In response, we introduce the
HISTAI dataset, a large, multimodal, open-access WSI collection comprising over
60,000 slides from various tissue types. Each case in the HISTAI dataset is
accompanied by extensive clinical metadata, including diagnosis, demographic
information, detailed pathological annotations, and standardized diagnostic
coding. The dataset aims to fill gaps identified in existing resources,
promoting innovation, reproducibility, and the development of clinically
relevant computational pathology solutions. The dataset can be accessed at
https://github.com/HistAI/HISTAI.
Learned Lightweight Smartphone ISP with Unpaired Data
Authors: name: Andrei Arhire, name: Radu Timofte
Published: 2025-05-15 15:37:51

The Image Signal Processor (ISP) is a fundamental component in modern
smartphone cameras responsible for conversion of RAW sensor image data to RGB
images with a strong focus on perceptual quality. Recent work highlights the
potential of deep learning approaches and their ability to capture details with
a quality increasingly close to that of professional cameras. A difficult and
costly step when developing a learned ISP is the acquisition of pixel-wise
aligned paired data that maps the raw captured by a smartphone camera sensor to
high-quality reference images. In this work, we address this challenge by
proposing a novel training method for a learnable ISP that eliminates the need
for direct correspondences between raw images and ground-truth data with
matching content. Our unpaired approach employs a multi-term loss function
guided by adversarial training with multiple discriminators processing feature
maps from pre-trained networks to maintain content structure while learning
color and texture characteristics from the target RGB dataset. Using
lightweight neural network architectures suitable for mobile devices as
backbones, we evaluated our method on the Zurich RAW to RGB and Fujifilm
UltraISP datasets. Compared to paired training methods, our unpaired learning
strategy shows strong potential and achieves high fidelity across multiple
evaluation metrics. The code and pre-trained models are available at
https://github.com/AndreiiArhire/Learned-Lightweight-Smartphone-ISP-with-Unpaired-Data .
HelpSteer3-Preference: Open Human-Annotated Preference Data across
  Diverse Tasks and Languages
Authors: name: Zhilin Wang, name: Jiaqi Zeng, name: Olivier Delalleau, name: Hoo-Chang Shin, name: Felipe Soares, name: Alexander Bukharin, name: Ellie Evans, name: Yi Dong, name: Oleksii Kuchaiev
Published: 2025-05-16 17:31:19

Preference datasets are essential for training general-domain,
instruction-following language models with Reinforcement Learning from Human
Feedback (RLHF). Each subsequent data release raises expectations for future
data collection, meaning there is a constant need to advance the quality and
diversity of openly available preference data. To address this need, we
introduce HelpSteer3-Preference, a permissively licensed (CC-BY-4.0),
high-quality, human-annotated preference dataset comprising of over 40,000
samples. These samples span diverse real-world applications of large language
models (LLMs), including tasks relating to STEM, coding and multilingual
scenarios. Using HelpSteer3-Preference, we train Reward Models (RMs) that
achieve top performance on RM-Bench (82.4%) and JudgeBench (73.7%). This
represents a substantial improvement (~10% absolute) over the previously
best-reported results from existing RMs. We demonstrate HelpSteer3-Preference
can also be applied to train Generative RMs and how policy models can be
aligned with RLHF using our RMs. Dataset (CC-BY-4.0):
https://huggingface.co/datasets/nvidia/HelpSteer3#preference
Fast, Not Fancy: Rethinking G2P with Rich Data and Rule-Based Models
Authors: name: Mahta Fetrat Qharabagh, name: Zahra Dehghanian, name: Hamid R. Rabiee
Published: 2025-05-19 11:11:12

Homograph disambiguation remains a significant challenge in
grapheme-to-phoneme (G2P) conversion, especially for low-resource languages.
This challenge is twofold: (1) creating balanced and comprehensive homograph
datasets is labor-intensive and costly, and (2) specific disambiguation
strategies introduce additional latency, making them unsuitable for real-time
applications such as screen readers and other accessibility tools. In this
paper, we address both issues. First, we propose a semi-automated pipeline for
constructing homograph-focused datasets, introduce the HomoRich dataset
generated through this pipeline, and demonstrate its effectiveness by applying
it to enhance a state-of-the-art deep learning-based G2P system for Persian.
Second, we advocate for a paradigm shift - utilizing rich offline datasets to
inform the development of fast, rule-based methods suitable for
latency-sensitive accessibility applications like screen readers. To this end,
we improve one of the most well-known rule-based G2P systems, eSpeak, into a
fast homograph-aware version, HomoFast eSpeak. Our results show an approximate
30% improvement in homograph disambiguation accuracy for the deep
learning-based and eSpeak systems.
Model Merging in Pre-training of Large Language Models
Authors: name: Yunshui Li, name: Yiyuan Ma, name: Shen Yan, name: Chaoyi Zhang, name: Jing Liu, name: Jianqiao Lu, name: Ziwen Xu, name: Mengzhao Chen, name: Minrui Wang, name: Shiyi Zhan, name: Jin Ma, name: Xunhao Lai, name: Yao Luo, name: Xingyan Bin, name: Hongbin Ren, name: Mingji Han, name: Wenhao Hao, name: Bairen Yi, name: LingJun Liu, name: Bole Ma, name: Xiaoying Jia, name: Zhou Xun, name: Liang Xiang, name: Yonghui Wu
Published: 2025-05-17 16:53:14

Model merging has emerged as a promising technique for enhancing large
language models, though its application in large-scale pre-training remains
relatively unexplored. In this paper, we present a comprehensive investigation
of model merging techniques during the pre-training process. Through extensive
experiments with both dense and Mixture-of-Experts (MoE) architectures ranging
from millions to over 100 billion parameters, we demonstrate that merging
checkpoints trained with constant learning rates not only achieves significant
performance improvements but also enables accurate prediction of annealing
behavior. These improvements lead to both more efficient model development and
significantly lower training costs. Our detailed ablation studies on merging
strategies and hyperparameters provide new insights into the underlying
mechanisms while uncovering novel applications. Through comprehensive
experimental analysis, we offer the open-source community practical
pre-training guidelines for effective model merging.
From Grunts to Grammar: Emergent Language from Cooperative Foraging
Authors: name: Maytus Piriyajitakonkij, name: Rujikorn Charakorn, name: Weicheng Tao, name: Wei Pan, name: Mingfei Sun, name: Cheston Tan, name: Mengmi Zhang
Published: 2025-05-19 08:57:30

Early cavemen relied on gestures, vocalizations, and simple signals to
coordinate, plan, avoid predators, and share resources. Today, humans
collaborate using complex languages to achieve remarkable results. What drives
this evolution in communication? How does language emerge, adapt, and become
vital for teamwork? Understanding the origins of language remains a challenge.
A leading hypothesis in linguistics and anthropology posits that language
evolved to meet the ecological and social demands of early human cooperation.
Language did not arise in isolation, but through shared survival goals.
Inspired by this view, we investigate the emergence of language in multi-agent
Foraging Games. These environments are designed to reflect the cognitive and
ecological constraints believed to have influenced the evolution of
communication. Agents operate in a shared grid world with only partial
knowledge about other agents and the environment, and must coordinate to
complete games like picking up high-value targets or executing temporally
ordered actions. Using end-to-end deep reinforcement learning, agents learn
both actions and communication strategies from scratch. We find that agents
develop communication protocols with hallmark features of natural language:
arbitrariness, interchangeability, displacement, cultural transmission, and
compositionality. We quantify each property and analyze how different factors,
such as population size and temporal dependencies, shape specific aspects of
the emergent language. Our framework serves as a platform for studying how
language can evolve from partial observability, temporal reasoning, and
cooperative goals in embodied multi-agent settings. We will release all data,
code, and models publicly.
Tiny QA Benchmark++: Ultra-Lightweight, Synthetic Multilingual Dataset
  Generation & Smoke-Tests for Continuous LLM Evaluation
Authors: name: Vincent Koc
Published: 2025-05-17 15:40:03

Tiny QA Benchmark++ (TQB++) presents an ultra-lightweight, multilingual
smoke-test suite designed to give large-language-model (LLM) pipelines a
unit-test style safety net dataset that runs in seconds with minimal cost. Born
out of the tight feedback-loop demands building the Comet Opik
prompt-optimization SDK, where waiting on heavyweight benchmarks breaks
developer flow. TQB++ couples a 52-item English gold set (less than 20 kB) with
a tiny synthetic-data generator pypi package built on provider-agnostic
LiteLLM. The generator lets practitioners mint their own tiny packs in any
language, domain, or difficulty, while ten ready-made packs already cover
Arabic, Chinese, French, German, Japanese, Korean, Portuguese, Russian,
Spanish, and Turkish. Every dataset ships with Croissant metadata and
plug-and-play files for OpenAI-Evals, LangChain, and standard CI tools, so
teams can drop deterministic micro-benchmarks directly into pull-request gates,
prompt-engineering loops, and production dashboards without touching GPU
budgets. A complete TQB++ run adds only a few seconds to pipeline latency yet
reliably flags prompt-template errors, tokenizer drift, and fine-tuning
side-effects long before full-scale suites like MMLU or BIG-Bench would finish
configuring. The entire framework is released to accelerate continuous,
resource-efficient quality assurance across the generative-AI ecosystem.
ViPlan: A Benchmark for Visual Planning with Symbolic Predicates and
  Vision-Language Models
Authors: name: Matteo Merler, name: Nicola Dainese, name: Minttu Alakuijala, name: Giovanni Bonetta, name: Pietro Ferrazzi, name: Yu Tian, name: Bernardo Magnini, name: Pekka Marttinen
Published: 2025-05-19 14:38:15

Integrating Large Language Models with symbolic planners is a promising
direction for obtaining verifiable and grounded plans compared to planning in
natural language, with recent works extending this idea to visual domains using
Vision-Language Models (VLMs). However, rigorous comparison between
VLM-grounded symbolic approaches and methods that plan directly with a VLM has
been hindered by a lack of common environments, evaluation protocols and model
coverage. We introduce ViPlan, the first open-source benchmark for Visual
Planning with symbolic predicates and VLMs. ViPlan features a series of
increasingly challenging tasks in two domains: a visual variant of the classic
Blocksworld planning problem and a simulated household robotics environment. We
benchmark nine open-source VLM families across multiple sizes, along with
selected closed models, evaluating both VLM-grounded symbolic planning and
using the models directly to propose actions. We find symbolic planning to
outperform direct VLM planning in Blocksworld, where accurate image grounding
is crucial, whereas the opposite is true in the household robotics tasks, where
commonsense knowledge and the ability to recover from errors are beneficial.
Finally, we show that across most models and methods, there is no significant
benefit to using Chain-of-Thought prompting, suggesting that current VLMs still
struggle with visual reasoning.
QVGen: Pushing the Limit of Quantized Video Generative Models
Authors: name: Yushi Huang, name: Ruihao Gong, name: Jing Liu, name: Yifu Ding, name: Chengtao Lv, name: Haotong Qin, name: Jun Zhang
Published: 2025-05-16 17:59:40

Video diffusion models (DMs) have enabled high-quality video synthesis. Yet,
their substantial computational and memory demands pose serious challenges to
real-world deployment, even on high-end GPUs. As a commonly adopted solution,
quantization has proven notable success in reducing cost for image DMs, while
its direct application to video DMs remains ineffective. In this paper, we
present QVGen, a novel quantization-aware training (QAT) framework tailored for
high-performance and inference-efficient video DMs under extremely low-bit
quantization (e.g., 4-bit or below). We begin with a theoretical analysis
demonstrating that reducing the gradient norm is essential to facilitate
convergence for QAT. To this end, we introduce auxiliary modules (Phi) to
mitigate large quantization errors, leading to significantly enhanced
convergence. To eliminate the inference overhead of Phi, we propose a
rank-decay strategy that progressively eliminates Phi. Specifically, we
repeatedly employ singular value decomposition (SVD) and a proposed rank-based
regularization gamma to identify and decay low-contributing
components. This strategy retains performance while zeroing out inference
overhead. Extensive experiments across 4 state-of-the-art (SOTA) video DMs,
with parameter sizes ranging from 1.3B sim14B, show that QVGen is the
first to reach full-precision comparable quality under 4-bit settings.
Moreover, it significantly outperforms existing methods. For instance, our
3-bit CogVideoX-2B achieves improvements of +25.28 in Dynamic Degree and
+8.43 in Scene Consistency on VBench.
MTVCrafter: 4D Motion Tokenization for Open-World Human Image Animation
Authors: name: Yanbo Ding, name: Xirui Hu, name: Zhizhi Guo, name: Yali Wang
Published: 2025-05-15 12:50:29

Human image animation has gained increasing attention and developed rapidly
due to its broad applications in digital humans. However, existing methods rely
largely on 2D-rendered pose images for motion guidance, which limits
generalization and discards essential 3D information for open-world animation.
To tackle this problem, we propose MTVCrafter (Motion Tokenization Video
Crafter), the first framework that directly models raw 3D motion sequences
(i.e., 4D motion) for human image animation. Specifically, we introduce 4DMoT
(4D motion tokenizer) to quantize 3D motion sequences into 4D motion tokens.
Compared to 2D-rendered pose images, 4D motion tokens offer more robust
spatio-temporal cues and avoid strict pixel-level alignment between pose image
and character, enabling more flexible and disentangled control. Then, we
introduce MV-DiT (Motion-aware Video DiT). By designing unique motion attention
with 4D positional encodings, MV-DiT can effectively leverage motion tokens as
4D compact yet expressive context for human image animation in the complex 3D
world. Hence, it marks a significant step forward in this field and opens a new
direction for pose-guided human video generation. Experiments show that our
MTVCrafter achieves state-of-the-art results with an FID-VID of 6.98,
surpassing the second-best by 65%. Powered by robust motion tokens, MTVCrafter
also generalizes well to diverse open-world characters (single/multiple,
full/half-body) across various styles and scenarios. Our video demos and code
are on: https://github.com/DINGYANB/MTVCrafter.