Orionnnn/hub-stats huggingface dataset:
Socratic-Zero : Bootstrapping Reasoning via Data-Free Agent Co-evolution
Authors: name: Shaobo Wang, name: Zhengbo Jiao, name: Zifan Zhang, name: Yilang Peng, name: Xu Ze, name: Boyu Yang, name: Wei Wang, name: Hu Wei, name: Linfeng Zhang
Published: 2025-09-29 12:54:07

Recent breakthroughs in large language models (LLMs) on reasoning tasks rely
heavily on massive, high-quality datasets-typically human-annotated and thus
difficult to scale. While data synthesis or distillation offers a promising
alternative, existing methods struggle with inconsistent data quality and an
inability to dynamically adapt to the evolving capabilities of the model,
leading to suboptimal training signals. To address these limitations, we
introduce Socratic-Zero, a fully autonomous framework that generates
high-quality training data from minimal seed examples through the co-evolution
of three agents: the Teacher, the Solver, and the Generator. The Solver
continuously refines its reasoning by learning from preference feedback on both
successful and failed trajectories; the Teacher adaptively crafts increasingly
challenging questions based on the Solver's weaknesses; and the Generator
distills the Teacher's question-design strategy to enable scalable,
high-fidelity curriculum generation. This closed-loop system produces a
self-improving curriculum-requiring no pre-existing tasks or labels.
Remarkably, starting from only 100 seed questions, our Socratic-Solver-8B
achieves an average gain of +20.2 percentage points over prior data synthesis
methods across seven mathematical reasoning benchmarks (AMC23, AIME24-25,
Olympiad, MATH-500, Minerva, and GSM8K), with consistent gains on both Qwen3
and GLM4 series models. Even more surprisingly, synthetic data from
Socratic-Generator-32B enables student LLMs to achieve superior performance
compared to other state-of-the-art (SOTA) commercial LLMs on these benchmarks,
including Qwen3-235B-A22B, DeepSeek-V3.1-671B, GPT-5, Gemini-2.5-Pro, Grok-4,
and Claude-4.1-Opus.
Generalized Correctness Models: Learning Calibrated and Model-Agnostic
  Correctness Predictors from Historical Patterns
Authors: name: Hanqi Xiao, name: Vaidehi Patil, name: Hyunji Lee, name: Elias Stengel-Eskin, name: Mohit Bansal
Published: 2025-09-29 16:19:01

Generating accurate and calibrated confidence estimates is critical for
deploying LLMs in high-stakes or user-facing applications, and remains an open
challenge. Prior research has often framed confidence as a problem of eliciting
a model's "self-knowledge", i.e., the ability of an LLM to judge whether its
own answers are correct; this approach implicitly assumes that there is some
privileged information about the answer's correctness that is accessible to the
model itself. However, our experiments reveal that an LLM attempting to predict
the correctness of its own outputs generally performs no better than an
unrelated LLM. Moreover, we hypothesize that a key factor in building a
"Correctness Model" (CM) is exposure to a target model's historical
predictions. We propose multiple methods to inject this historical correctness
information, creating a Generalized Correctness Model (GCM). We first show that
GCMs can be trained on the correctness data from many LLMs and learn patterns
for correctness prediction applicable across datasets and models. We then use
CMs as a lens for studying the source of correctness prediction ability and its
generalization, systematically controlling their training data and finding that
answer phrasing is a strong predictor for correctness. We further explore
alternative methods of injecting history without training an LLM, finding that
including history as in-context examples can help improve correctness
prediction, and post-hoc calibration can provide complementary reductions in
calibration error. We evaluate GCMs based on Qwen3-8B across 5 model families
and the MMLU and TriviaQA datasets, as well as on a downstream selective
prediction task, finding that reliable LLM confidence estimation is a
generalizable and model-agnostic skill learned by systematically encoding
correctness history rather than a model-specific skill reliant on
self-introspection.
Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement
  Learning
Authors: name: Xin Qiu, name: Yulu Gan, name: Conor F. Hayes, name: Qiyao Liang, name: Elliot Meyerson, name: Babak Hodjat, name: Risto Miikkulainen
Published: 2025-09-29 07:19:34

Fine-tuning pre-trained large language models (LLMs) for down-stream tasks is
a critical step in the AI deployment pipeline. Reinforcement learning (RL) is
arguably the most prominent fine-tuning method, contributing to the birth of
many state-of-the-art LLMs. In contrast, evolution strategies (ES), which once
showed comparable performance to RL on models with a few million parameters,
was neglected due to the pessimistic perception of its scalability to larger
models. In this work, we report the first successful attempt to scale up ES for
fine-tuning the full parameters of LLMs, showing the surprising fact that ES
can search efficiently over billions of parameters and outperform existing RL
fine-tuning methods in multiple respects, including sample efficiency,
tolerance to long-horizon rewards, robustness to different base LLMs, less
tendency to reward hacking, and more stable performance across runs. It
therefore serves as a basis to unlock a new direction in LLM fine-tuning beyond
what current RL techniques provide. The source codes are provided at:
https://github.com/VsonicV/es-fine-tuning-paper.
GRPO-MA: Multi-Answer Generation in GRPO for Stable and Efficient
  Chain-of-Thought Training
Authors: name: Hongcheng Wang, name: Yinuo Huang, name: Sukai Wang, name: Guanghui Ren, name: Hao Dong
Published: 2025-09-29 09:07:45

Recent progress, such as DeepSeek-R1, has shown that the GRPO algorithm, a
Reinforcement Learning (RL) approach, can effectively train Chain-of-Thought
(CoT) reasoning in Large Language Models (LLMs) and Vision-Language Models
(VLMs). In this paper, we analyze three challenges of GRPO: gradient coupling
between thoughts and answers, sparse reward signals caused by limited parallel
sampling, and unstable advantage estimation. To mitigate these challenges, we
propose GRPO-MA, a simple yet theoretically grounded method that leverages
multi-answer generation from each thought process, enabling more robust and
efficient optimization. Theoretically, we show that the variance of thought
advantage decreases as the number of answers per thought increases.
Empirically, our gradient analysis confirms this effect, showing that GRPO-MA
reduces gradient spikes compared to GRPO. Experiments on math, code, and
diverse multimodal tasks demonstrate that GRPO-MA substantially improves
performance and training efficiency. Our ablation studies further reveal that
increasing the number of answers per thought consistently enhances model
performance.
The Era of Real-World Human Interaction: RL from User Conversations
Authors: name: Chuanyang Jin, name: Jing Xu, name: Bo Liu, name: Leitian Tao, name: Olga Golovneva, name: Tianmin Shu, name: Wenting Zhao, name: Xian Li, name: Jason Weston
Published: 2025-09-29 17:50:31

We posit that to achieve continual model improvement and multifaceted
alignment, future models must learn from natural human interaction. Current
conversational models are aligned using pre-annotated, expert-generated human
feedback. In this work, we introduce Reinforcement Learning from Human
Interaction (RLHI), a paradigm that learns directly from in-the-wild user
conversations. We develop two complementary methods: (1) RLHI with User-Guided
Rewrites, which revises unsatisfactory model outputs based on users'
natural-language follow-up responses, (2) RLHI with User-Based Rewards, which
learns via a reward model conditioned on knowledge of the user's long-term
interaction history (termed persona). Together, these methods link long-term
user personas to turn-level preferences via persona-conditioned preference
optimization. Trained on conversations derived from WildChat, both RLHI
variants outperform strong baselines in personalization and
instruction-following, and similar feedback enhances performance on reasoning
benchmarks. These results suggest organic human interaction offers scalable,
effective supervision for personalized alignment.
LUMINA: Detecting Hallucinations in RAG System with Context-Knowledge
  Signals
Authors: name: Min-Hsuan Yeh, name: Yixuan Li, name: Tanwi Mallick
Published: 2025-09-26 04:57:46

Retrieval-Augmented Generation (RAG) aims to mitigate hallucinations in large
language models (LLMs) by grounding responses in retrieved documents. Yet,
RAG-based LLMs still hallucinate even when provided with correct and sufficient
context. A growing line of work suggests that this stems from an imbalance
between how models use external context and their internal knowledge, and
several approaches have attempted to quantify these signals for hallucination
detection. However, existing methods require extensive hyperparameter tuning,
limiting their generalizability. We propose LUMINA, a novel framework that
detects hallucinations in RAG systems through context-knowledge signals:
external context utilization is quantified via distributional distance, while
internal knowledge utilization is measured by tracking how predicted tokens
evolve across transformer layers. We further introduce a framework for
statistically validating these measurements. Experiments on common RAG
hallucination benchmarks and four open-source LLMs show that LUMINA achieves
consistently high AUROC and AUPRC scores, outperforming prior utilization-based
methods by up to +13% AUROC on HalluRAG. Moreover, LUMINA remains robust under
relaxed assumptions about retrieval quality and model matching, offering both
effectiveness and practicality.
Clean First, Align Later: Benchmarking Preference Data Cleaning for
  Reliable LLM Alignment
Authors: name: Min-Hsuan Yeh, name: Yixuan Li
Published: 2025-09-28 01:44:05

Human feedback plays a pivotal role in aligning large language models (LLMs)
with human preferences. However, such feedback is often noisy or inconsistent,
which can degrade the quality of reward models and hinder alignment. While
various automated data cleaning methods have been proposed to mitigate this
issue, a systematic evaluation of their effectiveness and generalizability
remains lacking. To bridge this gap, we introduce the first comprehensive
benchmark for evaluating 13 preference data cleaning methods in the context of
LLM alignment. PrefCleanBench offers a standardized protocol to assess cleaning
strategies in terms of alignment performance and generalizability across
diverse datasets, model architectures, and optimization algorithms. By unifying
disparate methods and rigorously comparing them, we uncover key factors that
determine the success of data cleaning in alignment tasks. This benchmark lays
the groundwork for principled and reproducible approaches to improving LLM
alignment through better data quality-highlighting the crucial but
underexplored role of data preprocessing in responsible AI development. We
release modular implementations of all methods to catalyze further research:
https://github.com/deeplearning-wisc/PrefCleanBench.
StableToken: A Noise-Robust Semantic Speech Tokenizer for Resilient
  SpeechLLMs
Authors: name: Yuhan Song, name: Linhao Zhang, name: Chuhan Wu, name: Aiwei Liu, name: Wei Jia, name: Houfeng Wang, name: Xiao Zhou
Published: 2025-09-26 11:32:51

Prevalent semantic speech tokenizers, designed to capture linguistic content,
are surprisingly fragile. We find they are not robust to meaning-irrelevant
acoustic perturbations; even at high Signal-to-Noise Ratios (SNRs) where speech
is perfectly intelligible, their output token sequences can change drastically,
increasing the learning burden for downstream LLMs. This instability stems from
two flaws: a brittle single-path quantization architecture and a distant
training signal indifferent to intermediate token stability. To address this,
we introduce StableToken, a tokenizer that achieves stability through a
consensus-driven mechanism. Its multi-branch architecture processes audio in
parallel, and these representations are merged via a powerful bit-wise voting
mechanism to form a single, stable token sequence. StableToken sets a new
state-of-the-art in token stability, drastically reducing Unit Edit Distance
(UED) under diverse noise conditions. This foundational stability translates
directly to downstream benefits, significantly improving the robustness of
SpeechLLMs on a variety of tasks.
Rethinking Large Language Model Distillation: A Constrained Markov
  Decision Process Perspective
Authors: name: Matthieu Zimmer, name: Xiaotong Ji, name: Tu Nguyen, name: Haitham Bou Ammar
Published: 2025-09-26 20:47:49

We introduce a novel approach to large language model (LLM) distillation by
formulating it as a constrained reinforcement learning problem. While recent
work has begun exploring the integration of task-specific rewards into
distillation processes, existing methods typically rely on ad-hoc reward
weighting. We propose a principled optimization framework that maximizes
task-specific rewards while constraining the divergence from the teacher model
to remain below a specified threshold. Our approach adapts constrained state
augmented reinforcement learning to the distillation setting, introducing a
modified reward function that maintains theoretical guarantees of constraint
satisfaction without requiring state augmentation or teacher model access
during deployment and without the computational overhead of the dual Lagrangian
methods. Through extensive experiments on mathematical reasoning tasks, we
demonstrate that our method achieves better constraint satisfaction rates and
better reasoning compared to the soft Lagrangian relaxation baselines while
maintaining competitive task performance. Our framework provides a
theoretically grounded and practically efficient solution for reward-aware
distillation in resource-constrained settings.
UniVid: The Open-Source Unified Video Model
Authors: name: Jiabin Luo, name: Junhui Lin, name: Zeyu Zhang, name: Biao Wu, name: Meng Fang, name: Ling Chen, name: Hao Tang
Published: 2025-09-29 02:31:36

Unified video modeling that combines generation and understanding
capabilities is increasingly important but faces two key challenges:
maintaining semantic faithfulness during flow-based generation due to
text-visual token imbalance and the limitations of uniform cross-modal
attention across the flow trajectory, and efficiently extending image-centric
MLLMs to video without costly retraining. We present UniVid, a unified
architecture that couples an MLLM with a diffusion decoder through a
lightweight adapter, enabling both video understanding and generation. We
introduce Temperature Modality Alignment to improve prompt adherence and
Pyramid Reflection for efficient temporal reasoning via dynamic keyframe
selection. Extensive experiments on standard benchmarks demonstrate
state-of-the-art performance, achieving a 2.2% improvement on VBench-Long total
score compared to EasyAnimateV5.1, and 1.0% and 3.3% accuracy gains on MSVD-QA
and ActivityNet-QA, respectively, compared with the best prior 7B baselines.
Charting a Decade of Computational Linguistics in Italy: The CLiC-it
  Corpus
Authors: name: Chiara Alzetta, name: Serena Auriemma, name: Alessandro Bondielli, name: Luca Dini, name: Chiara Fazzone, name: Alessio Miaschi, name: Martina Miliani, name: Marta Sartor
Published: 2025-09-23 14:06:09

Over the past decade, Computational Linguistics (CL) and Natural Language
Processing (NLP) have evolved rapidly, especially with the advent of
Transformer-based Large Language Models (LLMs). This shift has transformed
research goals and priorities, from Lexical and Semantic Resources to Language
Modelling and Multimodality. In this study, we track the research trends of the
Italian CL and NLP community through an analysis of the contributions to
CLiC-it, arguably the leading Italian conference in the field. We compile the
proceedings from the first 10 editions of the CLiC-it conference (from 2014 to
2024) into the CLiC-it Corpus, providing a comprehensive analysis of both its
metadata, including author provenance, gender, affiliations, and more, as well
as the content of the papers themselves, which address various topics. Our goal
is to provide the Italian and international research communities with valuable
insights into emerging trends and key developments over time, supporting
informed decisions and future directions in the field.
When Does Reasoning Matter? A Controlled Study of Reasoning's
  Contribution to Model Performance
Authors: name: Nicolas Boizard, name: Hippolyte Gisserot-Boukhlef, name: Kevin El-Haddad, name: Céline Hudelot, name: Pierre Colombo
Published: 2025-09-26 10:53:52

Large Language Models (LLMs) with reasoning capabilities have achieved
state-of-the-art performance on a wide range of tasks. Despite its empirical
success, the tasks and model scales at which reasoning becomes effective, as
well as its training and inference costs, remain underexplored. In this work,
we rely on a synthetic data distillation framework to conduct a large-scale
supervised study. We compare Instruction Fine-Tuning (IFT) and reasoning models
of varying sizes, on a wide range of math-centric and general-purpose tasks,
evaluating both multiple-choice and open-ended formats. Our analysis reveals
that reasoning consistently improves model performance, often matching or
surpassing significantly larger IFT systems. Notably, while IFT remains
Pareto-optimal in training and inference costs, reasoning models become
increasingly valuable as model size scales, overcoming IFT performance limits
on reasoning-intensive and open-ended tasks.
MMPB: It's Time for Multi-Modal Personalization
Authors: name: Jaeik Kim, name: Woojin Kim, name: Woohyeon Park, name: Jaeyoung Do
Published: 2025-09-26 18:24:48

Visual personalization is essential in user-facing AI systems such as smart
homes and healthcare, where aligning model behavior with user-centric concepts
is critical. However, recent large Vision-Language Models (VLMs), despite their
broad applicability, remain underexplored in their ability to adapt to
individual users. In this paper, we introduce MMPB, the first extensive
benchmark for evaluating VLMs on personalization. MMPB comprises 10k
image-query pairs and includes 111 personalizable concepts across four
categories: humans, animals, objects, and characters, with the human category
enriched with preference-grounded queries. We structure personalization into
three main task types, each highlighting a different key property of VLMs.
Using 23 widely used VLMs including both open- and closed-source models, we
evaluate personalization performance via a three-stage protocol: concept
injection, multi-turn dialogue, and personalized querying. Our findings
indicate that most VLMs (including some closed-source models) struggle with
personalization, particularly in maintaining consistency over dialogue,
handling user preferences, and adapting to visual cues. Our analysis reveals
that the challenges in VLM personalization (such as refusal behaviors and
long-context forgetting) highlight substantial room for improvement. By
identifying these limitations and offering a scalable benchmark, MMPB offers
valuable insights and a solid foundation for future research toward truly
personalized multi-modal AI. Project Page: aidaslab.github.io/MMPB
WirelessMathLM: Teaching Mathematical Reasoning for LLMs in Wireless
  Communications with Reinforcement Learning
Authors: name: Xin Li, name: Mengbing Liu, name: Yiyang Zhu, name: Wenhe Zhang, name: Li Wei, name: Jiancheng An, name: Chau Yuen
Published: 2025-09-27 09:58:03

Large language models (LLMs) excel at general mathematical reasoning but fail
catastrophically on specialized technical mathematics. In wireless
communications, where problems require precise manipulation of
information-theoretic bounds, optimization constraints, and signal processing
formulations, even state-of-the-art models struggle to achieve competent
performance. We present WirelessMathLM, demonstrating that compact models
(0.5B-7B parameters) can match or exceed much larger models through
domain-specific reinforcement learning with verifiable rewards. Our key insight
is that wireless mathematics problems possess a unique property--verifiable
correctness--that enables effective reinforcement learning without human
feedback. We construct WirelessMathBench-XL, a comprehensive benchmark of 4,027
problems from 970 papers. Using Group Relative Policy Optimization (GRPO) with
binary verification rewards, we train models directly from base checkpoints
without supervised warm-start. Our 7B model achieves 39.5% accuracy on
WirelessMathBench-XL, approaching GPT-4o (40.4%) while using about 100 times
fewer parameters than DeepSeek-R1 (671B, 57.4%). Remarkably, GRPO training
nearly doubles performance across all model scales (0.5B +11%, 3B +103%, 7B
+81%), with positive transfer to general mathematics benchmarks--our models
gain +8.4 points on average across MATH, Minerva-Math, OlympiadBench, AMC, and
AIME without any training on these tasks.
Local Success Does Not Compose: Benchmarking Large Language Models for
  Compositional Formal Verification
Authors: name: Xu Xu, name: Xin Li, name: Xingwei Qu, name: Jie Fu, name: Binhang Yuan
Published: 2025-09-27 02:33:08

We introduce DafnyCOMP, a benchmark for evaluating large language models
(LLMs) on compositional specification generation in Dafny. Unlike prior
benchmarks that focus on single-function tasks, DafnyCOMP targets programs
composed of multiple interacting functions with data dependencies, requiring
reasoning across component boundaries. The benchmark consists of 300
automatically synthesized multi-function programs. We evaluate several
state-of-the-art LLM families and find that, while they perform well on
single-function verification, their performance drops sharply on compositional
tasks. Analysis reveals systematic failures in cross-functional reasoning,
including fragile specifications, misalignment between implementations and
proofs, and unstable reasoning. DafnyCOMP thus provides a diagnostic tool for
measuring progress toward reliable, verifiable, and compositional code
generation with LLMs.
Detecting Corpus-Level Knowledge Inconsistencies in Wikipedia with Large
  Language Models
Authors: name: Sina J. Semnani, name: Jirayu Burapacheep, name: Arpandeep Khatua, name: Thanawan Atchariyachanvanit, name: Zheng Wang, name: Monica S. Lam
Published: 2025-09-27 10:32:41

Wikipedia is the largest open knowledge corpus, widely used worldwide and
serving as a key resource for training large language models (LLMs) and
retrieval-augmented generation (RAG) systems. Ensuring its accuracy is
therefore critical. But how accurate is Wikipedia, and how can we improve it?
  We focus on inconsistencies, a specific type of factual inaccuracy, and
introduce the task of corpus-level inconsistency detection. We present CLAIRE,
an agentic system that combines LLM reasoning with retrieval to surface
potentially inconsistent claims along with contextual evidence for human
review. In a user study with experienced Wikipedia editors, 87.5% reported
higher confidence when using CLAIRE, and participants identified 64.7% more
inconsistencies in the same amount of time.
  Combining CLAIRE with human annotation, we contribute WIKICOLLIDE, the first
benchmark of real Wikipedia inconsistencies. Using random sampling with
CLAIRE-assisted analysis, we find that at least 3.3% of English Wikipedia facts
contradict another fact, with inconsistencies propagating into 7.3% of FEVEROUS
and 4.0% of AmbigQA examples. Benchmarking strong baselines on this dataset
reveals substantial headroom: the best fully automated system achieves an AUROC
of only 75.1%.
  Our results show that contradictions are a measurable component of Wikipedia
and that LLM-based systems like CLAIRE can provide a practical tool to help
editors improve knowledge consistency at scale.