Orionnnn/hub-stats huggingface dataset:
Improving Inference-Time Optimisation for Vocal Effects Style Transfer
  with a Gaussian Prior
Authors: name: Chin-Yun Yu, name: Marco A. Martínez-Ramírez, name: Junghyun Koo, name: Wei-Hsiang Liao, name: Yuki Mitsufuji, name: György Fazekas
Published: 2025-05-16 14:40:31

Style Transfer with Inference-Time Optimisation (ST-ITO) is a recent approach
for transferring the applied effects of a reference audio to a raw audio track.
It optimises the effect parameters to minimise the distance between the style
embeddings of the processed audio and the reference. However, this method
treats all possible configurations equally and relies solely on the embedding
space, which can lead to unrealistic or biased results. We address this pitfall
by introducing a Gaussian prior derived from a vocal preset dataset, DiffVox,
over the parameter space. The resulting optimisation is equivalent to
maximum-a-posteriori estimation. Evaluations on vocal effects transfer on the
MedleyDB dataset show significant improvements across metrics compared to
baselines, including a blind audio effects estimator, nearest-neighbour
approaches, and uncalibrated ST-ITO. The proposed calibration reduces parameter
mean squared error by up to 33% and matches the reference style better.
Subjective evaluations with 16 participants confirm our method's superiority,
especially in limited data regimes. This work demonstrates how incorporating
prior knowledge in inference time enhances audio effects transfer, paving the
way for more effective and realistic audio processing systems.
CheXGenBench: A Unified Benchmark For Fidelity, Privacy and Utility of
  Synthetic Chest Radiographs
Authors: name: Raman Dutt, name: Pedro Sanchez, name: Yongchen Yao, name: Steven McDonagh, name: Sotirios A. Tsaftaris, name: Timothy Hospedales
Published: 2025-05-15 16:59:17

We introduce CheXGenBench, a rigorous and multifaceted evaluation framework
for synthetic chest radiograph generation that simultaneously assesses
fidelity, privacy risks, and clinical utility across state-of-the-art
text-to-image generative models. Despite rapid advancements in generative AI
for real-world imagery, medical domain evaluations have been hindered by
methodological inconsistencies, outdated architectural comparisons, and
disconnected assessment criteria that rarely address the practical clinical
value of synthetic samples. CheXGenBench overcomes these limitations through
standardised data partitioning and a unified evaluation protocol comprising
over 20 quantitative metrics that systematically analyse generation quality,
potential privacy vulnerabilities, and downstream clinical applicability across
11 leading text-to-image architectures. Our results reveal critical
inefficiencies in the existing evaluation protocols, particularly in assessing
generative fidelity, leading to inconsistent and uninformative comparisons. Our
framework establishes a standardised benchmark for the medical AI community,
enabling objective and reproducible comparisons while facilitating seamless
integration of both existing and future generative models. Additionally, we
release a high-quality, synthetic dataset, SynthCheX-75K, comprising 75K
radiographs generated by the top-performing model (Sana 0.6B) in our benchmark
to support further research in this critical domain. Through CheXGenBench, we
establish a new state-of-the-art and release our framework, models, and
SynthCheX-75K dataset at https://raman1121.github.io/CheXGenBench/
From Trade-off to Synergy: A Versatile Symbiotic Watermarking Framework
  for Large Language Models
Authors: name: Yidan Wang, name: Yubing Ren, name: Yanan Cao, name: Binxing Fang
Published: 2025-05-15 03:12:36

The rise of Large Language Models (LLMs) has heightened concerns about the
misuse of AI-generated text, making watermarking a promising solution.
Mainstream watermarking schemes for LLMs fall into two categories: logits-based
and sampling-based. However, current schemes entail trade-offs among
robustness, text quality, and security. To mitigate this, we integrate
logits-based and sampling-based schemes, harnessing their respective strengths
to achieve synergy. In this paper, we propose a versatile symbiotic
watermarking framework with three strategies: serial, parallel, and hybrid. The
hybrid framework adaptively embeds watermarks using token entropy and semantic
entropy, optimizing the balance between detectability, robustness, text
quality, and security. Furthermore, we validate our approach through
comprehensive experiments on various datasets and models. Experimental results
indicate that our method outperforms existing baselines and achieves
state-of-the-art (SOTA) performance. We believe this framework provides novel
insights into diverse watermarking paradigms. Our code is available at
https://github.com/redwyd/SymMark{https://github.com/redwyd/SymMark}.
MatTools: Benchmarking Large Language Models for Materials Science Tools
Authors: name: Siyu Liu, name: Jiamin Xu, name: Beilin Ye, name: Bo Hu, name: David J. Srolovitz, name: Tongqi Wen
Published: 2025-05-16 04:43:05

Large language models (LLMs) are increasingly applied to materials science
questions, including literature comprehension, property prediction, materials
discovery and alloy design. At the same time, a wide range of physics-based
computational approaches have been developed in which materials properties can
be calculated. Here, we propose a benchmark application to evaluate the
proficiency of LLMs to answer materials science questions through the
generation and safe execution of codes based on such physics-based
computational materials science packages. MatTools is built on two
complementary components: a materials simulation tool question-answer (QA)
benchmark and a real-world tool-usage benchmark. We designed an automated
methodology to efficiently collect real-world materials science tool-use
examples. The QA benchmark, derived from the pymatgen (Python Materials
Genomics) codebase and documentation, comprises 69,225 QA pairs that assess the
ability of an LLM to understand materials science tools. The real-world
benchmark contains 49 tasks (138 subtasks) requiring the generation of
functional Python code for materials property calculations. Our evaluation of
diverse LLMs yields three key insights: (1)Generalists outshine
specialists;(2)AI knows AI; and (3)Simpler is better. MatTools provides a
standardized framework for assessing and improving LLM capabilities for
materials science tool applications, facilitating the development of more
effective AI systems for materials science and general scientific research.
MMLongBench: Benchmarking Long-Context Vision-Language Models
  Effectively and Thoroughly
Authors: name: Zhaowei Wang, name: Wenhao Yu, name: Xiyu Ren, name: Jipeng Zhang, name: Yu Zhao, name: Rohit Saxena, name: Liang Cheng, name: Ginny Wong, name: Simon See, name: Pasquale Minervini, name: Yangqiu Song, name: Mark Steedman
Published: 2025-05-15 17:52:54

The rapid extension of context windows in large vision-language models has
given rise to long-context vision-language models (LCVLMs), which are capable
of handling hundreds of images with interleaved text tokens in a single forward
pass. In this work, we introduce MMLongBench, the first benchmark covering a
diverse set of long-context vision-language tasks, to evaluate LCVLMs
effectively and thoroughly. MMLongBench is composed of 13,331 examples spanning
five different categories of downstream tasks, such as Visual RAG and Many-Shot
ICL. It also provides broad coverage of image types, including various natural
and synthetic images. To assess the robustness of the models to different input
lengths, all examples are delivered at five standardized input lengths (8K-128K
tokens) via a cross-modal tokenization scheme that combines vision patches and
text tokens. Through a thorough benchmarking of 46 closed-source and
open-source LCVLMs, we provide a comprehensive analysis of the current models'
vision-language long-context ability. Our results show that: i) performance on
a single task is a weak proxy for overall long-context capability; ii) both
closed-source and open-source models face challenges in long-context
vision-language tasks, indicating substantial room for future improvement; iii)
models with stronger reasoning ability tend to exhibit better long-context
performance. By offering wide task coverage, various image types, and rigorous
length control, MMLongBench provides the missing foundation for diagnosing and
advancing the next generation of LCVLMs.
Multi-Token Prediction Needs Registers
Authors: name: Anastasios Gerontopoulos, name: Spyros Gidaris, name: Nikos Komodakis
Published: 2025-05-15 17:25:03

Multi-token prediction has emerged as a promising objective for improving
language model pretraining, but its benefits have not consistently generalized
to other settings such as fine-tuning. In this paper, we propose MuToR, a
simple and effective approach to multi-token prediction that interleaves
learnable register tokens into the input sequence, each tasked with predicting
future targets. Compared to existing methods, MuToR offers several key
advantages: it introduces only a negligible number of additional parameters,
requires no architectural changes--ensuring compatibility with off-the-shelf
pretrained language models--and remains aligned with the next-token pretraining
objective, making it especially well-suited for supervised fine-tuning.
Moreover, it naturally supports scalable prediction horizons. We demonstrate
the effectiveness and versatility of MuToR across a range of use cases,
including supervised fine-tuning, parameter-efficient fine-tuning (PEFT), and
pretraining, on challenging generative tasks in both language and vision
domains. Our code will be available at: https://github.com/nasosger/MuToR.
GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoning
Authors: name: Yue Liu, name: Shengfang Zhai, name: Mingzhe Du, name: Yulin Chen, name: Tri Cao, name: Hongcheng Gao, name: Cheng Wang, name: Xinfeng Li, name: Kun Wang, name: Junfeng Fang, name: Jiaheng Zhang, name: Bryan Hooi
Published: 2025-05-16 09:46:10

To enhance the safety of VLMs, this paper introduces a novel reasoning-based
VLM guard model dubbed GuardReasoner-VL. The core idea is to incentivize the
guard model to deliberatively reason before making moderation decisions via
online RL. First, we construct GuardReasoner-VLTrain, a reasoning corpus with
123K samples and 631K reasoning steps, spanning text, image, and text-image
inputs. Then, based on it, we cold-start our model's reasoning ability via SFT.
In addition, we further enhance reasoning regarding moderation through online
RL. Concretely, to enhance diversity and difficulty of samples, we conduct
rejection sampling followed by data augmentation via the proposed safety-aware
data concatenation. Besides, we use a dynamic clipping parameter to encourage
exploration in early stages and exploitation in later stages. To balance
performance and token efficiency, we design a length-aware safety reward that
integrates accuracy, format, and token cost. Extensive experiments demonstrate
the superiority of our model. Remarkably, it surpasses the runner-up by 19.27%
F1 score on average. We release data, code, and models (3B/7B) of
GuardReasoner-VL at https://github.com/yueliu1999/GuardReasoner-VL/
Simple Semi-supervised Knowledge Distillation from Vision-Language
  Models via texttt{D}ual-texttt{H}ead
  texttt{O}ptimization
Authors: name: Seongjae Kang, name: Dong Bok Lee, name: Hyungjoon Jang, name: Sung Ju Hwang
Published: 2025-05-12 15:39:51

Vision-language models (VLMs) have achieved remarkable success across diverse
tasks by leveraging rich textual information with minimal labeled data.
However, deploying such large models remains challenging, particularly in
resource-constrained environments. Knowledge distillation (KD) offers a
well-established solution to this problem; however, recent KD approaches from
VLMs often involve multi-stage training or additional tuning, increasing
computational overhead and optimization complexity. In this paper, we propose
texttt{D}ual-texttt{H}ead
texttt{O}ptimization (texttt{DHO}) -- a simple yet
effective KD framework that transfers knowledge from VLMs to compact,
task-specific models in semi-supervised settings. Specifically, we introduce
dual prediction heads that independently learn from labeled data and teacher
predictions, and propose to linearly combine their outputs during inference. We
observe that DHO mitigates gradient conflicts between supervised and
distillation signals, enabling more effective feature learning than single-head
KD baselines. As a result, extensive experiments show that DHO
consistently outperforms baselines across multiple domains and fine-grained
datasets. Notably, on ImageNet, it achieves state-of-the-art performance,
improving accuracy by 3% and 0.1% with 1% and 10% labeled data, respectively,
while using fewer parameters.
Humans expect rationality and cooperation from LLM opponents in
  strategic games
Authors: name: Darija Barak, name: Miguel Costa-Gomes
Published: 2025-05-16 09:01:09

As Large Language Models (LLMs) integrate into our social and economic
interactions, we need to deepen our understanding of how humans respond to LLMs
opponents in strategic settings. We present the results of the first controlled
monetarily-incentivised laboratory experiment looking at differences in human
behaviour in a multi-player p-beauty contest against other humans and LLMs. We
use a within-subject design in order to compare behaviour at the individual
level. We show that, in this environment, human subjects choose significantly
lower numbers when playing against LLMs than humans, which is mainly driven by
the increased prevalence of `zero' Nash-equilibrium choices. This shift is
mainly driven by subjects with high strategic reasoning ability. Subjects who
play the zero Nash-equilibrium choice motivate their strategy by appealing to
perceived LLM's reasoning ability and, unexpectedly, propensity towards
cooperation. Our findings provide foundational insights into the multi-player
human-LLM interaction in simultaneous choice games, uncover heterogeneities in
both subjects' behaviour and beliefs about LLM's play when playing against
them, and suggest important implications for mechanism design in mixed
human-LLM systems.
Group Think: Multiple Concurrent Reasoning Agents Collaborating at Token
  Level Granularity
Authors: name: Chan-Jan Hsu, name: Davide Buffelli, name: Jamie McGowan, name: Feng-Ting Liao, name: Yi-Chang Chen, name: Sattar Vakili, name: Da-shan Shiu
Published: 2025-05-16 10:40:35

Recent advances in large language models (LLMs) have demonstrated the power
of reasoning through self-generated chains of thought. Multiple reasoning
agents can collaborate to raise joint reasoning quality above individual
outcomes. However, such agents typically interact in a turn-based manner,
trading increased latency for improved quality. In this paper, we propose Group
Think--a single LLM that acts as multiple concurrent reasoning agents, or
thinkers. With shared visibility into each other's partial generation progress,
Group Think introduces a new concurrent-reasoning paradigm in which multiple
reasoning trajectories adapt dynamically to one another at the token level. For
example, a reasoning thread may shift its generation mid-sentence upon
detecting that another thread is better positioned to continue. This
fine-grained, token-level collaboration enables Group Think to reduce redundant
reasoning and improve quality while achieving significantly lower latency.
Moreover, its concurrent nature allows for efficient utilization of idle
computational resources, making it especially suitable for edge inference,
where very small batch size often underutilizes local~GPUs. We give a simple
and generalizable modification that enables any existing LLM to perform Group
Think on a local GPU. We also present an evaluation strategy to benchmark
reasoning latency and empirically demonstrate latency improvements using
open-source LLMs that were not explicitly trained for Group Think. We hope this
work paves the way for future LLMs to exhibit more sophisticated and more
efficient collaborative behavior for higher quality generation.
Mergenetic: a Simple Evolutionary Model Merging Library
Authors: name: Adrian Robert Minut, name: Tommaso Mencattini, name: Andrea Santilli, name: Donato Crisostomi, name: Emanuele Rodolà
Published: 2025-05-16 16:43:23

Model merging allows combining the capabilities of existing models into a new
one - post hoc, without additional training. This has made it increasingly
popular thanks to its low cost and the availability of libraries that support
merging on consumer GPUs. Recent work shows that pairing merging with
evolutionary algorithms can boost performance, but no framework currently
supports flexible experimentation with such strategies in language models. We
introduce Mergenetic, an open-source library for evolutionary model merging.
Mergenetic enables easy composition of merging methods and evolutionary
algorithms while incorporating lightweight fitness estimators to reduce
evaluation costs. We describe its design and demonstrate that Mergenetic
produces competitive results across tasks and languages using modest hardware.
Scaling Reasoning can Improve Factuality in Large Language Models
Authors: name: Mike Zhang, name: Johannes Bjerva, name: Russa Biswas
Published: 2025-05-16 11:39:33

Recent studies on large language model (LLM) reasoning capabilities have
demonstrated promising improvements in model performance by leveraging a
lengthy thinking process and additional computational resources during
inference, primarily in tasks involving mathematical reasoning (Muennighoff et
al., 2025). However, it remains uncertain if longer reasoning chains inherently
enhance factual accuracy, particularly beyond mathematical contexts. In this
work, we thoroughly examine LLM reasoning within complex open-domain
question-answering (QA) scenarios. We initially distill reasoning traces from
advanced, large-scale reasoning models (QwQ-32B and DeepSeek-R1-671B), then
fine-tune a variety of models ranging from smaller, instruction-tuned variants
to larger architectures based on Qwen2.5. To enrich reasoning traces, we
introduce factual information from knowledge graphs in the form of paths into
our reasoning traces. Our experimental setup includes four baseline approaches
and six different instruction-tuned models evaluated across a benchmark of six
datasets, encompassing over 22.6K questions. Overall, we carry out 168
experimental runs and analyze approximately 1.7 million reasoning traces. Our
findings indicate that, within a single run, smaller reasoning models achieve
noticeable improvements in factual accuracy compared to their original
instruction-tuned counterparts. Moreover, our analysis demonstrates that adding
test-time compute and token budgets factual accuracy consistently improves by
2-8%, further confirming the effectiveness of test-time scaling for enhancing
performance and consequently improving reasoning accuracy in open-domain QA
tasks. We release all the experimental artifacts for further research.
Visual Planning: Let's Think Only with Images
Authors: name: Yi Xu, name: Chengzu Li, name: Han Zhou, name: Xingchen Wan, name: Caiqi Zhang, name: Anna Korhonen, name: Ivan Vulić
Published: 2025-05-16 16:17:22

Recent advancements in Large Language Models (LLMs) and their multimodal
extensions (MLLMs) have substantially enhanced machine reasoning across diverse
tasks. However, these models predominantly rely on pure text as the medium for
both expressing and structuring reasoning, even when visual information is
present. In this work, we argue that language may not always be the most
natural or effective modality for reasoning, particularly in tasks involving
spatial and geometrical information. Motivated by this, we propose a new
paradigm, Visual Planning, which enables planning through purely visual
representations, independent of text. In this paradigm, planning is executed
via sequences of images that encode step-by-step inference in the visual
domain, akin to how humans sketch or visualize future actions. We introduce a
novel reinforcement learning framework, Visual Planning via Reinforcement
Learning (VPRL), empowered by GRPO for post-training large vision models,
leading to substantial improvements in planning in a selection of
representative visual navigation tasks, FrozenLake, Maze, and MiniBehavior. Our
visual planning paradigm outperforms all other planning variants that conduct
reasoning in the text-only space. Our results establish Visual Planning as a
viable and promising alternative to language-based reasoning, opening new
avenues for tasks that benefit from intuitive, image-based inference.
MPS-Prover: Advancing Stepwise Theorem Proving by Multi-Perspective
  Search and Data Curation
Authors: name: Zhenwen Liang, name: Linfeng Song, name: Yang Li, name: Tao Yang, name: Feng Zhang, name: Haitao Mi, name: Dong Yu
Published: 2025-05-16 07:56:03

Automated Theorem Proving (ATP) in formal languages remains a formidable
challenge in AI, demanding rigorous logical deduction and navigating vast
search spaces. While large language models (LLMs) have shown promising
performance, existing stepwise provers often suffer from biased search
guidance, leading to inefficiencies and suboptimal proof strategies. This paper
introduces the Multi-Perspective Search Prover (MPS-Prover), a novel stepwise
ATP system designed to overcome these limitations. MPS-Prover incorporates two
key innovations: a highly effective post-training data curation strategy that
prunes approximately 40% of redundant training data without sacrificing
performance, and a multi-perspective tree search mechanism. This search
integrates a learned critic model with strategically designed heuristic rules
to diversify tactic selection, prevent getting trapped in unproductive states,
and enhance search robustness. Extensive evaluations demonstrate that
MPS-Prover achieves state-of-the-art performance on multiple challenging
benchmarks, including miniF2F and ProofNet, outperforming prior 7B parameter
models. Furthermore, our analyses reveal that MPS-Prover generates
significantly shorter and more diverse proofs compared to existing stepwise and
whole-proof methods, highlighting its efficiency and efficacy. Our work
advances the capabilities of LLM-based formal reasoning and offers a robust
framework and a comprehensive analysis for developing more powerful theorem
provers.