Orionnnn/hub-stats huggingface dataset:
OpenTSLM: Time-Series Language Models for Reasoning over Multivariate
  Medical Text- and Time-Series Data
Authors: name: Patrick Langer, name: Thomas Kaar, name: Max Rosenblattl, name: Maxwell A. Xu, name: Winnie Chow, name: Martin Maritsch, name: Aradhana Verma, name: Brian Han, name: Daniel Seung Kim, name: Henry Chubb, name: Scott Ceresnak, name: Aydin Zahedivash, name: Alexander Tarlochan Singh Sandhu, name: Fatima Rodriguez, name: Daniel McDuff, name: Elgar Fleisch, name: Oliver Aalami, name: Filipe Barata, name: Paul Schmiedmayer
Published: 2025-10-02 09:58:23

LLMs have emerged as powerful tools for interpreting multimodal data. In
medicine, they hold particular promise for synthesizing large volumes of
clinical information into actionable insights and digital health applications.
Yet, a major limitation remains their inability to handle time series. To
overcome this gap, we present OpenTSLM, a family of Time Series Language Models
(TSLMs) created by integrating time series as a native modality to pretrained
LLMs, enabling reasoning over multiple time series of any length. We
investigate two architectures for OpenTSLM. The first, OpenTSLM-SoftPrompt,
models time series implicitly by concatenating learnable time series tokens
with text tokens via soft prompting. Although parameter-efficient, we
hypothesize that explicit time series modeling scales better and outperforms
implicit approaches. We thus introduce OpenTSLM-Flamingo, which integrates time
series with text via cross-attention. We benchmark both variants against
baselines that treat time series as text tokens or plots, across a suite of
text-time-series Chain-of-Thought (CoT) reasoning tasks. We introduce three
datasets: HAR-CoT, Sleep-CoT, and ECG-QA-CoT. Across all, OpenTSLM models
outperform baselines, reaching 69.9 F1 in sleep staging and 65.4 in HAR,
compared to 9.05 and 52.2 for finetuned text-only models. Notably, even
1B-parameter OpenTSLM models surpass GPT-4o (15.47 and 2.95). OpenTSLM-Flamingo
matches OpenTSLM-SoftPrompt in performance and outperforms on longer sequences,
while maintaining stable memory requirements. By contrast, SoftPrompt grows
exponentially in memory with sequence length, requiring around 110 GB compared
to 40 GB VRAM when training on ECG-QA with LLaMA-3B. Expert reviews by
clinicians find strong reasoning capabilities exhibited by OpenTSLMs on ECG-QA.
To facilitate further research, we provide all code, datasets, and models
open-source.
Less LLM, More Documents: Searching for Improved RAG
Authors: name: Jingjie Ning, name: Yibo Kong, name: Yunfan Long, name: Jamie Callan
Published: 2025-10-03 01:26:13

Retrieval-Augmented Generation (RAG) couples document retrieval with large
language models (LLMs). While scaling generators improves accuracy, it also
raises cost and limits deployability. We explore an orthogonal axis: enlarging
the retriever's corpus to reduce reliance on large LLMs. Experimental results
show that corpus scaling consistently strengthens RAG and can often serve as a
substitute for increasing model size, though with diminishing returns at larger
scales. Small- and mid-sized generators paired with larger corpora often rival
much larger models with smaller corpora; mid-sized models tend to gain the
most, while tiny and large models benefit less. Our analysis shows that
improvements arise primarily from increased coverage of answer-bearing
passages, while utilization efficiency remains largely unchanged. These
findings establish a principled corpus-generator trade-off: investing in larger
corpora offers an effective path to stronger RAG, often comparable to enlarging
the LLM itself.
Personalized Reasoning: Just-In-Time Personalization and Why LLMs Fail
  At It
Authors: name: Shuyue Stella Li, name: Avinandan Bose, name: Faeze Brahman, name: Simon Shaolei Du, name: Pang Wei Koh, name: Maryam Fazel, name: Yulia Tsvetkov
Published: 2025-09-30 18:55:28

Current large language model (LLM) development treats task-solving and
preference alignment as separate challenges, optimizing first for objective
correctness, then for alignment to aggregated human preferences. This paradigm
fails in human-facing applications where solving a problem correctly is
insufficient if the response mismatches the user's needs. This challenge
intensifies in just-in-time scenarios where no prior user interaction history
exists due to cold-start conditions or privacy constraints. LLMs need to
identify what they don't know about user preferences, strategically elicit
preference values through questioning, then adapt their reasoning processes and
responses accordingly -- a complicated chain of cognitive processes which we
term personalized reasoning. We introduce PREFDISCO, an evaluation methodology
that transforms static benchmarks into interactive personalization tasks using
psychologically-grounded personas with sparse preferences. Our framework
creates scenarios where identical questions require different reasoning chains
depending on user context, as optimal explanation approaches vary by individual
expertise and preferences while maintaining factual accuracy. Evaluation of 21
frontier models across 10 tasks reveals 29.0% of naive personalization attempts
produce worse preference alignment than generic responses, yet generic
responses also fail to serve individual user needs effectively. These findings
suggest personalized reasoning requires dedicated development rather than
emerging naturally. PREFDISCO establishes personalized reasoning as a
measurable research frontier and reveals fundamental limitations in current
LLMs' interactive capabilities, providing a foundation for developing systems
that can adapt to individual users in education, healthcare, and technical
domains where personalization is critical.
SoundReactor: Frame-level Online Video-to-Audio Generation
Authors: name: Koichi Saito, name: Julian Tanke, name: Christian Simon, name: Masato Ishii, name: Kazuki Shimada, name: Zachary Novack, name: Zhi Zhong, name: Akio Hayakawa, name: Takashi Shibuya, name: Yuki Mitsufuji
Published: 2025-10-02 15:18:00

Prevailing Video-to-Audio (V2A) generation models operate offline, assuming
an entire video sequence or chunks of frames are available beforehand. This
critically limits their use in interactive applications such as live content
creation and emerging generative world models. To address this gap, we
introduce the novel task of frame-level online V2A generation, where a model
autoregressively generates audio from video without access to future video
frames. Furthermore, we propose SoundReactor, which, to the best of our
knowledge, is the first simple yet effective framework explicitly tailored for
this task. Our design enforces end-to-end causality and targets low per-frame
latency with audio-visual synchronization. Our model's backbone is a
decoder-only causal transformer over continuous audio latents. For vision
conditioning, it leverages grid (patch) features extracted from the smallest
variant of the DINOv2 vision encoder, which are aggregated into a single token
per frame to maintain end-to-end causality and efficiency. The model is trained
through a diffusion pre-training followed by consistency fine-tuning to
accelerate the diffusion head decoding. On a benchmark of diverse gameplay
videos from AAA titles, our model successfully generates semantically and
temporally aligned, high-quality full-band stereo audio, validated by both
objective and human evaluations. Furthermore, our model achieves low per-frame
waveform-level latency (26.3ms with the head NFE=1, 31.5ms with NFE=4) on
30FPS, 480p videos using a single H100. Demo samples are available at
https://koichi-saito-sony.github.io/soundreactor/.
Continuously Augmented Discrete Diffusion model for Categorical
  Generative Modeling
Authors: name: Huangjie Zheng, name: Shansan Gong, name: Ruixiang Zhang, name: Tianrong Chen, name: Jiatao Gu, name: Mingyuan Zhou, name: Navdeep Jaitly, name: Yizhe Zhang
Published: 2025-10-01 18:00:56

Standard discrete diffusion models treat all unobserved states identically by
mapping them to an absorbing [MASK] token. This creates an 'information void'
where semantic information that could be inferred from unmasked tokens is lost
between denoising steps. We introduce Continuously Augmented Discrete Diffusion
(CADD), a framework that augments the discrete state space with a paired
diffusion in a continuous latent space. This yields graded, gradually corrupted
states in which masked tokens are represented by noisy yet informative latent
vectors rather than collapsed 'information voids'. At each reverse step, CADD
may leverage the continuous latent as a semantic hint to guide discrete
denoising. The design is clean and compatible with existing discrete diffusion
training. At sampling time, the strength and choice of estimator for the
continuous latent vector enables a controlled trade-off between mode-coverage
(generating diverse outputs) and mode-seeking (generating contextually precise
outputs) behaviors. Empirically, we demonstrate CADD improves generative
quality over mask-based diffusion across text generation, image synthesis, and
code modeling, with consistent gains on both qualitative and quantitative
metrics against strong discrete baselines.
Dale meets Langevin: A Multiplicative Denoising Diffusion Model
Authors: name: Nishanth Shetty, name: Madhava Prasath, name: Chandra Sekhar Seelamantula
Published: 2025-10-03 05:23:33

Gradient descent has proven to be a powerful and effective technique for
optimization in numerous machine learning applications. Recent advances in
computational neuroscience have shown that learning in standard gradient
descent optimization formulation is not consistent with learning in biological
systems. This has opened up interesting avenues for building biologically
inspired learning techniques. One such approach is inspired by Dale's law,
which states that inhibitory and excitatory synapses do not swap roles during
the course of learning. The resulting exponential gradient descent optimization
scheme leads to log-normally distributed synaptic weights. Interestingly, the
density that satisfies the Fokker-Planck equation corresponding to the
stochastic differential equation (SDE) with geometric Brownian motion (GBM) is
the log-normal density. Leveraging this connection, we start with the SDE
governing geometric Brownian motion, and show that discretizing the
corresponding reverse-time SDE yields a multiplicative update rule, which
surprisingly, coincides with the sampling equivalent of the exponential
gradient descent update founded on Dale's law. Furthermore, we propose a new
formalism for multiplicative denoising score-matching, subsuming the loss
function proposed by Hyvaerinen for non-negative data. Indeed, log-normally
distributed data is positive and the proposed score-matching formalism turns
out to be a natural fit. This allows for training of score-based models for
image data and results in a novel multiplicative update scheme for sample
generation starting from a log-normal density. Experimental results on MNIST,
Fashion MNIST, and Kuzushiji datasets demonstrate generative capability of the
new scheme. To the best of our knowledge, this is the first instance of a
biologically inspired generative model employing multiplicative updates,
founded on geometric Brownian motion.
Pretraining with hierarchical memories: separating long-tail and common
  knowledge
Authors: name: Hadi Pouransari, name: David Grangier, name: C Thomas, name: Michael Kirchhof, name: Oncel Tuzel
Published: 2025-09-29 17:59:50

The impressive performance gains of modern language models currently rely on
scaling parameters: larger models store more world knowledge and reason better.
Yet compressing all world knowledge into parameters is unnecessary, as only a
fraction is used per prompt, and impractical for edge devices with limited
inference-time memory and compute. We address this shortcoming by a
memory-augmented architecture and a pretraining strategy aligned with existing
hardware paradigms. We introduce small language models that access large
hierarchical parametric memory banks encoding world knowledge. During
pretraining and inference, we fetch a small, context-dependent memory block and
add it to the model. Our pretraining learns to store long-tail world knowledge
in the memory parameters, while the small language model acts as an anchor
capturing common knowledge and general reasoning abilities. Through
trillion-token-scale experiments, we show significant gains: a 160M-parameters
model augmented with an 18M-parameters memory fetched from a 4.6B memory bank
obtains comparable performance to a regular model with more than 2x the
parameters. Through extensive experiments, we study the optimal type and size
of parametric memories in transformers, scaling them to over 21B parameters. We
find that our proposed hierarchical feed-forward memories work robustly across
transformer architectures, whether added during pretraining or post-hoc.
Consolidating Reinforcement Learning for Multimodal Discrete Diffusion
  Models
Authors: name: Tianren Ma, name: Mu Zhang, name: Yibing Wang, name: Qixiang Ye
Published: 2025-10-03 10:36:24

Optimizing discrete diffusion model (DDM) with rewards remains a challenge:
the non-autoregressive paradigm makes importance sampling intractable and
rollout complex, puzzling reinforcement learning methods such as Group Relative
Policy Optimization (GRPO). In this study, we introduce MaskGRPO, the first
viable approach to enable scalable multimodal reinforcement learning in
discrete diffusion with effective importance sampling and modality-specific
adaptations. To this end, we first clarify the theoretical foundation for DDMs,
which facilitates building an importance estimator that captures valuable token
fluctuation for gradient updates. We then delicately tailored the rollout
method for visual sequences, which yields diverse completions and reliable
optimization gradients. Upon math reasoning, coding, and visual generation
benchmarks, MaskGRPO brings more stable and efficient updates, leading to
stronger reasoning performance and better generation quality. This study
establishes MaskGRPO as a systematic policy optimization approach and the first
practical way for discretized visual diffusion.
Bridging the Gap Between Promise and Performance for Microscaling FP4
  Quantization
Authors: name: Vage Egiazarian, name: Roberto L. Castro, name: Denis Kuznedelev, name: Andrei Panferov, name: Eldar Kurtic, name: Shubhra Pandit, name: Alexandre Marques, name: Mark Kurtz, name: Saleh Ashkboos, name: Torsten Hoefler, name: Dan Alistarh
Published: 2025-09-27 09:22:21

The recent hardware-accelerated microscaling 4-bit floating-point formats
such as MXFP4 and NVFP4, supported on NVIDIA and AMD GPUs, promise to
revolutionize large language model (LLM) inference. Yet, their practical
benefits remain unproven. We present the first comprehensive study of MXFP4 and
NVFP4 for post-training quantization, revealing gaps between their promise and
real-world performance. Our analysis shows that state-of-the-art methods
struggle with FP4, due to two key issues: (1) NVFP4's small group size provably
neutralizes traditional outlier mitigation techniques; (2) MXFP4's power-of-two
scale quantization severely degrades accuracy due to high induced error. To
bridge this gap, we introduce Micro-Rotated-GPTQ (MR-GPTQ), a variant of the
classic GPTQ quantization algorithm that tailors the quantization process to
FP4's unique properties, by using block-wise Hadamard transforms and
format-specific optimizations. We support our proposal with a set of
high-performance GPU kernels that enable the MR-GPTQ format with negligible
overhead, by rotation fusion into the weights, and fast online computation of
the activations. This leads to speedups vs. FP16 of up to 3.6x layer-wise, and
2.2x end-to-end on NVIDIA B200, and of 6x layer-wise and 4x end-to-end on
RTX5090. Our extensive empirical evaluation demonstrates that MR-GPTQ matches
or outperforms state-of-the-art accuracy, significantly boosting MXFP4, to the
point where it nears that of NVFP4. We conclude that, while FP4 is not an
automatic upgrade over INT4, format-specialized methods like MR-GPTQ can unlock
a new frontier of accuracy-performance trade-offs.
Large Reasoning Models Learn Better Alignment from Flawed Thinking
Authors: name: ShengYun Peng, name: Eric Smith, name: Ivan Evtimov, name: Song Jiang, name: Pin-Yu Chen, name: Hongyuan Zhan, name: Haozhu Wang, name: Duen Horng Chau, name: Mahesh Pasupuleti, name: Jianfeng Chi
Published: 2025-10-01 14:15:43

Large reasoning models (LRMs) "think" by generating structured
chain-of-thought (CoT) before producing a final answer, yet they still lack the
ability to reason critically about safety alignment and are easily biased when
a flawed premise is injected into their thought process. We propose RECAP
(Robust Safety Alignment via Counter-Aligned Prefilling), a principled
reinforcement learning (RL) method for post-training that explicitly teaches
models to override flawed reasoning trajectories and reroute to safe and
helpful responses. RECAP trains on a mixture of synthetically generated
counter-aligned CoT prefills and standard prompts, requires no additional
training cost or modifications beyond vanilla reinforcement learning from human
feedback (RLHF), and substantially improves safety and jailbreak robustness,
reduces overrefusal, and preserves core reasoning capability -- all while
maintaining inference token budget. Extensive analysis shows that RECAP-trained
models engage in self-reflection more frequently and remain robust under
adaptive attacks, preserving safety even after repeated attempts to override
their reasoning.
Game-Time: Evaluating Temporal Dynamics in Spoken Language Models
Authors: name: Kai-Wei Chang, name: En-Pei Hu, name: Chun-Yi Kuan, name: Wenze Ren, name: Wei-Chih Chen, name: Guan-Ting Lin, name: Yu Tsao, name: Shao-Hua Sun, name: Hung-yi Lee, name: James Glass
Published: 2025-09-30 15:23:39

Conversational Spoken Language Models (SLMs) are emerging as a promising
paradigm for real-time speech interaction. However, their capacity of temporal
dynamics, including the ability to manage timing, tempo and simultaneous
speaking, remains a critical and unevaluated challenge for conversational
fluency. To address this gap, we introduce the Game-Time Benchmark, a framework
to systematically assess these temporal capabilities. Inspired by how humans
learn a language through language activities, Game-Time consists of basic
instruction-following tasks and advanced tasks with temporal constraints, such
as tempo adherence and synchronized responses. Our evaluation of diverse SLM
architectures reveals a clear performance disparity: while state-of-the-art
models handle basic tasks well, many contemporary systems still struggle with
fundamental instruction-following. More critically, nearly all models degrade
substantially under temporal constraints, exposing persistent weaknesses in
time awareness and full-duplex interaction. The Game-Time Benchmark provides a
foundation for guiding future research toward more temporally-aware
conversational AI. Demos and datasets are available on our project website
https://ga642381.github.io/Game-Time.
TalkPlay-Tools: Conversational Music Recommendation with LLM Tool
  Calling
Authors: name: Seungheon Doh, name: Keunwoo Choi, name: Juhan Nam
Published: 2025-10-02 06:08:54

While the recent developments in large language models (LLMs) have
successfully enabled generative recommenders with natural language
interactions, their recommendation behavior is limited, leaving other simpler
yet crucial components such as metadata or attribute filtering underutilized in
the system. We propose an LLM-based music recommendation system with tool
calling to serve as a unified retrieval-reranking pipeline. Our system
positions an LLM as an end-to-end recommendation system that interprets user
intent, plans tool invocations, and orchestrates specialized components:
boolean filters (SQL), sparse retrieval (BM25), dense retrieval (embedding
similarity), and generative retrieval (semantic IDs). Through tool planning,
the system predicts which types of tools to use, their execution order, and the
arguments needed to find music matching user preferences, supporting diverse
modalities while seamlessly integrating multiple database filtering methods. We
demonstrate that this unified tool-calling framework achieves competitive
performance across diverse recommendation scenarios by selectively employing
appropriate retrieval methods based on user queries, envisioning a new paradigm
for conversational music recommendation systems.
DiffTester: Accelerating Unit Test Generation for Diffusion LLMs via
  Repetitive Pattern
Authors: name: Lekang Yang, name: Yuetong Liu, name: Yitong Zhang, name: Jia Li
Published: 2025-09-29 16:04:18

Software development relies heavily on extensive unit testing, which makes
the efficiency of automated Unit Test Generation (UTG) particularly important.
However, most existing LLMs generate test cases one token at a time in each
forward pass, which leads to inefficient UTG. Recently, diffusion LLMs (dLLMs)
have emerged, offering promising parallel generation capabilities and showing
strong potential for efficient UTG. Despite this advantage, their application
to UTG is still constrained by a clear trade-off between efficiency and test
quality, since increasing the number of tokens generated in each step often
causes a sharp decline in the quality of test cases. To overcome this
limitation, we present DiffTester, an acceleration framework specifically
tailored for dLLMs in UTG. The key idea of DiffTester is that unit tests
targeting the same focal method often share repetitive structural patterns. By
dynamically identifying these common patterns through abstract syntax tree
analysis during generation, DiffTester adaptively increases the number of
tokens produced at each step without compromising the quality of the output. To
enable comprehensive evaluation, we extend the original TestEval benchmark,
which was limited to Python, by introducing additional programming languages
including Java and C++. Extensive experiments on three benchmarks with two
representative models show that DiffTester delivers significant acceleration
while preserving test coverage. Moreover, DiffTester generalizes well across
different dLLMs and programming languages, providing a practical and scalable
solution for efficient UTG in software development. Code and data are publicly
available at https://github.com/wellbeingyang/DLM4UTG-open .
Scaling Policy Compliance Assessment in Language Models with Policy
  Reasoning Traces
Authors: name: Joseph Marvin Imperial, name: Harish Tayyar Madabushi
Published: 2025-09-27 13:10:21

Policy compliance assessment is a fundamental task of evaluating whether an
input case strictly complies with a set of human-defined rules, more generally
known as policies. In practice, human experts follow a systematic, step-by-step
process to identify violations with respect to specific stipulations outlined
in the policy. However, such documentation of gold-standard, expert-level
reasoning processes is costly to acquire. In this paper, we introduce Policy
Reasoning Traces (PRT), a form of specialized generated reasoning chains that
serve as a reasoning bridge to improve an LLM's policy compliance assessment
capabilities. Our empirical evaluations demonstrate that the use of PRTs for
both inference-time and training-time scenarios significantly enhances the
performance of open-weight and commercial models, setting a new
state-of-the-art for HIPAA and GDPR policies. Beyond accuracy gains, we also
highlight how PRTs can improve an LLM's ability to accurately cite policy
clauses, as well as influence compliance decisions through their high
utilization from the raw chains of thought.