Orionnnn/hub-stats huggingface dataset:
Paris: A Decentralized Trained Open-Weight Diffusion Model
Authors: name: Zhiying Jiang, name: Raihan Seraj, name: Marcos Villagra, name: Bidhan Roy
Published: 2025-10-03 18:53:12

We present Paris, the first publicly released diffusion model pre-trained
entirely through decentralized computation. Paris demonstrates that
high-quality text-to-image generation can be achieved without centrally
coordinated infrastructure. Paris is open for research and commercial use.
Paris required implementing our Distributed Diffusion Training framework from
scratch. The model consists of 8 expert diffusion models (129M-605M parameters
each) trained in complete isolation with no gradient, parameter, or
intermediate activation synchronization. Rather than requiring synchronized
gradient updates across thousands of GPUs, we partition data into semantically
coherent clusters where each expert independently optimizes its subset while
collectively approximating the full distribution. A lightweight transformer
router dynamically selects appropriate experts at inference, achieving
generation quality comparable to centrally coordinated baselines. Eliminating
synchronization enables training on heterogeneous hardware without specialized
interconnects. Empirical validation confirms that Paris's decentralized
training maintains generation quality while removing the dedicated GPU cluster
requirement for large-scale diffusion models. Paris achieves this using
14times less training data and 16times less compute than the prior
decentralized baseline.
Position: Privacy Is Not Just Memorization!
Authors: name: Niloofar Mireshghallah, name: Tianshi Li
Published: 2025-10-02 04:02:06

The discourse on privacy risks in Large Language Models (LLMs) has
disproportionately focused on verbatim memorization of training data, while a
constellation of more immediate and scalable privacy threats remain
underexplored. This position paper argues that the privacy landscape of LLM
systems extends far beyond training data extraction, encompassing risks from
data collection practices, inference-time context leakage, autonomous agent
capabilities, and the democratization of surveillance through deep inference
attacks. We present a comprehensive taxonomy of privacy risks across the LLM
lifecycle -- from data collection through deployment -- and demonstrate through
case studies how current privacy frameworks fail to address these multifaceted
threats. Through a longitudinal analysis of 1,322 AI/ML privacy papers
published at leading conferences over the past decade (2016--2025), we reveal
that while memorization receives outsized attention in technical research, the
most pressing privacy harms lie elsewhere, where current technical approaches
offer little traction and viable paths forward remain unclear. We call for a
fundamental shift in how the research community approaches LLM privacy, moving
beyond the narrow focus of current technical solutions and embracing
interdisciplinary approaches that address the sociotechnical nature of these
emerging threats.
Prosperity before Collapse: How Far Can Off-Policy RL Reach with Stale
  Data on LLMs?
Authors: name: Haizhong Zheng, name: Jiawei Zhao, name: Bedi Chen
Published: 2025-10-01 17:48:23

Reinforcement learning has been central to recent advances in large language
model reasoning, but most algorithms rely on on-policy training that demands
fresh rollouts at every update, limiting efficiency and scalability.
Asynchronous RL systems alleviate this by decoupling rollout generation from
training, yet their effectiveness hinges on tolerating large staleness in
rollout data, a setting where existing methods either degrade in performance or
collapse. We revisit this challenge and uncover a prosperity-before-collapse
phenomenon: stale data can be as informative as on-policy data if exploited
properly. Building on this insight, we introduce M2PO (Second-Moment Trust
Policy Optimization), which constrains the second moment of importance weights
to suppress only extreme outliers while preserving informative updates.
Notably, M2PO sharply reduces the fraction of clipped tokens under high
staleness (from 1.22% to 0.06% over training), precisely masking high-variance
tokens while maintaining stable optimization. Extensive evaluation across six
models (from 1.7B to 32B) and eight benchmarks shows that M2PO delivers stable
off-policy training even with data stale by at least 256 model updates and
matches on-policy performance.
Slow-Fast Policy Optimization: Reposition-Before-Update for LLM
  Reasoning
Authors: name: Ziyan Wang, name: Zheng Wang, name: Jie Fu, name: Xingwei Qu, name: Qi Cheng, name: Shengpu Tang, name: Minjia Zhang, name: Xiaoming Huo
Published: 2025-10-05 07:22:54

Reinforcement learning (RL) has become central to enhancing reasoning in
large language models (LLMs). Yet on-policy algorithms such as Group Relative
Policy Optimization (GRPO) often suffer in early training: noisy gradients from
low-quality rollouts lead to unstable updates and inefficient exploration. We
introduce Slow-Fast Policy Optimization (SFPO), a simple yet efficient
framework to address these limitations via decomposing each step into three
stages: a short fast trajectory of inner steps on the same batch, a reposition
mechanism to control off-policy drift, and a final slow correction. This
reposition-before-update design preserves the objective and rollout process
unchanged, making SFPO plug-compatible with existing policy-gradient pipelines.
Extensive experiments demonstrate that SFPO consistently improves stability,
reduces rollouts, and accelerates convergence of reasoning RL training.
Specifically, it outperforms GRPO by up to 2.80 points in average on math
reasoning benchmarks. It also achieves up to 4.93 fewer rollouts
and a 4.19 reduction in wall-clock time to match GRPO's best
accuracy.
Fine-Tuning on Noisy Instructions: Effects on Generalization and
  Performance
Authors: name: Ahmed Alajrami, name: Xingwei Tan, name: Nikolaos Aletras
Published: 2025-10-03 21:54:33

Instruction-tuning plays a vital role in enhancing the task-solving abilities
of large language models (LLMs), improving their usability in generating
helpful responses on various tasks. However, previous work has demonstrated
that they are sensitive to minor variations in instruction phrasing. In this
paper, we explore whether introducing perturbations in instruction-tuning data
can enhance LLMs' resistance against noisy instructions. We focus on how
instruction-tuning with perturbations, such as removing stop words or shuffling
words, affects LLMs' performance on the original and perturbed versions of
widely-used benchmarks (MMLU, BBH, GSM8K). We further assess learning dynamics
and potential shifts in model behavior. Surprisingly, our results suggest that
instruction-tuning on perturbed instructions can, in some cases, improve
downstream performance. These findings highlight the importance of including
perturbed instructions in instruction-tuning, which can make LLMs more
resilient to noisy user inputs.
MOSS-Speech: Towards True Speech-to-Speech Models Without Text Guidance
Authors: name: Xingjian Zhao, name: Zhe Xu, name: Qinyuan Cheng, name: Zhaoye Fei, name: Luozhijie Jin, name: Yang Wang, name: Hanfu Chen, name: Yaozhou Jiang, name: Qinghui Gao, name: Ke Chen, name: Ruixiao Li, name: Mingshu Chen, name: Ruiming Wang, name: Wenbo Zhang, name: Yiyang Zhang, name: Donghua Yu, name: Yang Gao, name: Xiaogui Yang, name: Yitian Gong, name: Yuanfan Xu, name: Yaqian Zhou, name: Xuanjing Huang, name: Xipeng Qiu
Published: 2025-10-01 04:32:37

Spoken dialogue systems often rely on cascaded pipelines that transcribe,
process, and resynthesize speech. While effective, this design discards
paralinguistic cues and limits expressivity. Recent end-to-end methods reduce
latency and better preserve these cues, yet still rely on text intermediates,
creating a fundamental bottleneck. We present MOSS-Speech, a true
speech-to-speech large language model that directly understands and generates
speech without relying on text guidance. Our approach combines a modality-based
layer-splitting architecture with a frozen pre-training strategy, preserving
the reasoning and knowledge of pretrained text LLMs while adding native speech
capabilities. Experiments show that our model achieves state-of-the-art results
in spoken question answering and delivers comparable speech-to-speech
performance relative to existing text-guided systems, while still maintaining
competitive text performance. By narrowing the gap between text-guided and
direct speech generation, our work establishes a new paradigm for expressive
and efficient end-to-end speech interaction.
Power Transform Revisited: Numerically Stable, and Federated
Authors: name: Xuefeng Xu, name: Graham Cormode
Published: 2025-10-06 16:32:22

Power transforms are popular parametric techniques for making data more
Gaussian-like, and are widely used as preprocessing steps in statistical
analysis and machine learning. However, we find that direct implementations of
power transforms suffer from severe numerical instabilities, which can lead to
incorrect results or even crashes. In this paper, we provide a comprehensive
analysis of the sources of these instabilities and propose effective remedies.
We further extend power transforms to the federated learning setting,
addressing both numerical and distributional challenges that arise in this
context. Experiments on real-world datasets demonstrate that our methods are
both effective and robust, substantially improving stability compared to
existing approaches.
Federated Computation of ROC and PR Curves
Authors: name: Xuefeng Xu, name: Graham Cormode
Published: 2025-10-06 16:16:46

Receiver Operating Characteristic (ROC) and Precision-Recall (PR) curves are
fundamental tools for evaluating machine learning classifiers, offering
detailed insights into the trade-offs between true positive rate vs. false
positive rate (ROC) or precision vs. recall (PR). However, in Federated
Learning (FL) scenarios, where data is distributed across multiple clients,
computing these curves is challenging due to privacy and communication
constraints. Specifically, the server cannot access raw prediction scores and
class labels, which are used to compute the ROC and PR curves in a centralized
setting. In this paper, we propose a novel method for approximating ROC and PR
curves in a federated setting by estimating quantiles of the prediction score
distribution under distributed differential privacy. We provide theoretical
bounds on the Area Error (AE) between the true and estimated curves,
demonstrating the trade-offs between approximation accuracy, privacy, and
communication cost. Empirical results on real-world datasets demonstrate that
our method achieves high approximation accuracy with minimal communication and
strong privacy guarantees, making it practical for privacy-preserving model
evaluation in federated systems.
Test-Time Scaling in Diffusion LLMs via Hidden Semi-Autoregressive
  Experts
Authors: name: Jihoon Lee, name: Hoyeon Moon, name: Kevin Zhai, name: Arun Kumar Chithanar, name: Anit Kumar Sahu, name: Soummya Kar, name: Chul Lee, name: Souradip Chakraborty, name: Amrit Singh Bedi
Published: 2025-10-06 17:16:41

Diffusion-based large language models (dLLMs) are trained flexibly to model
extreme dependence in the data distribution; however, how to best utilize this
information at inference time remains an open problem. In this work, we uncover
an interesting property of these models: dLLMs trained on textual data
implicitly learn a mixture of semi-autoregressive experts, where different
generation orders reveal different specialized behaviors. We show that
committing to any single, fixed inference time schedule, a common practice,
collapses performance by failing to leverage this latent ensemble. To address
this, we introduce HEX (Hidden semiautoregressive EXperts for test-time
scaling), a training-free inference method that ensembles across heterogeneous
block schedules. By doing a majority vote over diverse block-sized generation
paths, HEX robustly avoids failure modes associated with any single fixed
schedule. On reasoning benchmarks such as GSM8K, it boosts accuracy by up to
3.56X (from 24.72% to 88.10%), outperforming top-K margin inference and
specialized fine-tuned methods like GRPO, without additional training. HEX even
yields significant gains on MATH benchmark from 16.40% to 40.00%, scientific
reasoning on ARC-C from 54.18% to 87.80%, and TruthfulQA from 28.36% to 57.46%.
Our results establish a new paradigm for test-time scaling in diffusion-based
LLMs (dLLMs), revealing that the sequence in which masking is performed plays a
critical role in determining performance during inference.
Learning on the Job: Test-Time Curricula for Targeted Reinforcement
  Learning
Authors: name: Jonas Hübotter, name: Leander Diaz-Bone, name: Ido Hakimi, name: Andreas Krause, name: Moritz Hardt
Published: 2025-10-06 13:07:14

Humans are good at learning on the job: We learn how to solve the tasks we
face as we go along. Can a model do the same? We propose an agent that
assembles a task-specific curriculum, called test-time curriculum (TTC-RL), and
applies reinforcement learning to continue training the model for its target
task. The test-time curriculum avoids time-consuming human curation of datasets
by automatically selecting the most task-relevant data from a large pool of
available training data. Our experiments demonstrate that reinforcement
learning on a test-time curriculum consistently improves the model on its
target tasks, across a variety of evaluations and models. Notably, on
challenging math and coding benchmarks, TTC-RL improves the pass@1 of Qwen3-8B
by approximately 1.8x on AIME25 and 2.1x on CodeElo. Moreover, we find that
TTC-RL significantly raises the performance ceiling compared to the initial
model, increasing pass@8 on AIME25 from 40% to 62% and on CodeElo from 28% to
43%. Our findings show the potential of test-time curricula in extending the
test-time scaling paradigm to continual training on thousands of task-relevant
experiences during test-time.
Character Mixing for Video Generation
Authors: name: Tingting Liao, name: Chongjian Ge, name: Guangyi Liu, name: Hao Li, name: Yi Zhou
Published: 2025-10-06 17:57:39

Imagine Mr. Bean stepping into Tom and Jerry--can we generate videos where
characters interact naturally across different worlds? We study inter-character
interaction in text-to-video generation, where the key challenge is to preserve
each character's identity and behaviors while enabling coherent cross-context
interaction. This is difficult because characters may never have coexisted and
because mixing styles often causes style delusion, where realistic characters
appear cartoonish or vice versa. We introduce a framework that tackles these
issues with Cross-Character Embedding (CCE), which learns identity and
behavioral logic across multimodal sources, and Cross-Character Augmentation
(CCA), which enriches training with synthetic co-existence and mixed-style
data. Together, these techniques allow natural interactions between previously
uncoexistent characters without losing stylistic fidelity. Experiments on a
curated benchmark of cartoons and live-action series with 10 characters show
clear improvements in identity preservation, interaction quality, and
robustness to style delusion, enabling new forms of generative
storytelling.Additional results and videos are available on our project page:
https://tingtingliao.github.io/mimix/.
Optimal Scaling Needs Optimal Norm
Authors: name: Oleg Filatov, name: Jiangtao Wang, name: Jan Ebert, name: Stefan Kesselheim
Published: 2025-10-04 16:48:36

Despite recent progress in optimal hyperparameter transfer under model and
dataset scaling, no unifying explanatory principle has been established. Using
the Scion optimizer, we discover that joint optimal scaling across model and
dataset sizes is governed by a single invariant: the operator norm of the
output layer. Across models with up to 1.3B parameters trained on up to 138B
tokens, the optimal learning rate/batch size pair (eta^{ast}, B^{ast})
consistently has the same operator norm value - a phenomenon we term norm
transfer. This constant norm condition is necessary but not sufficient: while
for each dataset size, multiple (eta, B) reach the optimal norm, only a
unique (eta^{ast}, B^{ast}) achieves the best loss. As a sufficient
condition, we provide the first measurement of (eta^{ast}, B^{ast})
scaling with dataset size for Scion, and find that the scaling rules are
consistent with those of the Adam optimizer. Tuning per-layer-group learning
rates also improves model performance, with the output layer being the most
sensitive and hidden layers benefiting from lower learning rates. We provide
practical insights on norm-guided optimal scaling and release our Distributed
Scion (Disco) implementation with logs from over two thousand runs to support
research on LLM training dynamics at scale.
Code4MeV2: a Research-oriented Code-completion Platform
Authors: name: Roham Koohestani, name: Parham Bateni, name: Aydin Ebrahimi, name: Behdad Etezadi, name: Kiarash Karimi, name: Maliheh Izadi
Published: 2025-10-04 09:40:43

The adoption of AI-powered code completion tools in software development has
increased substantially, yet the user interaction data produced by these
systems remain proprietary within large corporations. This creates a barrier
for the academic community, as researchers must often develop dedicated
platforms to conduct studies on human--AI interaction, making reproducible
research and large-scale data analysis impractical. In this work, we introduce
Code4MeV2, a research-oriented, open-source code completion plugin for
JetBrains IDEs, as a solution to this limitation. Code4MeV2 is designed using a
client--server architecture and features inline code completion and a
context-aware chat assistant. Its core contribution is a modular and
transparent data collection framework that gives researchers fine-grained
control over telemetry and context gathering. Code4MeV2 achieves
industry-comparable performance in terms of code completion, with an average
latency of 200~ms. We assess our tool through a combination of an expert
evaluation and a user study with eight participants. Feedback from both
researchers and daily users highlights its informativeness and usefulness. We
invite the community to adopt and contribute to this tool. More information
about the tool can be found at https://app.code4me.me.
SAEdit: Token-level control for continuous image editing via Sparse
  AutoEncoder
Authors: name: Ronen Kamenetsky, name: Sara Dorfman, name: Daniel Garibi, name: Roni Paiss, name: Or Patashnik, name: Daniel Cohen-Or
Published: 2025-10-06 17:51:04

Large-scale text-to-image diffusion models have become the backbone of modern
image editing, yet text prompts alone do not offer adequate control over the
editing process. Two properties are especially desirable: disentanglement,
where changing one attribute does not unintentionally alter others, and
continuous control, where the strength of an edit can be smoothly adjusted. We
introduce a method for disentangled and continuous editing through token-level
manipulation of text embeddings. The edits are applied by manipulating the
embeddings along carefully chosen directions, which control the strength of the
target attribute. To identify such directions, we employ a Sparse Autoencoder
(SAE), whose sparse latent space exposes semantically isolated dimensions. Our
method operates directly on text embeddings without modifying the diffusion
process, making it model agnostic and broadly applicable to various image
synthesis backbones. Experiments show that it enables intuitive and efficient
manipulations with continuous control across diverse attributes and domains.
LLMSQL: Upgrading WikiSQL for the LLM Era of Text-to-SQL
Authors: name: Dzmitry Pihulski, name: Karol Charchut, name: Viktoria Novogrodskaia, name: Jan Kocoń
Published: 2025-09-27 15:08:43

Converting natural language questions into SQL queries (Text-to-SQL) enables
non-expert users to interact with relational databases and has long been a
central task for natural language interfaces to data. While the WikiSQL dataset
played a key role in early NL2SQL research, its usage has declined due to
structural and annotation issues, including case sensitivity inconsistencies,
data type mismatches, syntax errors, and unanswered questions. We present
LLMSQL, a systematic revision and transformation of WikiSQL designed for the
LLM era. We classify these errors and implement automated methods for cleaning
and re-annotation. To assess the impact of these improvements, we evaluated
multiple large language models (LLMs), including Gemma 3, LLaMA 3.2, Mistral
7B, gpt-oss 20B, Phi-3.5 Mini, Qwen 2.5, OpenAI o4-mini, DeepSeek R1 and
others. Rather than serving as an update, LLMSQL is introduced as an LLM-ready
benchmark: unlike the original WikiSQL, tailored for pointer-network models
selecting tokens from input, LLMSQL provides clean natural language questions
and full SQL queries as plain text, enabling straightforward generation and
evaluation for modern natural language-to-SQL models.
Imperceptible Jailbreaking against Large Language Models
Authors: name: Kuofeng Gao, name: Yiming Li, name: Chao Du, name: Xin Wang, name: Xingjun Ma, name: Shu-Tao Xia, name: Tianyu Pang
Published: 2025-10-06 17:03:50

Jailbreaking attacks on the vision modality typically rely on imperceptible
adversarial perturbations, whereas attacks on the textual modality are
generally assumed to require visible modifications (e.g., non-semantic
suffixes). In this paper, we introduce imperceptible jailbreaks that exploit a
class of Unicode characters called variation selectors. By appending invisible
variation selectors to malicious questions, the jailbreak prompts appear
visually identical to original malicious questions on screen, while their
tokenization is "secretly" altered. We propose a chain-of-search pipeline to
generate such adversarial suffixes to induce harmful responses. Our experiments
show that our imperceptible jailbreaks achieve high attack success rates
against four aligned LLMs and generalize to prompt injection attacks, all
without producing any visible modifications in the written prompt. Our code is
available at https://github.com/sail-sg/imperceptible-jailbreaks.