GenAI4ELab/papercli-papers huggingface dataset:
FedFACT: A Provable Framework for Controllable Group-Fairness Calibration in Federated Learning
Authors: Li Zhang; Zhongxuan Han; XiaoHua Feng; Jiaming Zhang; Yuyuan Li; Chaochao Chen
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: With emerging application of Federated Learning (FL) in decision-making scenarios, it is imperative to regulate model fairness to prevent disparities across sensitive groups (e.g., female, male).
Current research predominantly focuses on two concepts of group fairness within FL: *Global Fairness* (overall model disparity across all clients) and *Local Fairness* (the disparity within each client).
However, the non-decomposable, non-differentiable nature of fairness criteria pose two fundamental, unresolved challenges for fair FL: (i) *Harmonizing global and local fairness, especially in multi-class classification*; (ii) *Enabling a controllable, optimal accuracy-fairness trade-off*.
To tackle the aforementioned challenges, we propose a novel controllable federated group-fairness calibration framework, named FedFACT.
FedFACT identifies the Bayes-optimal classifiers under both global and local fairness constraints in multi-class case, yielding models with minimal performance decline while guaranteeing fairness.
To effectively realize an adjustable, optimal accuracy-fairness balance, we derive specific characterizations of the Bayes-optimal fair classifiers for reformulating fair FL as personalized cost-sensitive learning problem for in-processing, and bi-level optimization for post-processing.
Theoretically, we provide convergence and generalization guarantees for FedFACT to approach the near-optimal accuracy under given fairness levels.
Extensive experiments on multiple datasets across various data heterogeneity demonstrate that FedFACT consistently outperforms baselines in balancing accuracy and global-local fairness.
Transformer Key-Value Memories Are Nearly as Interpretable as Sparse Autoencoders
Authors: Mengyu Ye; Jun Suzuki; Tatsuro Inaba; Tatsuki Kuribayashi
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Recent interpretability work on large language models (LLMs) has been increasingly dominated by a feature-discovery approach with the help of proxy modules. Then, the quality of features learned by, e.g., sparse auto-encoders (SAEs), is evaluated.
This paradigm naturally raises a critical question: do such learned features have better properties than those already represented within the original model parameters, and unfortunately, only a few studies have made such comparisons systematically so far.
In this work, we revisit the interpretability of feature vectors stored in feed-forward (FF) layers, given the perspective of FF as key-value memories, with modern interpretability benchmarks.
Our extensive evaluation revealed that SAE and FFs exhibits a similar range of interpretability, although SAEs displayed an observable but minimal improvement in some aspects. 
Furthermore, in certain aspects, surprisingly, even vanilla FFs yielded better interpretability than the SAEs, and features discovered in SAEs and FFs diverged.
These bring questions about the advantage of SAEs from both perspectives of feature quality and faithfulness, compared to directly interpreting FF feature vectors, and FF key-value parameters serve as a strong baseline in modern interpretability research.
Beyond Oracle: Verifier-Supervision for Instruction Hierarchy in Reasoning and Instruction-Tuned LLMs
Authors: Sian-Yao Huang; Li-Hsien Chang; Che-Yu Lin; Cheng-Lin Yang
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Large language models (LLMs) are often prompted with multi-level directives, such as system instructions and user queries, that imply a hierarchy of authority. Yet models frequently fail to enforce this structure, especially in multi-step reasoning where errors propagate across intermediate steps. Existing methods rely on oracle completions but lack verifiable reward signals or intermediate traces, limiting their applicability. We introduce a unified supervision framework that embeds programmatically verifiable checkers into synthesized instruction-conflict instances. Each instance pairs a compliance directive with a conflicting one, along with an executable verifier that deterministically checks output adherence. This enables alignment without oracle labels or reasoning traces, supporting both instruction-tuned and reasoning models. The framework is instantiated via a synthesis pipeline that includes unit-test–based validation, LLM-assisted repair, and a probabilistic analysis of cleaning reliability. Fine-tuning on the resulting data improves instruction hierarchy adherence and boosts safety robustness, generalizing to adversarial safety benchmarks without task-specific supervision. This highlights verifiable supervision as a scalable foundation for robust alignment. All code, dataset, and verifier pipeline are publicly available at: https://github.com/cycraft-corp/BeyondOracle.
ASDSV: Multimodal Generation Made Efficient with Approximate Speculative Diffusion and Speculative Verification
Authors: Kaijun Zhou; Xingyu Yan; Xingda Wei; Xijun Li; Jinyu Gu
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Diffusion in transformer is central to advances in high-quality multimodal generation 
but suffer from high inference latency due to their iterative nature. 
Inspired by speculative decoding's success in accelerating large language models, 
we propose Approximate Speculative Diffusion with Speculative Verification (ASDSV), 
a novel method to enhance the efficiency of diffusion models. 
Adapting speculative execution to diffusion processes presents unique challenges. 

First, the substantial computational cost of verifying numerous speculative steps 
for continuous, high-dimensional outputs makes traditional full verification prohibitively expensive. 
Second, determining the optimal number of speculative steps $K$ 
involves a trade-off between potential acceleration and verification success rates. 

To address these, ASDSV introduces two key innovations: 
1) A speculative verification technique, which leverages the observed temporal correlation between draft and target model outputs, 
efficiently validates $K$ speculative steps by only checking the alignment of the initial and final states, significantly reducing verification overhead. 
2) A multi-stage speculative strategy that adjusts $K$ according to the denoising phase—employing smaller $K$ during volatile early stages 
and larger $K$ during more stable later stages to optimize the balance between speed and quality. 

We apply ASDSV to state-of-the-art diffusion transformers, 
including Flux.1-dev for image generation and Wan2.1 for video generation. 
Extensive experiments demonstrate that ASDSV achieves up to 1.77$\times$-3.01$\times$ speedup 
in model inference with a minimal 0.3\%-0.4\% drop in VBench score, 
showcasing its effectiveness in accelerating multimodal diffusion models without significant quality degradation. 
The code will be publicly available once the acceptance of the paper.
Towards Accurate Time Series Forecasting via Implicit Decoding
Authors: Xinyu Li; Yuchen Luo; Hao Wang; Haoxuan Li; Liuhua Peng; Feng Liu; Yandong Guo; Kun Zhang; Mingming Gong
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Recent booming time series models have demonstrated remarkable forecasting performance. However, these methods often place greater focus on more effectively modelling the historical series, largely neglecting the forecasting phase, which generates long-term forecasts by separately predicting multiple time points. Given that real-world time series typically consist of various long short-term dynamics, independent predictions over individual time points may fail to express complex underlying patterns and can lead to a lack of global views. To address these issues, this work explores new perspectives from the forecasting phase and proposes a novel Implicit Forecaster (IF) as an additional decoding module. Inspired by decomposition forecasting, IF adopts a more nuanced approach by implicitly predicting constituent waves represented by their frequency, amplitude, and phase, thereby accurately forming the time series. Extensive experimental results from multiple real-world datasets show that IF can consistently boost mainstream time series models, achieving state-of-the-art forecasting performance. Code is available at this repository: [https://github.com/rakuyorain/Implicit-Forecaster](https://github.com/rakuyorain/Implicit-Forecaster).
SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignment
Authors: Wonje Jeung; Sangyeon Yoon; Minsuk Kahng; Albert No
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Large Reasoning Models (LRMs) have become powerful tools for complex problem solving, but their structured reasoning pathways can lead to unsafe outputs when exposed to harmful prompts. Existing safety alignment methods reduce harmful outputs but can degrade reasoning depth, leading to significant trade-offs in complex, multi-step tasks, and remain vulnerable to sophisticated jailbreak attacks. To address this, we introduce SAFEPATH, a lightweight alignment method that fine-tunes LRMs to emit a short, 8-token Safety Primer at the start of their reasoning, in response to harmful prompts, while leaving the rest of the reasoning process unsupervised. Empirical results across multiple  benchmarks indicate that SAFEPATH effectively reduces harmful outputs while maintaining reasoning performance. Specifically, SAFEPATH reduces harmful responses by up to 90.0\% and blocks 83.3\% of jailbreak attempts in the DeepSeek-R1-Distill-Llama-8B model, while requiring 295.9x less compute than Direct Refusal and 314.1x less than SafeChain. We further introduce a zero-shot variant that requires no fine-tuning. In addition, we provide a comprehensive analysis of how existing methods in LLMs generalize, or fail, when applied to reasoning-centric models, revealing critical gaps and new directions for safer AI.
Diffusing DeBias: Synthetic Bias Amplification for Model Debiasing
Authors: Massimiliano Ciranni; Vito Paolo Pastore; Roberto Di Via; Enzo Tartaglione; Francesca Odone; Vittorio Murino
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: The effectiveness of deep learning models in classification tasks is often challenged by the quality and quantity of training data whenever they are affected by strong spurious correlations between specific attributes and target labels. This results in a form of bias affecting training data, which typically leads to unrecoverable weak generalization in prediction. This paper addresses this problem by leveraging bias amplification with generated synthetic data only: we introduce Diffusing DeBias (DDB), a novel approach acting as a plug-in for common methods of unsupervised model debiasing, exploiting the inherent bias-learning tendency of diffusion models in data generation. Specifically, our approach adopts conditional diffusion models to generate synthetic bias-aligned images, which fully replace the original training set for learning an effective bias amplifier model to be subsequently incorporated into an end-to-end and a two-step unsupervised debiasing approach. By tackling the fundamental issue of bias-conflicting training samples’ memorization in learning auxiliary models, typical of this type of technique, our proposed method outperforms the current state-of-the-art in multiple benchmark datasets, demonstrating its potential as a versatile and effective tool for tackling bias in deep learning models. Code is available at https://github.com/Malga-Vision/DiffusingDeBias
Escaping saddle points without Lipschitz smoothness: the power of nonlinear preconditioning
Authors: Alexander Bodard; Panagiotis Patrinos
Venue: NeurIPS 2025 (NeurIPS 2025 spotlight)

Abstract: We study generalized smoothness in nonconvex optimization, focusing on $(L_0, L_1)$-smoothness and anisotropic smoothness. The former was empirically derived from practical neural network training examples, while the latter arises naturally in the analysis of nonlinearly preconditioned gradient methods. We introduce a new sufficient condition that encompasses both notions, reveals their close connection, and holds in key applications such as phase retrieval and matrix factorization. Leveraging tools from dynamical systems theory, we then show that nonlinear preconditioning—including gradient clipping—preserves the saddle point avoidance property of classical gradient descent. Crucially, the assumptions required for this analysis are actually satisfied in these applications, unlike in classical results that rely on restrictive Lipschitz smoothness conditions. We further analyze a perturbed variant that efficiently attains second-order stationarity with only logarithmic dependence on dimension, matching similar guarantees of classical gradient methods.
Perturb a Model, Not an Image: Towards Robust Privacy Protection via Anti-Personalized Diffusion Models
Authors: Tae-Young Lee; Juwon Seo; Jong Hwan Ko; Gyeong-Moon Park
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Recent advances in diffusion models have enabled high-quality synthesis of specific subjects, such as identities or objects. This capability, while unlocking new possibilities in content creation, also introduces significant privacy risks, as personalization techniques can be misused by malicious users to generate unauthorized images. Although several studies have attempted to counter this by generating adversarially perturbed samples designed to disrupt personalization, they rely on unrealistic assumptions and become ineffective in the presence of even a few clean images or under simple image transformations. To address these challenges, we shift the protection target from the images to the diffusion model itself to hinder the personalization of specific subjects, through our novel framework called $\textbf{A}$nti-$\textbf{P}$ersonalized $\textbf{D}$iffusion $\textbf{M}$odels ($\textbf{APDM}$). We first provide a theoretical analysis demonstrating that a naive approach of existing loss functions to diffusion models is inherently incapable of ensuring convergence for robust anti-personalization. Motivated by this finding, we introduce Direct Protective Optimization (DPO), a novel loss function that effectively disrupts subject personalization in the target model without compromising generative quality. Moreover, we propose a new dual-path optimization strategy, coined Learning to Protect (L2P). By alternating between personalization and protection paths, L2P simulates future personalization trajectories and adaptively reinforces protection at each step.
Experimental results demonstrate that our framework outperforms existing methods, achieving state-of-the-art performance in preventing unauthorized personalization.
The code is available at https://github.com/KU-VGI/APDM.
Why Playing Against Diverse and Challenging Opponents Speeds Up Coevolution: A Theoretical Analysis on Combinatorial Games
Authors: Alistair Benford; Per Kristian Lehre
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Competitive coevolutionary algorithms (CoEAs) have a natural application to problems that are adversarial or feature strategic interaction. However, there is currently limited theoretical insight into how to avoid pathological behaviour associated with CoEAs. In this paper we use impartial combinatorial games as a challenging domain for CoEAs and provide a corresponding runtime analysis. By analysing how individuals capitalise on the mistakes of their opponents, we prove that the Univariate Marginal Distribution Algorithm finds (with high probability) an optimal strategy for a game called Reciprocal LeadingOnes within $O(n^2\log^3{n})$ game evaluations, a significant improvement over the best known bound of $O(n^5\log^2{n})$. Critical to the analysis is the introduction of a novel stabilising operator, the impact of which we study both theoretically and empirically.
Theoretical Guarantees for the Retention of Strict Nash Equilibria by Coevolutionary Algorithms
Authors: Alistair Benford; Per Kristian Lehre
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Most methods for finding a Nash equilibrium rely on procedures that operate over the entire action space, making them infeasible for settings with too many actions to be searched exhaustively. Randomised search heuristics such as coevolutionary algorithms offer benefits in such settings, however they lack many of the theoretical guarantees established for exhaustive methods such as zero-regret learning. We address this by developing a method for proving necessary and sufficient conditions for a coevolutionary algorithm to be stable, in the sense that it reliably retains a Nash equilibrium following discovery. As the method provides bounds that are adapted to both application and algorithm instance, it can be used as a practical tool for parameter configuration. We additionally show how bounds on regret may be deduced from our results and undertake corresponding empirical analysis.
Stochastic Principal-Agent Problems: Computing and Learning Optimal History-Dependent Policies
Authors: Jiarui Gan; R Majumdar; Debmalya Mandal; Goran Radanovic
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: We study a stochastic principal-agent model. A principal and an agent interact in a stochastic environment, each privy to observations about the state not available to the other. The principal has the power of commitment, both to elicit information from the agent and to signal her own information. The players communicate with each other and then select actions independently. 
Both players are {\em far-sighted}, aiming to maximize their total payoffs over the entire time horizon. 
We consider both the computation and learning of the principal's optimal policy. 
The key challenge lies in enabling {\em history-dependent} policies, which are essential for achieving optimality in this model but difficult to cope with because of the exponential growth of possible histories as the size of the model increases; explicit representation of history-dependent policies is infeasible as a result.
To address this challenge, we develop algorithmic techniques based on the concept of {\em inducible value set}. The techniques yield an efficient algorithm that computes an $\epsilon$-approximate optimal policy in time polynomial in $1/\epsilon$. 
We also present an efficient learning algorithm for an episodic reinforcement learning setting with unknown transition probabilities. The algorithm achieves sublinear regret $\widetilde{\mathcal{O}}(T^{2/3})$ for both players over $T$ episodes.
Large language models can learn and generalize steganographic chain-of-thought under process supervision
Authors: Robert McCarthy; Joey SKAF; Luis Ibanez-Lissen; Vasil Georgiev; Connor Watts; Hannes Whittingham; Lorena Gonzalez-Manzano; Cameron Tice; Edward James Young; Puria Radmard; David Lindner
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Chain-of-thought (CoT) reasoning not only enhances large language model performance but also provides critical insights into decision-making processes, marking it as a useful tool for monitoring model intent and planning. By proactively preventing models from acting on CoT indicating misaligned or harmful intent, CoT monitoring can be used to reduce risks associated with deploying models. However, developers may be incentivized to train away the appearance of harmful intent from CoT traces, by either customer preferences or regulatory requirements. However, recent works have shown that banning the mention of a specific example of reward hacking causes obfuscation of the undesired reasoning traces but the persistence of the undesired behavior, threatening the reliability of CoT monitoring. However, obfuscation of reasoning can be due to its internalization to latent space computation, or its encoding within the CoT. We provide an extension to these results with regard to the ability of models to learn a specific type of obfuscated reasoning: steganography. First, we show that penalizing the use of specific strings within load-bearing reasoning traces causes models to substitute alternative strings. Crucially, this does not alter the underlying method by which the model performs the task, demonstrating that the model can learn to steganographically encode its reasoning. This is an example of models learning to encode their reasoning. We further demonstrate that models can generalize an encoding scheme. When the penalized strings belong to an overarching class, the model learns not only to substitute strings seen in training, but also develops a general encoding scheme for all members of the class which it can apply to held-out testing strings.
PyraMotion: Attentional Pyramid-Structured Motion Integration for Co-Speech 3D Gesture Synthesis
Authors: Zhizhuo Yin; Yuk Hang Tsui; Pan Hui
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Generating full-body human gestures encompassing face, body, hands, and global movements from audio is crucial yet challenging for virtual avatar creation. Existing systems tokenize gestures frame-wise, predicting tokens of each frame from the input audio. However, expressive human gestures consist of varied patterns with different frame lengths, and different body parts exhibit motion patterns of varying durations. Existing systems fail to capture motion patterns across body parts and temporal scales due to the fixed frame-count setting of their gesture tokens. Inspired by the success of the feature pyramid technique in the multi-scale visual information extraction, we propose a novel framework named PyraMotion and an adaptive multi-scale feature capturing model called Attentive Pyramidal VQ-VAE (APVQ-VAE). Objective and subjective experiments demonstrate that the PyraMotion outperforms state-of-the-art methods in terms of generating natural and expressive full-body human gestures. Extensive ablation experiments highlight that the self-adaptiveness integration through attention maps contributes to performance.
Monoculture or Multiplicity: Which Is It?
Authors: Mila Gorecki; Moritz Hardt
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Two narratives about machine learning ecosystems grew out of recent algorithmic fairness discourse. In one, dubbed \emph{monoculture}, algorithmic ecosystems tend toward homogeneity akin to a single model making all decisions. Individuals then face the risk of systematic exclusion with no recourse. In the other, \emph{model multiplicity}, many models solve the same task with similar accuracy, causing excessive variation in outcomes. Both narratives are compelling, yet, seemingly at odds: model multiplicity can’t exist in a strict monoculture. In this work, we conduct a comprehensive empirical evaluation to test both claims. We work from the premise that increasingly decision makers will use large language models for consequential prediction tasks. We therefore examine 50 language models, open source models ranging in size from 1B to 141B parameters and state-of-the-art commercial models, under 4 different prompt variations, and across 6 different prediction tasks. Evaluating both new and old quantitative measures of monoculture and multiplicity, we find the empirical landscape sits between the two extremes. Each narrative finds some empirical support, but neither is dominant. Systematic exclusion with no recourse is rare, but model similarity is real. Even when starting from a single model, prompt variation induces some diversity in predictions. Our results contribute critical empirical grounding to ongoing debates and point toward a middle ground between monoculture and multiplicity as the most realistic outcome.