GenAI4ELab/papercli-papers huggingface dataset:
MDNS: Masked Diffusion Neural Sampler via Stochastic Optimal Control
Authors: Yuchen Zhu; Wei Guo; Jaemoo Choi; Guan-Horng Liu; Yongxin Chen; Molei Tao
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: We study the problem of learning a neural sampler to generate samples from discrete state spaces where the target probability mass function $\pi\propto\mathrm{e}^{-U}$ is known up to a normalizing constant, which is an important task in fields such as statistical physics, machine learning, combinatorial optimization, etc. To better address this challenging task when the state space has a large cardinality and the distribution is multi-modal, we propose **M**asked **D**iffusion **N**eural **S**ampler (**MDNS**), a novel framework for training discrete neural samplers by aligning two path measures through a family of learning objectives, theoretically grounded in the stochastic optimal control of the continuous-time Markov chains. We validate the efficiency and scalability of MDNS through extensive experiments on various distributions with distinct statistical properties, where MDNS learns to accurately sample from the target distributions despite the extremely high problem dimensions and outperforms other learning-based baselines by a large margin. A comprehensive study of ablations and extensions is also provided to demonstrate the efficacy and potential of the proposed framework. Our code is available at https://github.com/yuchen-zhu-zyc/MDNS.
Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning
Authors: Jaehun Jung; Seungju Han; Ximing Lu; Skyler Hallinan; David Acuna; Shrimai Prabhumoye; Mostofa Patwary; Mohammad Shoeybi; Bryan Catanzaro; Yejin Choi
Venue: NeurIPS 2025 (NeurIPS 2025 spotlight)

Abstract: Data diversity is crucial for training a strong language model. Yet metrics of diversity often diverge from this goal, measuring variations in heuristic features—like n-grams or embeddings—that are detached from how the model actually performs on a target task. This motivates us to ask: *Can we redefine data diversity—beyond measuring variations in heuristic features—in a way that better predicts model generalization?* Through large-scale empirical analyses spanning over 300 training runs, carefully controlled for data scale and quality, we show that data diversity can be a strong predictor of generalization in LLM reasoning—as measured by average model performance on unseen out-of-distribution benchmarks. We introduce **G-Vendi**, a metric that quantifies diversity via the entropy of model-induced loss gradients. G-Vendi scales to million-sample datasets and yet consistently outperforms heuristic alternatives, achieving strong correlation ($\text{Spearman's } \rho \approx 0.9$) with out-of-distribution (OOD) performance across both natural language inference (NLI) and math reasoning tasks. Building on this insight, we present **Prismatic Synthesis**, a framework for generating diverse synthetic data by targeting underrepresented regions in gradient space. Experimental results show that Prismatic Synthesis consistently improves model performance as we scale synthetic data—not just on in-distribution test but across unseen, out-of-distribution benchmarks—significantly outperforming state-of-the-art models in both domains. For example, PrismMath-7B, our model distilled from a 32B LLM without human verification, outperforms R1-Distill-Qwen-7B—trained on proprietary data generated by 671B R1—on 6 out of 7 challenging math benchmarks.
On the Closed-Form of Flow Matching: Generalization Does Not Arise from Target Stochasticity
Authors: Quentin Bertrand; Anne Gagneux; Mathurin Massias; Rémi Emonet
Venue: NeurIPS 2025 (NeurIPS 2025 oral)

Abstract: Modern deep generative models can now produce high-quality synthetic samples that are often indistinguishable from real training data. A growing body of research aims to understand why recent methods, such as diffusion and flow matching techniques, generalize so effectively. Among the proposed explanations are the inductive biases of deep learning architectures and the stochastic nature of the conditional flow matching loss. In this work, we rule out the noisy nature of the loss as a key factor driving generalization in flow matching.
First, we empirically show that in high-dimensional settings, the stochastic and closed-form versions of the flow matching loss yield nearly equivalent losses. Then, using state-of-the-art flow matching models on standard image datasets, we demonstrate that both variants achieve comparable statistical performance, with the surprising observation that using the closed-form can even improve performance.
Higher-Order Learning with Graph Neural Networks via Hypergraph Encodings
Authors: Raphaël Pellegrin; Lukas Fesser; Melanie Weber
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Higher-order information is crucial for relational learning in many domains where relationships extend beyond pairwise interactions. Hypergraphs provide a natural framework for modeling such relationships, which has motivated recent extensions of graph neural network (GNN) architectures to hypergraphs. Most of these architectures rely on message-passing to encode higher-order information. In this paper, we propose to instead use hypergraph-level encodings based on characteristics such as hypergraph Laplacians and discrete curvature notions. These encodings can be used on datasets that are naturally parametrized as hypergraphs and on graph-level datasets, which we reparametrize as hypergraphs to compute encodings. In both settings, performance increases significantly, on social networks by more than 10 percent. Our theoretical analysis shows that hypergraph-level encodings provably increase the representational power of message-passing graph neural networks beyond that of their graph-level counterparts. For complete reproducibility, we release our codebase: https://github.com/Weber-GeoML/Hypergraph_Encodings.
Subgraph Federated Learning via Spectral Methods
Authors: Javad Aliakbari; Johan Östman; Ashkan Panahi; Alexandre Graell i Amat
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: We consider the problem of federated learning (FL) with graph-structured data distributed across multiple clients. In particular, we address the common scenario of interconnected subgraphs, where interconnections between clients significantly influence the learning process. Existing approaches suffer from critical limitations, either requiring the exchange of sensitive node embeddings, thereby posing privacy risks, or relying on computationally-intensive steps, which hinders scalability.

To tackle these challenges, we propose FedLap, a novel framework that leverages global structure information via Laplacian smoothing in the spectral domain to effectively capture inter-node dependencies while ensuring privacy and scalability. We provide a formal analysis of the privacy of FedLap, demonstrating that it preserves privacy. Notably, FedLap is the first subgraph FL scheme with strong privacy guarantees. Extensive experiments on benchmark datasets demonstrate that the proposed method achieves competitive or superior utility compared to existing techniques.
AnaCP: Toward Upper-Bound Continual Learning via Analytic Contrastive Projection
Authors: Saleh Momeni; Changnan Xiao; Bing Liu
Venue: NeurIPS 2025 (NeurIPS 2025 spotlight)

Abstract: This paper studies the problem of class-incremental learning (CIL), a core setting within continual learning where a model learns a sequence of tasks, each containing a distinct set of classes. Traditional CIL methods, which do not leverage pre-trained models (PTMs), suffer from catastrophic forgetting (CF) due to the need to incrementally learn both feature representations and the classifier. The integration of PTMs into CIL has recently led to efficient approaches that treat the PTM as a fixed feature extractor combined with analytic classifiers, achieving state-of-the-art performance. However, they still face a major limitation: the inability to continually adapt feature representations to best suit the CIL tasks, leading to suboptimal performance. To address this, we propose AnaCP (Analytic Contrastive Projection), a novel method that preserves the efficiency of analytic classifiers while enabling incremental feature adaptation without gradient-based training, thereby eliminating the CF caused by gradient updates. Our experiments show that AnaCP not only outperforms existing baselines but also achieves the accuracy level of joint training, which is regarded as the upper bound of CIL.
Regression Trees Know Calculus
Authors: Nathan Wycoff
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Regression trees have emerged as a preeminent tool for solving real-world regression problems due to their ability to deal with nonlinearities, interaction effects and sharp discontinuities. In this article, we rather study regression trees applied to well-behaved, differentiable functions, and determine the relationship between node parameters and the local gradient of the function being approximated. We find a simple estimate of the gradient which can be efficiently computed using quantities exposed by popular tree learning libraries. This allows tools developed in the context of differentiable algorithms, like neural nets and Gaussian processes, to be deployed to tree-based models. To demonstrate this, we study measures of model sensitivity defined in terms of integro-differential quantities and demonstrate how to compute them for regression trees using the proposed gradient estimates. Quantitative and qualitative numerical experiments reveal the capability of gradients estimated by regression trees to improve predictive analysis, solve tasks in uncertainty quantification, and provide interpretation of model behavior.
How Benchmark Prediction from Fewer Data Misses the Mark
Authors: Guanhua Zhang; Florian E. Dorner; Moritz Hardt
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Large language model (LLM) evaluation is increasingly costly, prompting interest in methods that speed up evaluation by shrinking benchmark datasets. Benchmark prediction (also called efficient LLM evaluation) aims to select a small subset of evaluation points and predict overall benchmark performance from that subset. In this paper, we systematically assess the strengths and limitations of 11 benchmark prediction methods across 19 diverse benchmarks. First, we identify a highly competitive baseline: Take a random sample and fit a regression model on the sample to predict missing entries. Outperforming most existing methods, this baseline challenges the assumption that careful subset selection is necessary for benchmark prediction. Second, we discover that all existing methods crucially depend on model similarity. They work best when interpolating scores among similar models. The effectiveness of benchmark prediction sharply declines when new models have higher accuracy than previously seen models. In this setting of extrapolation, none of the previous methods consistently beat a simple average over random samples. To improve over the sample average, we introduce a new method inspired by augmented inverse propensity weighting. This method consistently outperforms the random sample average even for extrapolation. However, its performance still relies on model similarity and the gains are modest in general. This shows that benchmark prediction fails just when it is most needed: at the evaluation frontier, where the goal is to evaluate new models of unknown capabilities.
Secure and Confidential Certificates of Online Fairness
Authors: Olive Franzese; Ali Shahin Shamsabadi; Carter Luck; Hamed Haddadi
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: The "black-box service model" enables ML service providers to serve clients while keeping their intellectual property and client data confidential. Confidentiality is critical for delivering ML services legally and responsibly, but makes it difficult for outside parties to verify important model properties such as fairness. Existing methods that assess model fairness confidentially lack either (i) *reliability* because they certify fairness with respect to a static set of data, and therefore fail to guarantee fairness in the presence of distribution shift or service provider malfeasance; and/or (ii) *scalability* due to the computational overhead of confidentiality-preserving cryptographic primitives. We address these problems by introducing *online fairness certificates*, which verify that a model is fair with respect to data received by the service provider *online* during deployment. We then present OATH, a deployably efficient and scalable zero-knowledge proof protocol for confidential online group fairness certification. OATH exploits statistical properties of group fairness via a "cut-and-choose" style protocol, enabling  scalability improvements over baselines.
Scalable Neural Incentive Design with Parameterized Mean-Field Approximation
Authors: Nathan Corecco; Batuhan Yardim; Vinzenz Thoma; Zebang Shen; Niao He
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Designing incentives for a multi-agent system to induce a desirable Nash equilibrium is both a crucial and challenging problem appearing in many decision-making domains, especially for a large number of agents $N$.
Under the exchangeability assumption, we formalize this incentive design (ID) problem as a parameterized mean-field game (PMFG), aiming to reduce complexity via an infinite-population limit.
We first show that when dynamics and rewards are Lipschitz, the finite-$N$ ID objective is approximated by the PMFG at rate $\mathcal{O}(\frac{1}{\sqrt{N}})$.
Moreover, beyond the Lipschitz-continuous setting, we prove the same $\mathcal{O}(\frac{1}{\sqrt{N}})$ decay for the important special case of sequential auctions, despite discontinuities in dynamics, through a tailored auction-specific analysis.
Built on our novel approximation results, we further introduce our Adjoint Mean-Field Incentive Design (AMID) algorithm, which uses explicit differentiation of iterated equilibrium operators to compute gradients efficiently.
By uniting approximation bounds with optimization guarantees, AMID delivers a powerful, scalable algorithmic tool for many-agent (large $N$) ID.
Across diverse auction settings, the proposed AMID method substantially increases revenue over first-price formats and outperforms existing benchmark methods.
FOCUS: Internal MLLM Representations for Efficient Fine-Grained Visual Question Answering
Authors: Liangyu Zhong; Fabio Rosenthal; Joachim Sicking; Fabian Hüger; Thorsten Bagdonat; Hanno Gottschalk; Leo Schwinn
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: While Multimodal Large Language Models (MLLMs) offer strong perception and
reasoning capabilities for image-text input, Visual Question Answering (VQA)
focusing on small image details still remains a challenge. Although visual cropping
techniques seem promising, recent approaches have several limitations: the need
for task-specific fine-tuning, low efficiency due to uninformed exhaustive search,
or incompatibility with efficient attention implementations. We address these
shortcomings by proposing a training-free visual cropping method, dubbed FOCUS,
that leverages MLLM-internal representations to guide the search for the most
relevant image region. This is accomplished in four steps: first, we identify the
target object(s) in the VQA prompt; second, we compute an object relevance map
using the key-value (KV) cache; third, we propose and rank relevant image regions
based on the map; and finally, we perform the fine-grained VQA task using the top-ranked
region. As a result of this informed search strategy, FOCUS achieves strong
performance across four fine-grained VQA datasets and three types of MLLMs. It
outperforms three popular visual cropping methods in both accuracy and efficiency,
and matches the best-performing baseline, ZoomEye, while requiring 3 – 6.5 ×
less compute.
Deep Taxonomic Networks for Unsupervised Hierarchical Prototype Discovery
Authors: Zekun Wang; Ethan Haarer; Tianyi Zhu; Zhiyi Dai; Christopher J. MacLellan
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Inspired by the human ability to learn and organize knowledge into hierarchical taxonomies with prototypes, this paper addresses key limitations in current deep hierarchical clustering methods. 
Existing methods often tie the structure to the number of classes and underutilize the rich prototype information available at intermediate hierarchical levels. 
We introduce deep taxonomic networks, a novel deep latent variable approach designed to bridge these gaps.
Our method optimizes a large latent taxonomic hierarchy, specifically a complete binary tree structured mixture-of-Gaussian prior within a variational inference framework, to automatically discover taxonomic structures and associated prototype clusters directly from unlabeled data without assuming true label sizes.
We analytically show that optimizing the ELBO of our method encourages the discovery of hierarchical relationships among prototypes. 
Empirically, our learned models demonstrate strong hierarchical clustering performance, outperforming baselines across diverse image classification datasets using our novel evaluation mechanism that leverages prototype clusters discovered at all hierarchical levels.
Qualitative results further reveal that deep taxonomic networks discover rich and interpretable hierarchical taxonomies, capturing both coarse-grained semantic categories and fine-grained visual distinctions.
Stochastically Dominant Peer Prediction
Authors: Yichi Zhang; Shengwei Xu; Grant Schoenebeck; David Pennock
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Eliciting reliable human feedback is essential for many machine learning tasks, such as learning from noisy labels and aligning AI systems with human preferences. Peer prediction mechanisms incentivize truthful reporting without ground truth verification by scoring agents based on correlations with peers. Traditional mechanisms, which ensure that truth-telling maximizes the \textbf{expected scores} in equilibrium, can elicit honest information while assuming agents' utilities are \textbf{linear functions} of their scores. However, in practice, non-linear payment rules are usually preferred, or agents' utilities are inherently non-linear.

We propose \emph{stochastically dominant truthfulness (SD-truthfulness)} as a stronger guarantee: the score distribution of truth-telling stochastically dominates all other strategies, incentivizing truthful reporting for a wide range of monotone utility functions. Our first observation is that no existing peer prediction mechanism naturally satisfies this criterion without strong assumptions. A simple solution - rounding scores into binary lotteries — can enforce SD-truthfulness, but often degrades \emph{sensitivity}, a key property related to fairness and statistical efficiency. We demonstrate how a more careful application of rounding can better preserve sensitivity. Furthermore, we introduce a new enforced agreement (EA) mechanism that is theoretically guaranteed to be SD-truthful in binary-signal settings and, under mild assumptions, empirically achieves the highest sensitivity among all known SD-truthful mechanisms.
Unmasking Puppeteers: Leveraging Biometric Leakage to Expose Impersonation in AI-Based Videoconferencing
Authors: Danial Samadi Vahdati; Tai Duc Nguyen; Ekta Prashnani; Koki Nagano; david luebke; Orazio Gallo; Matthew Stamm
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: AI-based talking-head videoconferencing systems reduce bandwidth by transmitting a latent representation of a speaker’s pose and expression, which is used to synthesize frames on the receiver's end. However, these systems are vulnerable to “puppeteering” attacks, where an adversary controls the identity of another person in real-time. Traditional deepfake detectors fail here, as all video content is synthetic. We propose a novel biometric defense that detects identity leakage in the transmitted latent representation. Our metric-learning approach disentangles identity cues from pose and expression, enabling detection of unauthorized swaps. Experiments across multiple talking-head models show that our method consistently outperforms prior defenses, operates in real time on consumer GPUs, and generalizes well to out-of-distribution data. By targeting the latent features shared during normal operation, our method offers a practical and robust safeguard against puppeteering.
H-SPLID: HSIC-based Saliency Preserving Latent Information Decomposition
Authors: Lukas Miklautz; Chengzhi Shi; Andrii Shkabrii; Theodoros Thirimachos Davarakis; Prudence Lam; Claudia Plant; Jennifer Dy; Stratis Ioannidis
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: We introduce H-SPLID, a novel algorithm for learning salient feature representations through the explicit decomposition of salient and non-salient features into separate spaces. We show that H-SPLID promotes learning low-dimensional, task-relevant features. We prove that the expected prediction deviation under input perturbations is upper-bounded by the dimension of the salient subspace and the Hilbert-Schmidt Independence Criterion (HSIC) between inputs and representations. This  establishes a link between robustness and latent representation compression in terms of the dimensionality and information preserved. Empirical evaluations on image classification tasks show that models trained with H-SPLID primarily rely on salient input components, as indicated by reduced sensitivity to perturbations affecting non-salient features, such as image backgrounds.
SQLens: An End-to-End Framework for Error Detection and Correction in Text-to-SQL
Authors: Yue Gong; Chuan Lei; Xiao Qin; Kapil Vaidya; Balakrishnan Murali Narayanaswamy; Tim Kraska
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Text-to-SQL systems translate natural language (NL) questions into SQL queries, enabling non-technical users to interact with structured data. While large language models (LLMs) have shown promising results on the text-to-SQL task, they often produce semantically incorrect yet syntactically valid queries, with limited insight into their reliability. We propose SQLens, an end-to-end framework for fine-grained detection and correction of semantic errors in LLM-generated SQL. SQLens integrates error signals from both the underlying database and the LLM to identify potential semantic errors within SQL clauses. It further leverages these signals to guide query correction. Empirical results on two public benchmarks show that SQLens outperforms the best LLM-based self-evaluation method by 25.78% in F1 for error detection, and improves execution accuracy of out-of-the-box text-to-SQL systems by up to 20%.