dataset-review/dataset-review huggingface dataset:
Private Evolution Converges
NeurIPS 2025

Private Evolution Converges
Tomas Gonzalez
Carnegie Mellon University
tcgonzal@andrew.cmu.edu
Giulia Fanti
Carnegie Mellon University
gfanti@andrew.cmu.edu
Aaditya Ramdas
Carnegie Mellon University
aramdas@cs.cmu.edu
Abstract
Private Evolution (PE) is a promising training-free method for differentially private
(DP) synthetic data generation. While it achieves strong performance in some
domains (e.g., images and text), its behavior in others (e.g., tabular data) is less
consistent. To date, the only theoretical analysis of the convergence of PE depends
on unrealistic assumptions about both the algorithm’s behavior and the structure
of the sensitive dataset. In this work, we develop a new theoretical framework
to understand PE’s practical behavior and identify sufficient conditions for its
convergence. For d-dimensional sensitive datasets with n data points from a
convex and compact domain, we prove that under the right hyperparameter settings
and given access to the Gaussian variation API proposed in [33], PE produces an
(ε, δ)-DP synthetic dataset with expected 1-Wasserstein distance ˜O(d(nε)−1/d)
from the original; this establishes worst-case convergence of the algorithm as
n →∞. Our analysis extends to general Banach spaces as well. We also connect
PE to the Private Signed Measure Mechanism, a method for DP synthetic data
generation that has thus far not seen much practical adoption. We demonstrate the
practical relevance of our theoretical findings in experiments.
1
Introduction
Many modern machine learning applications rely on user data, making data privacy protection a
central concern. In this regard, differential privacy (DP) [17, 19] has become a de facto standard for
safeguarding sensitive information of individuals. Given a private dataset, many problems—including
regression [40, 12], deep learning [1], and stochastic optimization [6, 2]—can be performed in a DP
manner. While it is possible to design a different DP algorithm for each specific task, a reason...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000013. Conference bucket: NeurIPS.
Activation-Informed Merging of Large Language Models
NeurIPS 2025

Activation-Informed Merging of
Large Language Models
Amin Heyrani Nobari∗1, Kaveh Alim∗1, Ali ArjomandBigdeli2
Akash Srivastava3, Faez Ahmed1, Navid Azizan1
1Massachusetts Institute of Technology
2Stony Brook University
3MIT-IBM Watson AI Lab & Red Hat AI Innovation
Abstract
Model merging, a method that combines the parameters and embeddings of mul-
tiple fine-tuned large language models (LLMs), offers a promising approach to
enhance model performance across various tasks while maintaining computational
efficiency. This paper introduces Activation-Informed Merging (AIM), a technique
that integrates the information from the activation space of LLMs into the merging
process to improve performance and robustness. AIM is designed as a flexible,
complementary solution that is applicable to any existing merging method. It aims
to preserve critical weights from the base model, drawing on principles from con-
tinual learning (CL) and model compression. Utilizing a task-agnostic calibration
set, AIM selectively prioritizes essential weights during merging. We empirically
demonstrate that AIM significantly enhances the performance of merged models
across multiple benchmarks. Our findings suggest that considering the activation-
space information can provide substantial advancements in the model merging
strategies for LLMs with up to 40% increase in benchmark performance. Our code
is publicly available at https://github.com/ahnobari/ActivationInformedMerging.
1
Introduction
Foundation models are rapidly becoming the dominant force for building Artificial Intelligence (AI)
systems. In many cases, researchers build their machine learning models by starting from pre-trained
foundation models and fine-tuning (FT) these pre-trained models for some desired target task [27]. In
such a paradigm, numerous fine-tuned models are developed for various tasks. However, an important
opportunity is missed, as these fine-tuned task-specialized models typically operate in isolation
without leve...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000017. Conference bucket: NeurIPS.
Ascent Fails to Forget
NeurIPS 2025

Ascent Fails to Forget
Ioannis Mavrothalassitis∗
Pol Puigdemont∗
Noam Itzhak Levi∗
Volkan Cevher
LIONS, École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
{ioannis.mavrothalassitis, pol.puigdemontplana, noam.levi}@epfl.ch
Abstract
Contrary to common belief, we show that gradient ascent-based unconstrained
optimization methods frequently fail to perform machine unlearning, a phenomenon
we attribute to the inherent statistical dependence between the forget and retain
data sets. This dependence, which can manifest itself even as simple correlations,
undermines the misconception that these sets can be independently manipulated
during unlearning. We provide empirical and theoretical evidence showing these
methods often fail precisely due to this overlooked relationship. For random forget
sets, this dependence means that degrading forget set metrics (which, for the oracle,
should mirror test set metrics) inevitably harms overall test performance. Going
beyond random sets, we consider logistic regression as an instructive example
where a critical failure mode emerges: inter-set dependence causes gradient descent-
ascent iterations to progressively diverge from the oracle. Strikingly, these methods
can converge to solutions that are not only far from the oracle but are potentially
even further from it than the original model itself, rendering the unlearning process
actively detrimental. A toy example further illustrates how this dependence can
trap models in inferior local minima, inescapable via finetuning. Our findings
highlight that the presence of such statistical dependencies, even when manifest
only as correlations, can be sufficient for ascent-based unlearning to fail. Our
theoretical insights are corroborated by experiments on complex neural networks,
demonstrating that these methods do not perform as expected in practice due to
this unaddressed statistical interplay.
1
Introduction
Machine learning models have become an integral part of modern r...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000019. Conference bucket: NeurIPS.
Private Evolution Converges
NeurIPS 2025

Private Evolution Converges
Tomas Gonzalez
Carnegie Mellon University
tcgonzal@andrew.cmu.edu
Giulia Fanti
Carnegie Mellon University
gfanti@andrew.cmu.edu
Aaditya Ramdas
Carnegie Mellon University
aramdas@cs.cmu.edu
Abstract
Private Evolution (PE) is a promising training-free method for differentially private
(DP) synthetic data generation. While it achieves strong performance in some
domains (e.g., images and text), its behavior in others (e.g., tabular data) is less
consistent. To date, the only theoretical analysis of the convergence of PE depends
on unrealistic assumptions about both the algorithm’s behavior and the structure
of the sensitive dataset. In this work, we develop a new theoretical framework
to understand PE’s practical behavior and identify sufficient conditions for its
convergence. For d-dimensional sensitive datasets with n data points from a
convex and compact domain, we prove that under the right hyperparameter settings
and given access to the Gaussian variation API proposed in [33], PE produces an
(ε, δ)-DP synthetic dataset with expected 1-Wasserstein distance ˜O(d(nε)−1/d)
from the original; this establishes worst-case convergence of the algorithm as
n →∞. Our analysis extends to general Banach spaces as well. We also connect
PE to the Private Signed Measure Mechanism, a method for DP synthetic data
generation that has thus far not seen much practical adoption. We demonstrate the
practical relevance of our theoretical findings in experiments.
1
Introduction
Many modern machine learning applications rely on user data, making data privacy protection a
central concern. In this regard, differential privacy (DP) [17, 19] has become a de facto standard for
safeguarding sensitive information of individuals. Given a private dataset, many problems—including
regression [40, 12], deep learning [1], and stochastic optimization [6, 2]—can be performed in a DP
manner. While it is possible to design a different DP algorithm for each specific task, a reason...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000020. Conference bucket: NeurIPS.
Low Precision Streaming PCA
NeurIPS 2025

Low-Precision Streaming PCA
Sanjoy Dasgupta
University of California San Diego
sadasgupta@ucsd.edu
Syamantak Kumar
University of Texas at Austin
syamantak@utexas.edu
Shourya Pandey
University of Texas at Austin
shouryap@utexas.edu
Purnamrita Sarkar
University of Texas at Austin
purna.sarkar@utexas.edu
Abstract
Low-precision Streaming PCA estimates the top principal component in a stream-
ing setting under limited precision. We establish an information-theoretic lower
bound on the quantization resolution required to achieve a target accuracy for the
leading eigenvector. We study Oja’s algorithm for streaming PCA under linear and
nonlinear stochastic quantization. The quantized variants use unbiased stochas-
tic quantization of the weight vector and the updates. Under mild moment and
spectral-gap assumptions on the data distribution, we show that a batched version
achieves the lower bound up to logarithmic factors under both schemes. This
leads to a nearly dimension-free quantization error in the nonlinear quantization
setting. Empirical evaluations on synthetic streams validate our theoretical findings
and demonstrate that our low-precision methods closely track the performance of
standard Oja’s algorithm.
1
Introduction
Quantization (or discretization) is the mapping of a continuous set of values to a small, finite set
of outputs close to the original values; standard methods for quantization include rounding and
truncation. The current popularity of training large-scale Machine Learning models has brought a
renewed focus on quantization, though its origins go back to the 1800s. Some early examples include
least-squares methods applied to large-scale data analysis in the early nineteenth century [Sti86].
In 1867, discretization was introduced for the approximate calculation of integrals [Rie67], and the
effects of rounding errors in integration were examined in 1897 [She97]. For an excellent survey and
history of quantization, see [GKD`22].
In the context of efficie...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000021. Conference bucket: NeurIPS.
Private Evolution Converges
NeurIPS 2025

Private Evolution Converges
Tomas Gonzalez
Carnegie Mellon University
tcgonzal@andrew.cmu.edu
Giulia Fanti
Carnegie Mellon University
gfanti@andrew.cmu.edu
Aaditya Ramdas
Carnegie Mellon University
aramdas@cs.cmu.edu
Abstract
Private Evolution (PE) is a promising training-free method for differentially private
(DP) synthetic data generation. While it achieves strong performance in some
domains (e.g., images and text), its behavior in others (e.g., tabular data) is less
consistent. To date, the only theoretical analysis of the convergence of PE depends
on unrealistic assumptions about both the algorithm’s behavior and the structure
of the sensitive dataset. In this work, we develop a new theoretical framework
to understand PE’s practical behavior and identify sufficient conditions for its
convergence. For d-dimensional sensitive datasets with n data points from a
convex and compact domain, we prove that under the right hyperparameter settings
and given access to the Gaussian variation API proposed in [33], PE produces an
(ε, δ)-DP synthetic dataset with expected 1-Wasserstein distance ˜O(d(nε)−1/d)
from the original; this establishes worst-case convergence of the algorithm as
n →∞. Our analysis extends to general Banach spaces as well. We also connect
PE to the Private Signed Measure Mechanism, a method for DP synthetic data
generation that has thus far not seen much practical adoption. We demonstrate the
practical relevance of our theoretical findings in experiments.
1
Introduction
Many modern machine learning applications rely on user data, making data privacy protection a
central concern. In this regard, differential privacy (DP) [17, 19] has become a de facto standard for
safeguarding sensitive information of individuals. Given a private dataset, many problems—including
regression [40, 12], deep learning [1], and stochastic optimization [6, 2]—can be performed in a DP
manner. While it is possible to design a different DP algorithm for each specific task, a reason...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000024. Conference bucket: NeurIPS.
Private Evolution Converges
NeurIPS 2025

Private Evolution Converges
Tomas Gonzalez
Carnegie Mellon University
tcgonzal@andrew.cmu.edu
Giulia Fanti
Carnegie Mellon University
gfanti@andrew.cmu.edu
Aaditya Ramdas
Carnegie Mellon University
aramdas@cs.cmu.edu
Abstract
Private Evolution (PE) is a promising training-free method for differentially private
(DP) synthetic data generation. While it achieves strong performance in some
domains (e.g., images and text), its behavior in others (e.g., tabular data) is less
consistent. To date, the only theoretical analysis of the convergence of PE depends
on unrealistic assumptions about both the algorithm’s behavior and the structure
of the sensitive dataset. In this work, we develop a new theoretical framework
to understand PE’s practical behavior and identify sufficient conditions for its
convergence. For d-dimensional sensitive datasets with n data points from a
convex and compact domain, we prove that under the right hyperparameter settings
and given access to the Gaussian variation API proposed in [33], PE produces an
(ε, δ)-DP synthetic dataset with expected 1-Wasserstein distance ˜O(d(nε)−1/d)
from the original; this establishes worst-case convergence of the algorithm as
n →∞. Our analysis extends to general Banach spaces as well. We also connect
PE to the Private Signed Measure Mechanism, a method for DP synthetic data
generation that has thus far not seen much practical adoption. We demonstrate the
practical relevance of our theoretical findings in experiments.
1
Introduction
Many modern machine learning applications rely on user data, making data privacy protection a
central concern. In this regard, differential privacy (DP) [17, 19] has become a de facto standard for
safeguarding sensitive information of individuals. Given a private dataset, many problems—including
regression [40, 12], deep learning [1], and stochastic optimization [6, 2]—can be performed in a DP
manner. While it is possible to design a different DP algorithm for each specific task, a reason...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000025. Conference bucket: NeurIPS.
Embodied Cognition Augmented End2End Autonomous Driving
NeurIPS 2025

Embodied Cognition Augmented End2End
Autonomous Driving
Ling Niu
Tsinghua University
nl23@mails.tsinghua.edu.cn
Xiaoji Zheng
Tsinghua University
zhengxj24@mails.tsinghua.edu.cn
Han Wang
Tsinghua University
kelsulhgod@gmail.com
Chen Zheng
Tsinghua University
zhengchen@air.tsinghua.edu.cn
Ziyuan Yang
Tsinghua University
ziyuan86@uw.edu
Bokui Chen*
Tsinghua University
chenbokui@tsinghua.edu.cn
Jiangtao Gong∗
Tsinghua University
gongjiangtao2@gmail.com
Abstract
In recent years, vision-based end-to-end autonomous driving has emerged as a new
paradigm. However, popular end-to-end approaches typically rely on visual feature
extraction networks trained under label supervision. This limited supervision frame-
work restricts the generality and applicability of driving models. In this paper, we
propose a novel paradigm termed E3AD, which advocates for comparative learn-
ing between visual feature extraction networks and the general EEG large model,
in order to learn latent human driving cognition for enhancing end-to-end planning.
In this work, we collected a cognitive dataset for the mentioned contrastive learning
process. Subsequently, we investigated the methods and potential mechanisms
for enhancing end-to-end planning with human driving cognition, using popular
driving models as baselines on publicly available autonomous driving datasets.
Both open-loop and closed-loop tests are conducted for a comprehensive evalua-
tion of planning performance. Experimental results demonstrate that the E3AD
paradigm significantly enhances the end-to-end planning performance of baseline
models. Ablation studies further validate the contribution of driving cognition and
the effectiveness of comparative learning process. To the best of our knowledge,
this is the first work to integrate human driving cognition for improving end-to-end
autonomous driving planning. It represents an initial attempt to incorporate embod-
ied cognitive data into end-to-end autonomous driving, providing valuable i...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000026. Conference bucket: NeurIPS.
Generalizable, real-time neural decoding with hybrid state-space models
NeurIPS 2025

Generalizable, real-time neural decoding with hybrid
state-space models
Avery Hee-Woon Ryoo1,2,∗,B
Nanda H Krishna1,2,∗,B
Ximeng Mao1,2,∗,B
Mehdi Azabou3
Eva L Dyer4
Matthew G Perich1,2,†
Guillaume Lajoie1,2,5,†,B
1Mila – Quebec AI Institute
2Université de Montréal
3Columbia University
4University of Pennsylvania
5Canada CIFAR AI Chair
∗Co-first authors
†Co-senior authors
B{hee-woon.ryoo,nanda.harishankar-krishna,ximeng.mao,guillaume.lajoie}@mila.quebec
https://possm-brain.github.io
Abstract
Real-time decoding of neural activity is central to neuroscience and neurotech-
nology applications, from closed-loop experiments to brain-computer interfaces,
where models are subject to strict latency constraints. Traditional methods, includ-
ing simple recurrent neural networks, are fast and lightweight but often struggle
to generalize to unseen data. In contrast, recent Transformer-based approaches
leverage large-scale pretraining for strong generalization performance, but typically
have much larger computational requirements and are not always suitable for low-
resource or real-time settings. To address these shortcomings, we present POSSM, a
novel hybrid architecture that combines individual spike tokenization via a cross-
attention module with a recurrent state-space model (SSM) backbone to enable (1)
fast and causal online prediction on neural activity and (2) efficient generalization
to new sessions, individuals, and tasks through multi-dataset pretraining. We evalu-
ate POSSM’s decoding performance and inference speed on intracortical decoding
of monkey motor tasks, and show that it extends to clinical applications, namely
handwriting and speech decoding in human subjects. Notably, we demonstrate that
pretraining on monkey motor-cortical recordings improves decoding performance
on the human handwriting task, highlighting the exciting potential for cross-species
transfer. In all of these tasks, we find that POSSM achieves decoding accuracy
comparable to state-of-the-art...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000030. Conference bucket: NeurIPS.
Enforcing convex constraints in Graph Neural Networks
NeurIPS 2025

Enforcing convex constraints in Graph Neural
Networks
Ahmed Rashwan ˚
University of Bath
Keith Briggs
BT Research
Chris Budd
University of Bath
Lisa Kreusser
University of Bath and Monumo
Abstract
Many machine learning applications require outputs that satisfy complex, dy-
namic constraints. This task is particularly challenging in Graph Neural Network
models due to the variable output sizes of graph-structured data. In this paper,
we introduce ProjNet, a Graph Neural Network framework which satisfies input-
dependant constraints. ProjNet combines a sparse vector clipping method with the
Component-Averaged Dykstra (CAD) algorithm, an iterative scheme for solving
the best-approximation problem. We establish a convergence result for CAD and
develop a GPU-accelerated implementation capable of handling large-scale in-
puts efficiently. To enable end-to-end training, we introduce a surrogate gradient
for CAD that is both computationally efficient and better suited for optimization
than the exact gradient. We validate ProjNet on four classes of constrained op-
timisation problems: linear programming, two classes of non-convex quadratic
programs, and radio transmit power optimization, demonstrating its effectiveness
across diverse problem settings.
1
Introduction
In many machine learning (ML) applications, model outputs must satisfy certain constraints to en-
sure feasibility, safety, or interpretability [22, 40]. A common approach to enforcing constraints
is through activation functions (e.g., Sigmoid, Softmax) in the output layer [41], which restrict
outputs to a predefined set. However, while effective for simple linear or box constraints, this ap-
proach is not applicable for more complicated types of constraints. Many real-world applications
require complex, data-dependent constraints. For example, in robotics, constraints can encode fea-
sible movements or collision avoidance [21, 54, 63]. Similarly, in energy systems, power generation
limits vary with demand fluctua...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000033. Conference bucket: NeurIPS.