Title: Convex Approximation of Two-Layer ReLU Networks for Hidden State Differential Privacy
Venue: NeurIPS 2025
Accepted: yes
Keywords: Differential privacy

DP machine learning

hidden state analysis
Abstract: The hidden state threat model of differential privacy (DP) assumes that the adversary has access only to the final trained machine learning (ML) model, without seeing intermediate states during training. However, the current privacy analyses under this model are restricted to convex optimization problems, reducing their applicability to multi-layer neural networks, which are essential in modern deep learning applications. Notably, the most successful applications of the hidden state privacy analyses in classification tasks have only been for logistic regression models. We demonstrate that it is possible to privately train convex problems with privacy-utility trade-offs comparable to those of 2-layer ReLU networks trained with DP stochastic gradient descent (DP-SGD). This is achieved through a stochastic approximation of a dual formulation of the ReLU minimization problem, resulting in a strongly convex problem. This enables the use of existing hidden state privacy analyses and provides accurate privacy bounds also for the noisy cyclic mini-batch gradient descent (NoisyCGD) method with fixed disjoint mini-batches. Empirical results on benchmark classification  tasks demonstrate that NoisyCGD can achieve privacy-utility trade-offs on par with DP-SGD applied to 2-layer ReLU networks.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nSummary\nThis paper studies a scenario in which the adversary only has access to the final model and asks whether two-layer ReLU networks can be trained with strong hidden-state DP guarantees. The approach proceeds in three main steps:\n\n1. Convex reformulation\n\nUsing Pilanci \\& Ergen's dual perspective, the two-layer ReLU objective is rewritten as a convex program that enumerates all activation patterns. For sufficiently wide hidden layers, this convex formulation attains the same optimum as the original nonconvex problem.\n\n2. Stochastic, strongly convex approximation\n\nThe intractable full pattern set is replaced by $P \\ll M$ random hyperplanes, and an $\\ell_2$ regularization term is added. This yields the smooth, $\\lambda$-strongly convex objective in (3.6)-(3.7).\n\n3. Noisy Cyclic Gradient Descent (NoisyCGD)\n\nThe convex approximation is trained using fixed, disjoint mini-batches with added Gaussian noise. Privacy loss is tracked via Bok et al.'s $\\mu$-GDP accountant.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1. The authors tackle an important DP setting-where the attacker observes only the final model-and develop both theoretical and practical tools under convexity assumptions. Their techniques successfully extend to two-layer ReLU networks.\n\n2. Mapping a two-layer ReLU network to a convex program and integrating it with DP yields an improved privacy-utility trade-off. This insight is both novel and significant.\n\n3. The writing is well structured, and the related work section is comprehensive and detailed.\n\nWeaknesses:\n\nI did not identify any major flaws; overall, I find the work interesting and well executed. However, I have a few questions and suggestions for clarification:\n\nQUESTIONS:\nQuestions\n\n1. Line 194: Equation (3.2) includes a regularization term. Is this added specifically to prepare for the strongly convex approximation that follows?\n\n2. Line 202: Why does the bound on the matrices $D_X$ depend only on $\\operatorname{rank}(X)$ and not on the parameter $u$, given the definition of $D_X$ ?\n\n3. Line 205: Is equation (3.4) the dual form of (3.2)? It would be helpful to include a brief explanation of the roles of $v_i$ and $w_i$.\n\n4. Line 277 (Section 4.1): What does the constant $c$ represent? Additionally, what are the implications in the high-dimensional regime (when $n \\ll d$ )?\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-24"}

comments: SUMMARY:
The paper studies the hidden-state DP threat model, which has been limited to the Logistic Regression model. In particular, the work studies the threat model for the 2-layer ReLU network by introducing a stochastic approximation of a dual formulation of the ReLU minimization problem. The analysis helps further analysis of the NoisyCGD defense method.

STRENGTHS AND WEAKNESSES:
Strengths:
1) The paper is well-written: well-defined problem, clear prelims with rigorous theoretical results.
2) While the experiments are limited to MNIST and CIFAR10 only, they are well designed. I also find the results reasonable and supportive toward the theoretical claims.
3) I find the studied problem of hidden-state privacy is interesting and of great interests.

Weaknesses:
I find the practical implication of the theoretical analysis is limited: (1) while the authors showcase the usage of the theoretical analysis in NoisyCGD, I find that implication is niche and is not strictly beneficial (I find NoisyCGD is actually significantly lower in performance compared to DP-SGD, not "on par" as the authors stated). (2) I understand that improving the analysis from logistic regression to 2-layer ReLU is novelty, but the results are still very far-away from practicality for real-world neural networks. How is the analysis look like for N-layer networks?

QUESTIONS:
See weaknesses.

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies differentially private training of 2-layer ReLU networks under the hidden state threat model by leveraging a convex approximation of the original non-convex objective. Two previously distinct ideas are integrated: convex approximation of 2-layer ReLU networks and privacy amplification by iteration. They propose a strongly convex surrogate model to facilitate the privacy analysis and apply NoisyCGD with disjoint mini-batches to achieve both strong privacy and utility guarantees.\n\nSTRENGTHS AND WEAKNESSES:\nStrength: \n\nThe topic of improving the theoretical performance of DP-SGD is both interesting and meaningful, particularly in the context of privacy-preserving machine learning. This paper contributes to this effort by providing utility analysis for 2-layer ReLU networks problem which is nonconvex.  \n\n\n \n\nWeakness:\n\n1. The paper appears to be largely an integration and reapplication of results from previous studies. While the extension to ReLU networks is positioned as a novel contribution, the core theoretical results—such as the privacy bounds, convex reformulations, and the utility guarantees—are primarily based on established techniques and theorems from prior work. Although the context (applying these results to two-layer ReLU networks) may be new, the technical novelty and theoretical depth introduced by this paper appear limited. \n\n\n\n2. The paper studies DP-SGD for 2-layer ReLU networks. While it confused me that even the studied nonconvex problem is equal to a convex approximation problem under some conditions (e.g., the networks width $m$ is large enough), to achieve DP, in each iteration, DP-SGD adds Gaussian noise to each weight, that is, in each iteration, there are $m$ noise added to the gradient since there is $m$ neurons and $m$ weights. Since there are $m$ neurons in the hidden layer, this results in adding $m$ independent noise terms to the gradient at each step. Intuitively, as $m$ increases, the amount of injected noise also increases, which should degrade the performance of DP-SGD. However, both the privacy and utility bounds presented in the paper appear to be independent of $m$. This  puzzling me a lot.\n\nQUESTIONS:\nSee weakness, especially, \ncould the authors focus on answering the following two questions.\n\n1. Does network width affect the privacy and utility results in this setting? Is there some tradeoff between utility and network width $m$?\n\n2. Could the author conclude the novelty of the paper?\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}