Title: DP-SGD Without Clipping: The Lipschitz Neural Network Way
Venue: NeurIPS 2023
Accepted: no
Keywords: privacy

lipschitz

dpsgd

gnp

neural

framework
Abstract: State-of-the-art approaches for training Differentially Private (DP) Deep Neural Networks (DNN) faces difficulties to estimate tight bounds on the sensitivity of the network's layers, and instead rely on a process of per-sample gradient clipping. This clipping process not only biases the direction of gradients but also proves costly both in memory consumption and in computation cost. To provide sensitivity bounds and bypass the drawbacks of the clipping process, our theoretical analysis of Lipschitz constrained networks reveals an unexplored link between the Lipschitz constant with respect to their input and the one with respect to their parameters. By bounding the Lipschitz constant of each layer with respect to its parameters we guarantee DP training of these networks.  This analysis not only allows computing the aforementioned sensitivities at scale but also provides leads on to how maximize the gradient to noise ratio for fixed privacy guarantees. To facilitate the application of Lipschitz networks and foster robust and certifiable learning under privacy guarantees, we provide a Python package that implements building blocks allowing the construction and private training of such networks.

1

Abstract

State-of-the-art approaches for training Differentially Private (DP) Deep Neural Networks (DNN) faces difficulties to estimate tight bounds on the sensitivity of the network's layers, and instead rely on a process of per-sample gradient clipping. This clipping process not only biases the direction of gradients but also proves costly both in memory consumption and in computation. To provide sensitivity bounds and bypass the drawbacks of the clipping process, our theoretical analysis of Lipschitz constrained networks reveals an unexplored link between the Lipschitz constant with respect to their input and the one with respect to their parameters. By bounding the Lipschitz constant of each layer with respect to its parameters we guarantee DP training of these networks. This analysis not only allows the computation of the aforementioned sensitivities at scale but also provides leads on to how maximize the gradient-to-noise ratio for fixed privacy guarantees. To facilitate the application of Lipschitz networks and foster robust and certifiable learning under privacy guarantees, we provide a Python package that implements building blocks allowing the construction and private training of such networks.

1 Introduction

Machine learning relies more than ever on foundational models, and such practices raise questions about privacy. Differential privacy allows to develop methods for training models that preserve the privacy of individual data points in the training set. The field seeks to enable deep learning on sensitive data, while ensuring that models do not inadvertently memorize or reveal specific details about individual samples in their weights. This involves incorporating privacy-preserving mechanisms into the design of deep learning architectures and training algorithms, whose most popular example is Differentially Private Stochastic Gradient Descent (DP-SGD) [1]. One main drawback of classical DP-SGD methods is that they require costly per-sample backward processing and gradient clipping. In this paper, we offer a new method that unlocks fast differentially private training through the use of Lipschitz constrained neural networks. Additionally, this method offers new opportunities for practitioners that wish to easily "DP-fy" [2] the training procedure of a deep neural network.

Differential privacy fundamentals. Informally, differential privacy is a definition that quantifies how much the change of a single sample in a dataset affects the range of a stochastic function (here the DP training), called mechanism in this context. This quantity can be bounded in an inequality involving two parameters ϵ and δ . A mechanism fulfilling such inequality is said ( ϵ, δ ) -DP (see Definition 1). This definition is universally accepted as a strong guarantee against privacy leakages under various scenarii, including data aggregation or post-processing [3]. A popular rule of thumb suggests using ϵ ≤ 10 and δ < 1 N with N the number of records [2] for mild guarantees. In practice, most classic algorithmic procedures (called queries in this context) do not readily fulfill the definition for useful values of ( ϵ, δ ) , in particular the deterministic ones: randomization is mandatory. This randomization

comes at the expense of 'utility', i.e the usefulness of the output for downstream tasks [4]. The goal 37 is then to strike a balance between privacy and utility, ensuring that the released information remains 38 useful and informative for the intended purpose while minimizing the risk of privacy breaches. The 39 privacy/utility trade-off yields a Pareto front, materialized by plotting ϵ against a measurement of 40 utility, such as validation accuracy for a classification task. 41

Private gradient descent. The SGD algorithm consists of a sequence of queries that (i) take the 42 dataset in input, sample a minibatch from it, and return the gradient of the loss evaluated on the 43 minibatch, before (ii) performing a descent step following the gradient direction. The sensitivity (see 44 Definition 2) of SGD queries is proportional to the norm of the per-sample gradients. DP-SGD turns 45 each query into a Gaussian mechanism by perturbing the gradients with a noise ζ . The upper bound 46 on gradient norms is generally unknown in advance, which leads practitioners to clip it to C > 0 , in 47 order to bound the sensitivity manually. This is problematic for several reasons: 1. Hyper-parameter 48 search on the broad-range clipping value C is required to train models with good privacy/utility trade49 offs [5], 2. The computation of per-sample gradients is expensive: DP-SGD is usually slower and 50 consumes more memory than vanilla SGD, in particular for the large batch sizes often used in private 51 training [6], 3. Clipping the per-sample gradients biases their average [7]. This is problematic as the 52 average direction is mainly driven by misclassified examples, that carry the most useful information 53 for future progress. 54

An unexplored approach: Lipschitz constrained networks. We propose to train neural networks 55 for which the parameter-wise gradients are provably and analytically bounded during the whole 56 training procedure, in order to get rid of the clipping process. This allows for rapid training of models 57 without a need for tedious hyper-parameter optimization. 58

The main reason why this approach has not been experimented much in the past is that upper bounding 59 the gradient of neural networks is often intractable. However, by leveraging the literature of Lipschitz 60 constrained networks [8], we show that these networks allows to estimate their gradient bound. 61 This yields tight bounds on the sensitivity of SGD steps, making their transformation into Gaussian 62 mechanisms inexpensive - hence the name Clipless DP-SGD . 63

Informally, the Lipschitz constant quantifies the rate at which the function's output varies with respect 64 to changes in its input. A Lipschitz constrained network is one in which its weights and activations 65 are constrained such that it can only represent l -Lipschitz functions. In this work, we will focus our 66 attention on feed-forward networks (refer to Definition 3). Note that the most common architectures, 67 such as Convolutional Neural Networks (CNNs), Fully Connected Networks (FCNs), Residual 68 Networks (ResNets), or patch-based classifiers (like MLP-Mixers), all fall under the category of 69 feed-forward networks. We will also tackle the particular case of Gradient Norm Preserving (GNP) 70 networks, a subset of Lipschitz networks that enjoy tighter bounds (see appendix). 71

Contributions 72

73

74

75

76

77

78

While the properties of Lipschitz constrained networks regarding their inputs are well explored, the properties with respect to its parameters remain non-trivial. This work provides a first step to fill this gap: our analysis shows that under appropriate architectural constraints, a l -Lipschitz network has a tractable, finite Lipschitz constant with respect to its parameters. We prove that this Lipschitz constant allows for easy estimation of the sensitivity of the gradient computation queries. The prerequisite and details of the method to compute the sensitivities are explained in Section 2.

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

Our contributions are the following:

We extend the field of applications of Lipschitz constrained neural networks. So far the literature focused on Lipschitzness with respect to the inputs : we extend the framework to compute the Lipschitzness with respect to the parameters . This is exposed in Section 2.

Wepropose a general framework to handle layer gradient steps as Gaussian mechanisms that depends on the loss and the model structure. Our framework covers widely used architectures, including VGG and ResNets.

We show that SGD training of deep neural networks can be achieved without gradient clipping using Lipschitz layers. This allows the use of larger networks and larger batch sizes, as illustrated by our experiments in Section 4.

We establish connections between Gradient Norm Preserving (GNP) networks and improved privacy/utility trade-offs (Section 3.1).

Finally, a Python package 1 companions the project, with pre-computed Lipschitz constant and noise for each layer type, ready to be forked on any problem of interest (Section 3.2).

1.1 Differential Privacy and Lipschitz Networks

The definition of DP relies on the notion of neighboring datasets, i.e datasets that vary by at most one example. We highlight below the central tools related to the field, inspired from [9].

Definition 1 ( ( ϵ, δ ) -Differential Privacy) . A labeled dataset D is a finite collection of input/label pairs D = { ( x 1 , y 1 ) , ( x 2 , y 2 ) , . . . ... ( x N , y N ) } . Two datasets D and D ′ are said to be neighboring for the 'replace-one' relation if they differ by at most one sample: D ′ = D∪{ ( x ′ i , y ′ i ) } \ { ( x i , y i ) } . Let ϵ and δ be two non-negative scalars. A mechanism A is ( ϵ, δ ) -DP if for any two neighboring datasets D and D ′ , and for any S ⊆ range ( A ) :

A cookbook to create a ( ϵ, δ ) -DP mechanism from a query is to compute its sensitivity ∆ (see Definition 2), and to perturb its output by adding a Gaussian noise of predefined variance ζ 2 = ∆ 2 σ 2 , where the ( ϵ, δ ) -DP guarantees depends on σ . This yields what is called a Gaussian mechanism [3].

Definition 2 ( l 2 -sensitivity) . Let M be a query mapping from the space of the datasets to R p . Let N be the set of all possible pairs of neighboring datasets D , D ′ . The l 2 sensitivity of M is defined by:

Differentially Private SGD. The classical algorithm keeps track of ( ϵ, δ ) -DP values with a moments accountant [1] which allows to keep track of privacy guarantees at each epoch, by composing different sub-mechanisms. For a dataset with N records and a batch size b , it relies on two parameters: the sampling ratio p = b N and the 'noise multiplier' σ defined as the ratio between effective noise strength ζ and sensitivity ∆ . Bounds on gradient norm can be turned into bounds on sensitivity of SGD queries. In 'replace-one' policy for ( ϵ, δ ) -DP accounting, if the gradients are bounded by K > 0 , the sensitivity of the gradients averaged on a minibatch of size b is ∆ = 2 K/b ..

Crucially, the algorithm requires a bound on ∥∇ θ L (ˆ y, y ) ∥ 2 ≤ K . The whole difficulty lies in bounding tightly this value in advance for neural networks. Currently, gradient clipping serves as a patch to circumvent the issue [1]. Unfortunately, clipping individual gradients in the batch is costly and will bias the direction of their average, which may induce underfitting [7].

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

Lipschitz constrained networks. Our proposed solution comes from the observation that the norm of the gradient and the Lipschitz constant are two sides of the same coin. The function f : R m → R n is said l -Lipschitz for l 2 norm if for every x, y ∈ R m we have ∥ f ( x ) -f ( y ) ∥ 2 ≤ l ∥ x -y ∥ 2 . Per Rademacher's theorem [10], its gradient is bounded: ∥∇ x f ∥ ≤ l . Reciprocally, continuous functions gradient bounded by l are l -Lipschitz.

In Lipschitz networks, the literature has predominantly concentrated on investigating the control of Lipschitzness with respect to the inputs (i.e bounding ∇ x f ), primarily motivated by concerns of robustness [11]. However, in this work, we will demonstrate that it is also possible to control Lipschitzness with respect to parameters (i.e bounding ∇ θ f ), which is essential for ensuring privacy. Our first contribution will point out the tight link that exists between those two quantities.

Definition 3 (Lipschitz feed-forward neural network) . A feedforward neural network of depth D , with input space X ⊂ R n , output space Y ⊂ R K (e.g logits), and parameter space Θ ⊂ R p , is a parameterized function f : Θ ×X → Y defined by the sequential composition of layers f d :

f

(

θ, x

) := (

f

D

(

θ

d

)

◦

. . .

◦

f

2

(

θ

2

)

◦

f

1

(

θ

1

)) (

x

)

.

(3)

The parameters of the layers are denoted by θ = ( θ d ) 1 ≤ d ≤ D ∈ Θ . For affine layers, it corresponds to bias and weight matrix θ d = ( W d , b d ) . For activation functions, there is no parameters: θ d = ∅ .

Lipschitz networks are feed-forward networks, with the additionnal constraint that each layer x d ↦→ f d ( θ d , x d ) := y d is l d -Lipschitz for all θ d . Consequently, the function x ↦→ f ( θ, x ) is l -Lipschitz with l = l 1 × . . . × l d for all θ ∈ Θ .

In practice, this is enforced by using activations with Lipschitz constant l d , and by applying a constraint Π : R p → Θ on the weights of affine layers. This corresponds to spectrally normalized matrices [12, 13], since for affine layers we have l d = ∥ W d ∥ 2 := max ∥ x ∥≤ 1 ∥ W d x ∥ 2 hence Θ = {∥ W d ∥ ≤ l q } .

The seminal work of [8] proved that universal approximation in the set of l -Lipschitz functions was achievable by this family of architectures. Concurrent approaches are based on regularization (like in [14, 15, 16]) but they fail to produce formal guarantees. While they have primarily been studied in the context of adversarial robustness [11, 17], recent works have revealed additional properties of these networks, such as improved generalization [13, 18]. However, the properties of their parameter gradient ∇ θ f ( θ, x ) remain largely unexplored.

2 Clipless DP-SGD with l -Lipschitz networks

Our framework consists of 1. a method that computes the maximum gradient norm of a network with respect to its parameters to obtain a per-layer sensitivity ∆ d , 2. a moments accountant that relies on the per-layer sensitivities to compute ( ϵ, δ ) -DP guarantees. The method 1. is based on the recursive formulation of the chain rule involved in backpropagation, while 2. keeps track of ( ϵ, δ ) -DP values with RDP accounting. It requires some natural assumptions that we highlight below.

Requirement 1 (Lipschitz loss.) . The loss function ˆ y ↦→L (ˆ y, y ) must be L -Lipschitz with respect to the logits ˆ y for all ground truths y ∈ Y . This is notably the case of Categorical Softmax-Crossentropy.

The Lipschitz constants of common classification losses can be found in the appendix.

Requirement 2 (Bounded input) . There exists X 0 > 0 such that for all x ∈ X we have ∥ x ∥ ≤ X 0 .

While there exist numerous approaches for the parametrization of Lipschitz networks (e.g differentiable re-parametrization [19, 8], optimization over matrix manifolds [20] or projections [21]), our framework only provides sensitivity bounds for projection-based algorithms (see appendix).

Requirement 3 (Lipschitz projection) . The Lipschitz constraints must be enforced with a projection operator Π : R p → Θ . This corresponds to Tensorflow [22] constraints and Pytorch [23] hooks . Projection is a post-processing of private gradients: it induces no privacy leakage [3].

To compute the per-layer sensitivities, our framework mimics the backpropagation algorithm, where Vector-Jacobian products (VJP) are replaced by Scalar-Scalar products of element-wise bounds. For an arbitrary layer x d ↦→ f d ( θ d , x d ) := y d the operation is sketched below:

Vector-Jacobian product: backpropagate gradients

Scalar-Scalar product: backpropagate bounds

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

Xd-1

Forwardinputbounds

||≤Xd-1

Il.fa(0a,α)Il

PX

DP Layer

Backpropagate gradient bounds

Gd-1

fa(0d,α)

Gd

0fa (0d,x)

×

|αl≤Xd1

Sensitivity

ld,c

Ofa(0d,x)

×

00d

ld,0

|αl≤Xd1

The notation ∥ · ∥ 2 must be understood as the spectral norm for Jacobian matrices, and the Euclidean 163 norm for gradient vectors. The scalar-scalar product is inexpensive. For Lipschitz layers the spectral 164 norm of the Jacobian ∥ ∂f ∂x ∥ is kept constant during training with projection operator Π . The bound of 165 the gradient with respect to the parameters then takes a simple form: 166

Once again the operation is inexpensive. The upper bound ∥ ∥ ∥ ∂f ∂θ ∥ ∥ ∥ 2 typically depends on the supremum of ∥ x d ∥ 2 , that can also be analytically bounded, as exposed in the following section.

2.1 Backpropagation for bounds

The pseudo-code of Clipless DP-SGD is sketched in Algorithm 2. The algorithm avoids clipping by computing a per-layer bound on the element-wise gradient norm. The computation of this per-layer bound is described by Algorithm 1 (graphically explained in Figure 2). Crucially, it requires to compute the spectral norm of the Jacobian of each layer with respect to input and parameters.

Input bound propagation (line 2). We compute X d = max ∥ x ∥≤ X d -1 ∥ f d ( x ) ∥ 2 . For activation functions it depends on their range. For linear layers, it depends on the spectral norm of the operator itself. This quantity can be computed with SVD or Power Iteration [24, 19], and constrained during training using projection operator Π . In particular, it covers the case of convolutions, for which tight bounds are known [25]. For affine layers, it additionally depends on the amplitude of the bias ∥ b d ∥ .

Remark 1 (Tighter bounds in literature.) . Although libraries such as Decomon [26] or auto-LiRPA [27] provide tighter bounds X d via linear relaxations [28, 29], our approach is capable of delivering practically tighter bounds than worst-case scenarios thanks to the projection operator Π , while also being significantly less computationally expensive. Moreover, hybridizing our method with scalable certification methods can be a path for future extensions.

Computing maximum gradient norm (line 6). Webound the Jacobian ∂f d ( θ d ,x ) ∂θ d . In neural networks, the parameterized layers f ( θ, x ) (fully connected, convolutions) are bilinear operators. Hence we typically obtain bounds of the form:

where K ( f d , Θ d ) is a constant that depends on the nature of the operator. X d -1 is obtained in line 2 with input bound propagation. Values of K ( f d , θ d ) for popular layers are pre-computed in the library.

Backpropagate cotangeant vector bounds (line 7). We bound the Jacobian ∂f d ( θ d ,x ) ∂x . For activation functions this value can be hard-coded, while for affine layers it is the spectral norm of the linear operator. Like before, this value is constrained with projection operator Π .

2.2 Privacy accounting for Clipless DP-SGD

Two strategies are available to keep track of ( ϵ, δ ) values as the training progresses, based on accounting either a per-layer 'local' sensitivity, either by aggregating them into a 'global' sensitivity.

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

Algorithm 1 Backpropagation for Bounds ( f, X )

Algorithm 2 Clipless DP-SGD with local sensitivity accounting

Input

: Feed-forward architecture f ( θ, · ) = f D ( θ D , · ) ◦ . . . ◦ f 1 ( θ 1 , · )

Input

: Initial weights θ = ( θ 1 , θ 1 , . . . θ D ) , learning rate η , noise multiplier σ .

1: repeat

2: ∆ 1 , ∆ 2 . . . ∆ D ← Backpropagation for Bounds ( f, X ) .

3: Update Moment Accountant state with local sensitivities ∆ 1 , ∆ 2 , . . . ∆ d .

4: Sample a batch B = { ( x 1 , y 1 ) , ( x 2 , y 2 ) , . . . , ( x b , y b ) } .

5: Compute per-layer averaged gradient: g d := 1 b ∑ b i =1 ∇ θ d L ( f ( θ, x i ) , y i )) .

6: Sample local noise: ζ d ∼ N (0 , σ ∆ d ) .

7: Perform noisified gradient step: θ d ← θ d -η ( g d + ζ d ) .

8: Enforce Lipschitz constraint with projection: θ d ← Π( θ d ) .

9: until privacy budget ( ϵ, δ ) -DP budget has been reached.

The 'global' strategy. Illustrated in the appendix,this strategy simply aggregates the individual sensitivities ∆ d of each layer to obtain the global sensitivity of the whole gradient vector ∆ = √∑ d ∆ 2 d . The origin of the clipping-based version of this strategy can be traced back to [30]. With noise variance σ 2 ∆ 2 we recover the accountant that comes with DP-SGD. It tends to overestimate the true sensitivity (in particular for deep networks), but its implementation is straightforward with existing tools.

The 'local' strategy. Recall that we are able to characterize the sensitivity ∆ d of every layer of the network. Hence, we can apply a different noise to each of the gradients. We dissect the whole training procedure in Figure 3. At same noise multiplier σ , it tends to produce a higher value of ϵ per epoch than 'global' strategy, but has the advantage over the latter to add smaller effective noise ζ to each weight.

We rely on the autodp 2 library [32, 33, 34] as it uses the Renyi Differential Privacy (RDP) adaptive composition theorem [35, 36], that ensures tighter bounds than naive DP composition.

3 From theory to practice

Beyond the application of Algorithms 1 and 2, our framework provides numerous opportunities to enhance our understanding of prevalent techniques identified in the literature. An in-depth exploration of these is beyond the scope of this work, so we focus on giving insights on promising tracks based on our theoretical analysis. In particular, we discuss how the tightness of the bound provided by Algorithm 1 can be influenced by working on the architecture, the input pre-processing and the loss post-processing.

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

Adaptive

Stochastic

Composition

composition

batches

acrosstrainsteps

Layers (d)

(α,8)-RDP

(at,8)-RDP

(α,8)a-RDP

(aT,8)-RDP

Gaussianmechanism

Nonisotropic

Subsampling

of sensitivityd

Mechanism

Train steps

3.1 Gradient Norm Preserving networks

We can manually derive the bounds obtained from Algorithm 2 across diverse configurations. Below, we conduct a sensitivity analysis on l -Lipschitz networks.

Theorem (informal) 1. Gradient Norm of Lipschitz Networks. Assume that every layer f d is K -Lipschitz, i.e l 1 = · · · = l D = K . Assume that every bias is bounded by B . We further assume that each activation is centered in zero (e.g ReLU, tanh, GroupSort). We recall that θ = [ θ 1 , θ 2 , . . . θ D ] . Then the global upper bound of Algorithm 2 can be expanded analytically.

If K < 1 we have: ∥∇ θ L ( f ( θ, x ) , y ) ∥ 2 = O ( L ( K D ( X 0 + B ) + 1 )) . D

Due to the K ≪ 1 term this corresponds to a vanishing gradient phenomenon [37]. The output of the network is essentially independent of its input, and the training is nearly impossible.

If K > 1 we have: ∥∇ θ L ( f ( θ, x ) , y ) ∥ 2 = O ( LK D ( X 0 + B ) ) . D

Due to the K ≫ 1 term this corresponds to an exploding gradient phenomenon [38]. The upper bound becomes vacuous for deep networks: the added noise ζ is at risk of being too high.

If K = 1 we have: ∥∇ θ L ( f ( θ, x ) , y ) ∥ 2 = O L X 0 + D + BX 0 D + BD 3 / 2 , which for linear layers without biases further simplify to O ( L ( X 0 + √ D )) .

The formal statement can be found in appendix. From Theorem 1 we see that most favorable bounds are achieved by 1-Lipschitz neural networks with 1-Lipschitz layers. In classification tasks, they are not less expressive than conventional networks [18]. Hence, this choice of architecture is not at the expense of utility. Moreover an accuracy/robustness trade-off exists, determined by the choice of loss function [18]. However, setting K = 1 merely ensures that ∥∇ x f ∥ ≤ 1 , and in the worst-case scenario we have ∥∇ x f ∥ < 1 almost everywhere. This could result in a situation where the bound of case 3 in Theorem 1 is not tight, leading to an underfitting regime as in case K < 1 . With Gradient Norm Preserving (GNP) networks [17], we expect to mitigate this issue.

Controlling K with Gradient Norm Preserving (GNP) networks. GNP networks are 1-Lipschitz neural networks with the additional constraint that the Jacobian of layers consists of orthogonal matrices. They fulfill the Eikonal equation ∥ ∥ ∥ ∂f d ( θ d ,x d ) ∂x d ∥ ∥ ∥ 2 = 1 for any intermediate activation f d ( θ d , x d ) . Without biases these networks are also norm preserving: ∥ f ( θ, x ) ∥ = ∥ x ∥ .

As a consequence, the gradient of the loss with respect to the parameters is easily bounded by

which for weight matrices W d further simplifies to ∥∇ W d L∥ ≤ ∥∇ y D L∥×∥ f d -1 ( θ d -1 , x d -1 ) ∥ . We 243 see that this upper bound crucially depends on two terms than can be analyzed separately. On one 244 hand, ∥ f d -1 ( θ d -1 , x d -1 ) ∥ depends on the scale of the input. On the other, ∥∇ y D L∥ depends on the 245 loss, the predictions and the training stage. We show below how to intervene on these two quantities. 246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

Remark 2 (Implementation of GNP Networks) . In practice, GNP are parametrized with GroupSort activation [8, 39], Householder activation [40], and orthogonal weight matrices [17, 41]. Strict orthogonality is challenging to enforce, especially for convolutions for which it is still an active research area (see [42, 43, 44, 45, 46] and references therein). Our line of work traces an additional motivation for the development of GNP and the bounds will strengthen as the field progresses.

Controlling X 0 with input pre-processing. The weight gradient norm ∥∇ θ d L∥ indirectly depends on the norm of the inputs. This observation implies that the pre-processing of input data significantly influences the bounding of sensitivity. Multiple strategies are available to keep the input's norm under control: projection onto the ball ('norm clipping'), or projection onto the sphere ('normalization'). In the domain of natural images for instance, this result sheds light on the importance of color space such as RGB, HSV, YIQ, YUV or Grayscale. These strategies are natively handled by our library.

Controlling L with the hybrid approach, loss gradient clipping. As training progresses, the magnitude of ∥∇ f L∥ tends to diminish when approaching a local minima, quickly falling below the upper bound and diminishing the gradient norm to noise ratio. To circumvent the issue, the gradient clipping strategy is still available in our framework. Crucially, instead of clipping the parameter gradient ∇ θ L , any intermediate gradient ∇ f d L can be clipped during backpropagation. This can be achieved with a special ' clipping layer ' that behaves like the identity function at the forward pass, and clips the gradient during the backward pass. The resulting cotangeant vector is not a true gradient anymore, but rather a descent direction [47]. In vanilla DP-SGD the clipping is applied on the batched gradient ∇ W d L of size b × h 2 for matrix weight W d ∈ R h × h and clipping this vector can cause memory issues or slowdowns [6]. In our case, ∇ y D L is of size b × h which reduces overhead.

3.2 Lip-dp library

To foster and spread accessibility, we provide an opensource tensorflow library for Clipless DP-SGD training, named lip-dp . It provides an exposed Keras API for seamless usability. It is implemented as a wrapper over the Lipschitz layers of deel-lip 3 library [48]. Its usage is illustrated in Figure 1.

4 Experimental results

We validate our implementation with a speed benchmark against competing approaches, and we present the privacy/utility Pareto front that can be obtained with GNP networks.

Speed and memory consumption. We benchmarked the median runtime per epoch of vanilla DP-SGD against the one of Clipless DP-SGD, on a CNN architecture and its Lipschitz equivalent respectively. The experiment was run on a GPU with 48GB video memory. We compare against the implementation of tf_privacy , opacus and optax . In order to allow a fair comparison, when evaluating Opacus, we reported the runtime with respect to the logical batch size, while capping the physical batch size to avoid Out Of Memory error (OOM). Although our library does not implement logical batching yet, it is fully compatible with this feature.

An advantage of projection Π over per-sample gradient clipping is that the projection cost is

S

12

Optimizer

atch

10

[opacus]DP-SGD

bat

[lip-dp] Clipless DP-SGD

8

[optax]DP-SGD

p

[tf_privacy]DP-SGD

6

Runt

4

130K

510K

ian

2,000K

2

OutOf Memory Error

(limit:48GB)

M

0

BatchSize

10K

20K

30K

40K

50K

independent of the batch size. Fig 4 validates that our method scales much better than vanilla DP-SGD, and is compatible with large batch sizes. It offers several advantages: firstly, a larger batch size contributes to a decrease of the sensitivity ∆ ∝ 1 /b , which diminishes the ratio between noise and gradient norm. Secondly, as the batch size b increases, the variance decreases at the parametric rate O ( √ b ) (as demonstrated in appendix), aligning with expectations. This observation does not apply to DP-SGD: gradient clipping biases the direction of the average gradient, as noticed by [7].

(a)

MNIST

.

(b)

F-MNIST

.

(c)

CIFAR-10

.

1.00

1.00

0.70

0.90

0.90

0.80

0.80

0.60

Accura

0.70

Accurac

2

0.70

0.50

X

0.60

0.60

alidation

0.50

alidation

0.50

dation

0.40

0.40

Abadiet al.(2016)

ClipLessDP-SGD:Paretofront

0.40

0.30

Valida

0.30

ClipLessDP-SGD:Pareto front

Abadi et al. (2016)

0.30

Bu et al. (2019)

0.20

0.20

0.10

Chen&Lee(2020)

Feldman&Zrnic.(2020)

0.20

0.10

ClipLess DP-SGD:Pareto front

Chen&Lee(2020)

0.10

+

Chen&Lee(2020)

Papernot et al. (2021)

+

Papernotetal.(2021)

+

Papernotetal.(2021)

0.0

1.0

2.0

3.0

0.0

1.0

2.0

3.0

0.0

1.0

2.0

3.04.05.06.07.0

8.0

εat6=1e-5

εat6=1e-5

ε at =1e-5

Pareto front of privacy/utility trade-off. We performed a search over a broad range of hyper297 parameters values to cover the Pareto front between utility and privacy. Results are reported in 298 Figure 5. We emphasize that our experiments did not use the elements behind the success of most 299 recent papers (pre-training, data preparation, or handcrafted feature are examples). Hence our 300 results are more representative of the typical performance that can be obtained in an 'out of the 301 box' setting. Future endeavors or domain-specific engineering can enhance the performance even 302 further, but such improvements currently lie beyond the scope of our work. We also benchmarked 303 architectures inspired from VGG [52], Resnet [53] and MLP_Mixers [54] see appendix for more 304 details. Following standard practices of the community [2], we used sampling without replacement at 305 each epoch (by shuffling examples), but we reported ϵ assuming Poisson sampling to benefit from 306 privacy amplification [31]. We also ignore the privacy loss that may be induced by hyper-parameter 307 search, which is a limitation per recent studies [5], but is common practice. 308

309

310

311

312

313

314

315

316

317

318

319

320

321

5 Limitations and future work

Although this framework offers a novel approach to address differentially private training, it introduces new challenges. We primary rely on GNP networks, where high performing architectures are quite different from the usual CNN architectures. As emphasized in Remark 2, we anticipate that progress in these areas would greatly enhance the effectiveness of our approach. Additionally, to meet requirement 3, we rely on projections, necessitating additional efforts to incorporate recent advancements associated with differentiable reparametrizations [42, 43]. It is worth noting that our methodology is applicable to most layers. Another limitation of our approach is the accurate computation of sensitivity ∆ , which is challenging due to the non-associativity of floating-point arithmetic and its impact on numerical stability [55]. This challenge is exacerbated on GPUs, where operations are inherently non-deterministic [56]. Finally, as mentioned in Remark 1, our propagation bound method can be refined.

6 Concluding remarks and broader impact

Besides its main focus on differential privacy, our work provides (1) a motivation to further develop 322 Gradient Norm Preserving architectures . Furthermore, the development of networks with known 323 Lipschitz constant with respect to parameters is a question of independent interest, (2) a useful tool for 324 the study of the optimization dynamics in neural networks. Finally, Lipschitz networks are known 325 to enjoy certificates against adversarial attacks [17, 57], and from generalization guarantees [13], 326 without cost in accuracy [18]. We advocate for the spreading of their use in the context of robust and 327 certifiable learning. 328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

References

Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security , pages 308-318, 2016.

Natalia Ponomareva, Hussein Hazimeh, Alex Kurakin, Zheng Xu, Carson Denison, H Brendan McMahan, Sergei Vassilvitskii, Steve Chien, and Abhradeep Thakurta. How to dp-fy ml: A practical guide to machine learning with differential privacy. arXiv preprint arXiv:2303.00654 , 2023.

Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Theory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4-7, 2006. Proceedings 3 , pages 265-284. Springer, 2006.

Mário S Alvim, Miguel E Andrés, Konstantinos Chatzikokolakis, Pierpaolo Degano, and Catuscia Palamidessi. Differential privacy: on the trade-off between utility and information leakage. In Formal Aspects of Security and Trust: 8th International Workshop, FAST 2011, Leuven, Belgium, September 12-14, 2011. Revised Selected Papers 8 , pages 39-54. Springer, 2012.

Nicolas Papernot and Thomas Steinke. Hyperparameter tuning with renyi differential privacy. In International Conference on Learning Representations , 2022.

Jaewoo Lee and Daniel Kifer. Scaling up differentially private deep learning with fast perexample gradient clipping. Proceedings on Privacy Enhancing Technologies , 2021(1), 2021.

Xiangyi Chen, Steven Z Wu, and Mingyi Hong. Understanding gradient clipping in private sgd: Ageometric perspective. Advances in Neural Information Processing Systems , 33:13773-13782, 2020.

Cem Anil, James Lucas, and Roger Grosse. Sorting out lipschitz function approximation. In International Conference on Machine Learning , pages 291-301. PMLR, 2019.

Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science , 9(3-4):211-407, 2014.

Leon Simon et al. Lectures on geometric measure theory . The Australian National University, Mathematical Sciences Institute, Centre . . . , 1983.

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations , 2014.

Yuichi Yoshida and Takeru Miyato. Spectral norm regularization for improving the generalizability of deep learning. arXiv preprint arXiv:1705.10941 , 2017.

Peter L Bartlett, Dylan J Foster, and Matus Telgarsky. Spectrally-normalized margin bounds for neural networks. In Proceedings of the 31st International Conference on Neural Information Processing Systems , pages 6241-6250, 2017.

Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Improved training of wasserstein gans. In Advances in Neural Information Processing Systems , volume 30, pages 5767-5777. Curran Associates, Inc., 2017.

Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier. Parseval networks: Improving robustness to adversarial examples. In International Conference on Machine Learning , pages 854-863. PMLR, 2017.

Henry Gouk, Eibe Frank, Bernhard Pfahringer, and Michael J Cree. Regularisation of neural networks by enforcing lipschitz continuity. Machine Learning , 110:393-416, 2021.

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

Qiyang Li, Saminul Haque, Cem Anil, James Lucas, Roger B Grosse, and Jörn-Henrik Jacobsen. Preventing gradient attenuation in lipschitz constrained convolutional networks. In Advances in Neural Information Processing Systems (NeurIPS) , volume 32, Cambridge, MA, 2019. MIT Press.

Louis Béthune, Thibaut Boissin, Mathieu Serrurier, Franck Mamalet, Corentin Friedrich, and Alberto Gonzalez Sanz. Pay attention to your loss : understanding misconceptions about lipschitz neural networks. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems , 2022.

Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. arXiv preprint arXiv:1802.05957 , 2018.

P-A Absil, Robert Mahony, and Rodolphe Sepulchre. Optimization algorithms on matrix manifolds . Princeton University Press, 2008.

Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In International conference on machine learning , pages 214-223. PMLR, 2017.

Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. Tensorflow: Large-scale machine learning on heterogeneous distributed systems, 2015.

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems , 32, 2019.

Lloyd N Trefethen and David Bau. Numerical linear algebra , volume 181. Siam, 2022.

S Singla and S Feizi. Fantastic four: Differentiable bounds on singular values of convolution layers. In International Conference on Learning Representations (ICLR) , 2021.

Airbus. Decomon. https://github.com/airbus/decomon , 2023.

Kaidi Xu, Zhouxing Shi, Huan Zhang, Yihan Wang, Kai-Wei Chang, Minlie Huang, Bhavya Kailkhura, Xue Lin, and Cho-Jui Hsieh. Automatic perturbation analysis for scalable certified robustness and beyond. Advances in Neural Information Processing Systems , 33:1129-1141, 2020.

Gagandeep Singh, Timon Gehr, Markus Püschel, and Martin Vechev. An abstract domain for certifying neural networks. Proceedings of the ACM on Programming Languages , 3(POPL):130, 2019.

Huan Zhang, Tsui-Wei Weng, Pin-Yu Chen, Cho-Jui Hsieh, and Luca Daniel. Efficient neural network robustness certification with general activation functions. Advances in neural information processing systems , 31, 2018.

H Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang. Learning differentially private recurrent language models. In International Conference on Learning Representations , 2018.

Borja Balle, Gilles Barthe, and Marco Gaboardi. Privacy amplification by subsampling: Tight analyses via couplings and divergences. Advances in Neural Information Processing Systems , 31, 2018.

Yu-Xiang Wang, Borja Balle, and Shiva Prasad Kasiviswanathan. Subsampled rényi differential privacy and analytical moments accountant. In The 22nd International Conference on Artificial Intelligence and Statistics , pages 1226-1235. PMLR, 2019.

Yuqing Zhu and Yu-Xiang Wang. Possion subsampled rényi differential privacy. In International 424 Conference on Machine Learning , pages 7634-7642. PMLR, 2019. 425

Yuqing Zhu and Yu-Xiang Wang. Improving sparse vector technique with renyi differential 426 privacy. Advances in Neural Information Processing Systems , 33:20249-20258, 2020. 427

428

429

Ilya Mironov. Rényi differential privacy. In 2017 IEEE 30th computer security foundations symposium (CSF) , pages 263-275. IEEE, 2017.

430

431

432

433

434

435

436

437

438

439

440

441

442

Ilya Mironov, Kunal Talwar, and Li Zhang. R \ 'enyi differential privacy of the sampled gaussian mechanism. arXiv preprint arXiv:1908.10530 , 2019.

Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. On the difficulty of training recurrent neural networks. In International conference on machine learning , pages 1310-1318. Pmlr, 2013.

Yoshua Bengio, Patrice Simard, and Paolo Frasconi. Learning long-term dependencies with gradient descent is difficult. IEEE transactions on neural networks , 5(2):157-166, 1994.

Ugo Tanielian and Gerard Biau. Approximating lipschitz continuous functions with groupsort neural networks. In International Conference on Artificial Intelligence and Statistics , pages 442-450. PMLR, 2021.

Zakaria Mhammedi, Andrew Hellicar, Ashfaqur Rahman, and James Bailey. Efficient orthogonal parametrisation of recurrent neural networks using householder reflections. In International Conference on Machine Learning , pages 2401-2409. PMLR, 2017.

Shuai Li, Kui Jia, Yuxin Wen, Tongliang Liu, and Dacheng Tao. Orthogonal deep neural 443 networks. IEEE transactions on pattern analysis and machine intelligence , 43(4):1352-1368, 444 2019. 445

446

447

Asher Trockman and J Zico Kolter. Orthogonalizing convolutional layers with the cayley transform. In International Conference on Learning Representations , 2021.

448

449

450

451

452

453

454

455

456

457

458

459

460

461

462

463

464

465

Sahil Singla and Soheil Feizi. Skew orthogonal convolutions. In International Conference on Machine Learning , pages 9756-9766. PMLR, 2021.

El Mehdi Achour, François Malgouyres, and Franck Mamalet. Existence, stability and scalability of orthogonal convolutional neural networks. The Journal of Machine Learning Research , 23(1):15743-15798, 2022.

Sahil Singla and Soheil Feizi. Improved techniques for deterministic l2 robustness. arXiv preprint arXiv:2211.08453 , 2022.

Xiaojun Xu, Linyi Li, and Bo Li. Lot: Layer-wise orthogonal training on improving l2 certified robustness. arXiv preprint arXiv:2210.11620 , 2022.

Stephen P Boyd and Lieven Vandenberghe. Convex optimization . Cambridge university press, 2004.

Mathieu Serrurier, Franck Mamalet, Alberto González-Sanz, Thibaut Boissin, Jean-Michel Loubes, and Eustasio Del Barrio. Achieving robustness in classification using optimal transport with hinge regularization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 505-514, 2021.

Nicolas Papernot, Abhradeep Thakurta, Shuang Song, Steve Chien, and Úlfar Erlingsson. Tempered sigmoid activations for deep learning with differential privacy. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 9312-9321, 2021.

Florian Tramèr and Dan Boneh. Differentially private learning needs better features (or much 466 more data), 2021. 467

Soham De, Leonard Berrada, Jamie Hayes, Samuel L Smith, and Borja Balle. Unlock468 ing high-accuracy differentially private image classification through scale. arXiv preprint 469 arXiv:2204.13650 , 2022. 470

Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale 471 image recognition. arXiv preprint arXiv:1409.1556 , 2014. 472

473

474

475

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770-778, 2016.

476

477

478

479

480

481

482

483

484

485

486

487

Ilya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al. Mlpmixer: An all-mlp architecture for vision. Advances in neural information processing systems , 34:24261-24272, 2021.

David Goldberg. What every computer scientist should know about floating-point arithmetic. ACM computing surveys (CSUR) , 23(1):5-48, 1991.

Hadi Jooybar, Wilson WL Fung, Mike O'Connor, Joseph Devietti, and Tor M Aamodt. Gpudet: a deterministic gpu architecture. In Proceedings of the eighteenth international conference on Architectural support for programming languages and operating systems , pages 1-12, 2013.

Mahyar Fazlyab, Alexander Robey, Hamed Hassani, Manfred Morari, and George Pappas. Efficient and accurate estimation of lipschitz constants for deep neural networks. Advances in Neural Information Processing Systems , 32, 2019.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\n- The paper investigates the use of Lipschitz constrained networks to replace clipping functions and limit gradient sensitivity in DP-SGD.\n- Lipschitz constrained networks are utilized as an alternative to clipping in order to address the issues of clipping's impact on convergence and performance in DP-SGD.\n\nSTRENGTHS:\n- The idea of removing clipping as an alternative to clipping itself is promising, as clipping is known to have detrimental effects on convergence and performance of DP-SGD, even without noise addition [1].\n- The paper introduces the replacement of Vector-Jacobian product with Scalar-Scalar product to reduce computational complexity. The proposed methods outperform existing SGD approaches by a significant margin in terms of speed, which is crucial as memory usage and time inefficiency are major drawbacks of DP-SGD.\n\n[1] Differntially Private Shaprness-Aware Training (ICML’23)\n\nWEAKNESSES:\n- Please refer to the questions.\n- (Minor) There are several typos, such as the use of \"cotangeant vector\" which sounds little awkward, and inconsistencies in figure references (e.g., Fig 4 vs. Figure 5). Please carefully review the grammar and correct the typos.\n\nQUESTIONS:\n- The main concern is whether \"Clipless DP-SGD\" is truly effective compared to \"Clip DP-SGD\" in terms of training stability and performance. While it is understood that the main idea is to address the detrimental effects of clipping by proposing a clipping-free approach, the proposed method still relies on approximations or has drawbacks such as the Lipschitz constrained condition or gradient approximation with scalar-scalar form. Thus, (i) Can you provide mathematical bounds to support the strength of your methods? or (ii) Can you provide additional experimental results beyond Figure 5 to further demonstrate the performance of your approach?\n- Recent papers have explored different ways of using DP-SGD, particularly in fine-tuning tasks, to mitigate memory usage and time inefficiency [2,3]. Considering that their approaches may differ from training from scratch tasks, it would be worth to investigate whether your proposed networks are well-suited for fine-tuning tasks to improve generalization performance.\n- Can you provide more details on the implementation of GNP networks in DP-SGD? The implementation of 1-Lipschitz seems to be naive and detrimental to traininig, similar to selecting a norm clipping value in clip DP-SGD. However, the authors mention a problem in this implementation, stating, \"With Gradient Norm Preserving (GNP) networks, we expect to mitigate this issue\" (line number 236). Could you further explain how your methods address this problem? or let me know that my understanding is incorrect.\n\nI’d be happy to raise my score if the authors can address the weaknesses/questions in the rebuttal.\n\n[2] Large language models can be strong differentially private learners (ICLR’22)\n\n[3] Scalable and efficient training of large convolutional neural networks with differential\nprivacy (NeurIPS’22)\n\nLIMITATIONS:\nThe paper provides a detailed discussion of its limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nDifferentially Private (DP) Deep Neural Networks (DNNs) face challenges in estimating tight bounds on the sensitivity of the network’s layers. Instead, they rely on a per-sample gradient clipping process (as argued by the authors). This process not only biases the direction of the gradients but also proves costly in both memory consumption and computation. To provide sensitivity bounds and avoid the drawbacks of the clipping process, the authors provide a theoretical analysis of Lipschitz constrained networks, and uncovers a previously unexplored link between the Lipschitz constant with respect to their input and the one with respect to their parameters. By bounding the Lipschitz constant of each layer with respect to its parameters, the authors argue it will guarantee DP training of these networks.\n\nSTRENGTHS:\nThe paper is well-structured and clearly written.\n\nThe theoretical part is simple and easy to follow.\n\nWEAKNESSES:\nEstimating Lipschitzness with respect to parameters may not be necessary. If the network is Lipschitz continuous with respect to the input, its gradient will be bounded, and thus the weight update will also be bounded. So, the motivation may not be rational.\n\nExperimental results do not support the arguments. The validation accuracy of the DP-SGD is lower than several referenced works.\n\nQUESTIONS:\n1. Adam uses a normalized gradient between the first-order momentum and square root of the second-order momentum, which makes it robust to Lipschitz variations in different layers. Does it still need gradient clipping?\n\n2. Regarding per-sample gradient clipping, in the current deep learning framework, we usually do not use per-sample gradient clipping. Instead, we apply gradient clipping after obtaining the averaged gradients. In the abstract, the authors mention the difficulty of per-gradient clipping, but I don't understand the reason?\n\n3. Also, gradient clipping is a fast operation. I don't understand why it results in higher memory consumption and computation.\n\n4. Regarding the experiments, the results do not show that the DP-SGD achieves better performance (validation accuracy) than the referenced works. How should we evaluate the effectiveness of this work?\n\nLIMITATIONS:\nSee weaknesses and my questions", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses the problem of efficiently bounding the sensitivity of gradients in DP-SGD by using special architectures the layers of which can be proven to be Lipschitz with respect to the parameters, hence bounded gradient. They show how to recursively calculate the sensitivity of a sequence of layers, and incorporate the method in an algorithm to perform private SGD without clipping.\n\nSTRENGTHS:\nThe writing is exceptionally lucid. The concept is original and potentially significant, although there are at present many limitations.\n\nWEAKNESSES:\nThere are a lot of constraints on the architecture that severely limit the potential of the method for short-term impact. I'm torn, because introducing the concept at this stage is of value, but far more work must be done -- both theoretical, in establishing the requisite bounds for popular architectures -- and experimental, in demonstrating that the approach achieves good points on the privacy/utility/efficiency Pareto frontier -- before we can assess the significance of the work.\n\nI'm not convinced that it isn't a major problem that the gradients can vanish during training. This is the reason for the success of adaptive (layer-wise) clipping strategies. In particular see \"EXPLORING THE LIMITS OF DIFFERENTIALLY PRIVATE DEEP LEARNING WITH GROUP-WISE CLIPPING\" which would seem to enjoy the efficiency of your approach without the drawbacks of vanishing gradients or restricted architecture class.\n\nQUESTIONS:\nWhat is meant by “the activation is centered in zero”-- in particular, is ReLU really centered?\n\nExperimental results. Figure 4 is too small. I can't see tensorflow_privacy? Also it appears that the advantage over optax is negligible. In Figure 5, the other algorithms get one or a few points, while their own algorithm is given the benefit of thousands of hyperparameter choices to establish the Pareto frontier. Moreover the result is negative: for EMNIST the other algorithms match yours, and for the other datasets the other algorithms have higher accuracy.\n\nProbably the DP definition should be using add/remove-one neighborhoods, in accordance with the use of (pretend) Poisson sampling for accounting.\n\nLIMITATIONS:\nLimitations are honestly and adequately discussed. No potential negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the question of how to do differentially private optimization without using per-sample gradient clipping, in order to simplify and speedup the iteration cost.\nThe paper proposes to restrict the class of functions to feed-forward neural networks for which it is feasible to compute bound on the gradient norm (Lipshitz constant), and proposes to compute adaptively the bound on the gradient norm (layer-wise) at every step of DP-SGD depending on the current iterate point. Paper provides the description of the algorithm, as well as evaluates its practical behavior.\n\nSTRENGTHS:\n- An efficient implementation of the algorithm is provided.\n- Experiments show that per-iteration runtime of the proposed algorithm is indeed faster.\n- Overall the paper is interesting and novel and provides a new direction for future research.\n\nWEAKNESSES:\n1. No clear comparison of the proposed algorithm to the baseline method (DP-SGD) is given in terms of the final accuracy. When restricting to the same architecture, it is unclear if the proposed algorithm can still reach the good accuracy compared to the classical DP-SGD with gradient clipping. Without clipping the gradients, the amount of the added DP noise to each gradient is larger than if you clip the gradients, which might hurt the final performance. \n2. From the experiments on CIFAR10 one might conclude that for the same privacy $\\epsilon$ the final accuracy of the baselines is much better than of the proposed algorithm, which makes the proposed algorithm not applicable.\n3. In the “local” strategy (line 201), how exactly did you calculate the amount of the noise to be added? I did not find a clear description of the “local” strategy, and how it is different from the “global” strategy.\n4. Some parts of the paper are not very clearly written (see questions below).\n\nQUESTIONS:\n1. What do the green lines in pareto-front on Figure 5 represent ?\n2. Remark 1 says that the paper proposes a more efficient way to compute X_d, however I did not find the algorithm in the paper.\n3. It is unclear from the current presentation what is the connection of Theorem 1 with Algorithm 1, is it only for the easier computing of step 6 of Algorithm 1 ? What about steps 2 and 7? \n4. I did not understand the paragraph in lines 258-267. However some hybrid approach might be indeed a good solution: if the estimated worst case gradient norm is large, the training might benefit from clipping in order to reduce the amount of noise added to the gradients, however, when estimated worst case gradient norm is small, the training would benefit of using the estimates and have little added noise. \n5. Many experimental details are missing: which architecture/hyperparameters did you use, etc.\n6. A very minor comment: when printing on the paper, formulas in green and yellow color are not very visible, try using a different color.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-12"}