Making the Shoe Fit: Architectures, Initializations, and Tuning for Learning with Privacy

1 INTRODUCTION . Machine learning ( ML ) can be usefully applied to the analysis of sensitive data , e.g. , in the domain of healthcare ( Kononenko , 2001 ) . However , ML models may unintentionally reveal sensitive aspects of their training data , e.g. , due to overfitting ( Shokri et al. , 2017 ; Song & Shmatikov , 2019 ) . To counter this , ML techniques that offer strong privacy guarantees have been developed . Notably , the differentially private stochastic gradient descent , or DP-SGD , of Abadi et al . ( 2016 ) is an easyto-use , generally-applicable modification of stochastic gradient descent . In addition to its rigorous privacy guarantees , it has been empirically shown to stop the leaking of secrets ( Carlini et al. , 2019 ) . To strictly bound the impact of any training example , DP-SGD makes two changes to every gradient step : first , each example ’ s gradient contribution is limited to a fixed bound ( in practice , by clipping all per-example gradients to a maximum ` 2 norm ) ; second , random ( Gaussian ) noise of the scale of the clipping norm is added to each batch ’ s combined gradient , before it is backpropagated to update model parameters . Together , these changes create a new , artificial noise floor at each step of gradient descent , such that the unique signal of any individual example is below this new noise floor ; this allows differential privacy to be guaranteed for all training examples ( Dwork & Roth , 2014 ) . Training using DP-SGD is eminently practical and in addition to privacy offers advantages such as strong generalization and the promise of reusable holdouts ( Google , 2019 ; Dwork et al. , 2015 ) . Unfortunately , its advantages have not been without cost : empirically , the test accuracy of differentially private ML is consistently lower than that of non-private learning ( e.g. , see Papernot et al . ( 2018 ) ) . Such accuracy loss may sometimes be inevitable : for example , the task may involve heavy-tailed distributions and adding noise will definitely hinder visibility of examples in the tails ( Feldman , 2019 ; Bagdasaryan & Shmatikov , 2019 ) . However , this does not explain the accuracy loss of differentially private learning on standard benchmark tasks that are known to be relatively simple : MNIST ( Yann et al. , 1998 ) , FashionMNIST ( Xiao et al. , 2017 ) , CIFAR10 ( Krizhevsky et al. , 2009 ) , etc . This paper presents several new results for privacy-preserving learning that improve the state-ofthe-art in terms of both privacy and accuracy . Significantly , these new results stem from a single , simple observation : differentially-private learning with DP-SGD is different enough that all aspects of learning—model architecture , parameter initialization , and optimization strategy , as well as hyperparameter tuning—must be reconsidered . To achieve the best privacy/accuracy tradeoffs , we must tune our learning strategies to the specifics of privacy-preserving learning ; i.e. , we must “ learn to learn ” with privacy . Conversely , we concretely demonstrate how the architecture , initialization , and optimization strategy that gives the best accuracy for non-private learning can be a poor fit for learning with privacy . Instead , by revisiting our choices , we can reduce the information loss induced by clipping , limit the impact of added noise , and improve the utility of each gradient step when learning with privacy . Our contributions facilitate DP-SGD learning as follows : • We show how simple architecture changes , such as the use of tanh instead of ReLU activations , can improve a model ’ s private-learning suitability and achievable privacy/accuracy tradeoffs , by eliminating the negative effects of clipping and noising large gradients . • We explain how high-capacity models can be disadvantageous , as well as the advantages of models with a final , fully-connected layer that can be independently fine tuned , and how both help address the curse of dimensionality and high-dimensional noise . • We demonstrate the importance of finding good initializations , and show how this can be done with privacy using either transfer learning or weight scaling ( Raghu et al. , 2019 ) . • We show that better tradeoffs and increased wall-clock learning speeds can be achieved by tuning hyperparameters and choosing optimizers directly for DP-SGD learning . By applying the above , we advance the state of the art for MNIST , FashionMNIST , and CIFAR10 , significantly improving upon the privacy/accuracy tradoffs from prior work . On MNIST , we achieve 98.1 % test accuracy for a privacy guarantee of ( ε , δ ) = ( 2.93 , 10−5 ) , whereas the previous stateof-the-art reported in the TensorFlow Privacy library ( Google , 2019 ) was 96.6 % . On CIFAR10 , we achieve 72 % test accuracy at ( ε , δ ) = ( 2.1 , 10−5 ) in a setup for which to the best of our knowledge the previous state-of-the-art was achieved by Abadi et al . ( 2016 ) at 67 % accuracy . 2 TRAINING-DATA MEMORIZATION , DIFFERENTIAL PRIVACY , AND DP-SGD . Machine-learning models will easily memorize whatever sensitive , personal , or private data that was used in their training , and models may in practice disclose this data—as demonstrated by the attacks of Shokri et al . ( 2017 ) , Song & Shmatikov ( 2019 ) , and Carlini et al . ( 2019 ) . For reasoning about the privacy guarantees of algorithms such as training by stochastic gradient descent , differential privacy has become the established gold standard ( Dwork & Roth , 2014 ) . Informally , an algorithm can be differentially private if it will always produce effectively the same output ( in a mathematically precise sense ) , when applied to two input datasets that differ by only one record . Formally , a learning algorithmA that trains models from the set S is ( ε , δ ) -differentiallyprivate , if the following holds for all training datasets d and d′ that differ by exactly one record : Pr [ A ( d ) ∈ S ] ≤ eεPr [ A ( d′ ) ∈ S ] + δ ( 1 ) Here , ε gives the formal privacy guarantee , by placing a strong upper bound on any privacy loss , even in the worst possible case . A lower ε indicates a stronger privacy guarantee or a tighter upper bound ( Erlingsson et al. , 2019 ) . The factor δ allows for some probability that the property may not hold ( in practice , this δ is required to be very small , e.g. , in inverse proportion to the dataset size ) . A very attractive property of differential-privacy guarantees is that they hold true for all attackers— whatever they are probing and whatever their prior knowledge—and that they remain true under various forms of composition . In particular , the output of a differentially-private algorithm can be arbitrarily post processed , without any weakening of the guarantees . Also , if sensitive training data contains multiple examples from the same person ( or , more generally , the same sensitive group ) , ε-differentially-private training on this data will result in model with a kε-differential-privacy guarantee for each person , as long as at most k training-data records are present per person . Abadi et al . ( 2016 ) introduced DP-SGD as a method for training deep neural networks with differential-privacy guarantees that was able to achieve better privacy and utility than previous efforts ( Chaudhuri et al. , 2011 ; Song et al. , 2013 ; Bassily et al. , 2014 ) . DP-SGD bounds the sensitivity of the learning process to each individual training example by computing per-example gradients { gi } i∈0 .. n−1 with respect to the loss , for the n model parameters { θi } i∈0 .. n−1 , and clipping each per-example gradient to a maximum fixed ` 2 norm C. Subsequently , to the average of these perexample gradients , DP-SGD adds ( Gaussian ) noise that whose standard deviation σ is proportional to this sensitivity . In this work , we use the canonical implementation of DP-SGD and its associated analysis that has been made available through the TensorFlow Privacy library ( Google , 2019 ) . 3 MODEL ARCHITECTURES BETTER SUITED TO LEARNING WITH PRIVACY . We show here that learning with differential privacy imposes additional constraints that need to be taken into account when designing neural network architectures . They help us control the sensitivity of learning to training examples before the clipping operation is performed in DP-SGD , thus reducing the potential negative impact of clipping on the estimated gradient direction . 3.1 MODEL CAPACITY . The success of neural networks is in part explained by their ability to scale to complex tasks through an increase in model capacity . ResNets are an illustrative recent examples ( He et al. , 2016 ) . Here , we explain how additional capacity may not be beneficial when learning with privacy . One of the major challenges in training models with differential privacy is the curse of dimensionality ( Bassily et al. , 2014 ) . The accuracy of privately trained models typically degrades with the increase in the number of dimensions . Unfortunately , strong lower bounds suggest that this dependence on dimensionality is necessary ( Bassily et al. , 2014 ) . Consider the convolutional architecture described to the right . With all other architectural details being fixed , we can control the model ’ s capacity by varying the number of filters k in its two convolutional layers . While the relationship between generalization performance and the number of parameters is not always monotonic ( Neyshabur et al. , 2017 ) , we leave as future work a study of how different measures of capacity can inform the design of model architectures for private learning . We report the model ’ s accuracy when trained with SGD and DP-SGD in Figure 1 , both on MNIST ( left ) and FashionMNIST ( right ) . The test accuracy of models trained without privacy monotonically increases with the number of filters in their convolutional layers . Instead , we observe an inflection point at about 15 filters for which models trained with privacy achieve their highest test accuracy . Afterwards , the model ’ s generalization suffers as more filters are added . There are two competing explanations of this behavior , both compatible with the lower bound stated in Bassily et al . ( 2014 ) . First , recall that DP-SGD performs a clipping operation on each per-example gradient before the average gradients is used to update model parameters ; i.e. , each gradient is subject to the following transformation gi ← gi ·min 1 , C√∑n−1 i=0 g 2 i ( 2 ) where gi is the gradient corresponding to model parameter i . For a fixed clipping norm C ( corresponding to a certain , fixed privacy guarantee ) , the quantity C√∑n−1 i=0 g 2 i by which individual param- eters are multiplied decreases as the number n of parameters in a model increases . That is , the more parameters we have , the more likely DP-SGD is to clip the gradient ( or signal ) at each parameter . This can explain the presence of an inflection point in Figure 1 , after which learning with privacy becomes increasingly difficult as capacity is increased . Second , as the number of parameters ( i.e. , gi ’ s ) increases , the norm of the noise vector that DP-SGD must add to the gradient average to ensure privacy also increases . This noise norm increases as √ # parameters , and introduces another source of accuracy degradation with an increased number of parameters . Our observations may seem to contradict some of the findings in Abadi et al . ( 2016 ) . However , their limited experimental setup could offer few general lessons . First , they reduced data dimensionality using PCA to have inputs of only 60 dimensions ; second , they explored only a model architectures using a single layer perceptron with between 200 and 2 , 000 units . Instead , our experiments involve a realistic setting where the full input is passed to a convolutional neural network with a total of 3 hidden layers and over 26,000 parameters .

Summary: The paper methodically analyses the settings and choices used when training neural networks (specifically CNNs) via the DP-SGD algorithm and suggests changes to the standard procedures that empirically lead to higher accuracies despite the added noise. The main statement of the paper is quite simple: optimize hyperparameters for the model that you're training (DP-SGD) rather than the model it is inspired by. Yet, the findings an recommendations may be useful for practitioners.