Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: The paper considers ways of improving private versions of SGD in the context of image classification. The main finding is that providing "hand crafted" features can significantly improve the privacy/accuracy trade-off. In some cases, even a linear model built on top of such features (like those produced by ScatterNet), can improve over differentially private SGD. A plausible explanation for this phenomenon is that extra features can reduce the number of iterations required in SGD, resulting in better privacy and/or less noise. (It is also argued that having much more data similarly improves the trade-off, but this is unsurprising and, it seems, has been observed before by McMahan et al.)

The paper is quite well-written, and I found it easy to follow even though this is not my area of expertise. I also like that it presents a number of possible directions for further improving private SGD, including transfer learning from related, public data sets, and second-order optimization.

A possible criticism is that in principle the "hand crafted" features may have been built based on empirical work on MNIST and CIFAR-10, and the same goes for the architecture choices, so in theory there could be some privacy leakage from these choices. It would have been more impressive to demonstrate effectiveness of a newer data set, not known when ScatterNet and the used CNN architectures were proposed.

Two final comments:
- "Unlearned" usually means that you have (deliberately) forgotten something, so it is not the same as "not learned".
- It would be interesting to consider the setting where just the image *label* is private. Has DP SGD been considered in that setting?
review_title: Interesting experimental study of improved, differentially private image classification

final_score: rating: 7: Good paper, accept
review_content: This article is about a topical issue: performance degradation of of deep learning models trained with differential privacy (DP). Clipping of the gradients and addition of the noise, required to obtain DP guarantees, blur the models such that for moderate privacy guarantees (eps~7.0) CIFAR-10 test accuracy baseline is currently ~66% (Papernot et al., 2020b).

Lower bounds for this degradation have been shown theoretically (e.g. Bassily et al., 2014), and there has recently been also work on circumventing this issue, see e.g.

Kairouz, P., Ribero, M., Rush, K. and Thakurta, A., 2020. Dimension Independence in Unconstrained Private ERM via Adaptive Preconditioning. arXiv preprint arXiv:2008.06570,
Yingxue Zhou, Zhiwei Steven Wu, and Arindam Banerjee. Bypassing the ambient dimension: Private sgd with gradient subspace identification. arXiv preprint arXiv:2007.03813, 2020
(these references are not included in the paper). 

Although this paper does not introduce fundamentally anything new for DP learning (DP-SGD + Rényi DP accountant for obtaining eps,delta-guarantees are used), it does clearly beat the state-of-the-art for small epsilon values (eps up to 3.0) for MNIST, Fashion-MNIST and CIFAR-10. This is obtained by using so called Scattering Networks (Oyallon and Mallat, 2015), which have the property of converging very fast without privacy. This phenomenon is transferred to DP learning and thus high accuracies for shorter DP-SGD runs (i.e. smaller epsilons) are obtained. 

As expected, these 'handcrafted data-independent feature extractors' of Scatter Networks cannot beat CNN+DP-SGD when more private data is available, or when features can be extracted from public image data.

All in all, although I think the gist of the paper is simply combining these handcrafted feature extractors (ScatterNets) and DP-SGD, it does improve the baseline for DP CIFAR-10 for small / moderate eps-values (up to 3.0) and does provide new ideas / questions on how to improve DP learning (e.g. by accelerated convergence) also outside of image domain (handcrafted feature extraction for non-vision tasks).

The paper is very well written. A tiny remark:
You write "Gaussian noise of variance sigma^2 C^2 is added to the mean gradient."
Notice that sigma^2 C^2 - noise is added to the summed gradients, and sigma^2 C^2 / B^2 to the mean.


review_title: New baselines for differentially private vision tasks using handcrafted feature extractors

final_score: rating: 7: Good paper, accept
review_content: The paper shows that linear model on top of ScatterNet can outperform CNN for DPSGD training on a few generic image classification tasks. It analyzed the results, provides hypotheses to explain it, and concludes that more data / better feature is needed for DPSGD training.

People have been searching for good models for DPSGD training, so it is nice to see a new baseline (ScatterNet + linear model) that is simple but performs better. This can be pretty valuable for researchers and practitioners in the field.
The paper also did some quite interesting experiments to explain the advantage of ScatterNet + linear model and to suggest directions to improve DPSGD training. The experiments and discussions on the learning rate are quite interesting and inspiring to me. However, I feel like the results for having more data / transfer learning is not so surprising, though the experiments with models different from previous work are valuable.

More detailed comments:
- In Sec 4 "smaller models are not easier to train privately", you mentioned that the CNN is smaller than the linear model, so dimensionality is not an explanation for ScatterNet + linear's better performance. But I guess convex model (or maybe shallow model) might have some fundamental difference from nonconvex (or maybe deeper model). Maybe you could try something deeper than linear but shallower than the CNN to see if there is a sweet spot in between.
- In Sec 4 "Models with handcrafted features converge faster without privacy", I guess the results can be explained by the fact that simpler model (linear) has a lower capacity than more complicated model (CNN) so requires less training time even with lower learning rate. So maybe again it would worth trying something in between linear and CNN.
- In Sec 5.2, you showed results that are much better than previous results with transfer learning. It seems like the main difference is the model architecture. Is that the case? Do you have any comment on that?

The presentation is clear in general. I would love to see more details about ScatterNet as that is the important component of the proposed method.

review_title: Nice experiments

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper presents an analysis of differential privacy in machine learning, with a focus on neural networks trained via differentially private stochastic gradient descent (DPSGD). The main focus and the message in the paper is that the handcrafted features work better compared to learned features during training of NNs and having more training data results in better outcomes (i.e. a better privacy-utility trade-off).

Starting with the latter, this is apparent from the noise formulation in DPSGD, where the noise is reduced via sampling probability, which decreases as the data size grows. Hence, I do not consider this as a new insight or a contribution. Unless, I have misunderstood something, in which case, please do explain.

For the former, as the final model used (Table 3 and Figure 1) is a linear classifier, it outperforming an end-to-end CNN based model is intuitive, as it has far fewer number of parameters (which improve the noise scale, due to smaller gradient norm). This is slightly touched upon in subsection "Training CNNs on handcrafted features", where the comparison is made using CNNs on the handcrafted features, however there are no detailed results presented in the paper, I would have liked to see a similar table and figures as earlier.

The presentation of results (Table 3) is a bit strange. I would have further liked to see the comparison of both models on the *same* set of hyperparameters. Also, instead of stating that hyperparameter search's privacy budget was not accounted for as in prior works, it would have been nice to see some analysis, such as the section D(Appendix) in Abadi et al.
review_title: Review

Meta-review: This paper presents a very interesting investigation. While deep neural networks are typically best in non-private settings, the authors show that linear models with handcrafted features (ScatterNets) perform better in certain settings of the privacy parameter. The reviewers all found this to be important and insightful, with a thorough investigation, and I tend to agree, recommending acceptance.
Decision: Accept (Spotlight)