Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: Authors  identify the per-example gradient clipping in differentially private stochastic gradient descent (DP-SGD) as a major cause of miscalibration, and argue that existing baselines for improving private calibration only provide small improvements in calibration error while occasionally causing large degradation in accuracy. Authors propose using post-processing calibration. 


strength_and_weaknesses: Strengths:


Authors have a good grasp of some of the challenges of differentially private computations in the context of the paper. The paper is generally well-written. The empirical context is well motivated. 


Weakness:


This paper is primarily on empirical evidence for a recalibration idea. It is not clear to me that the recalibration preserves differential privacy, although there is claim in the paper about this. Perhaps additional discussion on this in an eventually revised manuscript is in order. There does not seem to be adequate theoretical justification about when, why and how the standard DP-SGD method fails, or when, why and how the proposed recalibration will work, or when it will fail. 

clarity,_quality,_novelty_and_reproducibility: The paper is clearly written, the quality of the empirical work seems fine but there is a lack of adequate theoretical developments. The algorithm may have some elements of novelty, and the results look reproducible.


summary_of_the_review: The main emphasis of this work is empirical evidence. More detailed analysis is needed.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studies the trade-off between differential privacy and classifier calibration. In particular, the main observation is that classifiers trained with the standard DP-SGD algorithm can be highly miscalibrated. To this end, the authors propose a natural and intuitive solution to fix this issue using differentially private recalibration. First, the algorithm runs DP-SGD in order to obtain the standard classification model with a subset of the data and then runs DP-SGD again in order to find a post-hoc recalibration function using the remaining data. Finally, it outputs the composition of the two functions.

strength_and_weaknesses: The paper provides extensive experimental evidence about the impact of differential privacy to the calibration property of classifiers. I find that the provided empirical observations are interesting and may raise some nice theoretical questions too. More to that, the paper is nicely written. The only issue is that the paper does not have any theoretical contribution.

A potential question is whether variants of DP-SGD or other training methods would also cause such miscalibration phenomena. Is gradient clipping the only obstacle towards private training that is well-calibrated?
clarity,_quality,_novelty_and_reproducibility: The problem and the experiments are clearly presented. The comparison with previous work is also discussed. 
summary_of_the_review: In short, even if there is no theoretical contribution, I believe that the paper is beyond the acceptance threshold, since its message can potentially raise some theoretical questions. The experimental observations are interesting.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studies the calibration of state-of-the-art classifiers trained using differentially private stochastic gradient descent (DP-SGD). It shows that there are miscalibration issues in these DP classifiers even when their accuracy matches that of their non-private counterparts. The likely cause of that is conjectured to be the repeated per-example gradient clipping in DP-SGD. The other contribution is addressing these miscalibrations using DP versions of temperature scaling (TS) and Platt scaling (PS), which are almost as good as their non-private counterparts. This work is mostly empirical in nature, and uses multiple datasets and models, with applications in text-processing and computer vision, and has experiments to evaluate both in-domain and out-of-domain calibrations.
strength_and_weaknesses: Strengths:
1. The paper successfully depicts the issue of miscalibrations in DP classifiers (trained via DP-SGD). All the experiments indicate the DP counterparts having more miscalibration than their respective non-private counterparts.
2. This paper also establishes its second goal of providing a DP method to lower the miscalibration errors significantly. These methods involve the use of DP-SGD again. The experimental results indicate very low miscalibration errors that are similar to those from the non-private methods.

Weaknesses:
1. I have some concerns about the writing quality. Please, see the section regarding clarity of the submission.
2. I'm also concerned about the significance of the problem being solved here. I'm curious as to why this problem under DP has not been studied much before this. In other words, why is reducing miscalibrations that important a problem to solve, especially under DP?
3. The algorithmic contributions don't seem to be that interesting, unfortunately -- just optimising a calibration function using DP-SGD. The techniques aren't novel, and simply involve blackbox uses of DP-SGD. Is the goal of this paper to bring the issue of miscalibrations in DP classifiers to people's attention?
clarity,_quality,_novelty_and_reproducibility: Writing Quality:
1. I think the paper is a bit vague at different places, and the notations are not well-defined. In Section 3.3 while defining canonical calibration, what is $h_{\theta}$? What is the probability over? Is $Y$ the true label or is it the prediction of $h$? Also, is Equation 2 like an analogue of $\ell_{\infty}$ error? In the same subsection, what is $\hat{p}_i$?
2. Might want to motivate in a paragraph why calibration is an important problem to study even if we have good accuracy. Understanding the motivation of this is important, and I don't think the authors do a great job at conveying that.
3. The DP-PS and DP-TS are unexplained in this draft. I know you use DP-SGD here, but giving some kind of an analysis or more detailed descriptions of the two would make it much more meaningful.
4. Might want to give a clear intuition of what calibration actually means or entails. Might make sense to explain the formal definitions a bit in words to show what it means to have bad calibration even if we have good accuracy.

Novelty:
There is novelty in the sense that there are previous works that address this issue under DP in settings different from the one in this paper.
summary_of_the_review: I am not convinced about the overall quality of this paper. The contributions don't seem that significant to me, unfortunately, especially given the lack of motivation of the problem in this draft and the not-so-novel technical ideas. The writing quality wasn't that great either, but that doesn't matter much to me as far as the quality of the work is concerned -- my feedback there is just to help the authors improve their manuscript. I do acknowledge that the authors seem to have achieved the claims of this paper. All this said, I will be open to changes in my final score based on the authors' responses though.

Update: Based on the responses, I have bumped the score.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studies the calibration of differentially private learners based on stochastic gradient descent. The paper first observes the miscalibration  is due to the per-example gradient clipping. Then, the paper provides differentially private recalibration to reduce  calibration errors. The basic idea is to divide the training dataset into two parts: one part is used to train a classifier, while the other is used to train a recalibration function. Extensive experimental results are reported to show the effectiveness of the differentially recalibration method. 
strength_and_weaknesses: **strength**

The proposed method is simple and general. It provides a framework which includes several post-hoc recalibration techniques. There are many experimental results, which show the proposed method works well in practice.

**weakness**

There is no theoretical analysis of the proposed method. There is no theoretical guarantee and therefore it is not clear how the method would behave in general.

The idea of extending recalibration to differentially private recalibration seems to be natural. It is not clear whether the extension is novel enough.
clarity,_quality,_novelty_and_reproducibility: The description of Algorithm 1 is very general. There is no detailed explanation of how DP-SGD is applied in both step 2 and step 3 of Algorithm 1. The authors should give more details.

The idea of calibration is not quite clear. It is not clear to me the underlying motivation on why the calibration is a big issue we need to consider in algorithm design. What is its connection to accuracy? There is not enough background on calibration.

In the abstract, the paper says their analysis identifies per-example gradient clipping as a major issue of miscalibration. However, I do not find such an analysis in the paper.

Below eq (1): $l_i$ should be $L_i$
summary_of_the_review: The paper presents a simple and general method for recalibration in a differentially private setting. The paper conducts many experiments to show the effectiveness of the method. The proposed algorithm is not presented in detail and there is no theoretical analysis.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper studies the calibration of differentially private learners based on stochastic gradient descent.
It first observes that the miscalibration is due to the per-example gradient clipping. Then, the paper provides differentially private recalibration to reduce calibration errors. The basic idea is to divide the training dataset into two parts: one part is used to train a classifier, while the other is used to train a recalibration function (standard or Platt scaling). Extensive experimental results are reported to show the effectiveness of the differentially recalibration method.


justification_for_why_not_higher_score: There does not seem to be adequate theoretical justification about when, why and how the standard DP-SGD method fails, or when, why and how the proposed recalibration will work, or when it will fail.
Extending recalibration to differentially private recalibration seems to be natural: it is  unclear whether the extension is novel enough.
justification_for_why_not_lower_score: It is a clearly written paper.
The empirical work is fine.
The algorithm may have some elements of novelty, and the results look reproducible.
Decision: Reject