Paper: An improved analysis of per-sample and per-update clipping in federated learning (ICLR-2024)

Reviewer comment (Theory):
The authors proved drastically different convergence results under per-sample clipping and per-update clipping, even though the two clipping methods are equivalent when $\tau = 1$ (despite a local step-size $\eta_l$). This seems counterintuitive and needs more clarification.

Author response:
We hope the discussion of the two algorithms in **Appendix E** has addressed your concern. 

Intuitively speaking, in per-update clipping, the local stepsize can control the norm of the model update $\Delta_i := -\eta_l\sum\nabla F_i(\mathbf{y}_i)$ such that $\Delta_i$ might not be clipped. However, if we remove $\eta_l$, we can no longer control the norm of $\Delta_i$ given a fixed $c$ and may risk converging to the neighbourhood. Please let us know if anything is unclear.

Reviewer comment (Theory):
Although the analysis holds for arbitrary clipping threshold, satisfying convergence to an accurate solution still relies on setting a large enough clipping threshold. This insight makes sense theoretically (as a larger clipping threshold enables the training process to be closer to unclipped training), yet it is quite different from practice. (For DP learning, generally, a small clipping threshold such as 0.1 enables good performance [a, b].)

Author response:
Thanks for this interesting question and the references! 

- Indeed, when we do not add any noise, the large clipping threshold allows Algorithm 1 to converge to a desired solution. 
- When we consider DP training, the noise scale should be proportional to the clipping threshold, i.e. $\sigma_{\text{DP}}^2 \ge \Omega(c^2)$. In other words, if we use a large clipping threshold, a large noise should be added, which might break the utility. In the DP setting, the optimal choice of $c$ is no longer $+\infty$. Instead, it depends on the privacy budget and many other factors. The optimal choice of $c$ can be found in **Appendix C.6.3 and D.5.3**. Therefore, as you pointed out, a small clipping threshold might often be a good choice in practice.

Reviewer comment (Novelty):
Another less critical weakness is that the main theorems (Theorem 1 and Theorem 2) seem to be direct extensions of the prior work [Koloskova 2023]. Consequently, it needs to be clarified how non-trivial are the additional efforts made in this work.

Author response:
Please see **section 3.3 (relation to Koloskova et al. 2023)** in the updated manuscript for a more detailed comparison against Koloskova et al. 2023.

Reviewer comment (Theory):
Could the author explain why this local update step-size $\eta_l$ would contribute to significantly different convergence behaviors between per-sample and per-update clipping? See weakness 1 for details.

Author response:
We hope our answers to your W1 has addressed this question.

Reviewer comment (Experiments):
Could the authors comment on this discrepancy between recommended large clipping threshold in this paper and the small clipping threshold used for practical DP learning?

Author response:
We hope the clarification in W2 has addressed your concern. When we consider DP training, the optimal choices of clipping threshold depend on the privacy budget and other factors. Please see **Appendix C.6.3 and D.5.3** for detailed optimal choices of $c$.

Reviewer comment (Experiments):
Is there a reason why, in the experiments of Figures 1 and 2, the clipping threshold for per-sample clipping is chosen to be much larger than the clipping threshold for per-update clipping?

Author response:
Thank you for your question. 

In the MNIST experiment, we observed that the norm of the mini-batch gradient i.e., $\nabla F_i(\mathbf{y}_i)$ (Line 8, Algorithm 1) is much larger than the norm of the model update, i.e., $\Delta_i:=-\eta_l\sum\nabla F_i(\mathbf{y}_i)$ (Line 11, Algorithm 2). Using a smaller clipping threshold than 1.5 will possibly give a similar convergence behaviour as when $c=1.5$ in Fig.1 (c), i.e., the norm of the gradient can not reach the target accuracy $||\nabla f(\mathbf{x}_t)||=0.18$.

Reviewer comment (Writing):
In Table 1, $L$ is not defined. In Theorem 1, $M$ is not defined. In Corollary 1, $g_{i, t}$ cannot be found in Algorithm 1.

Author response:
Thank you for pointing these out. We have clarified the meaning of $L$, $M$, and $g_{i,t}$ in our revised manuscript.

Reviewer comment (Evaluation):
It is unclear how the results are appropriate for privacy setting.

Author response:
Thank you for your question. We first refer to paragraph **Privacy discussion** under Corollary I and II, together with more detailed information in **Appendix C.6 and D.5** for a comprehensive discussion in private settings. 

- We totally agree that finite-sum is ideal for the DP-training settings. We hope the study in Appendix C.6 and D.5 has addressed this concern.
- We, unfortunately, do not understand why the bounded gradient assumption is better than the bounded variance assumption. We agree that, in practice, knowing the gradient norm may give some insights into choosing the clipping thresholds. However, as far as we understand, the bounded gradient assumption is a more restricted and less practical assumption theoretically, which can exclude simple quadratics. We refer to Table 1 for analysis that uses bounded gradient assumption. 

We hope we have addressed your concern. If not, we would like to engage in further discussion if the reviewer could elaborate more on this question.

Reviewer comment (Theory):
The results cannot recover the unclipped results when the clipping parameters goes to infinity.

Author response:
We apologize for the confusion. The complete convergence results of two algorithms, including the case where $c$ is extremely large, are presented in **Theorem III and Theorem IV in the Appendix**. Theorem III and IV recover the unclipped FedAvg when $c\to \infty$ (please see the explanation below Corollary III and Theorem IV).

Please let us know if there is anything that is unclear about recovering the standard unclipped results.

Reviewer comment (Theory):
It seems that when we choose $c$ as infinity, the results (e.g., Theorem I) cannot reduce to the unclipped results (e.g., LocalSGD Koloskova et al. 2020), and I'm wondering what steps cause this discrepancy?

Author response:
We apologize for the confusion. The complete convergence results of two algorithms, including the case where $c$ is extremely large, are presented in **Theorem III and Theorem IV in the Appendix**. Theorem III and IV recover the unclipped FedAvg when $c\to \infty$ (please see the explanation below Corollary III and Theorem IV).

Please let us know if there is anything that is unclear about recovering the standard unclipped results.

Reviewer comment (Theory):
I'm not sure how the results can be applied to the differentially private setting. The authors consider the stochastic setting, and thus the authors need to specify what is the dataset you want to protect when you apply Corollary I and Corollary II. From my understanding, it would be more meaningful if the authors can provide the finite sum results with bounded stochastic gradient assumption instead of the stochastic setting with bounded variance assumption for the application of differentially private setting.

Author response:
Thank you for your question. We first refer to paragraph **Privacy discussion** under Corollary I and II, together with more detailed information in **Appendix C.6 and D.5** for a comprehensive discussion in private settings. 

- We totally agree that finite-sum is ideal for the DP-training settings. We hope the study in Appendix C.6 and D.5 has addressed this concern.
- We, unfortunately, do not understand why the bounded gradient assumption is better than the bounded variance assumption. We agree that, in practice, knowing the gradient norm may give some insights into choosing the clipping thresholds. However, as far as we understand, the bounded gradient assumption is a more restricted and less practical assumption theoretically, which can exclude simple quadratics. We refer to Table 1 for analysis that uses bounded gradient assumption. 

We hope we have addressed your concern. If not, we would like to engage in further discussion if the reviewer could elaborate more on this question.

Reviewer comment (Theory):
I'm wondering when you consider the finite sum with bounded stochastic gradient assumption, how the lower bound result will look like in terms of the clipping parameter $c$ and the bounded gradient norm?

Author response:
We hope our response in Q2 has addressed this question. 

Under the bounded gradient norm assumption, there is no clipping bias if we pick $c$ to be larger than the bounded norm. See Zhang et al. 2022 for more detailed information.

Reviewer comment (Theory):
My main concern is that the authors do not make the implication of the theoretical results presented clear. I am puzzled by the motivation of studying Algorithm 1 with a fixed learning rate. It seems that Algorithm 1 can also apply both an inner and an outer learning rate and they should produce the similar bias control as what is claimed in Algorithm 2. So, this make the comparison between Algorithm 1 and 2 in Section 3.3 very confusing, given that after incorporated with the inner learning rate, both of them can achieve arbitrary accuracy. So, what do the analysis want to tell? Which clipping method should we apply in practice? Overall, this makes the practical impact of the theoretical results weak.

Author response:
We sincerely apologize for making you feel puzzled about the story and the motivation. Here is the clarification. 

Algorithms 1 and 2 with DP noise are commonly used to protect different levels of privacy. We study Algorithm 1 because it is a widely used algorithm [Liu et al. 2022, Yang et al. 2022, Koloskova et al. 2023] for achieving example-level local DP. Our rigorous analysis on per-update clipping (Algorithm 2) opens directions for designing a new per-sample clipping algorithm by incorporating an inner stepsize. However, this strategy is new and non-trivial. We will explore this in our future work.

Nevertheless, these two algorithms still share some similarities from a pure algorithmic point of view. Comparing these two algorithms using our analysis is still meaningful. Please see **section 3.3** in the updated manuscript.

Reviewer comment (Evaluation):
The privacy-utility tradeoff is not well studied in this paper. In Appendix D.4, the authors briefly discuss the iteration number T required to achieve the balanced point between convergence progress and utility loss resulting from noise. I would suggest plugging the expression of v with $\epsilon$ and $\delta$ into the bound, and compare with existing works, such as "Differentially private empirical risk minimization revisited: Faster and more general". It is not clear to me whether the proposed analysis brings any improvement either in terms of the optimal utility-privacy tradeoff or the efficiency, i.e., the convergence time. The clipping bias needs to be carefully controlled such that the three terms: convergence advancement, utility loss by noise, and the bias should be all in the same degree. From the complicated expression, I find it hard to figure out in practice, do we want a large bias but slow convergence with large DP noise or the converse way.

Author response:
We thank the reviewer for the reference and appreciate the insightful questions and suggestions. We refer the reviewer to the paragraphs **Privacy discussion** under Corollary I and II, **section 3.3**, for discussing the optimal privacy-utility trade-off and practical implications (see more detailed information in Appendix C.6 and D.5) in our updated manuscript. Please let us know if there is anything still unclear from your side.

Reviewer comment (Experiments):
For an ICLR paper, I feel the experiments need to be strengthened. The experiments on both MNIST and CIFAR10 do not report the test accuracy but only the loss. Still, we do not know **under the corresponding optimal parameter selection** of the two clipping methods, which one performs better in practice. There is no comparison presented with existing empirical works on DP-SGD, such as "Unlocking High-Accuracy Differentially Private Image Classification through Scale".

Author response:
Thank you for the suggestion. We have provided the test accuracy in sections H and I in the appendix. 

We agree with the reviewer that practical performance can vary depending on the choices of many other factors, as demonstrated in N. Ponomareva, 2023. We have clarified this at the beginning of **section 3.4** in the revised manuscript.

Reviewer comment (Theory):
What is the optimal utility-privacy tradeoff/utility bound under the optimal parameter selection of Algorithm 1 and 2 (in particular, Algorithm 1 with both inner and outer learning rate)? In particular, what is the tradeoff considering the bias caused.

Author response:
We refer the reviewer to paragraph **Privacy discussion** under Corollary I and II, section 3.3, and more detailed information in Appendix **C.6.3 and D.5.3** for all the discussions.

Note that when $\tau=1$, Algorithm 2 (the same as Algorithm 1 but with both inner and outer stepsize) achieves the same optimal privacy-utility trade-off as Algorithm 1 (single stepsize). Therefore, there is no obvious benefit in terms of optimal privacy-utility bound in using two stepsizes for DP training.

Reviewer comment (Experiments):
Can the different clipping methods produce better performance compared to existing works?

Author response:
We hope our answer in W1 has addressed your concern. We theoretically study the same clipping algorithms as the existing works [Liu et al. 2022, Zhang et al. 2022, Yang et al. 2022]. Therefore, we do not expect to achieve a better practical performance than them.