Paper: Federated Learning, Lessons from Generalization Study: Communicate Less, Learn More (ICLR-2024)

Reviewer comment (Evaluation):
**Counter-Intuitive Observation**: Moreover, if we consider a fixed n, each client ends up with n/R data for each communication round. To illustrate this with a special case, let's take R=1, which essentially reduces the scenario to centralized learning. Surprisingly, based on the results presented in the paper, which indicate that the generalization error increases with R, it might suggest that R=1 should be chosen. This finding appears counter-intuitive and raises questions about the practicality and relevance of the assumed setting for real-world federated learning applications.

Author response:
Note that the case $R=1$ is not the centralized learning algorithm, but the "one-shot" FL, where only one aggregation is performed. Indeed, in this case we obtain the best generalization performance, intuitively, since "the variance of the model" becomes small after one aggregation of the independent models. In contrast, it is hardly possible to minimize the empirical risk using only a single aggregation step. Therefore, $R=1$ is rarely the optimal value of $R$ when considering the population risk. In principle, there is a trade-off between the generalization error and the empirical error, as can be explicitly observed in Figure 3.

Reviewer comment (Experiments):
I suggest that the author undertake a more extensive exploration of the practical implications stemming from the theoretical findings. It is imperative to conduct an exhaustive examination of the interdependencies between key parameters across various scenarios. Such an in-depth analysis holds the potential to offer invaluable insights into the development of real-world federated learning algorithms. Personally, I hold the theoretical contribution of this paper in high regard, despite my reservations about the underlying settings. A more comprehensive analysis would significantly enhance the overall quality of this paper and can be translated into guidance for the refinement and enhancement of federated learning systems.

Author response:
- Regarding further experiments: We would like first to reiterate that the presented experiments on FSVM mainly aim at validating the behavior suggested by our Theorem 4, \ie that the generalization error of FSVM might increase with $R$. The additional experiments on neural networks are meant to suggest that this behavior may hold in other setups. It should be remarked that the experiment on ResNet-56, presented in the Section 5 of the paper, was reproduced for different sets of hyperparameters, giving similar results. As a complement, we added the reproduced experiments for two different values of the learning rate in Appendix E.6. Finally, we agree that extensive exploration of the influence of optimization hyperparameters on the generalization error as a function of $R$ is indeed an interesting question. We believe however, it is beyond the scope and motivation of this work and therefore left for future work.

Reviewer comment (Writing):
The definition of $P_{\mathbf{W},\mathbf{S}}$ is unclear. Is it a distribution over $\mathcal{W}^{(K+1)R}$ and then, is the use of the product $\Pi$ equivalent to $\otimes$? Otherwise, if it is a distribution over $\mathcal{W}$ can you make this explicit?

Author response:
- **Regarding $P_{\mathbf{S},\mathbf{W}}$:** The reviewer's understanding is correct. It is a probability distribution over $\mathcal{W}^{(K+1)R}$. We agree with the reviewer that $\bigotimes$ is a more accurate notation, that is used in the new version.

Reviewer comment (Theory):
In section 2, you said that 'the aggregation function at the PS is set to be deterministic and arbitrary' while in Theorem 1, $\bar{W}^{[r-1]}$ is drawn according to a probability distribution, what did I miss? Furthermore, how costful is this additional expectation in terms of computational time as it does not appear in many classical PAC-Bayes bounds?

Author response:
- **Regarding the aggregation:** First, we mention that there exists a recurrent typo in some of the bounds: $\overline{W}^{[r-1]}$ should be read as $\overline{W}^{(r-1)}$, which is corrected in the new version. We are deeply sorry for this. Now, concerning the question of the reviewer (by considering $\overline{W}^{(r-1)}$): The aggregation function is deterministic. This means that once $W_{[K]}^{(r-1)}$ is known, then $\overline{W}^{(r-1)}$ is deterministic. However, in general, $W_{[K]}^{(r-1)}$ are random due to the randomness in the way the mini-batches $\beta_{k,r,t}$ are picked from $S_k^{(r)}$, for all clients/rounds/iterations and also due to initialization randomness. Hence, $\overline{W}^{(r-1)}$ is also a random variable. Now, regarding the cost of expectation, we agree that if expectation were need to be taken with respect to $\overline{W}^{[r-1]}$, it would be costly. However, we only need to consider the expectation with respect to $\overline{W}^{(r-1)}$. If we now consider, for example, the approach of Wang et al. ("PAC-Bayes Information Bottleneck", ICLR 2022) on estimating the mutual information or KL divergence terms, then this means that we need an extra bootstrapping phase for $\overline{W}^{(r-1)}$. An initial idea is to aggregate a subset of the clients' models $\{W_k^{(r-1)}\}_{k \in [K]}$, drawn uniformly at random, and repeat the process a number of times. This is in the same spirit as the bootstrapping (for other variables) in Wang et al. This further investigation, which is indeed interesting, is left for future work.

Reviewer comment (Theory):
I found Theorem 1's proof poorly organised, for instance, $\mathbf{\nu}_{S}$ is defined in Lemma 1's proof, which has been put on another appendix. Similarly, I did not find a definition of $\mathcal{G}_S^{\delta}$ although somewhat inferable from context.

Author response:
- **Regarding $\mathcal{G}_{\textbf{S}}^{\delta}$:** We apologize for omitting the definition of  $\mathcal{G}_{\textbf{S}}^{\delta}$ (that contains $\nu_{\mathbf{S}}$) in the proof of Theorem 1, which happened after reorganization of that proof. We have added this in the new version.

Reviewer comment (Theory):
I don't understand the use of the subgausiannity assumption at the end of page 2022. Let rename $X= gen(s_k^{(r)}, \bar{W}^{(R)})$, then you affirm that because $X$ is $\sigma_{k,r}:= \sqrt{\frac{R}{4n}}$ subgaussian we have $\mathbb{E}[e^{\lambda X^2}]\leq \frac{2n}{R}$. This is highly non-standard, how do you prove it? To me, subgaussianity would only imply $\mathbb{E}[e^{\lambda X}]\leq \exp(\frac{\lambda^2 \sigma_{k,r}^2}{2})$.

Author response:
- **Regarding the subgausiannity:** This is a classical property of subgaussian random variables. Borrowing the reviewer's notation, if $X$ is $\sigma_{k,r}$-subgaussian, where $\sigma_{k,r}=\sqrt{\frac{R}{4n}}$, then by Theorem 2.6.IV. of Wainwright, 2019, (available at the link provided below), for any $\lambda_1 \in [0,1)$, $\mathbb{E}\left[e^{\frac{\lambda_1 X^2}{2\sigma_{k,r}^2}}\right]\leq \frac{1}{\sqrt{1-\lambda_1}}$. Now, letting $\lambda_1 =\frac{2n/R-1}{2n/R}=\frac{\lambda}{2n/R}$, we drive that
$$\mathbb{E}[e^{\lambda X^2}]=\mathbb{E}\left[e^{\frac{\lambda_1 X^2}{2\sigma_{k,r}^2}}\right]\leq \frac{1}{\sqrt{1-\lambda_1}}=\sqrt{2n/R}\leq 2n/R.$$
We added the reference in the proof for clarity. Moreover, previously for ease of expositions, we had reported the bound with the term $\log(2n/(R\delta))$ (and also since this term is rarely a dominant term). In the new version, we present them with the slightly tighter bound $\log(\sqrt{2n}/(\sqrt{R}\delta))$, for the clarity of the derivation.

The link: https://www.cambridge.org/core/books/highdimensional-statistics/basic-tail-and-concentration-bounds/30AF7B572184787F4C99715838549721

Reviewer comment (Theory):
The assumption that R < N and each data point is only visited in one round is not realistic and doesn't capture what happens for FL in practice. In most settings in FL, the number of samples per client (N) is small, and the number of rounds is much higher. Additionally, I don't see if the current analysis is possible to extend to these cases.

Author response:
We thank the reviewer for the new feedback. The comments by the reviewer on the practicality of the setup in which each sample can be used multiple times and what the reviewer believes to be the right behavior of the population risk as a function of the number of rounds in that case are certainly very interesting; and we would be happy to look at those aspects in a separate work. For now, we would be grateful if the contributions of this paper are considered and evaluated in their context. That is:

1) Analysis of Federated Learning from the view point of generalization error *is lacking* in both the general setup that the reviewer considers more practical or the one we assume in this paper). There exist no real bounds on the generalization error of FL that exploit its interactive structure, and of course none that is explicit in the number of rounds and system parameters. From this perspective, our bounds are the first of their kind for FL in the setup that we consider (which the reviewer considers un-practical)

2)  The very same setup that we consider was already considered in the context of Online Federated Learning, see, e.g

* Chen et al. "Asynchronous Online Federated Learning for Edge Devices with Non-IID Data" 2020 IEEE International Conference on Big Data, 2020
* * Mitra et al. "Online Federated Learning" 2021 60th IEEE Conference on Decision and Control (CDC)

and similar ones in the context of LocalSGD, see, e.g

* S. Stich, " Local SGD Converges Fast and Communicates Little",   (ICLR 2019) also available at https://arxiv.org/abs/1805.09767 
* Haddadpour et al.  Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronization (NeurIPS 2019)
* Gu et al. “Why (and when) does LocalSGD generalize better than SGD ?”, ICLR 2023

3) What we claim in this paper are:

* a bound on the generalization error of Federated SVM that increases with R; and which suggests that the generalization error of FSVM itself does so. This has also been validated experimentally.
* a direct consequence of this is that "the population risk decreases with R less fast than the empirical risk"
* a bound on the generalization error of general FL (under our setup); and experimental results that show that the population risk can even start to increase beyond some value of R. 

[Conclusion of the paper] At this point, we emphasize that, in particular, we here make no claim as for whether there should always be a tradeoff for the population risk that is similar to that that we observed in Figure 3(b), neither for general setups (where the entire dataset is used during each round) nor in our restricted setting. Actually, the example for FSVM shows no such  tradeoff; but in all presented experimental results the population risk decreases less fast with R than the empirical risk (from the experiment of Fig. 3(b), we know that in some cases the population risk may even start to increase start from some value of R). This latter finding was already observed in the context of Local SGD, see Figure 2 of Gu et al. Why (and when) does LocalSGD generalize better than SGD ?”, ICLR 2023.

One last point regarding whether the behavior reported in Fig(3) should or not be attributed to a decrease of the number of data points used for training locally at each round. In the opinion of the authors, it does not seem so. First, please observe that, for a large range of R, the population risk continues to decrease with R (less fast than ERM) even though the client uses fewer and fewer data points locally (n/R). It is only beyond some proper threshold R* that the change of the slope of population risk is observed. Second, the very same behavior was observed 

 Gu et al. Why (and when) does LocalSGD generalize better than SGD ?”, ICLR 2023.

where each data point is visited multiple times.

Reviewer comment (Evaluation):
The quantity of interest in ML is the true risk (population risk) and not the generalization. Even with the current assumptions, if the speed of decrease of training loss is more than the increase of generalization, it's not possible to argue the smaller number of rounds is better. In Fig 4. and Fig 8. of the appendix we can't see the increase in the population risk with the number of rounds for FL-SVM.

Author response:
We thank the reviewer for the new feedback. The comments by the reviewer on the practicality of the setup in which each sample can be used multiple times and what the reviewer believes to be the right behavior of the population risk as a function of the number of rounds in that case are certainly very interesting; and we would be happy to look at those aspects in a separate work. For now, we would be grateful if the contributions of this paper are considered and evaluated in their context. That is:

1) Analysis of Federated Learning from the view point of generalization error *is lacking* in both the general setup that the reviewer considers more practical or the one we assume in this paper). There exist no real bounds on the generalization error of FL that exploit its interactive structure, and of course none that is explicit in the number of rounds and system parameters. From this perspective, our bounds are the first of their kind for FL in the setup that we consider (which the reviewer considers un-practical)

2)  The very same setup that we consider was already considered in the context of Online Federated Learning, see, e.g

* Chen et al. "Asynchronous Online Federated Learning for Edge Devices with Non-IID Data" 2020 IEEE International Conference on Big Data, 2020
* * Mitra et al. "Online Federated Learning" 2021 60th IEEE Conference on Decision and Control (CDC)

and similar ones in the context of LocalSGD, see, e.g

* S. Stich, " Local SGD Converges Fast and Communicates Little",   (ICLR 2019) also available at https://arxiv.org/abs/1805.09767 
* Haddadpour et al.  Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronization (NeurIPS 2019)
* Gu et al. “Why (and when) does LocalSGD generalize better than SGD ?”, ICLR 2023

3) What we claim in this paper are:

* a bound on the generalization error of Federated SVM that increases with R; and which suggests that the generalization error of FSVM itself does so. This has also been validated experimentally.
* a direct consequence of this is that "the population risk decreases with R less fast than the empirical risk"
* a bound on the generalization error of general FL (under our setup); and experimental results that show that the population risk can even start to increase beyond some value of R. 

[Conclusion of the paper] At this point, we emphasize that, in particular, we here make no claim as for whether there should always be a tradeoff for the population risk that is similar to that that we observed in Figure 3(b), neither for general setups (where the entire dataset is used during each round) nor in our restricted setting. Actually, the example for FSVM shows no such  tradeoff; but in all presented experimental results the population risk decreases less fast with R than the empirical risk (from the experiment of Fig. 3(b), we know that in some cases the population risk may even start to increase start from some value of R). This latter finding was already observed in the context of Local SGD, see Figure 2 of Gu et al. Why (and when) does LocalSGD generalize better than SGD ?”, ICLR 2023.

One last point regarding whether the behavior reported in Fig(3) should or not be attributed to a decrease of the number of data points used for training locally at each round. In the opinion of the authors, it does not seem so. First, please observe that, for a large range of R, the population risk continues to decrease with R (less fast than ERM) even though the client uses fewer and fewer data points locally (n/R). It is only beyond some proper threshold R* that the change of the slope of population risk is observed. Second, the very same behavior was observed 

 Gu et al. Why (and when) does LocalSGD generalize better than SGD ?”, ICLR 2023.

where each data point is visited multiple times.

Reviewer comment (Theory):
As mentioned in the paper, another approach would be to just apply basic PAC-Bayes bounds such as McAllester's without explicitly considering the dynamics of FL. Some numerical comparison of these bounds is needed. It's not clear to me that the bound in theorem 1 gives a better guarantee. McAllester's bound doesn't have the structure of the FL explicitly, however, it would be applied to the output of the algorithm and that might be enough. Also, it doesn't require the assumptions mentioned in the weakness 1.

Author response:
Using the classic McAllester's PAC Bayes bound has two major shortcomings. First, considering the lossy version of it (which is more general) for FSVM leads to bounds of order $\mathcal{O}\left(\sqrt{\frac{B^2 \log(nK)}{nK\theta^2}}\right)$. This bound does not reflect the effect of the number of clients (since it depends only on $nK$, i.e., the total number of samples available, in contrast to our bound that depends on $nK^2$), nor does it include the effect of the number of rounds. More importantly, considering the entire learning algorithm as a single centralized algorithm and applying McAllester's bound results in a bound that can only be computed when all datasets are available on a server, and also only when the final model $\overline{W}^{(R)}$ is obtained. In contrast, our bound requires only the "local" availability of each client's data in each round, and requires only that client's "local" model at the end of the corresponding round. This is indeed a realistic assumption, and also helpful for "estimating" the total cumulative contributions to the generalization error at each round.

Reviewer comment (Experiments):
Based on my understanding of the analysis, I assume that the same approach can be applied to the number of local steps per round if each data point would participate in just one local round. As a result, the number of local steps would also appear in the bound. It would be interesting to see its effect, as in the experiments of the paper, the total number of SGD steps is fixed, and with the increase of R, the number of local steps decreases.

Author response:
**Regarding your point 5**: We don't see a straightforward extension of the current analysis used in proof of Theorem 4 to such a case. The reason is that one of the necessary steps in the proof of Theorem 4 is the perturbative analysis of the quantized local model of each client at the end of each round. This perturbative analysis is only valid if it is averaged over the model of other clients (or some similar "fair" aggregation method). Thus, it does not extend (at least trivially) to the case where different local steps of each client are considered separately.