From: ICLR.cc/2023/Conference/Program_Chairs
title: Paper Decision
decision: Reject
metareview:_summary,_strengths_and_weaknesses: This paper studies an important problem about "covariate shift" in federated learning, based on sample average approximation with use of importance reweighting for direct density ratio estimation. They develop the high-probability  generation guarantee and compute the excess risk for important weighting for ridge regression.   There were a lot of concerns in the initial reviews. After the rebuttal/revision and many rounds of discussions among reviewers and authors, the contribution of the paper has become more clear and several reviewers have raised their scores. Nevertheless, even without concerning about the writings, reviewers still feel this paper does not have sufficient novelty and the experiments are not strong. With my own reading of the paper, I agree with the reviewers that this paper in the current form is not good enough to be accepted to ICLR. We encourage the authors carefully revise the paper and improve the experiments for the next round of submission. 
justification_for_why_not_higher_score: The writing has issues which have caused a lot of confusions, the methods and the analysis are not novel, the experiments should be substantially improved for federated learning settings. 
justification_for_why_not_lower_score: N/A

From: ICLR.cc/2023/Conference/Paper4874/Authors
title: Thank you for re-evaluating our work and follow-up question
comment: We thank the reviewer for carefully reading all of the reviews and increasing the score.

Theoretically, we addressed two "generalization problems": one is the overall generalization across clients given true importance weights; the other is ratio estimation problem. We provide the theoretical results separately for better illustrations on each part, and  diagnose/understand the role of each part in theory. The overall take-home messages of our theory are that "our ratio estimation method generalizes to true ratios in FL even under imperfect estimates of $\overline r_k$'s" and "importance-weighted ERM results in smaller excess risk compared to classical ERM under certain conditions". 

It is an open question whether importance-weighted ERM is better than classical ERM under general data. We provided a counterexample in Appendix N to show how this is non-trivial.

Beyond our theory, we believe our algorithmic aspect will be beneficial to the FL community; to our knowledge, there has been no such algorithm in the past in FL.

Since our eventual goal is to make this paper more useful to the community, we are willing to improve our work further. Therefore, if the reviewer has any concrete comment on what kind of additional theory is beneficial and attainable in our case, we would be grateful.

From: ICLR.cc/2023/Conference/Paper4874/Reviewer_Yvya
title: Thanks for the response
comment: I want to acknowledge that I appreciate the response and have read them carefully. I am willing to update the score to 5 but I still have concerns about the theory. I still lean towards rejection as I think the theory is incremental to me.

From: ICLR.cc/2023/Conference/Paper4874/Authors
title: Happy to provide further clarification
comment: We hope that we have resolved the major concerns from your side. In particular, regarding "technical challenges and contributions", we hope that our explanation of algorithmic/theoretical/experimental challenges clarified your concerns. If possible, can you please rate the paper more positively? We would be happy to provide further clarifications if needed.

From: ICLR.cc/2023/Conference/Paper4874/Authors
title: Happy to provide further clarification
comment: We hope that we have resolved the major concerns from your side. In particular, regarding "cross-device FL", we hope that our explanation and additional experiments with 100 clients and client partial participation clarified your concerns. If possible, can you please rate the paper more positively? We would be happy to provide further clarifications if needed.

From: ICLR.cc/2023/Conference/Paper4874/Authors
title: Happy to provide further clarification
comment: We hope that we have resolved the major concerns from your side. In particular, regarding "technical challenges and contributions", we hope that our explanation of algorithmic/theoretical/experimental challenges clarified your concerns. If possible, can you please rate the paper more positively? We would be happy to provide further clarifications if needed.

From: ICLR.cc/2023/Conference/Paper4874/Authors
title: Follow-up
comment: Dear reviewer Yvya, 

We would like to check whether you have any questions/concerns? We also welcome any suggestions on improving our paper if the reviewer believes that it will make it easier for the reader to understand. Otherwise, we would appreciate it if the reviewer re-evaluates our submission based on the improvements we have already included (clarification of motivation, revisions in the notation, and newly added experiments).

From: ICLR.cc/2023/Conference/Paper4874/Authors
title: Follow-up 
comment: Dear reviewer Vgm5, 

We would like to check whether you have any questions/concerns? We also welcome any suggestions on improving our paper if the reviewer believes that it will make it easier for the reader to understand. Otherwise, we would appreciate it if the reviewer re-evaluates our submission based on the improvements we have already included (clarification of motivation, revisions in the notation, and newly added experiments).

From: ICLR.cc/2023/Conference/Paper4874/Authors
title: Follow-up 
comment: Dear reviewer S6qR, 

We would like to check whether you have any questions/concerns? We also welcome any suggestions on improving our paper if the reviewer believes that it will make it easier for the reader to understand. Otherwise, we would appreciate it if the reviewer re-evaluates our submission based on the improvements we have already included (clarification of motivation, revisions in the notation, and newly added experiments).

From: ICLR.cc/2023/Conference/Paper4874/Authors
title: New Experiments with 100 Clients and Client Partial Participation
comment: Dear Reviewer, 

We provided new experiments in settings closer to real-world FL. Please check the details [here](https://openreview.net/forum?id=Ho7W1yr8tV&noteId=FW1hGyW6qV).

Regards,   
Authors

From: ICLR.cc/2023/Conference/Paper4874/Authors
title: New Experiments with 100 Clients and Client Partial Participation
comment: To model a scenario closer to real-world FL, we consider a setting with 100 clients on CIFAR10 under partial participation of clients. We sub-sample 5 clients uniformly at random at every training step. The details can be found in Appendix O.1 where the data split is described in Table 5 and the results of the experiments are presented in Table 6 (and for convenience also in the table below). We observe that FIDEM uniformly improves the test accuracy when compared with FedAvg and that the gap is especially large between the worst-performing clients.


|                   	| *FIDEM* 	| *FedAvg* |
|-------------------------|--------------------|-----------------|
| Average client accuracy | **0.7658** | 0.7237      	|
| Worst client accuracy   | **0.6163** | 0.5403      	|
| Best client accuracy	| **0.9016** | 0.8904      	|

From: ICLR.cc/2023/Conference/Paper4874/Authors
title: Thank you for constructive comments and re-evaluating our work
comment: We thank the reviewer for their constructive comments, timely interactions, and increasing the score. In the uploaded revision, we have added a subsection in Appendix C to discuss the usefulness of importance weighting in the important and special case of our setting under no intra-client covariate shifts but inter-client covariate shifts.

From: ICLR.cc/2023/Conference/Paper4874/Authors
title: Response to Reviewer Vgm5
comment: Dear reviewer, 

This is a friendly reminder that to be able to properly address your concerns, we need further clarifications regarding your first comment “from distribution $−{\cal D}$ (covariance $−\Sigma$)”. 
We now address your other concerns:

------

- Sharing unlabelled test samples with the server… is too restrictive in FL:

Our privacy-preserving variant FIIDEM preserves the *same level of privacy* and *same communication costs* as those of the baseline FedAvg  but still achieves significant improvement over FedAvg in terms of overall test error in experiments. FIIDEM is described in Section 2.3 and Appendix E of the original submission. 

------

- In FL, … number of clients is … extremely large … each with possession of a small data set… error of … ratio estimation, is bounded by $O(1/\sqrt{n_i}+1/\sqrt{n_j})$ … more suitable to cross-silo setting with no privacy concern:

To show how our proposed methods can be potentially used in cross-device FL with a large number of clients each with small amounts of data, in Appendix E of the original submission, we provided a concrete example to show that the number of training data required for ratio estimation is reasonably low. In addition, communication overhead (as another major concern of cross-device FL) to estimate ratios is negligible compared to the communication costs for sharing high-dimensional stochastic gradients over the course of training. In particular, A total number of 1000 images are shown to be sufficient to learn density ratios on CIFAR10 (Kato & Teshima, 2021)[10, Section 5.1]. For cross-device FL, our privacy-preserving communication-efficient variant FIIDEM is a reasonable choice, which does not require any data sharing. Clients can estimate their ratios prior to training using a small portion of their own datasets. Then in each round of training, the participating clients can send the updates by applying ratios without additional computational costs and improve their overall test error. Our experiments show that FIIDEM significantly improves over FedAvg.

Concerning per-client generalization bounds, the rates match the **optimal minimax rates** for example for a density estimation problem when the density belongs to the Holder function class (Tsybakov 2008, Section 2) with a sufficiently large $\beta$ based on Definition 1.2 of Tsybakov (2008). The $\Omega(1/\sqrt{n})$ lower bounds are obtained for important problems including nonparametric regression, estimation of functionals, nonparametric testing, and finding a linear combination of $M$ functions to be as close as the target data generating function (Nemirovski 1998, Section 5.3).  

Alexandre B. Tsybakov. Introduction to Nonparametric Estimation. Springer Series in Statistics. Springer , 2008.

Arkadi Nemirovski. Topics in Non-parametric Statistics. Ecole d’Et´e de Probabilit´es de Saint-Flour XXVIII - 1998.

We have clarified this in Appendix J of the revision. 

----------

From: ICLR.cc/2023/Conference/Paper4874/Reviewer_rE23
title: Thanks for the clarification, score increased to 6.
comment: Thank you for the detailed response. As I indicated in my initial review my main concern was regarding no intra-client shift. You have now explained why even in this case, your method could be useful. As a result, I have made a relatively major change in my score from a 3 to a 6. 

I now have a much better understanding as to the benefit of FIDEM in the case of no intra-client shift. The main observation seems to be that now you have almost reduced the variance by a factor of $2$ (assuming the cross term in your expression for $\tilde \sigma^2_P, \tilde \sigma^2_Q$ is small enough). 

If accepted, my suggestion is that you make this point in the paper somewhere (perhaps around the discussion in section C of appendix), as it is quite counterintuitive that even in the case of no intra-client shift that there is something to be gained (potentially) with debiasing each client's loss rather than just minimizing the sum. Perhaps you can even give an example where $V_P + V_Q \leq \sigma^2_P + \sigma^2_Q$.

From: ICLR.cc/2023/Conference/Paper4874/Authors
title: Further Clarification on First Point 
comment: We thank the reviewer for further clarifications. Allow us to elaborate more so that it becomes clearer. Given two distributions, considering the case of no intra-client shift is a special case, where it is true that FedAvg is an unbiased estimate of the overal risk. However, this unbiasedness breaks as soon as there is only one client whose test and train dsitributions are different, which is very common in theory and practice (Sugiyama et al., 2007; Kanamori et al., 2009; Kato & Teshima, 2021; Uehara et al., 2020; Tripuraneni et al., 2021; Zhou & Levine, 2021). Please note that FIDEM is an unbiased estimate of the overal risk in a general FL setting without requiring any prior knwoledge/assumptions on the potential covariate shifts. For this more general and practical case, in Section 4.2, we showed that FIDEM results in smaller excess risk compared to FedAvg through a refined bias-variance decomposition.  


Even, in the special case of no intra-client shift, as we point out below, our method (FIDEM) has a smaller variance than FedAvg under certain conditions. Besides, our experimental results show significant improvement over FedAvg. We use the same notation as the reviewer: 

Let $\mathbb{E}_P[(\ell(z,\theta)-\mathbb{E}_P[\ell(z,\theta)])^2]=\sigma^2_P$ and $\mathbb{E}_Q[(\ell(z',\theta)-\mathbb{E}_P[\ell(z',\theta)])^2]=\sigma^2_Q$.

For FedAvg, we have  $\mathbb{E}_{P,Q}[(\ell(z,\theta)+\ell(z',\theta)-L(\theta))^2]=\sigma^2_P+\sigma^2_Q$. 

For FIDEM, we have
$\mathbb{E}_{P,Q}[(\frac{1}{2}(\hat L_P(\theta)+\hat L_Q(\theta))-L(\theta))^2]=V_P+V_Q$ where $V_P= \frac{1}{4}\mathbb{E}_P[(\hat L_P(\theta)-L(\theta))^2]$ and $V_Q= \frac{1}{4}\mathbb{E}_Q[(\hat L_Q(\theta)-L(\theta))^2]$. 

We now expand each term $V_P$ and $V_Q$: 

$V_P=\frac{1}{4}\mathbb{E}_P\Big[\Big((1+\frac{q(z)}{p(z)})\ell(z,\theta)-\mathbb{E}_P[\ell(z,\theta)]-\mathbb{E}_Q[\ell(z',\theta)]\Big)^2\Big]=\frac{\sigma^2_P+\tilde\sigma^2_P}{4}$ where 

$\tilde\sigma^2_P=\mathbb{E}_P\Big[\Big(\frac{q(z)}{p(z)}\ell(z,\theta)-\mathbb{E}_Q[\ell(z',\theta)]\Big)^2\Big]+2\mathbb{E}_P\Big[\Big(\ell(z,\theta)-\mathbb{E}_P[\ell(z,\theta)]\Big)\Big(\frac{q(z)}{p(z)}\ell(z,\theta)-\mathbb{E}_Q[\ell(z',\theta)]\Big)\Big]$. Similarly, we have 

$V_Q=\frac{1}{4}\mathbb{E}_Q\Big[\Big((1+\frac{p(z')}{q(z')})\ell(z',\theta)-\mathbb{E}_P[\ell(z,\theta)]-\mathbb{E}_Q[\ell(z',\theta)]\Big)^2\Big]=\frac{\sigma^2_Q+\tilde\sigma^2_Q}{4}$ where 
$\tilde\sigma^2_Q=\mathbb{E}_Q\Big[\Big(\frac{p(z')}{q(z')}\ell(z',\theta)-\mathbb{E}_P[\ell(z,\theta)]\Big)^2\Big]+2\mathbb{E}_Q\Big[\Big(\ell(z',\theta)-\mathbb{E}_Q[\ell(z',\theta)]\Big)\Big(\frac{p(z')}{q(z')}\ell(z',\theta)-\mathbb{E}_P[\ell(z,\theta)]\Big)\Big]$. 

We note if $\tilde\sigma^2_P+\tilde\sigma^2_Q\leq 3(\sigma^2_P+\sigma^2_Q)$ then, FIDEM will have smaller variance than FedAvg, i.e., $V_P+V_Q\leq \sigma^2_P+\sigma^2_Q$. The exact conidition depends on the loss and densities. 


In sum, unbiasedness of FedAvg breaks as soon as there is only one client whose test and train dsitributions are different, and under no intra-client shift, FIDEM has a smaller variance than FedAvg under certain conditions. We hope this answers your question and we would be happy to clarify if you have any further concerns.

From: ICLR.cc/2023/Conference/Paper4874/Reviewer_rE23
title: Clarification on first point?
comment: My observation is very simple. 

Consider you have the following situation, two clients, with distribution $P$ and $Q$, respectively. Moreover, I assume they have no intra distribution shifts, so that their train and test distributions are (respectively) $P$ and $Q$. To simplify my discussion, suppose you have one sample $z \sim P$, and one sample $z' \sim Q$. 

The sum of losses $\ell(z, \theta) + \ell(z', \theta)$ is unbiased for $L(\theta) = \mathbb{E}_P \ell(z, \theta) + \mathbb{E}_Q \ell(z', \theta)$. 

But the approach in your paper is the following: form 

$\hat L_P(\theta) = (1 + \frac{q(z)}{p(z)}) \ell(z, \theta)$ and $\hat L_Q(\theta) = (1 + \frac{p(z')}{q(z')}) \ell(z', \theta)$. 

While it is true that $\hat L_P, \hat L_Q$ are each individually unbiased for $L$, the clear disadvantage of your approach is having to estimate the likelihood ratios. Moreover, there is an increased variance for using likelihood ratio in the solution to the IW-ERM problem that results. 

Can you please explain what is the envisioned benefit of using your approach, when one is only interested in computing the minimizer of $L$ over my naive approach of minimizing $\ell(z, \cdot) + \ell(z', \cdot)$?

From: ICLR.cc/2023/Conference/Paper4874/Authors
title: Response to Reviewer rE23
comment: We thank the reviewer for their thoughtful comments which we address one by one below. 

------

- Under no intra covariate shift, … FIDEM leads to greater variance but with no gain in de-biasing: 

In this important setting, FIDEM does not lead to additional variance. Due to the double sum $\sum_{k=1}^K\sum_{l=1}^K$, we have more terms with each $F_k$ outputs an unbiased estimate of the overall risk minimizer (Each $F_k$ in (2.2) converges to the minimizer of $\sum_{i=1}^K R_i$ for $k\in[K]$ ). Now we have $K$ such terms. So the overall variance will be $O(\sigma^2/K)$ where $\sigma^2$ denotes the variance of the estimate for the overall risk minimizer using data of only one client.

Please note that our proposed privacy-preserving FIIDEM will be equivalent to ERM in this setting. 
We hope that this clarifies your concerns.
 

------

- Main technical contribution of this paper is the identification of the quantity $r_k({\bf x})$ : 

Please check our detailed and common response to this question [here](https://openreview.net/forum?id=Ho7W1yr8tV&noteId=Mbd8ONpyQLY).

------

-  Prior work on IW-ERM uses just a two client setting: 

We respectfully disagree. The prior work only considers one client, not even two  (Zadrozny, 2004; Sugiyama & Müller, 2005; Huang et al., 2006; Sugiyama et al., 2007; Kanamori et al., 2009; Sugiyama et al., 2012; Fang et al., 2020; Zhang et al., 2020; Kato & Teshima, 2021). If the reviewer has any links to works that do consider such a setting, we will be able to include it in our evaluation. 

------

- Train and/or test distributions of different clients have different supports: 

We can indeed relax the assumption to have $p_k^{\mathrm{tr}}({\bf x}^{\mathrm{tr}})>0$ for $k\in[K]$ and  ${\bf x}^{\mathrm{tr}}\in {\cal X}^{\mathrm{tr}}$ with ${\cal X}^{\mathrm{te}}\subseteq{\cal X}^{\mathrm{tr}}$.
 
In particular, we need a common data domain with strictly positive train density $p_k^{\mathrm{tr}}({\bf x}^{\mathrm{tr}})$ in the support similar to (Kato & Teshima, 2021), (Kanamori et al., 2009, Section 2.1). 
 
In FL, this is quite mild, e.g., the common data domain can include images with $W\times H\times C$ pixels where $W$, $H$, and $C$ denote the width, height, and number of channels.

We have addressed this issue in the revised version. 

------

- Provide additional experiments:

We are currently working on providing experiments in settings closer  to real-world FL. We will update the rebuttal and paper in the next few days. 

------

From: ICLR.cc/2023/Conference/Paper4874/Authors
title: Response to Reviewer Yvya
comment: We thank the reviewer for their thoughtful comments which we address one by one below. 

------

- Experiments to validate Thm. 2: 

In Appendix O.3, for both datasets used in Table 1 and 2, we have verified that the eigenvalues of the training distribution and test distribution for each client satisfy $\sqrt{\frac{\lambda_i'}{\lambda_i}} \in \Big(0, \frac{1+\sqrt{1+4 \xi_i}}{2}\Big)$ in Thm. 2. 

------

- Clarify challenges in generalizing analysis on the nnBD to FL:

Learning on multiple clients in FL under covariate shifts via importance-weighted ERM  is challenging due to multiple data owners with own learning objectives, multiple potential but unpredictable train/test shift scenarios, privacy, and communication costs (Kairouz et al., 2021). To be specific, 
it is non-trivial to control privacy leakage to other clients while estimating ratios and relax the requirement to have a perfect estimate of $\overline r_k=\sup_{{\bf x}\in {\cal X}^{\mathrm{tr}}}r_k^*({\bf x})$, which is a key step for non-negative BD (nnBD) DRM.
 **Our work is the first step towards handling inter/inter-client distribution shifts in FL**. While we focus on covariate shifts, it is possible to use our techniques and obtain similar FI(I)DEM formulations under *more general distribution shifts for FL*. 

**Theoretical challenges**: It is challenging to obtain per-client generalization bounds for a general nnBD DRM with multiple clients and imperfect estimates of $\overline r_k$'s due to intra/inter-client couplings in ratios. Note that previous generalization bounds in the literature are for only one client, **under a perfect estimate of $\overline r_k$**, and typically for a special DRM, which can be recovered as special cases of our bounds.
 

We have clarified technical challenges and contributions in Section 1.1 of the revision. 

------

-  What is the connection between analyses on nnBD and excess risk? 


Someone may question the usefulness of importance weighting regardless of the quality of estimated density ratios. While improved ratio estimates have been proposed in the past two decades, it was not clear whether importance-weighted ERM results in smaller excess risk compared to classical ERM  *even if we have access to perfect estimates of density ratios*. Our work gives an initial attempt by providing an affirmative answer for ridge regression. 

Please check our detailed and common response to this question [here](https://openreview.net/forum?id=Ho7W1yr8tV&noteId=Mbd8ONpyQLY).


------
- Experiments in real FL settings:

We are currently working on providing experiments in settings closer  to real-world FL. We will update the rebuttal and paper in the next few days.  

------
- Handling target shift does not need nnBD-based methods. Why did you include it in Section 5? 

We indeed did not use nnBD ratio estimation for experiments in Table 1. They are included to validate that our method can be extended beyond covariate shift. The text clarifiers this as follows:

“While we focus on covariate shift for density ratio estimation and establishing theory, it is possible to use our techniques and obtain a similar IWERM formulation as Eq. (3.1) under *more general distribution shifts for FL*.”

------

From: ICLR.cc/2023/Conference/Paper4874/Authors
title: Response to Reviewer S6qR
comment: We thank the reviewer for their quick response and thoughtful comments which we address one by one below. 

------

- Proof of Prop. 1 and $\lambda$: 


In the revision, we added “The consistency of FIDEM, i.e., convergence in probability, is immediately followed the standard arguments in e.g., (Shimodaira, 2000)[Section 3] and (Sugiyama et al., 2007)[Section 2.2] using the law of large numbers.” to Appendix C. 

Regarding the regularization parameter, $\hat{\bf \theta}$ is the estimation of *regularized* least squares, and thus the regularization parameter will occur in the bias-variance decomposition. For better illustration, in Lemma 1, we provided the explicit expression for regularized least square estimation $\hat{\bf \theta}= \arg\min_{\bf \theta} \sum_{i=1}^n w_i ({\bf \theta}^\top{\bf x}_i-y_i)^2+\lambda||\bf \theta||_2^2$ in Section 4.2. 


------

- Clarify technical challenges and contributions: 

Please check our detailed and common response to this question [here](https://openreview.net/forum?id=Ho7W1yr8tV&noteId=Mbd8ONpyQLY).


------

From: ICLR.cc/2023/Conference/Paper4874/Authors
title: Common Response
comment: We are thankful for the effort and the feedback of the reviewers. A common concern raised by the reviewers is the technical challenges and contributions w.r.t. problem/algorithm, theoretical guarantees, and experiments. We address this concern below: 

**Algorithmic challenges**: Learning on multiple clients in FL under covariate shifts via importance-weighted ERM  is challenging due to multiple data owners with their own learning objectives, multiple potential but unpredictable train/test shift scenarios, privacy, and communication costs (Kairouz et al., 2021). To be specific, 
it is non-trivial to control privacy leakage to other clients while estimating ratios and relax the requirement to have a perfect estimate of $\overline r_k=\sup_{{\bf x}\in {\cal X}^{\mathrm{tr}}}r_k^*({\bf x})$, which is a key step for non-negative BD (nnBD) DRM.
 **Our work is the first step towards handling inter/inter-client distribution shifts in FL**. While we focus on covariate shifts, it is possible to use our techniques and obtain similar FI(I)DEM formulations under *more general distribution shifts for FL*. 

**Theoretical challenges**: It is challenging to obtain per-client generalization bounds for a general nnBD DRM with multiple clients and imperfect estimates of $\overline r_k$'s due to intra/inter-client couplings in ratios. Note that previous generalization bounds in the literature are for only one client, **under a perfect estimate of $\overline r_k$**, and typically for a special DRM, which can be recovered as special cases of our bounds.


Someone may question the usefulness of importance weighting regardless of the quality of estimated density ratios. While improved ratio estimates have been proposed in the past two decades, it was not clear whether importance-weighted ERM results in smaller excess risk compared to classical ERM  *even if we have access to perfect estimates of density ratios*. It is challenging to provide guarantees on achieved generalization gains using importance weighting decoupled from the density ratio estimation problem. 

**Experimental challenges**: Even though well-established benchmarks for multi-client FL have been used, they are usually designed in a way that each client has test samples drawn uniformly from a set of classes. However, we believe this might not be the case in real-world applications. Hence, we had to devise an algorithm and method for this challenging case.

Overall, proposing an intuitive framework to minimize average **test error** in FL, which can be extended to general intra/inter-client distribution shifts, designing efficient mechanisms to control privacy leakage while estimating ratios (FIDEM) along with a privacy-preserving and communication-efficient variant (FIIDEM), improving non-negative BD DRM under FL without requiring perfect knowledge of $\overline r_k=\sup_{{\bf x}\in {\cal X}^{\mathrm{tr}}}r_k^*({\bf x})$ along with establishing generalization guarantees for  general nnBD DRM with multiple clients **under imperfect estimates of $\overline r_k$**, which unifies a number of DRMs, showing benefits of importance weighting in terms of excess risk decoupled from density ratio estimation problem, and finally demonstrating more than 16% overall test accuracy improvement over existing FL baselines when training ResNet-18 on imbalanced CIFAR-10 are our main contributions.

We appreciate the thorough reviews that aim at achieving a high quality contribution to FL. However, we have done our due diligence and believe that this is the *first work addressing inter and intra-client distribution shifts in FL*. As such, we had all types of challenges in defining our loss, guarantees, and framework. We have strived to make the ideas intuitively explained; however this should not be confused with trivial contributions.

In the uploaded revision, we have elaborated on those challenges and main contributions (highlighted in blue). We are eager to understand if there are any other questions or concerns on this topic from the reviewers.

From: ICLR.cc/2023/Conference/Paper4874/Reviewer_S6qR
title: Thanks for the clarification 
comment: I thank the authors for their timely response & clarification. 

However, I still believe this paper should be given a thorough revision and clarify the above misunderstandings in their paper. For example, the two references  (Shimodaira 2000, Section 3) and (Sugiyama et al., JMLR2007, Section 2.2)  are not even mentioned in their proof and the current proof is simply a calculation of the expectation. Also, the definitions of $\lambda$ should be given in a more explicit way. Apart from the above-mentioned places, other places suffer from similar issues.

Also, I would recommend highlighting the technical challenges and their technical contributions, which is not clear in this submission. 

In summary, I think the current version is below the bar of this conference and should be thoroughly revised. I am open to a rating change, conditional on that a carefully-revised version is updated before the response deadline.

PS: in Lemma~$1$, the definition of $L(\theta)$ does not contain the regularizers let alone $\lambda$. Why it will appear in the bias and variance?

From: ICLR.cc/2023/Conference/Paper4874/Authors
title: Clarification Needed on "from distribution $−{\cal D}$ (covariance $−\Sigma$)"
comment: We thank the reviewer for their comments. In the one-dimensional case, covariance $-\Sigma$ implies sampling data points from a distribution with a negative variance, which is not possible. Could you please double check and if we have misunderstood your criticism on our side, please clarify what specifically you mean?

From: ICLR.cc/2023/Conference/Paper4874/Authors
title: Clarifications of Misunderstandings
comment: We thank the reviewer for their comments. We would like to clarify some seemingly misunderstandings regarding our methods/proofs:  

------

-  FIDEM contains the probability $p_{\ell}^{\mathrm{te}}$... This information is w.r.t. the testing data and should not be used in training:

We do not sample from test distribution when training a model. Ratios in e.g., Eq. (2.1) and Eq. (2.2) are outputs after computing density ratios at *training points*. In particular, $p_{\ell}^{\mathrm{te}}$ is used to denote the probability density function in Eq. (2.1). Please note that computing density ratios at training samples is the very basic idea of  importance weighting (Zadrozny, 2004; Sugiyama & Muller, 2005; Huang et al., 2006; Sugiyama et al., 2007; Kanamori et al., 2009; Sugiyama et al., 2012; Fang et al., 2020; Zhang et al., 2020; Kato & Teshima, 2021). 

------

-  Convergence in probability in Prop. 1:

We believe our proof is correct. The consistency of FIDEM in Prop. 1 including special cases discussed in Appendix C is immediately followed the standard arguments in e.g., (Shimodaira 2000, Section 3) and (Sugiyama et al., JMLR2007, Section 2.2) using the law of large numbers. We hope this answers your question and we would be happy to clarify if you have any further concerns.


------

-  $\lambda$ is not stated explicitly :

It is indeed introduced in Lemma 1 in the original submission.

------

-  Prop. 1 in the main context is denoted as Prop. 2 in the supplementary material:

They are indeed different. In Appendix C, we include special cases in (C.1)-(C.3). We opt to not discuss all cases in the main draft and focus only on a simplified version of the general scenario (CS on multi.) in Prop. 1. Please note that the statement of Prop. 1 is slightly different from that of Prop. 2. But its proof follows that of Prop. 2. 

------

We would be happy to clarify further to Reviewer S6qR if their concerns are not addressed.

From: ICLR.cc/2023/Conference/Paper4874/Reviewer_S6qR
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: This paper focuses on the covariate shift in the federated learning framework. In sec 2.2, they formulate FIDEM and show that it is consistent. Also, they briefly discuss its modifications in terms of privacy. The modified objective function is named as FIIDEM. In Sec 3, they discuss the estimation of the ratio $r_k(x)$ in Eq. (3.1). The detailed estimation procedure is discussed in Sec. 3.1 and Sec. 3.2. The theoretical guarantee is in Sec. 4. In Sec 4.1, they study the estimation of $\hat{r}_k$ and put the result in Theorem 1. In Sec 4.2, they consider the ridge regression as an illustrative example to show FIDEM outperforms the classical ERM in reducing the regret. In Sec. 5, they present the numerical results and draw the conclusions iin Sec. 6.
strength_and_weaknesses: Strength: 
First, this paper consider a very interesting issue. Also, it is well written and can convey its basic ideas to the readers. 

Weakness:
1. The biggest concern is that FIDEM contains the probability $p^{\textup{te}}_{\ell}$ (c.f. Eq. (2.1)). This information is w.r.t. the testing data and should not be used in training. 

2. Certain statement is not correct. In Prop. 1, they claim FIDEM __converges in probability to the optimal solution__. Notice that __converge in probability__ has a specific meaning in probability. It is used to distinguish with the concepts __converge in distribution__, __almost sure converge__, and $\ell_p$ converge. After checking their proof, I think the authors actually mean __convergence in distribution__. Also, their proof is not complete. They should also prove that they can switch the order of $\min$ and $\lim$. 

3. The notation is a little messy. In Thm.2, the regularization coefficient $\lambda$ is not stated explicitly and it is in a very similar form of the $\lambda_i$ (the probability).

4. Also, the indexing is not consistent between the main context and supplementary material. For example, Proposition 1 in the main context is denoted as Proposition 2 in the supplementary material.


clarity,_quality,_novelty_and_reproducibility: This paper is well-written & can convey its basic idea to the readers.
summary_of_the_review: Although this paper is well written, it contains serious problem such as using the testing information during training. 
Also, certain claims are not accurate (see my comments above). 
correctness: 2: Several of the paper’s claims are incorrect or not well-supported.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 3: reject, not good enough

From: ICLR.cc/2023/Conference/Paper4874/Reviewer_Vgm5
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This paper proposes an important sampling approach to handle inter and intra client distribution shifts in the federated learning setting. To this end, the authors propose Federated Importance-weighteD Empirical risk Minimization (FIDEM) algorithm to optimize a global FL model, along with new variants of density ratio matching methods, in particular 
a histogram-based density ratio matching method. Then, we introduce another variant dubbed as FIIDEM) that  does not require any form of data sharing among clients and enjoys the same level of privacy and same communication costs as FedAvg.

strength_and_weaknesses: The proposed density ratio matching method is a natural generalization of existing methods to federated settings with a binning technique to achieve this goal which can be implemented in parallel in a simple way: clients first share unlabelled test samples with the server. The server returns the randomly shuffled pool of samples to all clients. On the theoretical side, the authors provide bounds on the error of estimating ratios, and generalization of  FIDEM. I have a few concerns about the results:

- Consider a simple setting where there are $n$ users, $n/2$ with data sampled from distribution $\mathcal{D}$ (e.g., covariance $\Sigma$) and the rest from $-\mathcal{D}$ ($-\Sigma$). In this case, understanding it is an extreme case, if we shuffle the data, with high probability no client can make informative ratio estimates due to discrepancy among distributions. But I was not able to infer this from the derived bound.

- I appreciate authors who honestly mention that  clients need to compromise some level of privacy and share unlabelled test samples with the server, but this is too restrictive in FL. Moreover, in an FL setting, usually the number of clients is assumed to be extremely large (at least in cross-silo) each with possession of a small data set. Since the error of standard empirical discrepancy estimation methods between pair of data sources $i$ and $j$, which also applies to ratio estimation, is bounded by $O\left(1/\sqrt{n_i} + 1/\sqrt{n_j}\right)$ where $n_i$ and $n_j$ are number of samples in data sources, I am not sure how these methods can be applied to cross-device FL (would be more suitable to cross-silo setting with no privacy concern).
 

clarity,_quality,_novelty_and_reproducibility: I appreciate the author's effort to make the paper as readable as possible, but in most cases it is hard to follow the results and presentation could be improved.

summary_of_the_review: Overall, the paper studies an interesting question in learning from multiple heterogeneous data sources, but I did not find the theoretical results and proposed algorithm novel/exciting enough due to issues discussed above. Also, I have concern about the applicability of algorithms to FL as advertised by authors which makes me lean toward rejection.

correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper4874/Reviewer_Yvya
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: This paper focuses on federated learning under covariate shift using direct density ratio estimation. This work establishes high-probability generalization guarantees and the benefit of importance weighting in terms of excess risk through bias-variance decomposition in a ridge regression problem. The experimental results of the proposed method is significantly better than the vanilla federated averaging in Fashion-mnist and C-mnist datasets.
strength_and_weaknesses: Strength:

Even though the theory is built on a set of previous works, the results under the federated setting is new.

The empirical results are impressive.

The result also covers cases when the covariate shift correction is working better than ERM.

Weakness:
The experiments are not relevant to the theory. For example, for theorem 2, it would be nice to have it validated in the experiments.

The new federated learning setting is similar to the covariate shift to me. Most of the analysis under vanilla covariate shift can be easily applied to the federated learning setting. I may be wrong, but at least it is not that clear in the paper what is the hard part of generalizing analysis on the nnBD to federated learning.

The analysis on nnBD is general while the analysis for the excess risk is for rigid regression and also for one-hot cases. They are not that connected, so we do not know how to think about the whole federated learning in an end-to-end fashion. From the literature, it seems the two analyses are rooted in different lines of previous research. It would be nice to build a more natural connection between them.

The experiments are limited that more real federated learning settings should support the proposed method. 

Target shift is a slightly different setting than covariate shift even though important weighting is still an effective method there. However, in target shift, you do not need nnBD methods for density ratio estimation. So I am not sure why it is included in the experiments.
clarity,_quality,_novelty_and_reproducibility: The technical sections are relatively sound.

The presentation of the math is relatively clear, however the writing can be significantly improved. The transition between sections can do a better job of preparing the readers for different content. 

I do not find code repositories. There are limited details mentioned in the main paper. Appendix includes enough details for reproducibility.

summary_of_the_review: This paper will benefit from a significant restructure and rewriting to make the theoretical results more connected with each other and expand the experiments. How federated learning imposes new challenges to the analysis and methodology should also be emphasized. 
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper4874/Reviewer_rE23
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: The authors propose FIDEM, a framework (i.e., modeling choice) to compute a parameter which minimizes the average population risk over all clients. Their proposal is essentially sample average approximation with use of importance reweighting. They identify a quantity which is useful in this computation: the sum of density of the test distributions over the train distribution of a given client (see eqn. 3.1). They then propose DRM for estimating this ratio. 
strength_and_weaknesses: My main concern with this paper is that the formulation is problematic in the case of no intra covariate shift. In particular, imagine that you have K clients, all of which have no internal shift (i.e., their train and test distributions are the same), yet their covariate laws are different. In the notation of this paper you have $p_i := p^{\rm tr}_i = p^{\rm te}_i$ for all i, but $p_i \neq p_j$ for distinct $i,j \in [K]$. 

In the setting described above, FIDEM is attempting to minimize the average risks across all the clients, but still employs likelihood-reweighting, which will lead to greater variance but with no gain in de-biasing. This is not necessary. In this case, one can simply minimize the average empirical risks (i.e., minimize the sum of the client loss over all the network only reweighting by the number of samples on each client) --- and this will be an unbiased approximation of the average empirical risk. The authors need to justify why this formulation is beneficial in this (important) setting. For instance, look at equation (C.1). The approach suggested here amounts to debiasing the estimate for R_1 and R_2, even though this does not matter from the perspective of minimizing the sum of the two risks. 

Otherwise, my concern is that this paper is not very original. The main contribution of this paper (from a technical perspective) is the identification of the quantity r_k(x) as given in equation (3.1). This is very straightforward, and is essentially moving one sum inside another. Not only that, the authors themselves cite papers of Sugiyama et al, which has proposed reweighting ERM. This is not at all a new observation beyond moving a sum inside another for $K > 2$ (typically prior work on IW-ERM uses just a two client setting, but this is a very minor technical detail.)

Finally, the authors' assumptions regarding their likelihood ratio estimation procedure are very strong. Their condition assumes that all the train and test distributions are mutually absolutely continuous with respect to one another. This for instance, precludes the possibility that the train and/or test distributions of different clients have any difference in their supports. We know this is not necessary, for instance, in the setting of a well-specified linear model, in which case, the only requirement to learn under covariate shift is that the distributions all place some mass in all subspaces (but note that is is possible to have different supports). 

I would also point out that the experimental evaluation is lacking greatly. The majority of the paper studies covariate shift, yet only one example is given and the result is quite similar to FIIDEM. Finally, the authors should provide more evaluations on e.g. image tasks, as have been proposed by others in the community (e.g. the WILDS datasets, LEAF datasets, etc.)

clarity,_quality,_novelty_and_reproducibility: Please see above regarding quality and originality. 

The paper is well written and the theoretical statements are clearly stated. 
summary_of_the_review: I currently recommend that this paper is rejected. The methodology is relatively standard (importance reweighting). While the authors do propose some new HDRM estimator for the likelihood ratio, the formulation in the case of only inter-client shift (but no intra-client shift), as well motivated by recent surveys (e.g. Kairouz et al) on FL, is highly problematic, as it introduces greater variance (e.g. from estimating a likelihood ratio). 


correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold