Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper presents an adaptive kernel approach for learning causal effects from multiple sources. Random Fourier features further extend the kernel to a federated setting.
strengths_and_weaknesses: Strength:

This paper on federated causal effect is timely and the presented method is reasonable.

Weakness:

1. Whether using a variational auto-encoder (VAE) can preserve the identifiability is unclear because it is known that the common VAE model is unidentifiable [1]. The cited previous work [2] did not claim any identifiability result.

2. The settings regarding the latent variables are not consistent between Lemma 1 and 2. Lemma 1 assumes latent variable but Lemma 2 does not.

3. There are only 5 sources in the experimental setting, which is too few for a federated learning paper.

Reference

[1] Khemakhem, Ilyes et al. “Variational Autoencoders and Nonlinear ICA: A Unifying Framework.” ArXiv abs/1907.04809 (2020).

[2] Louizos, Christos et al. “Causal Effect Inference with Deep Latent-Variable Models.” NIPS (2017).

questions: Is it possible to scale up the synthetic data experiment regarding clients and data samples?
limitations: N/A.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper studies the treatment effect estimation from decentralized, heterogeneous data sources that could not be combined together at the individual level. This paper proposes an algorithm based on Random Fourier Features that adaptively learns the similarities among data sources and optimally uses other data sources to learn the treatment effect parameters on each data set. This paper provides minimax lower bounds for the parameters estimated from its proposed algorithm. 
strengths_and_weaknesses: This paper lies at the intersection of causal inference and federated learning, which is an underexplored, but rapidly growing research topic, and has broad applications, especially in healthcare. 

This paper accounts for the heterogeneity across data sets, and proposes a novel adaptive algorithm to optimally leverage other data sets to estimate the treatment effect on each source data. Importantly, this paper allows for unobserved confounders and uses variational inference that can identify treatment effects in the presence of unobserved confounders. Note that Vo et al (2021), Xiong et al (2021), and Han et al (2021) assume that all confounders are observed. Therefore, allowing for unobserved confounders is a significant contribution to the literature on federated causal inference.

Moreover, I like the idea of using adaptive kernels and Random Fourier Features to account for heterogeneous data sets and avoid negative transfer, which I think is an important complement to the adaptive method in Han et al (2021). 

Overall I think this paper is well-written and I enjoy reading the paper. 


questions: 1. Privacy guarantee: Given that the paper is in a federated setting, what can be shared across data sets and which cannot? It will be helpful to elaborate on the privacy constraints (e.g., data cannot be shared at the individual level). It will also be helpful to clarify why the proposed method satisfies the privacy constraints. 

2. Effect of heterogeneity on parameter estimation: I really appreciate that this paper provides the minimax lower bounds for the proposed method. However, I think some interpretations of the bounds will make them more accessible to readers. For example, how do the lower bounds vary with the distance between true parameter values (a measure of heterogeneity) across data sets? Some discussion will be helpful.
limitations: I think this paper can benefit from improving the exposition of its method and main results, especially in Section 4. For example, the authors may consider incorporating the answers to my questions above in the updated version of their paper.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This work is an important step toward privacy-preserving causal inference.
The authors introduce a new approach to causal inference under a specified causal graph from a federated setting with multiple, decentralized, and dissimilarly distributed data sources.
The authors assume the confounders are latent (unobservable) and use use variational inference in the spirit of the variational auto-encoder (VAE) to recover the latent confounders. Further, they propose a kernel-based approach to learn the variational posterior distribution and adapt Random Fourier Features to rewrite the objective function as a summation of local objective functions in each source.

strengths_and_weaknesses: > originality:

It seems to me that the problem is original and important (causal inference in federated setting), while the methodology part seems a mixture of existing results (e.g., identifiability of $\boldsymbol{Z}$, solution of $f_c$ in equation (9), the application of Random Fourier Features in equation (10), (12), (13) ). If I misunderstood, please point out which part is original.

> quality&significance:

This is a solid work with contributions to causal inference in federated learning.

> clarity:

The paper is clear and well written.  But it seems overstate the contributions in methodology.


questions: >1. why assume the causal structure in Figure 1.

I have to say the assumption of $\textit{Causal Structure}$ in Figure 1 seems a rather strong assumption to me. 
For one thing, it assumes all the confounders are latent, while in practice the confounders are usually partially observed (some of $Z$ belong to the covariates $X$).

>2. Is it appropriate to use one vertex to denote $X$ in the causal structure

In the causal structure in Figure 1, all the covariates are denoted by one vertex. While in practice (e.g. your real data example), each dimension of $X$ might denote a variable with specific physical meaning, then it is inappropriate to put all the covariates in one vertex. Sorry to stick on the causal structure assumption, I suppose it is quite important since all the content is based on this assumption.

>3. The identification of $Z$

The authors have introduced the identification results in Louizos et al. (2017) in section 3.1. A further question, whether the federated setting would benefit the identification of $Z$, as the assumptions in Louizos et al. (2017) seems very rigid. The authors may refer to the following reference for the identification of latent causal factor in multiple-distributions setting.

Sun X, Wu B, Zheng X, et al. Recovering latent causal factor for generalization to distributional shifts[J]. Advances in Neural Information Processing Systems, 2021, 34: 16846-16859.

>4. the originality of the method

Could the authors explain whether it is the first time to apply the Fourier Features to the optimization process to enable the assertive form of the objective function (equation (13))? As it seems to me a very important innovation of this work.

>5. Experiment

In the real data experiment, whether the ground truth is obtained by raw difference? (since it mentioned IHDP is a randomized study). If so, then I suppose the result is not convincing enough, since it is a very special (degenerative) case with no confounders.

limitations: The original contribution sees unclear.
The fundamental assumption seems too strong.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The authors propose a new approach called CausalRFF for causal inference (CATE and ATE) based on data from different sources with dissimilar distributions. The approach is built upon Federated Learning such that the risk of privacy leakage can be minimized. 
strengths_and_weaknesses: Observed data can come from different sources with dissimilar distributions, the authors thus propose a model to learn the distributions separately and adaptively. They use a kernel based model and adapt Random Fourier Features into the model. The authors undergo comprehensive experiments to demonstrate that the proposed approach outperforms other existing methods.

There are some grammatical mistakes. For example, line 202 ‘requires collecting all data points from all source’ should be ‘requires collecting all data points from all sources’, line 188-189, ‘In particular, we propose a kernel-based approach to learn these distribution’ should be ‘In particular, we propose a kernel-based approach to learn these distributions’

questions: The paper is generally well-written, but still, I have some questions to ask and want the authors to clarify:

1. In line 200, you write ‘kappa(u^s,u^v)=lambda^{s,v}k(u^s,u^v) if s1\neq s2’. Should it be ‘kappa(u^s,u^v)=lambda^{s,v}k(u^s,u^v) if s\neq v’? If not, how to interpret the formula here? Furthermore, what is the purpose of lambda^{s,v}? Why don’t you introduce lambda^{s,v}k(u^s,u^v) in the case when s1=s2?
2. In line 219, what is L^(s)? Should it be the empirical ELBO from source s? In each source s, should we optimize according to J^(s), and in the federated leverl, we should use Eqn. (13) instead of Eqn. (8)? 
limitations: To my knowledge, this submission has no potential negative societal impact.
review_title: null

Meta-review: metareview: All reviewers found the problem setting, which spans federated learning and causal estimation, important and well motivated, and felt the paper made a solid technical contribution in this space. The reviewers raised questions about topics such as identifiability, the nature of the privacy guarantees, and some technical details. The author responses addressed these questions adequately; one reviewer raised their score. Overall, no major weaknesses were raised. Congratulations to the authors on a solid and well received paper.

confidence: Certain
Decision: Accept