Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposed a CATE estimation method for graph-structured treatment by Robinson Decomposition. It is an extension of Robinson Decomposition, and the experiment shows the effectiveness of GIN on two small world datasets. The writing is good, but some contents are missing, so that the paper is not that easy to read. My detailed comments and concerns are described in the following.
main_review: 1. My biggest concern is about m(x). m(x) is related to the  e^h(x), according to the definition. However, in the training algorithm, the estimated function m(x) is fixed after Stage 1, only with the observational x and y. The design of training mismatch Eq(11), and e^h(x) is always updating when training the model. How can m(x) be fixed in the first place?

2. how to mitigate dependency between X and T towards unbiased treatment effect estimation? It is not clear in the manuscript.

3. GIN seems to work well on mitigating the selection bias according to the experiments. Which component of GIN contributes to controlling it? It is also not clear in the paper.

4. GIN uses GNN to handle the graph treatment, i.e., developing the function h(T). However, its detailed mapping process is missing. For example, how to get the readout output of the graph treatment?  

5. the paper aims to estimate the CATE, so how to select the covariate values x in the training set? or use all values shown in the training set?

6. Under the setting of the GIN model, the identification of causal effect is another concern.

7. The number of baseline methods is a bit limited. Authors can use more treatment effect estimation models and expand them to the graph-treatment setting. For example, using the CFR model to obtain covariates' representation and use GNN to map the treatment into low-dimensional space, then concatenate them to predict potential outcomes.

8. BTW, as far as I know, GIN usually means "Graph Isomorphism Network", which is published in ICLR 2019.
limitations_and_societal_impact: None
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes graph intervention network (GIN) to estimate conditional average causal effects when treatments are graph-structured. GIN builds on an extension of traditional Robinson decomposition to arbitrary treatment types. In particular, GIN factorizes the outcome Y into an inner product of two terms $g(X)^\top h(T)$. During training, GIN first estimates a mean outcome model $m(X)$ and then estimate $g,h$ using the estimated $m(X)$ function. The method is evaluated on two datasets with synthetic outcome functions.
main_review: Overall, the proposed method is an interesting algorithmic solution to graph-structured treatment effect estimation. The method is a novel approach and the assumption on product effect is reasonable. My major concern of the paper are the following:

1. Assumption on overlap. When treatment itself is graph-structured or high-dimensional, it is hard to satisfy the overlap assumption in general. There are many unseen combinations of covariates, treatments, and outcomes $(x,t,y)$. Therefore, for many treatments $t$, the empirical estimate of $p(t|x)=0$. I don't think this paper has sufficiently addressed this issue. 

2. The TCGA experimental setup is too synthetic. The outcome function is purely simulated (a predefined function). In addition, molecules from the QM9 datasets are not really drug like and they are typically not qualified as chemotherapy compounds. Using drug like molecules from the ChEMBL database is a more realistic setup.
limitations_and_societal_impact: The paper sufficiently addressed its limitations and societal impacts.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper considers the problem of estimating conditional average treatment effect when the treatment is a graph. The paper generalizes the Robinson decomposition, adapts the R-learner to this graph-treatment setting and introduces a two-stage algorithm. The performance of the proposed algorithm is compared to a few benchmarks through numerical experiments. 
main_review: Originality: The proposed method is original. The paper generalizes the Robinson decomposition and adapts the R-learner in a very interesting and novel way. The two-stage training algorithm is different from the cross-fitting method considered in R-learner. 

Quality: As far as I checked, all claims are well supported. The experimental results are explained very clearly. One question I have is on cross-fitting. My impression is that cross-fitting is usually very helpful (sometimes even necessary) in developing theories. In practice, it is not the most data efficient way, since it uses only part of the data to train things. I’m wondering in the experiments, how much improvement in accuracy owes to not doing cross-fitting? Do you think there are specific settings where not doing cross-fitting would make the method fail? Would theories still go through without cross-fitting? I understand there is no guarantee of convergence so far, but I think a slightly deeper discussion on these would be very helpful.

Clarity: The paper is well written and I enjoyed reading it. 

Significance: The problem this paper considers is very important. The paper also provides a strong solution to the problem. The proposed algorithm appears to perform very good empirically. Indeed, I believe the proposed method works for any non-binary treatment variables. It seems that both the Robinson decomposition and the two-stage algorithm can be generalized to other types of treatment, e.g., real numbers, vectors (potentially high dimensional), or categorical variables. I wonder whether this is true and how much the algorithm needs to be modified to adapt to other types of variables. If this can be discussed in depth, I think this can make the conclusions of the paper even stronger. 
limitations_and_societal_impact: The authors have adequately addressed the limitations and potential negative societal impact of their work. They have discussed limitations including lack of a convergence guarantee and potential hidden confounders.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper studies the estimation of conditional average treatment effect (CATE) with multiple treatments that are graph structured. The proposed method extends the R-learner (Nie and Wager, 2020), which is designed for binary treatment, by learning a representation that reduces the dimension of the treatments. The representation is parametrized by a graph neural network that takes the graph structure into account. Unlike the standard R-learner that learns the nuisance components $E[Y|X]$ and $E[T|X]$ separately from the CATE, the proposed method only estimates $E[Y|X]$ separately while learns the representation $h(T)$, the propensity features $E[h(T)|X]$, and CATE jointly via gradient-based methods. 
main_review: The idea of combining R-learner and representation learning to reduce the dimension of complex treatments is quite interesting. The method can be useful in many applications that involve graph-structured treatments, such as those in two-sided markets. The paper is well-written overall and the technical arguments are sound. 
limitations_and_societal_impact: (1) I would suggest stating the product effect model (9) directly without describing the general version (8), which looks odd to me. In fact, R-learner, or Robinson's method, is designed for the product effect model which breaks the response function $f(x, t)$ into the target parameter (i.e., CATE in this case) and two nuisance functions. It is shown that (e.g., Nie and Wager, 2020) R-learner yields a faster rate for the target parameter if only the two nuisance functions are estimated at an $O(n^{-1/4})$ rate. Without this structure, there is no reason to estimate $f$ through (8); instead, one can directly estimate it by minimizing $(1/N)\sum_{i=1}^{N}(y_i - f(x_i, t_i))^2$. 

(2) The overlap assumption (Assumption 2) is arguably too strong when the treatment is complex. Under the model (9), is it fine to just assume overlap on $p(h(t)|x)$?

(3) Algorithm a (GIN training) updates $\eta$ only once for every $K$ updates of $\phi$ and $\eta$. Is there any specific reason for this asymmetry?

(4) It would be nice to provide a slightly more detailed description on how to parametrize the treatment via a graph neural network. 

(5) An advantage of cross-fitting is to reduce bias and enable valid inference (e.g., valid pointwise confidence intervals). Is there any approach for inference with the proposed method?
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposed a CATE estimation approach when the treatment variable $T$ is a graph. Under a separable assumption (A3) between confounders $X$ and the treatment $T$ in the outcome structural equation ($Y$), Robinson's decomposition of CATE for binary $T$ can be applied. The performance of the authors' proposal was evaluated in both synthetic experiments and real data analysis.
main_review: The authors set out to address a very important problem in causal inference, which is methodology development for non-Euclidean treatment variables. In particular, the authors considered graph-valued treatment $T$. However, from my own perspective, there are several major limitations in this work.

(1) The type of intervention on graph-valued treatment is not spelled out clearly in the paper (or at least in the introduction section).

As a referee, when I saw the title of this paper, the immediate question I had in mind is what type of intervention on graphs the authors were considering. But until Section 4, I finally realized that the intervention is simply changing one graph $\mathcal{G}$ to another $\mathcal{G}'$. I think this should be made much clearer at the beginning of the paper.

(2) As a follow-up comment of (1), I would think it will be more interesting to consider problems like the direct effect of changing one set of vertices on a graph and the indirect effect of changing this set of vertices on the overall network structure, further on to the outcome. Such effect decomposition might be more interesting to applied researchers. For example, there might be targeted cancer therapy that can inhibit the expression of one particular gene. Then it is very interesting to understand whether the effect of such a drug is manifested through its target gene or through the entire gene regulatory networks. Directly comparing the change of the network may not be entirely meaningful to applied researchers.

(3) Kennedy (2020) proposed a novel CATE estimation method based on the efficient influence function. I am wondering if this or related method can also be used in the graph-valued treatment setup. 

Some of the following minor limitations might have been pointed out by the authors themselves. If so, I will not describe in more detail:
(1) Lack of theoretical guarantees. But personally, I do not think this is a very important limitation for a really novel idea.
(2) In Section 4.4, with partial data, I thought the best one can do is to produce bounds on the treatment effect. But maybe I am wrong.

References:
Kennedy EH. Optimal doubly robust estimation of heterogeneous causal effects. 2020
limitations_and_societal_impact: Yes. The authors listed a few limitations and what can be done in future work and wrote a quite thorough paragraph on the potential negative societal impact of applying causal inference methods in real life.
review_title: null

Meta-review: The proposed method extends the R-learner (Nie and Wager, 2020) to deal with multiple treatments that are graph-structured. Dealing with non-Euclidean treatments is a very important problem that is understudied in the literature. The proposed two-stage algorithm is an extension of the Robinson decomposition that several reviewers found promising (kq9J, rvHB). An important stepping stone of the approach is to generalize the propensity scores to propensity features. Reviewers rvHB and 1AJd criticized the overlap assumption, which is very strong if the treatment is graph-structured. In the discussion, it has been clarified that the overlap assumption is only required for the features maps of X and T. In addition, the authors provided a sketch for a quasi-oracle regret bound. Reviewers kq9J, rvHB and fdib agree that the paper is well-written. Reviewer fdib pointed to some issues in the presentation. Overall, the authors have addressed the main issues in their responses and the theoretical results have been strengthened during the review process.
Decision: Accept (Poster)