Venue: CLeaR 2022 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary: The paper studies estimation of causal effects using a combination of experimental and observational data. The authors propose the integrative R-learner, an extension of the existing R-learner estimator, for the purpose. Under the assumptions of effect transportability and ignorability in the experimental data, this is shown to lead to consistent estimation with asymptotic variance at least as good as a comparable estimator based only on experimental data. The approach is evaluated on synthetic and real-world data and compared with relevant baselines. 
Main review: **Originality:**

The paper approaches a previously explored problem under less restrictive assumptions, with an estimator that generalizes a previously known one. This provides sufficient originality. 

**Significance:**

The result that strict variance improvement over RCT data alone depends on the nature of basis functions for the effect and confounding functions is interesting—but not expanded upon. 

It is slightly disappointing that advantage over RCT-only estimates is studied only in terms of asymptotic variance for a fixed proportion of RCT samples. If anything, this study is justified by the fact that RCT data is rare and observational data is plentiful. Although p(S=1) may be small, I find it hard to see that “n” grows linearly with “m” in practice. Admittedly, the comparison is made with an estimator that is more wasteful—which uses all N samples from an RCT. Nevertheless, it is unclear how the result scales with p(S=1) or for fixed “n” with growing “m”. Surely these questions are of greater practical concern in the settings used for motivation?

The comparison with Kallus et al. (2018) and Nie & Wager (2021) in section 5 is OK, it makes for a more self-contained paper, but does not add much new insight. For example, the instability of the U-learner is noted in existing work. Would it be possible to provide a rate comparison with the U-learner?

In simulation study I, the authors study the finite-sample properties of their estimator. This is a very low-dimensional setting (d=2) where the sample size from the RCT is proportionally quite large. Additionally, the model size is very small as well. The integrative r learner is the clear winner in this setting, yes, but it does not tell us much about where this edge comes from and if/when it is lost. 

**Technical quality:**

Overlap between observational data and RCT is not discussed other than in experiments. However, is it not necessary for objective (5) to be uniquely identified in general? Specifically, if RCT and OS are completely disjoint, it would seem c and tau could be invariant to a mutliplication/division with a  constant on the observational data, since \tau does not have to be consistent with the RCT in these points. 

The authors equate “If all the confounding variables are captured in the OS” with exchangeability w.r.t X. These are not equivalent, e.g., if the OS contains other post-treatment variables than the outcome. 

In Assumption 1, \pi_A(X) is not defined. 

**Clarity:**

The introduction is clear, but perhaps a little too long given its current content. This is not the first work studying a combination of RCT and OS data, and the problem does not need to be justified from scratch—the arguments that RCTs are expensive and OS data is plentiful but can lead to biased estimates have been made many times before. I appreciate that this gives a more self-contained description, but I think the space could be better spent. In particular, I’m missing references (in the introduction) to e.g., Bareinboim & Pearl (2016), Kallus, Puli & Shalit (2018)—especially since it is used heavily later in the paper, Peysakhovich & Lada (2016), Gentzel, Pruthi & Jensen (2021), etc. While these may not address the same exact setting as is studied here, they are more closely relevant than some of the other references in the introduction. In particular, this could help contextualize the contributions of the present work. 

The Setup section clearly defines the problem and the main assumptions. 
Section 3.1 justifies the proposed learning objective well. 

Unless I’m mistaken, Algorithm 1 describes a special case of grid search. I’m not sure an algorithm box is necessary here—although it is certainly clear!

I think the abbreviations NK_linear and NK_lasso could be interpreted as “Nathan Kallus_linear”. Surely, it would be more appropriate to also give credit to the two co-authors of that paper than to list both first and last name of the first author?

The original R-learner studies a non-parametric kernel-based estimator. Here, a series approximation is used. Was there a particular reason for this departure?
review_title: null

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary: This paper proposes a new method, integrative R-learner, for estimating causal effects from both observational and experimental data in the binary contextual bandit setting. Importantly, the observational data is subject to causal confounding, and both the observational and experimental data are subject to covariate shift (different distribution of the context variables). The authors show their method asymptotically converges to an unbiased estimator (under assumptions), at least as fast as the estimator that uses the experimental data.
Main review: ### Originality

The approach proposed in the paper seems novel, as far as I can tell. However, I think a few citations to related works might be missing. How does the approach relate to, e.g.

Zhang and Bareinboim (2017) Transfer learning in multi-armed bandits: A causal approach.

Zhang and Bareinboim (2021) Bounding causal effects on continuous outcomes.

### Significance

This paper addresses an important problem, that of merging observational and experimental data in causal inference, which is very relevant to CLeaR. However, it is also very dense mathematically for a quite limited in scope (binary contextual bandits). And most importantly, I remain unconvinced, from the presented experimental results, of the utility of the method. Thus, I am unsure this paper is likely to have a broad impact.

### Technical quality

The technical quality of the paper is rather poor. The presented approach is backed-up by theoretical justifications, however it is hard to judge the validity of those, or how restrictive the required assumptions are. The paper is very dense and mathematically heavy (although I noted the notation was also a bit sloppy at times), and the proposed approach includes many bells and whistles (different models for different parts of the model, cross-fitting, series approximation, regularization with cross-validation) which highly obfuscate the interpretability of the paper. On the experimental side also, I found the experiments rather limited and potentially flawed (see my detailed comments), and thus not convincing enough to support the claims of the paper.

### Clarity

Although the proposed approach does not seem that complicated and can be summarized in two equations (5 and 6), the mathematical developments in the paper are hard to parse and interpret intuitively. Also, I found the experiments not very clear. For example, I could not clearly understand what the compared methods actually correspond to (e.g., the rct_x and os_x variants).

### Detailed comments

p.3 Ass.1: $c_1,c_2 \in (0, 1)$ -> This notation is confusing to me. Do you mean the interval $[0,1]$, or the set $\\{0,1\\}$ ? This should be clarified.

p.3 Ass.1: $\pi_A(X)$ -> this concept has not been introduced yet. What is $\pi_A(X)$ ? Furthermore, if I assume that this represents the policy used in the observational study, $p(A|X,S=0)$, then this statement is trivial. Since $A$ is binary, we necessarily have that $p(A=a|X,S=0) \in [0,1], \forall a \in \\{0,1\\}$.

p.3 Ass.1: I have a hard time understanding the meaning of (c). I suppose you mean $Y=Y(a), \forall a$ ? Then, this turns your assumption (a) into $A \perp Y \mid (X,S=1)$, which means that the treament has no causal effect on the outcome. Am I missing something ?

p.3 Ass.2: $s=0,1$ -> do you mean $\forall s \in \\{0,1\\}$ ?

p.3 eq.2: $A\\{\dots\\}$ -> this notation is again new. What does $A\\{\dots\\}$ mean ?

p.3: combining the RCT and OS can identify the HTE and confounding function -> the first part of this statement is trivial, since the RCT alone can identify the HTE.

p.3: $Y(a) \perp S \mid X,(a=0,1)$ -> this notation again is rather sloppy. Do you mean $Y(a) \perp S \mid X,A=a, \forall a \in {0,1}$ ?

p.3 Ass.2: $\mathbb{E}\\{...\\}=\tau(X)$ -> notation is inconsistent, should be $\mathbb{E}(...)$.

p.6: the possible smooth or sparsity of $c(X)$ leads to the efficiency gain of the integrative R-learner compared to using the RCT alone -> Do the gains of your method only come from regularization ? And what if the RCT $\tau(X)$, estimated alone from RCT data, is also regularized ? Do you still obtain gains with your method ?

p.6: Thus, it is critical to impose different regularization parameters for $\tau(X)$ and $c(X)$ -> I do not see the implication. Why would these regularizations need to be different ?

p.7 Ass.3: $||\tau(X)||_\infty$ and $||c(X)||_\infty$ have not been defined.

p.7: the integrative $R$-learner has strictly larger efficiency -> what is meant here by efficiency ? This concept has not been properly defined. Do you mean that regularizing $c(X)$ imposes an implicit regularization on $\tau(X)$, which reduces the capacity of the RCT model, thus reducing model variance (at the cost of increasing model bias) ?

p.9: It is unclear to me what the "rct_x" and "os_x" methods are. The methods described in Section 3 and 5 seem to use both observational and experimental data. How are those methods modified to include only experimental (rct) or observational (os) data ?

p.9 Sec.6: the main claim of your paper is "without any prior knowledge of hidden confounding in the OS, the proposed integrative R-learner is consistent and asymptotically at least as efficient as the estimator using only the RCT". To support that claim experimentally, it is crucial that in your experiments, at least in the synthetic ones, you let the number of RCT samples vary from a small to a large number so as to verify that a) both estimators indeed converge to the same thing; and b) your estimator converges faster.

p.9: $U_{rct}$ has not been defined.

p.10 Tab.1: This experiment seems flawed. How is it that the unbiased RCT estimator "rct_rf" performs worse than the confounded one, "os_rf" ? It seems like the combination of RCT and OS data has a much lower impact on the final performance than the choice of the model family used to learn the estimator. I am not sure what "rct_oa", "os_oa", "rct_rlearner" and "os_rlearner" are, but if you want to evaluate the hypothesis that "int_rlearner" brings improvements due to the use of the OS data, then it is crucial that you also evaluate against a model that is learned from the RCT data only, which is adequate and properly regularized (e.g., using cross-validation).

p.10 Tab.1: Another comment, it seems crucial to me to include another important important baseline, which assumes unconfoundedness in the OS data, and estimates the RCT from a naive combination of RCT and OS data. I suspect that such a baseline, with a properly regularized model, could perform very well in this experiment.

p.10: generate population -> generate a population

p.10: slection -> typo

p.11 Tab.2: Here again, how is it that the biased estimators, using OS data only, perform better than the unbiased estimators, which use RCT data only ?

p.11: taking samples with $T=0$ -> what is $T$ here ?

p.11: an unbiased estimate from all samples -> $X$ is missing from this formula. How can this be an unbiased estimate of the RCT ?

p.12: the RMSE of our integrative R-learner increases -> how is it that the best RMSE obtained by your method is at iteration 1? Why event proposing an iterative learning procedure, if your method performs best without doing iterations ?
review_title: null

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: Summary: The current paper builds on the R-learner method developed by Nie & Wager. The Nie & Wager method provides a flexible framework for jointly using a variety of statistical and machine learning methods to predict treatment effects from observational data under the assumption of no unmeasured confounding of the treatment and outcome. The current paper develops the Integrative R-learner method which uses both observational (e.g., EMR) and experimental (e.g., RCT) data on a common set of real-valued covariates to predict treatment effects of a binary treatment on a real-valued outcome, when unmeasured confounders are possible. The method uses the experimental data to reduce bias in the causal predictions, and it uses the observational data to reduce variance, when doing so is possible. An evaluation using simulated and real data support that the method predicts treatment effects better than several other state-of-the-art methods.
Main review: Originality:

* The approach leverages the flexibility of the R-learner to use multiple machine learning methods, while also supporting the integration of observational and experimental data to predict treatment effects.

* The approach uses different regularization terms for the HTE and confounding functions.

* The paper includes in the supplement a proof that the integrative R-learner is consistent and that asymptotically it is at least as efficient as the estimator that uses only experimental data.

Significance:

* The problem of leveraging both observational and experimental data to improve the prediction of treatment effects is quite important in many fields. In healthcare, for example, it could support better predictions of treatment outcomes for individual patients, which can help optimize their care.

* The ability to use a wide range of supervised statistical and machine learning methods to address this problem is important, because it allows current and future methods to be immediately applied to help improve the prediction of treatment effects.

* The paper does not discuss the issue of model interpretability. How feasible would it be to develop an automated explanation capability that describes the factors in the model that are driving its predictions? For the real-data experiments described in the paper, what were the factors driving the predictions?

* To what extent are the assumptions made in the simulation studies in alignment with the assumptions made by the Integrative R-Learner method? For the simulation study, did the authors “stress test” their method by making assumptions that are (1) counter to those made by their Integrative R-Learner method, and (2) compatible with the assumptions made by other methods that were evaluated? The paper does not discuss this issue, and thus, it seems that they did not.  

* The Discussion section is quite brief. It would be useful to include, for instance, some discussion of the limitations of the described approach and directions for future research. For example, how would the method handle a treatment outcome that is naturally represented using a nominal variable (e.g., a binary variable representing in-hospital death)? How would it handle the common situation in which the covariates in an RCT are a small subset of the available observational covariates in the electronic medical record (EMR) system? The paper doesn’t have to solve these problems of course, but some discussion of such issues and open problems would be helpful.

Technical quality:

* The work looks to be of high quality, but I did not carefully check the proofs. 

* In the real experiments, what is the standard error of the predictions shown in Figure 1?

Clarity:

* Overall, the paper is grammatically well written. 

* The paper is written in a style that will be easiest for readers with a strong background in mathematical statistics to understand. Several parts of the paper may be difficult for readers with other backgrounds to understand clearly and completely.
review_title: null

Meta-review: The paper studies an extension of the R-learner to settings where both experimental and observational data are available. The author show that asymptotically the proposed estimator converges to the truth at least as fast as the estimator that uses only experimental data. This is an interesting extension of important work and thus well-suited for CLEAR. However, as some reviewers point out, the paper is hard to read due to use of dense notation and there are some issues with the simulations. It would be great if in the final revision the authors could make an attempt at improving readability, for example by addressing the comments by reviewer 8X2B.
Decision: Accept (Poster)