Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper addresses the problem of heterogeneous transfer learning for CATE estimation by using representation learning and a multi-task architecture to transfer information between potential outcome functions across domains, generalizing several existing CATE estimators to the transfer learning perspective.
strengths_and_weaknesses: Strengths:
- The paper deals with an important problem that is understudied. I appreciate the motivation to study multitask architectures for CATE transfer learning when CATE has often been examined in isolated examples which real-world medical data provide compelling motivation for multitask and transfer learning settings.
- The paper effectively generalizes several CATE frameworks to the transfer learning setting which.
- The empirical results are sufficient and effectively show individual effects (e.g. the information sharing and the selection bias in Figure 6 and Figure 7), although not extensive experiments.

Weaknesses:
- The framework deals with fixed datasets with source and target domains. How does this framework extend to more complicated settings?  For example, what if we have multiple domains, the datasets are streaming, or the domains are unknown at training time? What about personalized feature spaces (e.g. different features are available for different samples)? Of course, I don’t expect all of these situations to be addressed in this manuscript but discussions about extensions would be helpful.

questions: See weaknesses above.
limitations: Limitations and ethics are addressed appropriately. Some of the limitations and directions for future work discussed in the conclusion could also be mentioned more explicitly throughout the paper to make tradeoff decisions clear when introducing the framework.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This work aims to solve the heterogeneous transfer learning problem for CATE estimation by introducing several building blocks that use representation learning to handle the heterogeneous feature spaces and a flexible multi-task architecture with shared and private layers to transfer information between potential outcome functions across domains. Besides, they propose several building blocks to construct HTCE-learner, similar to the most common CATE learners. 
strengths_and_weaknesses: Strengths:
1.  The paper is overall well written and it clearly defines the problem.
2.  These building blocks involve handling the heterogeneous feature spaces, sharing information between PO functions across domains and sharing information between PO functions within a single domain.

Weaknesses:
1. the model design is lack of innovation and major idea is based on the meta-learner.
2. limited baseline models.


questions: limited baseline models
limitations: To the best of my knowledge, the work is not as novel as the authors claim it is. Please refer to the following works:
* Pearl and Bareinboim, 2011, Transportability of Causal and Statistical Relations, https://ftp.cs.ucla.edu/pub/stat_ser/r372-a.pdf
* Bareinboim and Pearl, 2016, Causal inference and the data-fusion problem, https://ftp.cs.ucla.edu/pub/stat_ser/r450-reprint.pdf
* Magliacane et al, 2017, Causal Transfer Learning, https://www.semanticscholar.org/paper/Causal-Transfer-Learning-Magliacane-Ommen/b650e5d14213a4d467da7245b4ccb520a0da0312
* Mooij et al., 2016, Joint Causal Inference from Multiple Contexts, https://arxiv.org/abs/1611.10351
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes a Heterogeneous Transfer Causal Effect (HTCE) framework to improve treatment effect estimations on the target dataset under the heterogeneous transfer learning problem.
strengths_and_weaknesses: Originality: 3. This problem has already been noticed by some works. The basic structure of HTCE is actually the combination of layer sharing + FlexTENet.

Quality: 4. I think it is good when authors combine HTCE structure with meta-learners and TARNet. They are explicit.

Clarity: 5. This paper is well written. Section 4 is especially easy to follow. Good!

Significance: 3. I think the argument and setting of estimating TEs for heterogeneous transfer learning is meaningful, but I don't think the experiment parts show the significance of the proposed method aiming to this problem.
questions: 1. In lines 110-111, you mentioned that [32] do not assume access to label information. But in my view, I think that should be the true definition of "transfer learning". In general, your problem is similar to shared machine learning.

2. In lines 230-231, do you mean $||$ instead of $+$?

3. Your figures 3.a and 3.b are inconsistent with the text. It is just a small mistake.

4. I think the most interesting parts are i) solving the problem when the target dataset is substantially different from the source dataset, especially when the target dataset is in a small sample size; ii) combining HTCE with different learners. But your experiments do not show the importance and significance. For both your simulated and benchmark datasets, the distributions of source and target datasets are very similar. Specifically, the DGP or the empirical distribution is fixed, and the only difference is that you split them into source and target. It is undoubted that involving shared information will give better results than losing shared information. That is inconsistent with your claim in lines 149-152. I suggest you show how small the sample size of the target dataset is, and how big the differences between the distributions of source and target datasets are before reporting your experimental results.

5. In your Appendix, there are some typos, e.g, there exists "size size". Also, is twins 11400 or 114000?
limitations: The limitations are well dicusessed.
review_title: null

Meta-review: metareview: The paper studies methods for estimating conditional average treatment effects (CATE) under a shift in domain where source and target feature spaces are heterogenous. It is assumed that the (respective) CATEs in both source and target domains are identifiable through ignorability and overlap. No formal assumptions are made regarding the similarity of potential outcome distributions across domains, but implicitly that there exists a shared structure in the outcome functions. A number of heuristics are proposed to modify popular neural network CATE estimators to this setting, including a wide array of meta-learners such as propensity weighting, doubly robust estimators and TARNet. 

Reviewers appreciated the setting of heterogenous feature domain adaptation which is understudied in the literature and representative of many transfer tasks of interest, such as transfer from a clinical trial to an observational cohort. Typically, the feature set collected in trials is smaller than in, say, a registry. However, as pointed out by one reviewer, the empirical evaluation does not consider such applications. In addition, no details are given in the main paper for how the heterogenous feature spaces are constructed for experiments (this is only given in the Appendix). The uniform sampling is quite unrealistic and most likely less challenging than real-world cases. 

The authors make assumptions of ignorability and overlap, referring to previous work that this renders the causal effect identifiable. While this is true, the interesting complication in this work is that no assumptions are made regarding similarities of feature sets or outcome functions; these are left implicit. As a result, no claims can be made about the usefulness of source data for this task, see e.g., [1] for a discussion on hardness of transfer. In other words, the authors rely on empirical evidence to demonstrate this usefulness. In semi-synthetic experiments, the authors find that their proposed approach improves significantly over using only shared features, even when the number of target samples is minimal. 

Reviewers were concern with the contextualisation of the work in the literature, given previous work on transportability of causal effects and on domain adaptation. Adding to this list, I would suggest that the authors refer to previous work on heterogenous-feature transfer learning. Under ignorability and overlap, the settings are not much different from each other, not least demonstrated by the fact that the T-learner solution performs well. 

The authors propose several "building blocks" but don't evaluate the importance of these in isolation, using, for example, an ablation study. This makes it difficult to assess which components are necessary and which are not.

In summary, the considered setting is interesting and the algorithmic contributions appear useful empirically. The theoretical and methodological contributions are rather small, and the work should be better contextualised in the related topics of domain adaptation and transportability.

[1] Ben-David, Shai, and Ruth Urner. "On the hardness of domain adaptation and the utility of unlabeled target samples." International Conference on Algorithmic Learning Theory. Springer, Berlin, Heidelberg, 2012.
confidence: Less certain
Decision: Accept