From: ICLR.cc/2023/Conference/Program_Chairs
title: Paper Decision
decision: Reject
metareview:_summary,_strengths_and_weaknesses: Since this was a borderline paper, I have carefully read the paper myself. I agree with some of the major concerns of the reviewers, even after reading the author responses. 

I do think that the contribution is incremental in terms of novelty. The authors essentially introduce an extra divergence distance measure on top of the wasserstein distance, in the same spirit as DCGans, with a discrimination loss, that amounts to a propensity estimator. 

The theoretical result seems to be technically straightforward and follow similar (of course not identical) analysis to the proof approach in Shalit et al, plus extra basic properties of the H-divergence. 

The final method proposed is an ad-hoc combination of the two distance measures and the experimental performance is only marginally better than the prior best method (the MBRL method). The real world evaluation is only done on IHDP which is a very over-analyzed data set and hence not convincing that the same marginally better performance would stand other data sets qualitatively different in nature. For instance, I would advice the authors to try more extensive ITE benchmarks, such as the semi-synthetic datasets offered by the RealCause package.
justification_for_why_not_higher_score: The method and results are incremental.
justification_for_why_not_lower_score: N/A

From: ICLR.cc/2023/Conference/Paper4353/Reviewer_JQVc
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: This work seeks to improve upon representation learning modeling for inferring individualized treatment effects which employs a discrepancy constraint between the representations of the treatment and control groups. The authors propose to use H-divergences to achieve this. This is motivated by a bound on the ITE error by an H-divergence. Empirical results show gains relative to prior art. 
strength_and_weaknesses: The strengths of this paper are in the relative simplicity and efficacy of the proposed approach. The use of H-divergences is both natural and well motivated with theory. I thought the authors did a nice job of presenting both the motivation and solution as well. I think the empirical results are also compelling. 

In terms of weakness, the main issue for me is a relative lack of novelty, though I don't think that is enough to vote for rejection at this time. 
clarity,_quality,_novelty_and_reproducibility: I found the paper to be well written and organized. As I mention above this paper follows prior work fairly directly, but improves upon prior art so I think it is still a valuable contribution.
summary_of_the_review: As I mention above, I think this is a nice, simple idea with good execution. The authors do a nice job of explaining the relative merits of using an H-divergence and provide compelling empirical evidence for their claims as well. While this work may border on incremental in terms of novelty, I think it is still a valuable addition to the literature. 
correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold

From: ICLR.cc/2023/Conference/Paper4353/Reviewer_ncJx
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: The paper proposes a framework that accounts for observed confounding in treatment effect estimation via (i) balanced representation learning of treatment groups and (ii) adversarial propensity score learning. Experimental results on synthetic and semi-synthetic data sets demonstrate improved treatment effect estimation.
strength_and_weaknesses: **Strengths**
- The paper is well-written and easy to follow
- The proposed approach seems easy to implement
- Leveraging both propensity score and balanced representation in a unified framework is interesting

**Weakness**

*Novelty*
- The paper seems to be a straightforward combination of DANN (Ganin et al., 2016) and CFR (Shalit et al., 2017)
- The paper should highlight how the proposed approach differs from related works 

*Unconvincing Experiments*
- It may be challenging to optimize the four objectives, which also include adversarial learning
- The paper should also compare with similar approaches incorporating balanced representation and propensity score learning, *e.g.*, Zhang et al., 2020; Assaad et al., 2021; Huang et al., 2022a;  Hassanpour & Greiner, 2019b; *etc.*
- How are the $\alpha$ weights selected?
- The paper should provide an expanded discussion  on the chosen baselines and Table 3 results
clarity,_quality,_novelty_and_reproducibility: See above.
summary_of_the_review: The paper combines previously proposed DANN and  CFR in a unified framework. However, the performance gain of the proposed approach seems marginal when compared to the  DANN framework alone (DINet). Additionally, several baselines are missing from the experimental evaluation.

**Post Author (s) Response**
- The author (s) addressed most of my concerns, and I have adjusted my score accordingly.

correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold

From: ICLR.cc/2023/Conference/Paper4353/Reviewer_eA2v
confidence: 2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: This paper proposes a framework for treatment effect estimation that consists of two components. The first is to balance representations by minimizing group distance and maximizing individual propensity confusion (i.e., PDIG). The second is to use both pre-balancing and post-balancing representations for outcome prediction (i.e., PPBR). 
strength_and_weaknesses: Strength: 

The proposed method is supported by two bounds of the expected estimation error of heterogeneous treatment effects. The first bound of treatment effect estimation error uses the Wasserstein distance between treated and control groups. The second bound of treatment effect estimation error uses H-divergence as a measure of propensity confusion.

Weakness:

The connection between the two components (PDIG and PPBR) could have been better elaborated. For example, how is PDIG helpful for PPBR? Furthermore, it would be nice to provide an example of why it is important to account for pre-balancing representations for outcome prediction. A simple numerical example could be useful to understand intuition. 

clarity,_quality,_novelty_and_reproducibility: The paper is generally well-written, and the contributions are clearly stated. The proposed method seems to be a useful treatment effect approach. The simulation results look promising. 
summary_of_the_review: This paper proposes a method for treatment effect estimation. This method balances representations, and uses both pre-balancing and post-balancing representations for outcome prediction. This method is demonstrated to have good performance in simulations. 
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 6: marginally above the acceptance threshold