Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors propose a novel method for conducting instrumental variable regression by learning the ratio of conditional and unconditional densities, and using this estimate when learning the causal relationship between cause and effect variables. The authors present error analyses for their estimation tasks, as well as empirical comparison to previous works.
main_review: Instrumental variable regression is an important problem to tackle for a number of application fields. Therefore an improvement of the methodology in this area should be welcome for the conference audience. 

The paper presents a method that is novel in terms of the estimand it proposes, the density ratio, in order to learn the structural function. Their method can be used with various models and objective functions in learning the causal function. The authors present error analyses for the density ratio and the causal function, as well as experiment results comparing their work to recent algorithms in the field. These constitute the strengths of the present work. 

One of the weaknesses of the present work is in its presentation. Which shortcomings of the original methods are addressed by the present work, and how, is never clearly presented. The lack of description regarding this issue leads the reader to look for a clear performance increase from the present work. 

This leads me to the second weakness of the paper, which is that the experiments do not demonstrate a unanimous advantage for a specific method proposed by the authors. The (potential) advantages of the paper could be more accurately depicted by extending the experiments, e.g. as in Xu et al. (2021).

Though I recommend the acceptance of the paper, I think that the paper would benefit from the aforementioned issues being addressed.
summary_of_the_review: The authors present a novel method for instrumental variable regression that promises to be useful in various applications. However the motivation for the method and its empirical testing should be improved.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper presents a method for estimating causal effects (or more generally non/parametric functions) under conditional moment restrictions, focusing in this case on nonparametric IVs. The main idea is casting these conditional restrictions to an unconditional version through importance resampling using a conditional density estimator (e.g. based on a least-squares method with a NN). The paper also provides a theoretical analysis of the estimation error, providing am error bound (Thm1). The method is evaluated based on three econometrics datasets from literature, showing a smaller MSE than the compared methods.
main_review: Improving non-parametric instrumental variables methods is an important problem in causal effect estimation. Based on my limited knowledge, I think this paper is providing a very interesting and promising result, and I do appreciate the theoretical analysis of the estimation error. 

My only concern is that it is a bit technical and hard to follow for the general ICLR community, so I would suggest the authors to try to make it a bit more accessible and readable and therefore even more impactful.

In particular, as a minor point, I think the abstract is a bit too vague and would benefit from a more precise description of what the paper is about.
summary_of_the_review: The paper seems to provide a novel, interesting and applicable method that improves the accuracy of an important causal inference tool (instrumental variables).
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The present paper proposes an importance weighting approach to address the issue of regression under conditional moment restrictions in the context of non-parametric instrumental variables. The authors provide error bounds on the learned structural function. They show that their approach has a convergence rate of $\mathcal{O}(1/\sqrt{n})$.
The main contributions of this work are presented in Sections 4 and 5, where they first introduce their method based on re-weighting by an estimated conditional density ratio function and then provide an estimation error analysis.
The theoretical results are complemented with two synthetic experiments which show that the proposed method can compete with and in certain cases improves upon state of the art.
main_review: ## Strong points: 
- The approach of using importance weighting to overcome the difficulty of learning under conditional moment restrictions is an elegant solution to solve the presented NPIV regression problem.
- The article is well written and the balance between motivation, theory and experiments allows to get a good understanding of the proposal.
- The experiments are well described and motivated. I appreciate the honest assessment of the relevance of the chosen settings for the proposed method.

## Issues/Points that require clarification:
- In Section 3.1, there should be references given for the cited methods (general reference for IV methods, for two-step GMM, CU estimator, etc.)
- The assumptions part is sometimes vague and only phrases certain assumptions without formalizing them. To list a few examples:
  - In Section 2, do you assume i.i.d. samples? If so, please add this when introducing the available dataset and if not, please specify why you don't need i.i.d. samples.
  - In Section 4, the definition of the conditional density ratio function $r^*$ implicitly assumes that $p(y,x)\neq0$ for all $(x,y)\in\mathcal{X}\times \mathbb{R}$. This should be made explicit, and also be discussed w.r.t. the density ratio estimator $\hat{r}$ and maybe also w.r.t. the motivating example. (Your Assumption 2 in Section 5 excludes this case of $p(y,x)=0$ but this should maybe be stated already at the moment where you define $r^*$.)
  - In Section 5, you introduce Assumptions 1 and 2 with the remark "For ease of discussion". Does that mean that these are not necessary?
- Since the authors suggest that their approach should be relevant for learning causal relationships in the context of computer vision and NLP tasks (Section 4.3 and Section 6.2), the claimed added value of their proposal in practice would be more convincing if they could provide an example from either of these contexts where such an IV approach would be suitable.
- At the end of Section 4.3, the authors conjecture that their proposed heuristic works well when $f^*$ is close to $\mathbb{E}[Y_i|X_i]$. If I understood correctly, this means that in the case where $\mathbb{E}[\varepsilon|X_i]\approx 0$ the heuristic should work, but from what I understood this approach is motivated by cases where the assumption $\mathbb{E}[\varepsilon|X_i]= 0$ is clearly violated, motivating the use of IVs. Is that correct or did I misunderstand the heuristic?  
- In Lemma 2, what is the role of $T$? From Appendix A.3, it is not clear what $T$ represents. In the proof of Lemma 2, it is written ``_[...] we consider $T$ i.i.d. random variables $W_i=(X_i,Y_i), i=1,\dots,n$ [...]''. What is $T$ in relation to $n$ in this definition? Is it the same $T$ as in Section 5.3?
- Since the authors provide a theoretical bound on the MSE of $f^*$, it would be interesting and a valuable addition to empirically assess this bound.
- Did the authors also assess the performance of the classical linear 2SLS estimator in both experiments (Sections 6.1 and 6.2)? It would be interesting to see the impact of the nonlinear form of the structure function $f^*$ on the performance of this classical estimator to compare with the neural network based approaches.

### Minor comments (that did not impact the score):
- p.1: when learning causal (structural) relationships _are_ $>>$ when learning causal (structural) relationships _is_
- p.1: please add a (standard) reference such as Wooldridge (2010) when you first mention the instrumental variable approach (second paragraph of the introduction). 
- p.1: we focus on _on_ nonparametric IV $>>$ we focus on nonparametric IV 
- p.2, Section 3.1: the parameter $\theta$ of the linear model: shouldn't it rather be $\theta^*$?
- p.2, Section 3.1: if there are IVs of dimension $d_X$ or more that _satisfies_ $>>$ that _satisfy_
- p.2, Section 3.1: $\tilde{X}=...$: shouldn't it rather be $\hat{X}$?
- Please try to avoid alternating between past and present tense when referring to other works (e.g., p. 3 ``Newey \& Powell (2003) proposed [...] They first define [...]''. I would suggest using only present tense.
- p. 3, Section 3.3: approaches recently _propose_ $>>$ approaches recently _proposed_
- p.5, Section 4.3: as a recent finding, such as Bartlett et al. (2020) in linear regression $>>$ as a recent finding suggests in the case of linear regression (Bartlett et al., 2020).
- p. 6, Section 5: the distribution of $(Y_i, X_i, Z_i$ by $O$ $>>$ the distribution of $(Y_i, X_i, Z_i)$ by $O$

##########################################################################
### Post-rebuttal update

I thank the authors for their detailed, pertinent and timely responses. The reformulated assumptions and other paragraphs, the changed title and the requested additional experimental results have addressed my concerns. I have therefore decided to increase my rating and to recommend this paper to be accepted. 

##########################################################################

### References:
[1] Guido W. Imbens and Donald B. Rubin. _Causal inference in statistics, social, and biomedical sciences_. Cambridge University Press, 2015.

[2] Judea Pearl. _Causality_. Cambridge university press, 2009.

[3] Jeffrey Wooldridge. _Econometric Analysis of Cross Section and Panel Data, 2nd edition_. MIT Press, 2010.

summary_of_the_review: In summary, I am rather convinced that the contribution of this paper is important and its theoretical as well as empirical findings are well described and motivated. However the authors should try to provide relevant applications or settings for their method that would indeed benefit from this novelty (compared to existing works). The experiments do not support the authors' claim that their method outperforms other competitors. Also, they should maybe clarify the paper's content in the title by specifying the context of IVs instead of ``learning causal relationships''. Indeed, from the title one expects to find references to the classical causal inference frameworks (either Imbens \& Rubin (2015) or Pearl (2009)) and specification of causal estimands. 
I will read the rebuttal carefully and am willing to increase the score if the authors address the raised concerns.
review_title: null

Meta-review: This paper proposes a method that uses conditional moment restriction methods to estimate causal parameters in non-parametric instrumental variable settings.  This is done by converting to an unconditional moment restriction setting common in the econometrics causal inference literature.

The paper was reviewed quite favorably by reviewers, and the authors updated the manuscript to address specific issues raised by reviewers.
Decision: Accept (Spotlight)