Regularized DeepIV with Model Selection

Authors: Zihao Li, Hui Lan, Vasilis Syrgkanis, Mengdi Wang, Masatoshi Uehara
Primary area: learning theory
Keywords: Nonparametric estimator, instrumental variables, model selection, causal inference.

Abstract:
In this paper, we study nonparametric estimation of instrumental variable (IV) regressions. While recent advancements in machine learning have introduced flexible methods for IV estimation, they often encounter one or more of the following limitations: (1) restricting the IV regression to be uniquely identified; (2) requiring minimax computation oracle, which is highly unstable in practice; (3) absence of model selection procedure. In this paper, we analyze a Tikhonov-regularized variant of the seminal DeepIV method, called Regularized DeepIV (RDIV) regression, that can converge to the least-norm IV solution, and overcome all three limitations. RDIV consists of two stages: first, we learn the conditional distribution of covariates, and by utilizing the learned distribution,  we learn the estimator by minimizing a Tikhonov-regularized loss function. We further show that RDIV allows model selection procedures that can achieve the oracle rates in the misspecified regime. When extended to an iterative estimator, we prove that RDIV matches the current state-of-the-art convergence rate. Furthermore, we conducted numerical experiments to justify the efficiency of RDIV empirically.  Our results provide the first rigorous guarantees for the empirically well-established DeepIV method,  showcasing the importance of regularization which was absent from the original work.

Decision: reject
Reviewer scores: 5, 5, 6, 5

Reviews and discussion:
<thread>
creation date:
1729973432333

thread_type:
official_review

<thread_object>
creator: reviewer_p9wu

<content>
summary:
This manuscript is basically a technical paper, discussed about two-stage non-parametric IV and model-selection in the second stage when equipped with an additional $L_2$ regularization.

strengths:
* The authors discussed lots of the aspects for non-parametric clearly.

weaknesses:
* I feel the problem studied in this paper is with limited novelty. The transformation between one-stage and two-stage algorithms and analysis is in general only a technical problem and have been discussed in different places like DualIV, and the $L_2$ regularization itself makes the model-selection easier (with the strongly convexity).

questions:
* To convert the MLE guarantee into an $L_2$ guarantee, the authors assumed a minimum density on the conditional density. What’s the benefits/drawbacks compared with the conditional mean embedding based methods (although it also requires some assumptions like HS operators).
</content>
</thread_object>

</thread>
<thread>
creation date:
1730351385486

thread_type:
official_review

<thread_object>
creator: reviewer_hewd

<content>
summary:
The paper studies the nonparametric instrumental variable regression with Tikhonov regularization (RDIV), and proves that RDIV allows model selection procedures and matches the SOTA convergence rate. I agree with the author's claim that this work is the first attempt to provide rigorous guarantees for DeepIV. With Tikhonov regularization, the model selection procedure achieves the oracle rate and iterative RDIV matches the SOTA rate.

strengths:
The paper is well-written, and the results are motivated well. I didn't go through the proofs, but the explanations after each result are insightful, and ease the reading.  The theoretical contribution is solid.

weaknesses:
The numerical experiments are only based on simulated data. It would be better to have some results from real data to demonstrate the strength of the proposal.

questions:
How is Tikhonov regularization related to a function space parametrized by the neural network? It seems not straightforward to relate it to weight decay.

Is there a computational gain when minimax optimization is no longer needed?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730749518797

thread_type:
official_review

<thread_object>
creator: reviewer_q5qb

<content>
summary:
This paper addresses the problem of nonparametric instrumental variable (IV) regression, a framework with wide applications across fields such as causal inference, handling missing data, and reinforcement learning. The objective in IV regression is to solve the conditional moment equation, 𝐸 [ 𝑌 − ℎ ( 𝑋 ) ∣ 𝑍 ] = 0, where 𝑍  serves as the instrument. The authors introduce RDIV, a regularized variant of the DeepIV method, marking the first work to provide rigorous theoretical guarantees for DeepIV. RDIV enhances generalization by incorporating Tikhonov regularization. Methodologically, RDIV follows a two-stage approach. The first stage involves learning the conditional distribution of covariates, while the second stage refines the estimator by minimizing a Tikhonov-regularized loss function.

strengths:
RDIV offers several key advantages over existing methods. It addresses three significant limitations of prior literature: it eliminates the need for unique IV regression identification, avoids reliance on the often unstable minimax computation oracle, and supports model selection procedures.

weaknesses:
It is unclear how the method compares for example to recently developed methods (see arxiv:2405.19463; to appear at NeurIPS 2024) that completely avoids minimax formulations, as well as avoiding the need for two-stage procedures.

questions:
please see above
</content>
</thread_object>

</thread>
<thread>
creation date:
1730812254279

thread_type:
official_review

<thread_object>
creator: reviewer_mfbt

<content>
summary:
This paper studies a two stage procedure for regression in the scenario where the errors are not conditionally independent. They first learn a conditional density to make use of instrumental variables and consequently solve a square loss erm problem weighted by the learned conditional density. They show that this procedure attains mostly standard nonparametric rates.

strengths:
* With the (unfortunate) exception of the introduction, I found the paper mostly well-written and clear.

* The paper studies an interesting problem, proposes a natural solution, and proceeds to analyze said solution. While I am not familiar with the immediately preceding related work (in IV), this seems clean to me.

weaknesses:
* The organization of the paper is hard to follow and the introduction is way too terse. As someone well-versed in nonparametric statistics but not necessarily IV methods, I had to skip ahead to section 4 to really understand what was going on.  Stating that you are trying to solve some fixed point equation in the introduction is not conducive to most people's understanding of the problem you are solving. 

* My overall feeling is that the result is somewhat incremental. To my understanding the main difficulty lies making standard guarantees for MLE in Hellinger^2 compatible with the square loss. I could not entirely follow why this is so challenging and would encourage the authors to further explain why this is the case (for instance, in the very last paragraph of section 1, you mention this difficulty but do not really expand on it, nor do you reference the lemmata which might be useful for understanding this difficulty).

questions:
* is it really fair to say that your algorithm is more computationally tractable when it is based on MLE?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732742292121

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Thank you for your review

comment:
Dear Reviewers,

We sincerely appreciate the time and effort you have dedicated to reviewing our paper. During the initial phase of the rebuttal process, we carefully addressed your concerns and provided an updated version of the manuscript. We would like to reiterate our response in the following:

>Q1. The technical novelty of this paper, and the contribution it made to IV regression

**A1**. (1) Our technique for bounding the $L_1$ norm instead of the $L_2$ norm of $(\mathcal{T} - \hat{\mathcal{T}})(h - \hat{h})$ is essential for the **fast rate result**. Particularly, directly applying the Hellinger distance to $L_2$-norm results in a bound of $||(\mathcal{T} - \hat{\mathcal{T}})(h - \hat{h})||_2 \leq \delta_n^{1/2} \cdot ||h - \hat{h}||_2$, which further results in a slower convergence rate of $|| h_0 - h^*||_2 \leq O(\delta_n^{\frac{\min(\beta,2)}{\min(\beta,2) + 2}})$. Our approach in Lemma 1 and Theorem 5 manages to bound $\hat{\mathcal{T}})(h - \hat{h})||_1 \leq O(\delta_n \cdot || \hat{h} - h ||_2)$, and carefully applying this result to the second stage guarantee results in a rate of $|| h_0 - h^*||_2 \leq O(\delta_n^{\frac{2\min(\beta,2)}{\min(\beta,2) + 2}})$, which is way much faster than the trivial result obtained with $L_2$ norm and comparable to the existing state of the art. 

(2) Our work establishes the **first theoretical guarantee for the popular DeepIV type method** [2], which has been lacking for years, and establishes a statistical convergence rate that **almost matches the SOTA convergence rate** in existing minimax approaches while remaining **computationally tractable**. We also point to a crucial modification that is required for that method to be made rigorously efficient (regularization) from both theoretical and empirical studies. Moreover, we establish a **theoretically grounded model selection** procedure for RDIV in Sec. 7, namely the Best-ERM and the Convex-ERM methods. Although widely discussed in machine learning literature, a provably efficient model selection guarantee has been lacking in existing works such as DeepIV, AGMM [4], and DFIV. 

>Q2. Comparison with TOSG-IVaR [2] and DualIV [3] 

**A2**. In our paper, we studied RDIV under **general function approximation**, and how to perform the model selection procedure, with both guarantees from theoretical and empirical perspectives. In contrast, [2] only studies stochastic optimization for IV regression under **parametric function classes**, including linear and generalized linear function classes, which are less general compared to our setting. Based on such a setting, the optimization scheme proposed in [2] cannot be directly extended to nonparametric function classes. When our function class is restricted to parametric function classes, RDIV allows a convergence rate of $O(1/ \sqrt{n})$, identical to [2].

[3] studies minimax optimization within a **kernel function class**. However, their method (i) provides no convergence guarantee, (ii) suffers computational intractability when transferred to the general function class due to the absence of a closed-form solution for the dual problem, and (iii) does not provide a theoretically grounded model selection procedure. Notably, [3] is a special case of the AGMM method [4], the latter studies minimax methods under general function approximation, and is discussed in detail in our related work section. 

>Q3. Computational efficiency for MLE 

**A3**. We believe that the application of MLE does not hurt the efficiency of our method. In our study, we have already conducted experiments on low-dimensional and median-dimensional data, in which MLE is naturally tractable by function approximation such as Gaussian mixture models. For more specific high-dimensional data such as images, since our method only requires sampling from $P(x|z)$,  Step 1 could be easily conducted by Flow-based models [5] or diffusion models [6]. 

As we approach the midpoint of the rebuttal period, we are eager to hear your feedback on the revised paper. We are committed to addressing any remaining questions or concerns you may have and further improving the work based on your valuable insights.

Thank you again for your thoughtful and constructive feedback.

[1] Deep IV: A Flexible Approach for Counterfactual Prediction, Jason Hartford, Greg Lewis, Kevin Leyton-Brown, Matt Taddy. ICML 2017

[2] Stochastic Optimization Algorithms for Instrumental Variable Regression with Streaming Data, Xuxing Chen, Abhishek Roy, Yifan Hu, Krishnakumar Balasubramanian

[3] Dual Instrumental Variable Regression, Krikamol Muandet, Arash Mehrjou, Si Kai Lee, Anant Raj

[4] Minimax Estimation of Conditional Moment Models, Nishanth Dikkala and Greg Lewis and Lester Mackey and Vasilis Syrgkanis. NeurIPS 2020

[5] Danilo Jimenez Rezende, and Shakir Mohamed. “Variational inference with normalizing flows.” ICML 2015.

[6] Denoising Diffusion Probabilistic Models, Jonathan Ho, Ajay Jain, Pieter Abbeel.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This study represents a solid and good contribution. However, the novelty and usefulness of the approach have not been fully appreciated, at least by the reviewers. The paper's improvements in norm selection and error evaluation are not presented in a form that is easily accepted by the majority of the machine learning community. The lack of experiments with real and large data sets is also noted as a problem. Since this is solid and good research, it would be necessary to make it more appealing to the community.