Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper discusses the Gateaux derivative for a parameter functional of a probability distribution. The authors propose a method for approximating the Gateaux derivative empirically, with applications to causal inference. The authors also suggest dynamic treatment regimes as an interesting application.
strengths_and_weaknesses: First, I could not fully understand this study. In particular, I would like the authors to elaborate on the motivation more. Therefore, I would like to make a final evaluation after a discussion with the authors.

I think that the authors' proposed method for approximating the Gateaux derivative is interesting. As the authors mention, the Gateaux derivative and the influence function play an important role in statistical inference. For example, in (semiparametric) counterfactual mean estimation, we usually analytically derive the (semiparametric) influence function and use it as a lower bound of the asymptotic variance; then, we propose an estimator that achieves the lower bound. However, I could not understand what it means to approximate the Gateaux derivative empirically, and I would appreciate a careful explanation.

The influence function is helpful in constructing the estimator, but I do not think it is necessary to be concerned with its estimation as long as it can be analytically derived. For example, Hirano, Imbens, and Ridder (2003) propose an IPW-based estimator that achieves the lower bound derived from Hahn (1998)'s (AIPW-style) influence function. Of course, defining the influence function for a broad class of estimators is an important issue, and several previous studies have attempted it. However, once the Gateaux derivative is defined, it is not clear to what extent its empirical approximation needs to be discussed. At least, it does not seem to be necessary for counterfactual mean estimation and many OPE tasks.

In summary, while I acknowledge the contribution of the method, I would like to know more about the motivation.
questions: Following on from the above question, I have the following minor question.
- Is it possible to apply the proposed method to robust statistics, such as Koh and Liang (2017)?
- I think one of the parentheses in Lemma 1 is not closed.
- What estimator (method) is used to estimate the nuisance parameter in the counterfactual mean estimation in Section 4.1?
limitations: See the above comments.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: Gateaux derivatives arise in a number of statistical estimation problems, and are used to improve asymptotic efficiency in estimation (e.g., in one-step estimators).  For a given problem, the form of the Gateaux derivative is typically derived analytically, and the relevant components then estimated from data (e.g., certain conditional expectations).  Here, the form of the derivative is known, and typical analysis focuses on how different rates of estimation (in the relevant nuisance parameters) lead to appropriate rates of convergence in the original problem.

This paper considers "empirical" Gateaux derivatives, which do not involve such analytic derivations, but instead directly seek to estimate the derivative via finite-differencing.  This introduces an additional source of error in computing the Gateaux derivatives (numerical approximation error), but has the potential to broader the utility of one-step estimators, by not requiring explicit derivations in the case of new statistical functionals.

While other work has considered the problem of numerical approximation error, this work analyzes how both numerical and statistical errors interact in the context of several examples: One of the main results, for instance, gives rates on numerical approximation hyperparameters that preserve parametric rates in the one-step estimator for a standard causal inference problem (counterfactual mean estimation).  Two other illustrative estimation problems are considered as well.

strengths_and_weaknesses: [**REBUTTAL UPDATE**: See discussion below.  My main concerns were addressed during the response, and I am convinced that the relevant clarifications are straightforward to incorporate into a camera-ready, if accepted.  My score has been updated as follows:
* Presentation: 2 -> 3
* Overall Score: 5 (borderline accept) -> 7 (accept)
]

### Strengths
This paper tackles an interesting and highly relevant problem, and provides novel technical results. Theorem 1 in particular is a clean and informative result, characterizing rates of perturbation and smoothing that preserve $\sqrt{n}$-consistency of counterfactual mean estimation with empirical Gateaux derivatives.  Overall, one of the main take-away messages is that the structure of the statistical problem can lead to weaker approximation requirements (e.g., rates on $(\epsilon, \lambda)$) in some cases, but not in others, which I thought was an interesting point.

### Weaknesses

The clarity and quality of presentation could be improved, which is a weakness that is hopefully straightforward to fix.  Below, I will detail some points that were particularly unclear to me, in the hope that it is useful for the authors as they craft a response. I put explicit questions in "bullet points", with the remaining text as context.  I am willing to increase my score if these points can be sufficiently clarified.

**(1) Unclear significance of analysis of required rates for $(\epsilon, \lambda)$**

As I understand it, the main goal of the analysis is to understand the extra error incurred by using a finite-differencing approach, above and beyond using analytic derivatives:  As in Theorem 1, if this error decays fast enough, then one can use empirical derivatives while preserving $O_p(n^{-1/2})$ rates of estimation.  This seems like a very clear type of result, but for an application where the analytic derivative is already well known.  Meanwhile, the results in the remaining sections do not go "all the way" to a result like Theorem 1, but instead stop at giving some side-by-side comparison of the analytic and empirical derivatives.

It seems that there are two conclusions the reader is supposed to draw from this analysis, in the context of the remainder of the work:  First, that $(\epsilon, \lambda)$ can decay slower than we might generically "expect", for problems with special structure. Second, that this special structure is present in the dynamic treatment regime (DTR) functional, but not in the policy optimization functional.  This second point is implied to be surprising, because all three problems exhibit some double-robustness structure (see lines 279-280).

I had trouble drawing such conclusions, though I suspect this is mostly an issue of presentation / clarity.

(1a) First, it seems in several places that the reader should have a "baseline" result in mind, to contrast with the results presented here, but this baseline was not entirely clear.  A few examples:
+ Line 188: "can be a slower rate than implied by the generic analysis of finite differences".  What kind of rate would that be, and is there a reference for such results?
+ Lines 190-191: "potential improvement...could be on the order of generic rate improvements implied by a central difference scheme".  What is this referring to?
+ Lines 278-280: "does not appear that rate-double robustness would admit weaker numerical requirements on $\epsilon$".  Weaker requirements than what?  The "generic analysis" referenced above?
+ Are there "conservative" rates on $(\epsilon, \lambda)$ that will always preserve $O_P(n^{-1/2})$ rates of estimation, obtainable via some generic analysis?

The first three questions are about contextualizing the results, but the last one this is important for clarifying whether (a) there is always a generic approach to derive rates on $(\epsilon, \lambda)$ that preserve $O_P(n^{-1/2})$ convergence, and this is just an improved analysis for specific estimands that shows slower rates are possible, or (b) it is generally necessary to do a "Theorem 1-style" analysis to verify $O_P(n^{-1/2})$ convergence.  The latter conclusion seems much more restrictive than the former.

(1b) Second, conclusions are often made by comparing the form of the empirical and analytical derivatives directly, but these were somewhat difficult to follow:
* Proposition 2 (DTR) "verifies that the requirements...are similar in $\epsilon$ as in the case of a single-timestep", but there is no $O(\epsilon^2)$ term in either Proposition 1 or Corollary 1.  Could you clarify what is meant here?
* Propositions 3 & 4 differ not only in an additive term, but also in the usage of perturbed nuisances, which makes them difficult to compare directly (this also applies to Corollary 1, as noted on line 170).  Is there a reason why a direct comparison (e.g., isolating only an additive difference) is unnecessary here?

**(2) Unclear significance of limitations of Empirical Gateaux derivatives**: 

As outlined in the introduction, constructive / algorithmic approaches to bias adjustment are very appealing, particularly for problems where small changes require re-derivation of the analytic derivative. This paper strikes an appropriate note of humility in the conclusion, giving limitations of Empirical Gateaux derivatives as a "completely general approach" (lines 329-335), namely the fact that (a) pathwise differentiability and (b) the second-order nature of the remainder must be verified analytically. However, these limitations do seem to undercut the general value of the approach. With that in mind, a few relevant questions
* Are there existing scenarios where the analytic form of the gateaux derivative is non-obvious, but where these conditions (pathwise differentiability, second-order remainder) can nonetheless be verified to hold? Or does verifying these conditions *always* require derivation of the analytic form?
* More broadly, are there scenarios where we can apply this approach (with appropriately conservative rates on $(\epsilon, \lambda)$) and have confidence in achieving $O_P(n^{-1/2})$ rates, **without** deriving the analytic derivative?  E.g., would the constrained MDP with arbitrary linear constraints be such an example?

If there exist some space of problems where the answer to these questions is "yes", then it would go a long way towards mitigating the impact of these limitations.

**Comments on soundness**: Regarding technical soundness, I have a (hopefully minor) question or two on Lemma 1
* In the display following line 563, it is claimed that the following holds due to Cauchy-Schwarz.  I'm not sure I see the application of CS here: Is there another reason why we would expect the cross term $2 E[(\tilde{\mu}_{\epsilon}(X) - \tilde{\mu}(X))(\tilde{\mu}(X) - \mu(X))]$ to be non-positive?

$$ E(\tilde{\mu}\_{\epsilon(X)} - \mu(X))^2 \leq E(\tilde{\mu}_{\epsilon}(X) - \tilde{\mu}(X))^2 + E(\tilde{\mu}(X) - \mu(X))^2 $$
* In the display following line 562, the last inequality seems like a non-trivial jump, would you mind walking through the logic explicitly?

Otherwise, the proofs seem correct to me.  Note that I only read through the proofs for Section 3 in depth, and only skimmed the proofs of other relevant results (e.g., Propositions 3 and 4).  While I am well-versed in the causal inference literature, I am not otherwise an expert on non-parametric / semi-parametric statistics, so I may have missed something. 

As an aside, it may be helpful to include a citation in the proof for some of the assumed results regarding kernels. [58] is referenced in the main text, referring to kernel smoothing more broadly, but it seems some of the prerequisite results could be cited more precisely (e.g., Lemma 25.1 of [58] appears to be a relevant result)

### Other Minor Feedback

I consider the following points to be minor feedback re: presentation / notation / possible typos, and they did not meaningfully influence my score, and they **do not require an explicit response from the authors** (some are stated as questions only because I am unsure if they are typos).

**Suggestions on clarity**:
* (Lines 15-19) This and some other sentences are a bit long and difficult to parse, and could perhaps be split into multiple sentences.
* (Line 71) Is the introduction of projections onto the semi-parametric model necessary, given Remark 1's statement that this work focuses on nonparametric models?  CLvdL (Equation 2.2) seems to refers to Luedtke, Carone, and van der Laan (2015) as a reference for the equation on between lines 71-72 holding generally in a non-parametric model.

**Other typos / inconsistencies**
* Example 1 uses $\mathbb{E}_P$ for the outer expectation, but not for the inner expectation.
* Proposition 1 uses $\tilde{\mathbb{E}}_{\tilde{P}_\epsilon}$ in one place, perhaps the tilde on $\mathbb{E}$ was not intended?
* Line 65, what is the observation $\tilde{o}$?  This is not referenced anywhere, I assume this is meant to be $o'$.
* Footnote 1, should the kernel be $\lambda^{-d} K(u / \lambda)$ instead of $h^{-d} K(u / \lambda)$?  There is also a reference to an $O(h^J)$ error term on line 568 that should perhaps be $O(\lambda^{\beta})$?
* Algorithm 1: Should it be $\tilde{P}$ on lines 3-4?  Should it likewise be $P_{\epsilon, \lambda}^i$ instead of $P_{\epsilon}^i$?
* Lemma 1: Should $\tilde{e}\_{\epsilon}(X) = \tilde{p}\_{\epsilon}(A = 1, X) / \tilde{p}(X)$ instead of the current formulation, which uses $\tilde{p}_{\epsilon}(A = 1 \mid X)$ in the numerator?  This would also make it consistent with usage in the proof (see e.g., line 561)
* Line 154, there seems to be a missing parenthesis
* Line 149 "analyses of from kernel density estimation"
* Assumption 1 (iv), should the equation refer to $\tilde{\mu}_{\epsilon}$ or just $\tilde{\mu}$?  Additionally, should the product-rate condition be $o_p(n^{-1})$ as written or $o_p(n^{-1/2})$?
* Line 561, says to bound perturbed $e$ one should "argue similarly", seemingly in reference to the (later) bound on the perturbed $\mu$, perhaps the order was swapped.
* Line 562, following equation, third equality, missing a square on the final $\tilde{p}(A=1,x)$ term.
* Line 572, $E[\Gamma(O; e_{\epsilon}, \mu)]$ should be $E[\Gamma(O; \tilde{e}_{\epsilon}, \tilde{\mu})]$ 
* Line 637, indicator is missing an $\epsilon$ on the left-hand side
* Supplement, Section D.2, the reference is to citation [20], but I believe this should be citation [18]

**Undefined notation**:
* Line 66, I did not see a definition of the function $g(u)$.
* I'm unsure if the dimension $d$ was precisely defined prior to usage in Lemma 1, though it is fairly obvious from context.
* Eq. 7: I'm not sure if $\mu_a$ is defined anywhere, aside from being the optimization variable, and similar for $\mu^*(s, a)$ in Equation 8.
* $\nu$ is used a few times in the proofs without being defined (end of equation starting on 576, end of equation starting on 562), presumably referring to a strong-overlap constant.

questions: My questions can be found in the "Strengths and Weaknesses" section, where every bullet point corresponds to a question.  I collect them (verbatim) below.

* Line 188: "can be a slower rate than implied by the generic analysis of finite differences".  What kind of rate would that be, and is there a reference for such results?
* Lines 190-191: "potential improvement...could be on the order of generic rate improvements implied by a central difference scheme".  What is this referring to?
* Lines 278-280: "does not appear that rate-double robustness would admit weaker numerical requirements on $\epsilon$".  Weaker requirements than what?  The "generic analysis" referenced above?
* Are there "conservative" rates on $(\epsilon, \lambda)$ that will always preserve $O_P(n^{-1/2})$ rates of estimation, obtainable via some generic analysis?
* Proposition 2 (DTR) "verifies that the requirements...are similar in $\epsilon$ as in the case of a single-timestep", but there is no $O(\epsilon^2)$ term in either Proposition 1 or Corollary 1.  Could you clarify what is meant here?
* Propositions 3 & 4 differ not only in an additive term, but also in the usage of perturbed nuisances, which makes them difficult to compare directly (this also applies to Corollary 1, as noted on line 170).  Is there a reason why a direct comparison (e.g., isolating only an additive difference) is unnecessary here?
* Are there existing scenarios where the analytic form of the gateaux derivative is non-obvious, but where these conditions (pathwise differentiability, second-order remainder) can nonetheless be verified to hold? Or does verifying these conditions *always* require derivation of the analytic form?
* More broadly, are there scenarios where we can apply this approach (with appropriately conservative rates on $(\epsilon, \lambda)$) and have confidence in achieving $O_P(n^{-1/2})$ rates, **without** deriving the analytic derivative?  E.g., would the constrained MDP with arbitrary linear constraints be such an example?
* In the display following line 563, it is claimed that the following holds due to Cauchy-Schwarz.  I'm not sure I see the application of CS here: Is there another reason why we would expect the cross term $2 E[(\tilde{\mu}_{\epsilon}(X) - \tilde{\mu}(X))(\tilde{\mu}(X) - \mu(X))]$ to be non-positive?

$$ E(\tilde{\mu}\_{\epsilon(X)} - \mu(X))^2 \leq E(\tilde{\mu}_{\epsilon}(X) - \tilde{\mu}(X))^2 + E(\tilde{\mu}(X) - \mu(X))^2 $$
* In the display following line 562, the last inequality seems like a non-trivial jump, would you mind walking through the logic explicitly?

limitations: I think the authors do a fine job of explaining limitations of the approach.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: Efficient / doubly-robust estimation in causal inference typically requires constructing estimators based on the (efficient) influence function, which is derived case by case. The paper considers numerical approximation to the influence function with *empirical Gateaux derivatives*, which are obtained through finite differencing and smoothed perturbation. The approximation uses plugin estimate of the probability density. The paper studies conditions on the smoothing and perturbation parameter such that the resulting one-step estimator retains desired statistical properties. The paper first examines the case of estimating the counterfactual mean and then extends to more complex dynamic settings. 
strengths_and_weaknesses: ### Strengths
1. The paper is built upon recent efforts in "automating / computerizing" influence-function-based estimation and studies an important question. 
2. The paper provides a clean analysis for the basic case of estimating the counterfactual mean. 

### Weaknesses
1. The approximation is based on a density estimator of the underlying data distribution, which seems to bear serious practical limitations. It is unclear if this choice is necessary, and if so, how to overcome the limitations. 
2. The conditions impose a product-form rate condition on the smoothed estimates; see (iv) of Assumption 1. It is unclear when these conditions can be met in practice. 
3. Other than a simple simulation, the paper does not seem to provide concrete practical recipes for the running example that can flexibly incorporate different underlying distributions, dimensions, smoothness, etc. This weakens the relevance for the results developed for the more complex, dynamic settings. 
4. The clarity of certain sections can be improved. 
questions: 1. Gateaux differentiable vs influence function

(1) The definition of the influence function ($\S$2) in terms of Gateaux derivative comes from the robust statistics literature. How is it related to the influence function used for semiparametric inference?

(2) page 2, line 59-60: in the direction of $H$ or for all $H$?

2. On plugin estimation through density estimation

(1) Given the curse of dimensionality, it is unclear to me why the approximation must be based on a density estimate. I do not parse the argument provided in line 117-121: can the authors elaborate on this? More specifically, for the running example, it seems that one can easily simulate data from the perturbed distribution. Can the approximation be based on an estimator using the simulated perturbed data (e.g., empirical average of an estimated conditional mean, where the distribution is that of the simulated data)?

(2) Suppose density estimation is necessary. Then it is worth discussing when the kernel-based estimators can fulfill the conditions required, e.g., (iv) in Assumption 1. 

(3) To really "bolster confidence in the numerical procedure", the case study of the running example should be expanded accordingly to show that the proposed method works in a range of settings (e.g., at least when dimension is not high). 

3. I have a hard time trying to follow the extension to dynamic settings in $\S$4. Can the authors summarize the take-away message there?

4. A few points worth clarification. 

(1) How is $\tilde{P}$ used in Algorithm 1?

(2) line 71: it is unclear what "$\Psi(P_{\epsilon,\lambda}^{\ast})$ is the projection of $\Psi(P_{\epsilon,\lambda})$ ..." means. 

5. On the relation to previous works in “automatic” or “computerized” semiparametric inference

The paper features a long list of references. Since the focus here is the influence function, I would suggest adding a paragraph after $\S$2, which clarifies the relation to previous papers on "computerizing" influence-function based estimations, e.g., [9] and [17] in the bibliography. 
limitations: I do not foresee a potential negative societal impact of the proposed work.
review_title: null

Meta-review: metareview: The authors make a solid contribution in the literature on computerized estimation of gateaux derivatives and automatic debiasing, with applications to causal inference, providing theorems on the level of numerical approximation that preserves root-n statistical rates. Therefore this should be an interesting paper for the causal ml community. The authors have addressed most major concerns raised by reviewers in their original evaluation.

On a minor note the authors should also relate to the recent prior work on automatic debiased machine learning for dynamic effects https://arxiv.org/abs/2203.13887 which seems to be capturing their main application, contrary to what is claimed in their related work.
confidence: Certain
Decision: Accept