Automated Efficient Estimation using Monte Carlo Efficient Influence Functions
Authors: Raj Agrawal
Sam Witty
Andy Zane
Eli Bingham
Venue: NeurIPS 2024 spotlight (2024)
Keywords: Efficient influence function
semiparametric statistics
double robustness
automatic differentiation
Primary area: causal_inference

Abstract: Many practical problems involve estimating low dimensional statistical quantities with high-dimensional models and datasets. Several approaches address these estimation tasks based on the theory of influence functions, such as debiased/double ML or targeted minimum loss estimation. We introduce \textit{Monte Carlo Efficient Influence Functions} (MC-EIF), a fully automated technique for approximating efficient influence functions that integrates seamlessly with existing differentiable probabilistic programming systems. MC-EIF automates efficient statistical estimation for a broad class of models and functionals that previously required rigorous custom analysis. We prove that MC-EIF is consistent, and that estimators using MC-EIF achieve optimal $\sqrt{N}$ convergence rates. We show empirically that estimators using MC-EIF are at parity with estimators using analytic EIFs. Finally, we present a novel capstone example using MC-EIF for optimal portfolio selection.

Automated Efficient Estimation using Monte Carlo Efficient Influence Functions
Raj Agrawal
Sam Witty
Basis Research Institute, Broad Institute rajobasis.an
Basis Research Institute, Broad Institute sam@basis.o
Andy Zane
Eli Bingham
Basis Research Institute, UMass Amherst andy@basis.. ai
Basis Research Institute, Broad Institute eli @basis..i
Abstract
Many practical problems involve estimating low dimensional statistical quantities with high-dimensional models and datasets. Several approaches address these esti- mation tasks based on the theory of influence functions, such as debiased/double ML targeted minimum loss estimation. We introduce Monte Carlo Efficient Influence Functions (MC-EIF), a fully automated technique for approximating efficient influence functions that integrates seamlessly with existing differentiable probabilistic programming systems. MC-EIF automates efficient statistical estima- tion for broad class of models and functionals that previously required rigorous custom analysis. We prove that MC-EIF is consistent, and that estimators using MC-EIF achieve optimal VN convergence rates. We show empirically that estima- tors using MC-EIF are parity with estimators using analytic EIFs. Finally, we present a novel capstone example using MC-EIF for optimal portfolio selection.
Introduction
Over the past several decades, there has been remarkable progress robust and efficient statistical estimation, especially for high dimensional problems. A particularly compelling class of such finfttinnceinnnooons methods are built foundation of efficient influence functions (EIF), i.e., functional derivatives in the space of probability distributions [en22]. These methods have been particularly fruitful in causal inference applications, where estimating quantities such the average treatment effect require modeling digh-diimessooall nuisance parameters relating confounders to treatment and outcome variables. Intuitively, these methods focus finite statistical resources quantities that matter, and on nuisance parameters that only indirectly inform the statistical quantities we wish to estimate.
Despite their successes, estimation methods based on the EIF have lagged behind the kinds of automation that machine learning practitioners have grown accustomed to, instead requiring complex manual derivation
case-by-case basis. This is contrasted with the generality of automatic differentiation (AD) systems [BPRST8] and probabilistic programming languages (PPLs) such as Pyro [BCJ+ 19 Gen [CTSLMI9, which automate numerical computations for probabilistic inference. ElFeb estimators have historically eluded this level of automation and generality, because exact recovery of the EIF requires solving high- tigh-dimensional integral equations.
Contributions. We introduce Monte Carlo Ffficient Influence Functions (MC-EIF), general and automated technique for numerically computing EIFs using only quantities that are already available from existing AD and PPL systems. Our key insight is that EIFs can be expressed equivalently as product of (i) the gradient of the functional, (ii) the inverse Fisher information matrix, and (iii) the
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

gradient of the log-likelihood, as shown in Theorem 3.4 in Section 3 In Section MC-EIF can be used automatically construct variety of efficient estimators for a broad class of models and functionals, avoiding the need for complex manual and error-prone derivations.
4 we show how
In summary, we show that: (i) MC-EIF provides accurate estimates of the true EIF, enabling efficient estimation, and (ii) MC-EIF is very general, applying to many functionals and models that can be written as probabilistic programs. In Section 3 we introduce MC-EIF and provide a non-asymptotic error bound on the quality of our approximation. We show how estimators using MC-EIF achieve the same asymptotic guarantees as using analytic EIFs in Section 4 In Section 5
we show empirically that MC-EIF produces more accurate estimates of the EIF than existing automated approaches, and using MC-EIF as a drop-in replacement for the analytic EIF does not degrade estimation accuracy in a variety of benchmarks, including a novel capstone on optimal portfolio allocation.
Related Work. Influence function-based estimators have rich history in the statistics and machine learning literature [BKB+99) 93| Tsi06 Ken22 [HDDOV22]. Despite their effectiveness, these methods have historically required custom and complex mathematical analysis for specific combinations of models and functional. Even an incomplete recent survey of the influence siuction-based estimation literature yields a large collection of complex scenario-specific research. For targeted minimum loss estimation (TMLE) (VDLRU6); cluster-randomized trials [BvdLA+233 continuous time-dependent interventions 22vdL22) mixed experimental and observational data (DTA 22) mediation analysis with longitudinal data [WydLP" 23). subgroup treatment effect estimation [WPvdL 231, survival and competing risks analysis [RvdL24), continuous timeett-vvent outcomes [33v and variable im- portance measures for effect estimates [L 2.3]. For double/debiased machine learning [CCD+18] ieiference-inddiffeeeee
[Cha20], instrumental variable designs [TBB21a], and mediation analy- sis [FHL+ Importantly, our work does not introduce novel efficient estimators; instead it aims to lower the mathematical burden for practitioners who wish to use existing influence function- based efficient estimator templates (see Section 4 with custom models and/or functionals.
Our work is not the first to attempt automate and generalize computations for efficient statistical estimation. Perhaps the closest technique we are aware of is approximating the influence function using finite differences on kernel-smoother empirical distributions [FOWD55| We provide thorough comparison with this method in Section S demonstrating how MC-EIF automates and scales better to high dimensional problems, exactly the settings where efficient estimation is most useful. Recent work has made progress towards general efficient estimators, but still impose strong restrictions on models and/or functionals. DML-ID [TB21b] extends double machine learning to nonparametric causal graphs and marginal density under intervention functionals. Similarly, the kernel debiased plugin estimator [CGMM23] implements a version of TMLE that bypasses the influence function computations for models defined in a RKHS. Finally, other recent work [CNS22, FS23 CNQMS2T] CNQMS22) approximates the efficient influence function for generalized method-of-moneet estimators.
JWZ22al
Finally, we note that there are number of intriguing connections between three related but distinct mathematical objects: the efficient influence function in semiparametric statistics (si06), the natural gradient in information geometry [Ama16], and the so-called empirical influence function in robust statistics and machine learning [aw86]. A comprehensive review of these connections is beyond the scope of this paper, and we focus here on two particularly important points for contextualizing our work. First, we emphasize that the efficient and empirical inffuence functions are not equivalent: the efficient influence function a fundamental mathematical object in semiparametric statistical theory which quantifies the effect of perturbing functional in an arbitrary direction in the space of probability measures, while the empirical influence function is a distinct and somewhat more specialized [BNL 22) object quantifying the effect of perturbing individual training points in a para- metric statistical model. More specifically, in the parametric setting the efficient influence function is defined as shown in Theorem 3.4 in terms of the Fisher information matrix [Tii06), whereas the empirical influence function is defined in terms of the Hessian of a model at the training points, two quantities with very different mathematical and statistical properties that are not straightforwardly interchangeable [KHB19] Second, we note that existing algorithms for computing natural gradi- ents and empirical influence functions cannot immediately be adapted to efficient estimation. Specifically, our algorithm described in Section 3 for Monte Carlo approximation of the efficient influence function is similar in structure some previous algorithms developed for estimating the This would seem to
natural gradient (TR19 and empirical influence function [KL17 suggest porting other methods that compute more heavily biased approximations of the natural gradi-
2

---

ent GLB 18) and empirical influence function to computing the efficient influence function, some of these methods are known scale to even the largest neural network models deployed in practice today [GBA++3) However, these approximations are not directly applicable in our setting because provably efficient estimation is only known to possible with tight control over any approximation error introduced in computing the efficient influence function, as discussed in Section below and in [WW222a]. Relaxing these restrictions to enable similarly scalable variations on the basic MC-EIF framework of Section 3 is an important direction for future work.
Problem Statement
General Problem. We consider the estimation of some estimand 0" RR, where L E N denotes the
dimension of the target quantity. Typically. we can express 0 = V (P*) for some known functional Mr
where V maps a probability distribution vector R. L, and P (x) denotes the true data generrenngaaiing
distribution over some vector of observables x E RD D N. Many estimation tasks involve high- dimensional muisance parameters, or quantities of no immediate value to the analyst. For example, to
estimate the average treatment effect, one might need to adjust for high-dimensional confounders.
Semiparametric Solution. Semiparametric statistics provides mathematical framework for opti- mally estimating 0* in the presence of potentially complex, high-dimensional nuisance parameters. A standard way to estimate 8* is with the plug-in in approach; construct an estimate of IP* and report @ = (TP). Unfortunately. the plug-in approach can lead to provably sub-optimal estimates of 6* due to poor estimates of IP" [Tsi06]]]]] CCD+ 18 FS23 Instead, a general recipe for efficiently estimating
0r from finite data {a.,,,m where I.n.
1,.., .N, is given by the following three-steps: (i) use N/2 samples to construct an initial estimate f of IP* (ii) compute the influence function (to be defined shortly) of v at the estimate P. and (iii) evaluate the influence function at the held out N/2 datapoints to derive a corrected estimate _ In Section 4 we elaborate on how influence functions are used to construct several popular efficient estimators.
iid (x:) for
Influence Functions. A central premise of this paper is that to automate efficient estimation, it suffices to automate the computation of influence functions, which can be thought of as gradients in function space. We make this precise below.
Definition 2.1. (Gateaux derivative) Consider the e-perturbed probability distribution P. = (1 c)P + eQ = TP + cQQ - P), where Q some probability distribution. is Gateaux differentiable at @ if the following limit exists:
((P.)) (P)
(TP)) de
lim c-+0
r=0
The Gateaux derivative can be viewed a generalization of the directional derivative from ordinary calculus; it characterizes how much a functional changes at a point I' in the direction Q - P.
Definition 2.2. (Influence function) Suppose there exists square integrable function (IP such that
= -pp,9 - p) 10.4-p)12-- = (P.) c==
- -(())dd)
de
2EERD
for all Q M and Ex<p((<x) = O, where M denotes some space of probability distributions and p(-) and g(-) are the density functions for Q and IP, respectively. Then, & is called an influence function for T at P.
influence function is a re-centered "functional gradient" in L (P( just as the Euclidean inner product between the gradient of a function and vector yields the directional derivative in ordinary differential calculus. the (P) inner product between the influence function and perturbation Q P yields the Gateaux directional derivative. Influence functions, however, are not always unique [si06 Ken16] - some may lead to higher asymptotic variance estimators than others. The optimal influence function minimizes asymptotic variance, and is called the efficient fuftttnncefunction
function
'There are some small variations to this three-step recipe such as using cross-fittin fotts-fittig (CCD 18 instead of a
simple equal split of the data; also Section 4
3

---

(EIF). When the EIF exists, it is I almost everywhere unique, and found through a Hilbert space projection onto what is known as the nuisance tangent space. We defer details to [Ti006] and [Ken
As the influence function in Definition 2.2 is defined implicitly as a solution to an infinite set of integral constraints over M, it is often hard to find. Entire papers have been written to analytically derive influence functions; see, for example, the papers listed in Section [ For even experts in machine learning and statistics, such derivations are out-of-reach, time consuming, and error prone.
3 Monte Carlo Efficient Influence Function
Much of the work in semiparametric statistics and efficient estimation has focused on scenarios where the nuisance function is modeled nonparametrically [i00 VDL CCD+
Ken22]. However, practitioners often use high-dimensional parametric models such as generalized linear models, neural networks, and tensor splines in practice due their flexibility and ability to scale large datasets. Due to the richness of these high-dimensional spaces, inference is still statistically challenging and benefits from efficient estimation; see, for example, Table 1 in 1C) Specifically, in contrast to traditional wow-dimeniionll parametric models where maximum likelihood estimation is typically efficient [FR22 | high -ighdiimessional parametric models often exhibit distinct asymptotic behaviors [vd V98. KBB 13 [TWI5).. In these high-dimenn dineesiimnsi models, estimates may converge slower than classic Op( vV rates without the application of efficient inference methods [VDLROG
CCD+18 Ken22]. A key question we address is whether using high-dimensional parametric model simplifies the process of solving Definition 2.2 We show that does below.
Notation. We let 6 E & C. I" denote a fimite-dieenionnl parameter specifying a distribution on the observed random variables
R. for p < oc, p E N. Po (x) corresponds to a distribution in this space, and P (x) the true distribution, or the one closest to the true data- generating distribution in Bellbackk-eibber distance. We let (o))) denote function I2 + IR' that equals the evaluation of the functional V (IP,, () for all á e 4. Under mild differentiability assumptions, we provide the analytic formula for the EIF in Theorem 3.4
The first assumption states that the density of TP (x) is continuous and differentiable with respect to 6. a condition satisfied by many parametric model families. For example, the univariate Gaussian density 1 exp --05((z x)²) is a continuous and differentiable function of its mean, & R.
2m
Assumption 3.1. Va E R,,, the map & + P (1x) is continuous and differentiable with respect to ø.
The next assumption is also satisfied for many functionals. For example, consider the mean functional L (P))) = Ezwe, [z:]. Continuing with the univariate Gaussian example from above, where the mean is unknown, we have (6)) = Φ, which is a continuous and differentiable function of sb. Assumption 3.2.
(b)) ()) a continuous and differentiable function of é.
The last assumption requires that the Fisher information matrix be invertible, which is necessary for to be identifiable [Tsi66). Assumption 3.3. Fisher information (()) == Ez-Pa(s) V log P (z) log (2)) is invertible.
Theorem 3.4. (Theorem 3.5 in (Ti06) Suppose Assumption 3.7 Assumption 3.2 and Assumption 3.3
hold. Then the efficient function
function po at Φ evaluated at the point & E RDD equals ( I(0) I(( 'V log (.))
While Equation | Thas been around for many decades, has mainly been used as a theoretical tool for mathematical statisticians. In particular, Equation| 1 is typically evaluated at the true data generating parameter 6" to characterize the theoretical asymptotic variance of an estimator. In other instances, it is used to derive approximate confidence intervals; see, for example, Chapter 3 in [Ti06)) In the following Sections, we discuss how Equation [ provides a key ingredient in automating efficient estimation in high-dimensional parametric models.
3.1 Numerically Approximating the EIF
Given a model (.) and functional V(.), we seek to automatically compute Equation ! Our Monte Carlo efficient influence function (MC-EIF) estimator achieves this automation by replacing (0) and

---

I()), which are typically unknown, with stochastic approximations &Ar and [0) computed from M E N Monte Carlo samples:
Po.M (z) = IV (o) (o)) Pog Pa
(2)
Here, we show that Equation 2 leaads to an automated and accurate approach to numerically computing EIFs using only quantities provided by existing AD and PPL systems.
Approximating IM (a() log Po (+((()) (.). We draw I'm de Pa(x), 1
S
m M for M E N, and let
1
(Ø)
V log Pa (o Pog
M
A naive approach for computing (o)" V log P. (i) is calculating the full p matrix in Equation 3 inverting it, and then taking its product with the score vector V log (x(m)) E I computed from AD. This naive approach takes (M?2 + p²)) time and O(p²) memory which might be too expensive for large p. Instead, we exploit AD and numerical linear algebra techniques to avoid explicitly storing and inverting the approximate Fisher information matrix, similar to (KL17. Suppose that we have a black-box method to compute Fisher vector products I ( v for arbitrary vectors v E I²,. Then, we could use the conjugate gradient algorithm to iteratively find ( log (T( where the cost of each conjugate gradient step is determined by the cost to compute T (o)e. While the number of conjugate gradient steps needs be p for an exact inverse, often far fewer iterations required for close approximate solution WPG 19). To make computing I (o) efficient, we collect the M simulated datapoints in the matrix XM
IR MDD and let
log Po (X(x)) (log CogPPplan Pa(x)), ,log Po(()))) E ERM.
Then, Ls (Ø)v equals
JT JMM v
M JM [JMu],
(4)
MM
where JM V (X E RMxx is the Jacobian matrix. We use Pearlmutier's trick to avoid computing the entire Jacobian matrix (Pea94). In particular, this method allows us to compute the Jacobian vector product UM = [Jaru] E R in time proportional to a single evaluation of log P. (X) and O(M + p) memory. Similarly, we use the vector Jacobian product to compute JK
Approximating V v Instead, it only requires a sequence of gradient estimators
(Ø). Robust estimation with MC-EIF does not require exact gradients.
of ((((d)) whose error can be bounded above by some Am > O, where the Mth iterate Vohar (Ø) is used in Equation 2F Using such a sequence guarantees that the approximation error of Equation 2 is not dominated by V Pewa((), In practice, the target functional (0) might be quite complex, making gradient estimation challenging. For example, it might involve taking expectations with respect to conditionals of & (x). or be defined implicitly as a solution to an optimization problem as in [W722al
One particularly simple and general way to address this challenge is to implement a Monte Carlo estimator of y that can be transformed via automatic differentiation into an efficient Monte Carlo estimator for its gradient, review. We note that for the very wide class of functionals that can be written as nested expectations, recent work [RCY+ 18 SW23 LHSM33 gives formal statements of smoothness assumptions and theoretical results sufficient to obtain the oracle rate Am in terms of numbers of samples, as well as algorithms that can be implemented using automatic differentiation software like PyTorch (PGC+ 17 For example code snippets of functionals, see Appendix B
well-understood problem that is beyond the scope of this paper to
3.2 Theoretical Guarantees for MC-EIF
We conclude by deriving a asy-tsympto error bound for how well Equation 2 approximates Equation]
For fixed input dimension D and model sizes p, Equation converges to Equation [ at
In Section 3.2 we require that Am = (Vm -mplogg
5

---

Op(1/VM) rate by the Law of Large Numbers. As we are interested in sigh-dimensional parametric families, we analyze the behavior of our upproximation as a function of both input dimension D and model size To prove our result, we use standard tools and assumptions from empirical process theory such as the requirement of sub-Gaussian tails [vdV98].
Assumption 3.5. Suppose ~ IP (ar). There exists universal constant 0 < C₁ < oo such that the normalized score vector I = VD V log IP (x)) is sub-Gaussian random vector with parameter Cj-
As Elll 6, log lo O(D), 1 j p. the division by vD in Assumption 3.5 ensures that the variance of the score does not grow unboundedly as - oo. Thus, our assumption that sub-Gaussian is mild. Assumption 3.6 below ensures that the functional and score are smooth enough by bounding their gradients.
Assumption 3.6. There exist universal constants C2, C3 < oo such that [Voo(()]] C2 and Volog
< C3 for any x' E RRD, for any p and D.
Unlike our Monte Carlo approximation to the Fisher information matrix, we do not assume a particular type of estimator for V (Ø). To prove convergence of MC-EIF the true EIP, we assume that Volu (Φ) converges to V (1)) at the following rate:
Assumption 3.7. Let 6m = V (ov(()) V VorM (0) E IR LxP denote the approximation error. There exists universal constant C oo such that for M C. for any O.
P
| (oaulle
p plog p +
< exp(-c), and P (IV (oVol Cz) exp(-e),
M
In Appendix A.1
prove that Monte Carlo estimators of V (0)) with gradient clipping (ZHS]20
satisfy Assumption 3.7 Hence, Assumption 3.7 is mild condition. Under these three assumptions, and the ones in Theorem 3.4 prove the following result in Appendix(A.1] which states that M must scale linearly with log to guarantee close pointwise approximation.
Theorem 3.8. Suppose Assumption 3.7 Assumption 3.2 Assumption 3.3
Assumption 3.5 Assump- < C4 and Cs oc, such
tion 3.0 and Assumption 3.7 hold. Then, there exists universal constants that for any e and M max(Cs(p (CC,CC))
plog +
1xp(x*) - G CaAmar (0
(5)
M
for x" E RR with probability at least - 2 (exp(-e), where 2 = @ō) and Amax (-) denotes the largest eigenvalue of a matrix.
4 MC-EIF for Automated Efficient Inference
In Theorem 3.8. we proved that MC-EIF is close to the true efficient influence function pointwise. In this Section we; (i) show how MC-EIF be used automate the construction of popular efficient estimators, and (ii) prove how many Monte Carlo samples are needed to ensure that key statistical properties hold when MC-EIF is used instead of the true EIF. In doing so, MC-EIF brings conceptual clarity to the practice of constructing efficient estimators, and how these estimators can be implemented using existing differentiable probabilistic programming languages like Pyro [BCJ 19)
All three of the efficient estimator templates we explore this Section involve some combination of plug-in estimation and ElF-based computations. A key practical benefit of our work is that MC-EIF-based efficient estimators are entirely modular: advances in general-purposs purposse probabilistic inference technology directly translate to advances in efficient estimation under framework.
4.1 Von Mises One Step Estimator
We start with the simple Von Mises One Step Estimator, which corrects the plug-in estimator in Section by adding the average value of the efficient influence function on held out dataset. Despite its simplicity, this estimator achieves optimal statistical rates [Ken22]. Our one step estimator using MC-EIF (z))) instead of the true efficient influence function (()((z)) )) is provided in Algorithm

---

Algorithm 1 MC-EIF one step estimator
Input: Target functional s) initial estimate of parameters O. held out datapoints { A. N=241+11
Number of Monte Carlo samples M
Oplug-in + (()
{plug-in in estimate }
C= 2 E Em=N/2+1 (F
[MC-EIF one step correction) ]
Return: Optng-in +C
Theoretical Guarantees. We call the step estimator that the true EIF instead of MC-EIF in Algorithm T the analytic one step estimator. Below we prove how many MC samples are needed to ensure estimator for finite M has the same statistical properties the analytic one step estimator.
Proposition 4.1. Let 6, denote the output of the analytic one step estimator and Ô the output of Algorithm for M = oc and M < co, respectively. If M = ((Nplog)) > O(log N) and the assumptiions in Theorem 3.8 hold, then ||0. 0||2 (p(1/VN).
By Proposition 4.1 MC-EIF is asymptotically efficient when the number of Monte Carlo samples in Algorithm grows faster than Np logp.
4.2 Debiased/Double ML
Next, we express debiased/double ML (DML) [CCD + terms of MC-EIF. To rewrite DML explicitly in terms of MC-EIF, we largely follow [CCD* 18 [N22]
Algorithm 2 MC-EIF debiased ML
Input: Vector of estimating equations g. initial estimate of parameters 3 held out datapoints (Ie N N/2+1 Number of Monte Carlo samples M
[(0) & S N/2+1 g(T, n(p3), 0) + &8 (Fn,0) (MC-EIF orthogonal moment function)
Return: ((0) 0}
Construction of Orthogonal Generalized Method Moment (GMM) Estimators. GMM-based estimators are defined by nuisance functional 17(-) E IR,' J E N, and set of K E N functions (a,, (P (P))) { often called estimating equations. These estimating equations are selected so that their roots uniquely identify 0* when the nuisance parameters my(P))) are estimated correctly:
Ex (ax) (x)]g((, ((P ), 0)] + 0",
(6)
where g (g1.-... 9K). As an example. g might be the gradient of the log-likelihood function. To make GMM-based estimators less sensitive to incorrect estimation of the nuisance parameters, (CCD+18 18 IN22 CNS22 replace g() with the orthogonal moment function, constructed using influence functions. In our setting the orthogonal moment function equals the following:
g(v, (PP)), 8) (xx 0).
where (x 0) is the efficient influence function associated with the functional (a( Exer, [g(z, (Pp,) a), 0)] for fixed by Equation 2.6 in [N22]. By Theorem 3.4
Po(z, 0) = [V [V)no())] ((6) VV log (x(x).
(8)
Since g is a known by assumption, we can readily use the Monte Carlo methods in kW SHWAT 15
to automatically approximate (o)) We summarize the DML algorithm Algorithm 2 which replaces Equation 8 with our MC-EIF approximation.
Theoretical Guarantees. For general estimating equations, is difficult quantity how errors in our MC-EIF approximation to Equation g lead to changes in final estimates. When the estimating equations have more structure, however, we obtain a similar result as in Proposition 4.1
3 DML can handle functionals which are not pathwise differentiable, we only consider parametric model families, however, we can assume without loss of generality that the functional is pathwise differentiable [FS23 and that the range space of the nuisance functional is dimteesdiessinal.... Consequently, we can express DML in terms of efficient influence functions.
7

---

Assumption 4.2. [[ (PP)), 0) = m(xn, (F +) 0 for some vector of known functions m(-).
Assumption 4.2 was made several works CCS222 N22 already. We prove an analogous rate
guarantee as Proposition 4.1 under Assumption 4.2
Proposition 4.3. Let 0. denote the output of the analytic DMI. estimator and 0 the outpur of Algorithm
2 for M oc and M 00, respectively. If M (2Nplogg p Oloo N) and the assumptions in Theorem
1.meeemm.8mmm 3.8und Assumption 4.2 hold, then |10. 0||2 0p (1/VNN.
4.3 Targeted Minimum Loss Estimation
We conclude by writing targeted minimum loss estimation (TMLE) [VDLR06] explicitly in terms of MC-EIF. Unlike the one step estimator or DML, TMLE directly corrects the estimated distribution Pz(x) and then plugs in the corrected distribution into the functional Vr as the final estimate. To perform this correction it perturbs Pá in the direction of the influence function, searching for the optimal step size by maximizing the perturbed likelihood on the held out dataset. Intuitively, TMLE can be viewed as a form of gradient ascent in function space. We show one step TMLE [VDLR06) in Algorithm | The multi-step TMLE version is computed by iterating Algorithm 3 multiple times until e approximately equals [VDLR06]
Algorithm 3 MC-EIF one step TMLE
Input: Target functional V, initial estimate of parameters c. held out datapoints {zn [zn)n=N/241 N= Number of Monte Carlo samples M
p(c,x) + (1 + e" Pa.M
+ (=))p)))))))
(MC-EIF projected e-perturbed density function)
é t arg max.eR N Em=N/2+1 logp(e,æ,)
{ [Maximum likelihood search over e)
Return: ()((,))
5 Experiments
We start by comparing the quality of MC-EIF against other methods for influence function approxima- tion. Then, we show how MC-EIF behaves when; (i) the number of Monte Carlo samples is varied, (ii) the dimensionality of the input is varied, and (iii) the efficient estimator type is varied. Our empirical results ultimately validate our theoretical results in Section and Section 4 Finally, show how MC-EIF can be used to automate the construction of efficient estimators for new functionals by revisiting classic problem in optimal portfolio theory. Our MC-EIF implementation is publicly available in the Python package ChiRho All results shown here are end-to-end reproducible.
In JWZ22al, the authors target the nonparametric influence function, which is the unique influence function when M = (IP) in Definition 2.1 By contrast, we target the efficient influence function. Thus, for evaluation, we compare how well the empirical Gateaux method from [JZ22al approx- imates the nonparametric influence function and how well our MC-EIF method approximates the efficient influence function the same geterateeerggggg process.
To have a ground truth for comparison, we select simple model and functional where we can analytically compute the nonparametric and efficient influence functions. To this end, consider the problem of estimating the expected density, (IP) = " as in [BR88] CLvdL.19. We further suppose that Jr ~ N o), We consider two parametric model families: one where is unknown but a = 1, and one where both jl and a are unknown, which we call M₁ and M2 respectively. As the nonparametric influence function makes assumptions on the underlying model family, it remains fixed across M1 and M2 and always equals ( - V (IP)) [CLvdL99. However, the EIF equals zero for Mj. the expected density does not depend on Hence, any plug-in estimate for models in Mj will result in a correct value of the expected density, and thus no distributional perturbations produce any change. In M2, the efficient influence function for the expected density depends on the unknown a. See Figure in the Appendix for further intuition around the expected density influence functions in parametric (in unknown σ)) and non-parametric settings.
Figure summarizes how well the empirical Gateaux derivative method approximates the nonpara- metric influence function and how well our MC-EIF method approximates the EIF at the point

---

Median Relative Error
Extimaned Emor Rete Oggiity
Our Therr Olphir)
MCKEP Monte ssmp
known o²
10-1
Inflweco
10-2
10 102
10a
Vake (c) Empirical Gateaux
Model Dimension (p)
(b) unknown o²
Figure 2: Empirical evidence for conver- gence theory. Increasing p for the ATE ex- periments produces MC-EIF approximation errors that closely match and sit below the worst-case error rates given by Theorem 3.8
Figure 1: Comparison between MC-EIF and empirical Gateaux approximation. MC-EIF (a and b) is less sensitive to hyperparameters (e and A) than the empirical Gateaux baseline (c).
P = N0, 1). We see that MC-EIF is able to approximate the efficient influence function very well (M = 10 samples). By contrast, the empirical Gateaux derivative is highly sensitive to the choice of two kernel smoothing hyperparameters, e and X. As the true influence function is not known, always clear how to select and X Such numerical instability was already discussed in [CLvdL19] where the precision necessary must get exponentially smaller with input dimension, making it infeasi- ble when D x 10 MC-EIF, however, has only a single tunable parameter (M), where larger M unambiguously provides better approximation. In Theorem 3.8 we provided conditions for this improvement, and Figure of the Appendix corroborates the unsurprising improvement empirically. We further discuss challenges automating the empirical Gateaux method in Appendix [ We attempted to use the empirical Gateaux derivative baseline for other experiments, but were unable to achieve numerically stable solutions for any p > 2 without prohibitively long run-times.
not
Next, we focus on a classic model consisting of a binary treatment, high-dimensional continuous confounders, and Gaussian distributed response; see Appendix E for the precise model formula. We assume that the analyst is interested in estimating the average treatment effect (ATE), where the true ATE is zero but unknown. All influence function computations are relative to an initial point estimate o, found through maximum a posteriori estimation using 500 training datapoints. Due to the exponential runtime in dimension for the methods in (JWZ22al [L) we focus on comparing MC-EIF with the analytic influence function for ATE below.
Sensitivity to Dimensionality. Theorem 3.mmmmpiieeeee M. the quality of the approximation degrades with the square root of model dimension p. In Figure 2 we empirically show how approximation quality degrades as increases for M = 10 fixed. Based on Figure 2 the empirical results closely match the theoretical behavior predicted by Theorem 3.8 We also show how the computational complexity of MC-EIF scales as p increases in Figure
implies that for fixed number of Monte Carlo samples
Sensitivity to Estimator Type. Here, consider a nigh-dimensional setup where there are 200 confounders but only 500 training datapoints. We simulate 100 different datasets with this configura- tion approximate the sampling distribution of different efficient estimators. In Figure 3 see that across estimators, using MC-EIF instead of the true EIF results in minimal downstream error. This is consistent with our theoretical results in Section While MC-EIF is agnostic to the choice of
*One suggestion is to visually inspect an spsilon-lambde plot (like the one in Figure 1 for a "o.siblsiilly curvilinear triangular region nested in the upper left portion of the [plot]. In this region, the finiteddifference approximation of the EIF value should be essentially constant (CLvvL119 $4.3)."
the data unit is = di + d2, current theory requires e < X1 or even e & X in some largely avoidable cases ([L 19). $4.1). If functional evaluation is approximate, smaller requires significantly more compute for the finite difference to dominate Monte Carlo error in estimating the difference between the functional evaluated the plugin distribution. and the functional evaluated on the e-perturbed distribution. might even be so small as overrun floating point accuracy on modern machines even 1 exceeds standard precision recommendations for b4t floating point values. As discussed in Section 3.2 to make the error not grow with p. we would need M p log.. p.
"Specifically, where the non-parametric dimensionality

---

Metric
One Step MC-EIF
Plug-in
REV
1.86 + .35
2.60 .35
RMSE
.08 + .02
MG-EIF
MC-EIF
MCEF
(a) One Step (b) TMLE
(c) Double ML
Table 1: Empirical results for Markowitz optimal portfolio optimization. Using MC- EIF, Algorithm [ lower relative ex- pected volatility (REV) and RMSE compared to the oracle estimator.
Figure 3: Comparison between ATE estimators using MC-EIF and analytic EIF. MC-EIF pro- duces ATE estimates very close to the diagonal, representing an oracle estimator of the EIF.
efficient estimator, one may prefer some over others depending on the problem. See Figures 5 and of the Appendix for an example performance comparison between efficient estimators of the ATE.
Ability to Handle New Functionals. To illustrate MC-EIF's flexibility, we revisit a classic problem in optimal portfolio theory. Suppose that E P is a vector of asset returns. We are interested in estimating the optimal portfolio weights l" E TR. that maximize the expected return while minimizing the variance of the portfolio. Then, the Markowitz optimal portfolio [Mar52] is given by:
D 0
V x (P&) arg max 6" Eg, [x] >o Cov(x;;o)) BEED
6"E,,
subject to
where the tradeoff between expected return and variance (measure of risk), and (ov(:; PP denotes the covariance matrix with respect to P&- Hence, the optimal weights functional x (TP, (T)) depends on a high-dimensional nuisance, namely the x D covariance matrix of returns. The target 0 / (P') is much lower D-dimensional target parameter. Setting À = corresponds to the global minimum variance portfolio [HB9T] [M03] ARC20) for which there is (to our knowledge) no efficient estimator in the literature. We show results in Table iindicating substantial improvement in synthetic data experiment; detailed description of this experiment may be found in Appendix F.2
6 Limitations
As discussed in Assumptions 3.1 and 3.2 both the likelihood and the target functional must be differentiable with respect o. In practice, especially if the model involves latent discrete random variables, some degree of relaxation, marginalization, or reparameterization may be required to ensure differentiability [TG17]. Recall also that while MC-EIF operates models with finite parametrizations (Section 3), its capacity to handle nigh-dimensional nuisance parameters means can likely apply for example, function approximators that recover some of the value proposition offered by non-parametric model components [HSW89] Additionally, as discussed in Appendix [ infinite- dimensional models (like the Gaussian process) can often be reduced finite ones where MC-EIF can be applied. That said, future work is needed to fully explore the practical and empirical capabilities of MC-EIF in these settings, including how the polynomial complexity of Fisher information matrix inversions plays out in practice.
7 Conclusion
We have shown both theoretically and empirically that MC-EIF can reliably used to automate efficient estimation. Our key contributions include MC-HIF's consistency and capability to achieve optimal convergence rates. Empirical evidence shows that MC-EIF performs comparably to tradi- tional estimators using analytic EIFs. Additionally, we illustrate the practical application of MC-EIF in scenarios where the analytic EIF is not known. Given these contributions, there are many exciting areas of future work. For example, one may with to construct more powerful provably efficient estimators on top of MC-EIF (see Appendix(D () and explore the growing connection between semi- parametric theory and heuristic methods in deep learning [VACB22] BNL 22 DKSM21] [DJ 2 Additionally, there are many methods that could be used to accelerate the calculation of the Fisher information matrix, which computational bottleneck in MC-EIF. Given its foundational role in statistics, various techniques-sun as using Kronecker-factored approximations [GM16 -culdd improve efficiency without sacrificing performance.
10

---

Acknowledgments and Disclosure of Funding
The authors would like to thank DARPA for funding this work through the Automating Scientific Knowledge Extraction and Modeling (ASKEM) program, Agreement No. HR001 1262087. The views, opinions and/or findings expressed are those of the authors and should not be interpreted as representing the official views policies of the Department of Defense or the U.S. Government. The authors would also like to thank Tamara Broderick, David Burt, and Ryan Giordano for helpful discussions.
References
[Ama16] Shun-ichi Amari. Information geometry and its applications, volume 194. Springer, 2016.
[ARU20] Raj Agrawal, Uma Roy, and Caroline Uhler. Covariance matrix estimation under total positivity for portfolio selection. Journal Financial Econometrics, 20(2)367-389, 09 2020.
[BC 19] Eli Bingham, Jonathan P Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul Szerlip, Paul Horsfall, and Noah D Goodman. Pyro: Deep universal probabilistic programming. The Journal of Machine Learning Research, 20(1):973-978, 2019.
(BKB+93) Peter J Bickel, Chris AJ Klaassen, Peter Bickel, Ya' acov Ritov, J Klaassen, Jon A Wellner, and YA' Acov Ritov. Efficient and adaptive estimation for semiparametric models, volume 4. Springer, 1993.
[BKW23] Sivaraman Balakrishnan, Edward H Kennedy, and Larry Wasserman. The fundamental limits of structure-agnostic functional estimation. arXiv preprint arXiv:2305. 14116, 2023.
[2] Juhan Bae, Nathan Ng, Alston Lo, Marzyeh Ghassemi, and Roger B Grosse. If influence functions are the answer, then what is the question? Advances in Neural Information Processing Systems, 35:17953-17967 2022.
[BPRS18] Atilim Gunes Baydin, Barak Pearlmutter, Alexey Andreyevich Radul, and Jef- frey Mark Siskind. Automatic differentiation in machine learning: survey, Journal of Marchine Learning Research, 18: 18:1-43, 2018.
[BR88] P. J. Bickel and Y. Ritov. Estimating integrated squared density derivatives: Sharp best order of convergence estimates. Sankhyā: The Indian Journal of Statistics, 50(3):381-392 1988.
[BvdLA+22 Laura B Balzer, Mark van der Laan, James Ayieko, Moses Kamya, Gabriel Chamie, Joshua Schwab, Diane Havlir, and Maya L Petersen. Two-stage tmle to reduce bias and improve efficiency in cluster randomized trials. Biostatistics, 24(2):502-517, 2023.
[CCD+18] Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21(1):C1-668, 01 2018.
[CGMM23] Brian Cho, Kyra Gan, Ivana Malenica, and Yaroslav Mukhin. Kernel debiased plug-in estimation. arXiv preprint arXiv:2306.08598 2023.
[Cha20) Neng-Chieh Chang. Double/dehiased machine learning for differrence-in-diffeennes models. The Econometrics Journal, 1372):177-191, 2020.
[CLvdL19) Marco Carone, Alexander R. Luedtke, and Mark J. van der Laan. Toward computerized efficient estimation infiniteddimensionall models. Journal of the American Statistical Association, 1 114(527) 2019.

---

[CNQMS21] Victor Cheroozuukov, Whimey K Newey, Victor Guintas-Marting and Vasilis Syrgka- nis. Automatic debiased machine learning via neural nets for generalized linear regression. arXiv preprint arXiv:2 14737, 2021.
[CNQMS22] Victor Cheroozhuovv, Whitney Newey. Victor M Quintas-Martines and Vasilis Syrgkanis. Riesznet and forestriesz: Automatic debiased machine learning with neural nets and random forests. In International Conference on Machine Learning. pages 3901-3914, PMLR, 2022.
[CNS22] Victor Chemozhukov, Whitnyy Newey, and Rahul Singh. Automatic debiased machine learning of causal and structural effects. Econometrica, 90(3):967-1027, 2022.
[CTSLM19] Marco F Cusumano- Feras A Saad, Alexander K Lew, and Vikash K Mans- inghka. Gen: general-purpose probabilistic programming system with programmable inference. In Proceedings of the 40th acm sigplan conference on programuming lan- guage design and implementation, pages 221-236, 2019.
[DJS08] Arnak S. Dalalyan, Anatoly Juditsky, and Vladimir Spokoiny, A new algorithm for estimating the effective dimension-reduction subspace. Journal of Machine Learning Research, 9(53): 167-11677 2008.
[DKSM21] Tri Dao, Govinda M Kamath, Vasilis Syrgkanis, and Lester Mackey. Knowledge distillation as semiparametric inference. arXiv e-prints, pages arXiv-2104, 2021.
[DTA+22] Lauren Eyler Dang, Jens Magelund Tarp, Trine Julie Abrahamsen, Kajsa Kvist, John B Buse, Maya Petersen, and Mark van der Laan. A cross-validated targeted maximum likelihood estimator for data-adeptive experiment selection applied to the augmentation of rct control arms with external data. arXiv preprint urXiv:2210.05800 2022.
[FHL+22] Helmut Farbmacher, Martin Huber, Lukáš Lafférs, Henrika Langen, and Martin Spindler. Causal mediation analysis with double machine learning. The Econometrics Journal, 25(2):277-300, 2022.
[FQWD15] Constantine E Frangakis, Tianchen Qian, Zhenke Wu, and Ivan Diaz. Deductive deriva- tion and prriin-coopuueriizaton of semiparametric efficient estimation. Biometrics, 71(4):867-774 2015.
[FR22] R. A. Fisher and Edward John Russell. On the mathematical foundations of theoretical statistics. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of Mathematical or Physical Character, 222(594-604:3309-333 1922.
[FS23] Dylan J. Foster and Vasilis Syrgkanis. Orthogonal statistical learning. The Annals of Statistics, 51(3):879 908, 2023.
[GBA+23] Roger Grosse, Juhan Bae, Cem Anil, Nelson Elhage, Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Esin Durmus, Ethan Perez, et al. Studying large lan- guage model generalization with influence functions. arXiv preprint arXiv:2308.032900 2023.
[GLB+18] Thomas George, César Laurent, Xavier Bouthillier, Nicolas Ballas, and Pascal Vincent. Fast approximate natural gradient descent a kronecker factored eigenbasis. Advances in Neural Information Processing Systems, 31, 2018.
[GM16] Roger Grosse and James Martens. A Kronecker-factoree approximate fisher matrix for convolution layers. In International Conference on Machine Learning, pages 573-582, 2016.
[GSL+19] Ryan Giordano, William Stephenson, Runjing Liu, Michael Jordan, and Tamara Broderick. A swiss army infinitesimal jackknife. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 1139-1 1339-1147, 147, PMLR, 2019.
12

---

[HB91] Robert A. Haugen and Nardin L. Baker. The efficient market inefficiency of capitalization-weighhe weighted stock portfolios. Journal of Portfolio Management, 17:35-40, 1991.
[HDDOV22] Oliver Hines, Oliver Dukes, Karla Diaz-Ordar and Stijn Vansteelandt. Demystifying statistical learning based on efficient influence functions. The American Statistician, 76(3):292-304, 2022.
[Hill1] Jennifer L Hill. Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphical Statistics, 20(1):217-244 2011.
(HSW89) Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks are universal approximators. Neural Networks, 2(5):35--36,, 1989.
[HTW15] Trevor Hastie, Robert Tibshirani, and Martin Wainwright. Statistical Learning with Sparsity: The Lasso and Generalizations. Chapman Hall/CRC, 2015.
[IN22] Hidehiko Ichimura and Whitney K. Newey. The influence function of semiparametric estimators. Quantitative Economics, 13(1):29-46 61,, 2022.
[JP117] Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel- softmax, 2017.
[JM03] Ravi Jagannathan and Tongshu Ma. Risk reduction in large portfolios: Why imposing the wrong constraints helps. The Journal of Finance, 18(4):1651-1683 2003.
[TB21a] Yonghan Jung, Jin Tian, and Elias Bareinboim. Double machine learning density esti- mation for local treatment effects with instruments. Advances in Neural Information Processing Systems, 1221-218331311 2021.
[TT221b] Yonghan Jung, Jin Tian, and Elias Bareinboim. Estimating identifiable causal effects through double machine learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 12113-12122, 2021.
[JW722a] Michael Jordan, Yixin Wang, and Angela Zhou. Empirical gateaux derivatives for causal inference. Advances in Neural Information Processing Systems, 35:8512-8525. 2022.
[JW222b] Michael Jordan, Yixin Wang, and Angela Zhou. Data-driven influence functions for optimization- causal inference. arXiv preprint arXiv:2208. 13701, 2022.
[KBB+13] Noureddine Karoui, Derek Bean, Peter J. Bickel, Chinghway Lim, and Bin Yu. On robust regression with high-dimensional predictors. Proceedings of the National Academy of Sciences, 110036): 2013.
[Ken16] Edward H. Kennedy. Semiparametric Theory and Empirical Processes in Causal Inference, pages 141-167. Springer International Publishing, 2016.
[Ken22] Edward H Kennedy. Semiparametric doubly robust targeted double machine learning: review. arXiv preprint arXiv:2203.064669 2022.
[KHB19] Frederik Kunstner, Philipp Hennig, and Lukas Balles. Limitations of the empirical fisher approximation for natural gradient descent. Advances in neural information processing systems, 32, 2019.
[KK21] Zeljko Kereta and Timo Klock. Estimating covariance and precision matrices along subspaces. Electronic Journal of Statistics, 15(1):554-533 2021.
15) Kirthevasan Kandasamy, Akshay Krishnamurthy, Barnabas Poczos, Larry Wasserman, et Nonparametric von mises estimators for entropies, divergences and mutual informations. Advances in Neural Information Processing Systems, 28, 2015.
[KL17] Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. International conference on machine learning, pages 1885-1894. PMLR, 2017.
13

---

[KW14] Diederik P. Kingma and Max Welling. Auto-encodin variational Bayes. In Interna- tional Conference on Learning Representations, 2014.
[Law86] John Law. Robust statistics-thhe approach based on influence functions, 1986.
[LHSM23] Alexander Lew, Mathieu Huot, Sam Staton, and Vikash K Mansinghka. Adev: Sound automatic differentiation of expected values of probabilistic programs. Pro- ceedings of the ACM on Programming Languages, 7(POPL):121-153, 2023.
[LHvdL23] Haodong Li, Alan Hubbard, and Mark van der Laan. Targeted learning on variable im- portance measure for heterogeneous treatment effect. arXiv preprint arXiv:2309.13322 2023.
[Mar52] Harry Markowitz. Portfolio selection. The Journal of Finance, 7(1):77-91, 1952.
[Pea94] Barak A. Pearlmutter. Fast exact multiplication by the hessian. Neural Computation, 6(1):147-160, 1994.
[PGC*17] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch, 2017.
[Rao45] Calyampudi Radhakrishna Rao. Information and the accuracy attainable in the es- timation of statistical parameters. Bulletin of the Calcutta Mathematical Society, 37(3):81-9) 1945.
[RCY+18 18] Tom Rainforth, Rob Cornish, Hongscok Yang, Andrew Warrington, and Frank Wood. On nesting monte carlo estimators. In International Conference on Machine Learning, pages 4267-4276. PMLR, 2018.
[REvdL223] Helene CW Rytgaard, Frank Eriksson, and Mark van der Laan. Estimation of time-specific intervention effects on continuously distributed time-to-event outcomes by targeted maximum likelihood estimation. Biometrics, 79(4):3038-304) 2023.
[RGvdL22] Helene C Rytgaard, Thomas Gerds, and Mark van der Laan. Continuous- targeted minimum loss-based estimation of intervention-specinn mean outcomes. The Annals of Statistics, 50(5):2469-24 2491, 2022.
[RLT James Robins, Lingling Li, Eric Tchetgen, Aad van der Vaart, et al. Higher order influence functions and minimax estimation of nonlinear functionals. In Probubility and statistics: essays honor of David A. Freedman, volume 2, pages 335-422 Institute of Mathematical Statistics, 2008.
[RvdL24] Helene CW Rytgaard and Mark J van der Laan. Targeted maximum likelihood estimation for causal inference in survival and competing risks analysis. Lifetime Data Analysis, 30(1 )33 2024.
[SHWA15] John Schulman, Nicolas Heess, Theophane Weber, and Pieter Abbeel. Gradient estimation using stochastic computation graphs. In Neural Information Processing Systems, 2015.
[SW23] Yasa Syed and Guanyang Wang. Optimal randomized multilevel monte carlo for repeatedly nested expectations. arXiv preprint arXiv:2301.04009 04095, 2023.
[TR19] Da Tang and Rajesh Ranganath. The variational predictive natural gradient. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference Machine Learning, volume 97 of Proceedings Machine Learning Research, pages 6145-6154, PMLR, 09-15 Jun 2019.
[Tsi06] Anastasios A Tsiatis. Semiparametric theory and missing data. Springer, 2006.
[VACB22] Matthew I Vowels, Sina Akbari, Necati Cihan Camgoz, and Richard Bowden. A free lunch with influence functions? improving neural network estimates with concepts from semiparametric statistics. arXiv preprint
2022.
14

---

[VDLR06] Mark J Van Der Laan and Daniel Rubin. Targeted maximum likelihood learning. The international journal of biostatistics, 2(1), 2006.
[vdV98] A. W. van der Vaart. Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 1998.
[vdV14] Aad van der Vaart. Higher order tangent spaces and influence functions. Statistical Science, pages 679-686, 2014.
[Wai19] Martin (Martin J.) Wainwright. Iligh-dimensional statistics non-asymplotic viewpoint. Cambridge University Press, 2019.
[WPG+19] Wang, Geoff Pleiss, Jacob Gardner, Stephen Tyree, Kilian Q Weinberger, and Andrew Gordon Wilson. Exact gaussian processes on a million data points. Advances in neural information processing systems, 32, 2019.
[WPvdL+23] Waverly Wei, Maya Petersen, Mark van der Laan, Zeyu Zheng, Chong Wu, and Jingshen Wang. Efficient targeted learning of heterogeneous treatment effects for multiple subgroups. Biometrics, 79(3):1934-1944 2023.
[WvdLP+23] Zeyi Wang, Lars der Laan, Maya Petersen, Thomas Gerds, Kajsa Kvist, and Mark van der Laan. Targeted maximum likelihood based estimation for longitudinal mediation analysis. arXiv preprint arXiv:2304.c 204.024004.04944 044944 2023.
[3D]+23] Banghua Zhu, Mingyu Ding, Philip Jacobson, Ming Wu, Wei Zhan, Michael Jordan, and Jiantao Jiao. Doubly robust self-training. arXiv preprint arXiv:2306.00266 2023.
[ZHSJ20] Jingzhao Zhang, Tranxing He, Suvrit Sra, and Ali Jadbabaic. Why gradient clip- ping accelerates training: A theoretical justification for adaptivity. In International Conference on Learning Representations, 2020.
A Proofs
A.1 Proof of Theorem 3.8
Proof. Let [ denote the M Monte Carlo samples in Equation 3 and let Im = VD V log (2m) for I s M. Let E m 2m=1 Zm denote the sample covariance matrix. Then, E = ((0) and E [(d). Hence,
V log
V PogP)((")
(p((x) - Po,M (0) =
V Vorna M (0)) (0) 1-1)
+ d E-1
D
D
V log P 4 (xx))))))))))))))
V log (
< V S (0)) 1
+ SM
D
D
(10)
V log P (x*)
V Vorn (0)) (E* - )
+ 1 Csl
D
V log P
(xx)
< v (E) - 1
+ CaAman ( 111's11rlllr
D
By Assumption 3.7
1881/1
V
E with probability greater than 1 1- exp(-e) when M > Cz. If
M7
we can prove that there exists a constant C4
( 1- log"'("")
p ploop+ +
S C.Amaax ( (E:))
(11)
M
with probability greater than 2 exp(-e), the claim follows by application of the union bound. By Theorem 10 in [KK21] the claim follows we can prove that there exists a universal constant C₁ such that
V log P. (x") T
VovAr(o)
< <Cddmaxx2" (E
(12)
D
4/
15

---

with probability greater than 1 - exp( exp(-e), where || ll,s, denotes the Orlicz mub-Sausiian norm: see Equation in [K22]] for a precise definition the Orlicz norm of random vector. With probability greater than - exp( -), | ( < C2 by Assumption 3.7 Hence, with probability greater than exp(-t),
V log Po Pa(x*)
VORI 1
C2 |0
2/2
(13)
CiagC3ammm (2
where the first inequality follows from Assumption 3.6 the third by [DIS08] and Assumption 3.5 and last by the definition of the spectral norm of a matrix. The result now follows by setting C1 CCCC3.
Assumption 3.7 Holds for Monte Carlo Estimators. Here we show if dM is also approximated with M Monte Carlo samples, then Assumption 3.7 holds. To this end, suppose
M
(
(ogo(um)
Wm
(w)) I m M, s.t. E ([Vos((um) V (o)()))
M
(14) for some distribution q(w) and function 9G. Such decomposition exists, for example, when the functional is expressible as a stochastic computation graph (SHWAI5] or for reparameterizable densities [KW14). Suppose further that there exists universal constant such that V (0 E R. a sub-Gaussian random vector with parameter as for S s p. Then,
||
(I(V 20 1VV vi(wm))l)
1=1
(15)
S VpL
[[ (Vo,go((um)) (V (wm))
max 1SISL1SSSP
By Exercise 2.12 in Wail9],
[[(V (VVogg(umll ((V,, (w Op ( σ
log(pL)
max IILLSS
(16)
M
(Vologtoo) Thus, the first equation in Assumption 3.7
Hence, since L is constant, || =0p
Oz
holds. Under Assumption 3.6, the second equation in Assumption 3.7 trivially holds using gradient clipping with C2.
A.2 Proof of Theorem 4.1
Proof. We want to prove that difference between the analytic one step estimator and Algorithm ! with finite M decays a Op (+)))
rate. Their difference equals
N
2
Σ
-
maa N/241,
(p(((n)
G8
(17)
N
N/241
Let c > O and suppose =² max Cs, 1) ()(pog(() + max 1 max
(Σ OplooppN)
where C is defined in Assumption 3.5 and Cs and E- are defined in Theorem
3.8 If
P"
max n=N/2+1,""
.N (pg(zn) - Φ (Z
>
< exp(-e)
(18)
N
16

---

holds, then the proof is complete since M grows faster than M'. Let = + log(N/2). By Theorem 3.8 and the union bound.
p plog(p) + EN
p
(P(lan) - (o)ar((a)) CaAmax
num
1,",
M'
N
(Pea(onn) - -aa) Ca
( (E-t))/ plog(p) EN
Σ
P
2
M'
(19)
n=N/2+1
< N/2 2/2expp--EN)
= exp( -EN + log N/2)
exp(-c)
Now,
plog(p) +EN M'
plog(p) + + log(N/2)
C4dmax (ammxx(--1) (2
C4Amax (21)
max max 1) + <) (aax((() 1)N Max (Σ-1)
plog(p) + log(N/2) (p log((p) + N
2
(20)
The proof now follows from Equation
19 and Equation
20
A.3 Proof of Theorem
Lemma A.1. Suppose Assumption 4.2 holds. Then, dra 2 Em=N/2+1 m(In, (Pp))
ΣA (In), where pa(x:) is the influence function associated with the functional v0))
E, Ea ((P)),
Proof. We claim po(a, 0) = Po(x) for all 0. To prove this claim, notice that V (o) V Volur(d (Ø) for arbitrary A and 6' since (o( = Ez-Pa [m( nfP ( 6. Hence, the claim follows from Equation g Now,
N
N
2
[a(an. n(p3), 0) + ggglnn 0)]
Σ
m(zn. n(z)) + pz(En) - 8.
(21)
N
N
N=N/2+1
Hence, PDML N m(xn, (Pp))
EN (x))))
Proof of Proposition By Lemma A.I Algorithm 2 uses the same correction term C/ in Algo- rithm Hence, the proof of Proposition now follows from Proposition 4.1
Remark A.2. By Lemma
the only difference between Algorithm and Algorithm different value for the initial estimate of 0* Specifically, in DML, the initial estimate of 8" is
N 2 D N=241 m(nn, n(pa)), which averages over datapoints drawn from the distribution. By
contrast, Algorithm uses iplugiin which averages datapoints simulated from Pa-
B
Code Examples
B.1
Automatically Differentiable Functionals
The implementation of differentiable functional approximations is fairly straightforward when using modern autodifferentation tools. For example, the squared density functional for mean-zero, univariate normal can be approximated using Monte Carlo as follows:
17

---

N N (2); on ~ N (0,00)
(22)
This can implemented pytorch PPGC+ and thus automatically differentiated with respect to (called "scale" in the code block below) using, for example, torch. autograu.rrd grad.
Listing 1: Automatically Differentiable Monte Carlo Approximation of Integrated Squared Normal Density
import torch
def diffable_ ww__mm_inn snteg_squared, density (scale: torch. Tensor, monte_carlop int): assert scale requires_gas Sample from the density
samples torch. distributions. Normal scale sample (umum monte__crlo Evaluate those samples under the density.
logprobs torch. distributions. NNrmmll scale log-prob (samples) Return the mean pdf value in numerically stable way.
return torch exp(torch logsumexp (logprobs dim=0)) torch numel (amplees)
C So, What's Automatic?
Derivation
Considerations for Numerical Approximation
Implementation Computer Code
(or approximation)
Analytic
Straightforward
Non Par-marrmtticc
IF
Empirical Gauteaux
Straightfowaard
Manual
Manual
Ours
Efficient IF
Automated
Automated
Automated
requires V
Figure 4: We taxonomize the workflow of robust estimation into three stages: the derivation of an (approximate and/or efficient) influence function, numerical derivation and analysis required for its computation, and the code required to compute it. For the analytic workflow, the derivation of the IF results Equation 24 This largely involves terms already required by the original plug-in (Equation 23 but still must be implemented on case-by-case basis in code. For the "Empirical Gateaux" workflow, the first stage requires only the general purpose Equation 25 but demands case-specific numerical considerations and derivations like the one shown in Equation 26 In stark contrast, given a differentiable approximation to the functional interest, MC-EIF "automates" each stage through use of an end-to-end, general purpose solution.
The work required to perform robust estimation can be subdivided into few key steps. The process begins with functional of interest, V. With this functional in hand, analyst must first derive the influence function (or an approximation thereof), and consider any nuances in numerically approximating that quantity. Finally, an engineer must implement that approximation code. Different approaches boast varying levels of "automation" for each step. We claim that in problems where our conditions hold (as outlined in Section 3.2.. MC-EIF provides end-to-end automation via general- purpose solution at each stage. Here, we contrast our approach with both general-purpose
executable
the analytic (see e.g. (6) and "Empirical Gateaux" workflows [JWZ22b] CLvdL19). We will track workflow's 's "products" at each of the three stages: first, the derivation of the (efficient and/or approximate) influence function; second, a tractable version the influence function that properly considers its numerical nuances; third, executable code that computes the influence function. Because our approach uses a general purpose formulation for each of these three stages, we call our approach "automated."
18

---

We assume the workflow starts having identified functional of interest and having implemented, in code. a plug-in estimator for Throughout, we will use the mean-potential outcome (MPE) functional as our working example
/ XXAA D p(A= 1, / p (x) dydr
- X,A
(23)
C.1 Analytic Workflow
The analytic workflow begins by deriving closed form influence function a challenging task even for seasoned experts. This first stage culminates the following analytic influence function for the MPE [6
I) 6 (0; P(A= ([A=1|X) (Y Ey |X,
X, A= 1]} + [Y X,A= v (P)
(24)
For general functionals, the derivation resulting in Equation 24 challenging, even for experts but given such a derivation, it is often the case that the computation of the quantities composing it can share code and numerical considerations developed to estimate the original. plug-in functional (e.g. Equation 23). Indeed, the influence function of the MPE (Equation 24) involves only terms that an analyst has already considered and implemented for the plug-in in (Equation 23). For this reason, we say that in the "analytic" workflow, most of the labor must be allocated to deriving the influence function tractable, well behaved computation of that influence function tends involve straightforward extensions of tooling and analysis that already exists for the plug-in.
The last stage the implementation of that tooling in computer code, which will always require some work on case-by-case basis.
C.2 Empirical Gateaux Workflow
The workflow presented by (WZ22bb and [CyvL199 significantly reduces the resources required in the first stage the derivation of the influence function - by providing a general purpose, finiteddifference approximation to the influence function (Equation 25p. PE,.. here, represents perturbation of the estimated distribution P of size in the direction of X-smooth kernel centered at observation O
(0::) =
1 (w (P%) (P))
(25)
At first glance, it seems that computing this term would follow easily given a general purpose framework for the perturbation of P. and then applying the plug-in functional to that perturbed density. Unfortunately, computing V/ (Po)) presents a number of numerical challenges in practice.
As exhibited in Figure (which echoes figure 1 in the work by [CLvdL19]), selecting appropriate perturbation parameters 6 and À priori is challenging, and a battle-tested framework for doing so has not yet been developed. Further, in high dimensions (where MC-EIF excels), the required can be so small as to quickly overrun floating point accuracy on modern computers when even Rs 10 [CLvdL19. Indeed, [WZ22bb] have explicitly left thorough numerical analysis of this approach to further work. footnote 7, they anecdotally report that quadrature methods were overly sensitive in evaluating perturbed densities in the MPE functional, and instead present Monte Carlo approach tailored to the task. Unfortunately, neither the numeric considerations or pode-inplemennatioonn the plug-in estimator easily translate when computing the plug-in with respect to the perturbed data distribution. Below, we show their numeric approximation of Equation 25 for the MPF, where observation comprises (x., a, y). the perturbation kernel K has bandwidth X, and they N Monte Carlo samples from a uniform kernel over confounder r.
of
For simplicity in exposition, but without loss of generality in regards this description of what is and isn't automated, we follow [JW222b] assuming that outcome and confounders are continuous real numbers, while the treatment is binary.
*This is their equation 73 in appendix E.3.
19

---

(1-c) (E (iiA,=KK(X, - -x)Y)) P +) 11
GAx (0)
1 N
Σ
(1-e)p(A = 1 + (a= =
(26)
1
p( Pr = (A=
|at (y E;|Y|A 1,5x]}
Indeed, just like finite differencing can simplify multivariate calculus, but introduce numeric chal- lenges, the empirical- gateaux approach makes variational calculus easier, but introduces numeric challenges. In sum, we consider the second. "numerical," stage of this workflow to be both labor and expertise intensive, even when the curse of dimensionality does not render moot.
Like the analytic workflow, the "coding" stage of course requires case-by-case implementations. Moreover, added numerical challenges here introduce significant nuance in implementation that isn sn't present in the analytic case.
C.3 Our Workflow
In stark contrast, our workflow exploits general solutions all three stages for the "price" of differentiability of parametric plug-in estimator. When our general conditions are met (as outlined provides the general purpose solution to first stage of deriving an (approximate, efficient) influence function, and the second stage is achieved with the Monte Carlo approximation in Equation
in Section 3) Equation [
The third stage is met in software implementing this general purpose solution that operates on functional implementations using one of many intt-diiferrnntitiion tools now ubiquitous machine learning (see Appendix B.I for a simple example). As discussed at the end of Section [3. this sometimes requires the ability to exploit methods like the reparameterization trick for the functional of interest. In many cases, modern automatic differentiation software makes this trivial (as shown in Appendix B.I]. For some functionals, however, like those involving inner optimizations, this may be more challenging.
This general purpose approximation underpins the end-to-end automation of MC-EIF, and is the best of our knowledge the only such general purpose approximation for efficient influence functions.
D Towards an EIF Cookbook
As generalization of the gradient operator ordinary functions, the EIF viewed operator functionals can be shown to have a number of convenient algebraic properties [Ken 16, Ken22), many of which are inherited directly by the MC-EIF estimator. In this section, speculate on several ways in which these properties could be used to extend the basic MC-EIF framework (and the MC-EIF. based robust estimators in Section to new classes of models and functionals, significantly increasing the range of practical use cases addressable by an implementation MC-EIF in a differentiable probabilistic programming language like Pyro BCJ 19
Multi-argument functionals Many important quantities statistics and machine learning, like the mutual information T[X; Y| or KI -ivviergee KT[[P; Q] are functionals of more than one probability distribution. As shown in 15)++15) we can define partial EIFs analogous to partial derivatives for these quantities (which can then be plugged into the efficient estimators of Section by treating all but one argument as part of the functional and computing the ordinary EIF with respect to that argument.
Higher-order EIFs Although all of the efficient estimators of Section are derived from the first-order EIF. there are some circumstances where incorporating higher-order EIFs can be shown to theoretically necessary for achieving certain statistical properties [RLT+ BK33.. Just as ordinary higher-order derívatives are computed by recursively applying a first-order derivative operator its output, higher-order EIFs can be computed by recursively applying first-order EIF operator to its own output [vdV14), property straightforwardly inherited by MC-EIR.
20

---

Models with latent variables Thus far, we have assumed that we can exactly simulate from model predictive distributions ~ Ps. and compute log-densities log Po (z), score functions V, log Pa (x) and Hessian-vecte products. However, our MC-EIF estimator can be extended straighfforwadly to models with latent variables and intractable densities and score functions by using a nested Monte Carlo procedure [RCY+ 18 [W233 to approximate the prior predictive or posterior predictive distributions and plugging the resulting stochastic estimates into the vanilla MC-EIF framework. We expect our theoretical results to extend to this case provided the approximation error can be made small relative to the Monte Carlo error in estimating the Fisher matrix from finite set of model Monte Carlo samples.
diiesiond models and targets Semiparametric statistics by definition fundamentally concerned with models that contain infiniteddimenionnt (i.e. dunction-valuedd components. There also intense interest in deriving efficient, doubly robust estimators for infiniteddimensional target functionals like the conditional average treatment effect (CATE) in causal inference. Fortunately, in many of these settings the infiniteddimensional quantities can be reduced to finite ones (and ultimately must be to be representable on a digital computer) to which MC-EIF may be applied in straightforward way. For example, Gaussian process is fully characterized by the latent function's 's values finite set of test points; computing the EIF for functional of the GP reduces to computing the EIF of the mnite-dimensional joint distribution on function values at the test points, which is straightforward to estimate with MC-EIF. Similarly, in the case of the CATE, we are ultimately interested in the values of the CATE function on a finite set of test inputs, reducing the problem to ordinary MC-EIF for finite-dmeensionl target functional.
E
Additional Experiment Details
E.1
Model Details
In Section
5
we consider the following model with confounders c, treatment t, and response y:
No ~ ~N,
(intercept)
E N~N
1F
(outcome weights)
0,
IF
(propensity weights)
VF
(27)
NO0, 1), On ~ NO, ID), (confounders)
(treatment weight)
tn Cn, ~ Bermouli(logits = =x =,,,), (treatment assignment) In M(tt, + ET + Mo; 1), (response)
R,,, where D = F F 2 and á (Ao, &. ,T,T) T) E R2F+2 To obtain point estimate in Section 5 we take the maximum posteriori estimate. In Section we vary the model dimension p by varying since p 2F + 2.
where F N denotes the number of confounders. In this example, x (c. t, y)
E.2 Portfolio optimization details
We assume æ drawn from multivariate Gaussian distribution with unknown covariance matrix for D 25 and N = 1000 datapoints. We randomly sample the true covariance matrix using Lewandowski-Kel -uwiiskiiikowwiaaa--aaa
distribution on positive definite matrices. We evaluate MC-EIF and the one step estimator using the relative expected volatility (REV) and the root mean- seeun--quaee--rree (RMSE) between the estimated and the true optimal portfolio weights. Here, the expected volatility calculated by applying the estimated weights with the actual covariance to the objective in Equation
9 Repeating our experiment using randomly generated datasets, we find that MC-EIF enables substantially improved estimates, shown in Table
21

---

E.3
Additional Figures
Here, we provide experimental results that provide interesting insight, but do not directly support the key claims of our paper.
One-Step
DoubieML
Plug
Ground Truth
Density
ATE Estimate
Figure 5: Comparison of plug-in estimator and efficient estimators using MC-EIF and analytic EIF for estimating ATE with synthetic data. The true ATE O. Closer to zero the better. The distribution is over 100 simulated datasets. Dashed lines represent the estimates using the analytic EIF, and the solid lines represent using MC-EIF (when applicable). Given the high-dimensionality of the problem, the estimation leads to non-zero centering (i.e., some bias remains even after influence function based corrections). Importantly, this is property of the influence corrected estimators, and not artifact introduced by MC-EIF. Instead, chose empirical study demonstrate that MC-EIF produces near-identical results for a diversity of statistical tasks, with a diversity of statistical implications.
22

---

One Step
DoubleML
Plug-in
Ground Truth
Density
34 366
38
4.0
42 44
ATE Estimate
Figure 6: Comparison of plug-in estimator and efficient estimators using MC-EIF and analytic ETF for estimating ATE with real data. Here, we use the Infant Health and Development Program semi-synthetic data Нi commonly used for effect estimation benchmarking with the same causal GLM as our synthetic data experiments. Here, MC-EIF produces estimates that are closely aligned with the analytic EIF estimators and, in the case of double machine learning, produce estimates that are much closer to the true ATE, Again, we emphasize that the choice of influence corrected estimator separable from the choice of how to estimate the efficient influence function, which is focus in this work.
0.00
Influence Function at Ir
0.25
-0.50
-0.75
-1.00
Efficient Influence (Known SD)
1.25
Efficient Influence (Unknown SD)
Nonparametric Influence
1.50
2
3
Value of
Figure 7: Nonparametric and efficient influence functions for expected density.
23

---

MC-EIF (10k Monte Carlo samples)
Model Fitting Time
Runtime (s)
15
10
200
400
600
800
1000
Model Dimension (p)
Figure Runtime of fitting point estimate and computing MC-EIF as function of model size.
Median Relative Error
-0
10'
10
10
Number of Monte Carlo EIF Samples
Figure 9: Median relative between MC-EIF and true efficient influence function for unknown variance model and expected density functional. Median absolute error computed by randomly sampling points to evaluate EIF, computing the relative error each point, and then taking the median.
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Our abstract and introduction clearly state the claims, contributions, and concise description of assumptions. We also discuss the evidence for these claims, which is supported later in the paper. Specifically, we discuss MC-EIF's asymptotic theoretical guarantees, as well the empirical study on finite data a variety of case studies.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We clearly describe the limitation of our work both by; (i) clearly stating the assumptions behind the theoretical results, and (ii) emphasizing throughout that MC-EIF produces approximate evaluations of the true EIF We discuss the computational efficiency of the approach in detail, and provide explicit bounds on its accuracy.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model vell-specifiiattin, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
25

---

3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: We provide an appropriately formal description of each assumption and the theorem statements in the main body of the paper, and provide proofs in the appendix. We also provide some insight into why the theorems hold throughout.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We clearly explain our experiments in Section and provide hyperparameter configurations in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
26

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide documented implementation with substantial unit tests, well as code to reproduce experiments, and a README.md.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cct
Srriides/CoodSubmissstt
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) is recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We clearly explain our experiments in Section and provide hyperparameter configurations Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: the figures we either show the entire distribution (Figure or provide standard errors (see Table showcase the uncertainty of presented results.
Guidelines:
27

---

The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes].
Justification: All experiments were run on an Apple M2 pro. In Figure we plot the runtime of our method under various conditions.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: We have reviewed the NeurIPS Code of Ethics and confirm that our practices conform to them.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
28

---

Justification: The research presented in this submission does not represent societal impacts except for those shared by all fundamental research. Our work represents general method- ological progress for estimating statistical quantities using modern automatic differentiation and probabilistic programming systems.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negatíve societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Not applicable
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification: Not applicable.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
29

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperssi thcode com/ com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's ' creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Not applicable.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the datasct/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Not applicable.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Not applicable.
Guidelines:
30

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
31


Reviews:
[{"review_id": "DoMHCrqCbP", "reviewer": "Reviewer_ebo4", "summary": "The paper proposes Monte Carlo Efficient Influence Functions (MC-EIF), an automated technique for numerically computing efficient influence functions using existing differentiable probabilistic programming systems. MC-EIF simplifies efficient statistical estimation for high-dimensional models, achieving optimal convergence rates and consistency without the need for complex manual derivations. The approach is validated both theoretically and empirically.", "questions": "Q1. In Assumption 3.5, authors assume that the normalized score vector is sub-Gaussian with a parameter $C_1$. I want to know if this constant scales with respect to the dimension $D$ and model size $p$. If yes, what is the scaling? If no, can you clarify why it does not scale?\n\nQ2. The authors assume that the map $\\phi$ to $P_{\\phi}$ is continuous. My question is, in order to approximate the Fisher information $\\hat{I_{m}}$, do we need to know what $P_{\\phi}$ actually is?\n\n\nQ3. The authors back up their theoretical results with synthetic data experiments. Given the assumptions, it is not clear how applicable the proposed method is to real datasets, and how should the results be interpreted if these assumptions don't hold?\"", "limitations": "The authors clearly mention the assumptions before stating their theoretical guarantees. Also, the assumptions are explained clearly. I don't see any potential negative societal impact of their work.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well-written and can be followed easily. The authors introduce Monte Carlo Efficient Influence Functions (MC-EIF), a technique for numerically computing EIFs using existing AD and PPL system quantities. They express EIFs as a product of the gradient of the functional, the inverse Fisher information matrix, and the gradient of the log-likelihood. This method automates the construction of efficient estimators, avoiding manual derivations, and provides accurate, generalizable estimates applicable to various functionals and models. They also provide a non-asymptotic error bound on the quality of their approximation, showing how estimators using MC-EIF achieve the same asymptotic guarantees as using analytic EIFs. Empirical results show MC-EIF outperforms existing approaches without degrading estimation accuracy.", "weaknesses": "I am not fully familiar with this line of work, so I am unable to identify a major weakness. However, I have some questions regarding the assumptions  and the effectiveness of the proposed approach on real datasets, which I have added to the Question sections.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are very grateful to receive such positive feedback on our submission’s exposition, and empirical and theoretical evidence. We are also very grateful to receive actionable feedback we can use to further improve our work.\n\n**(1) Assumption 3.5**. The constant C will not grow with D in Assumption 3.5. If there exists a universal constant k such p <= kD, then C will not grow with p either but this is not a necessary assumption. We will include this discussion in the revision but for completeness we provide a quick sketch below: \n\nThe score vector $\\nabla \\log P_{\\phi}(x)$ is a p-dimensional vector so its expected squared norm grows linearly with $p$. The expected squared norm of the normalized score $\\frac{1}{\\sqrt{D}} \\nabla \\log P_{\\phi}(x)$ scales as $O((\\frac{1}{\\sqrt{D}})^2 \\times p) = O(p/D)$. Hence, if $p \\leq kD$, then the expected squared norm is bounded. Hence, the sub-Gaussian constant in Assumption 3.5 for the normalized score will not grow with p or D. \n\n**(2) Continuity assumption**. MC-EIF addresses the problem of efficient estimation with parametric models using efficient influence functions. Here, $P_{\\phi}$ is that parametric model itself, and is known by construction for any fixed set of parameters $\\phi$. For example, if we wish to use MC-EIF to construct an efficient estimator for a linear regression model with Gaussian errors, then $\\phi$ would be the fitted regression coefficients and $P_{\\phi}$ is the induced Gaussian distribution over outcomes. We will add a note clarifying this in the revision.\n\n**(3) Real data**. Since submission, we received other feedback to add real data experiments so we evaluated MC-EIF on real data from the UCI machine learning repository. We find that the approximation quality is similar to the good performance on synthetic data in Section 5 of our paper! This is not surprising, as previous work on efficient estimation has already demonstrated the benefit of influence function based estimation with real data, and MC-EIF closely approximates the efficient influence function.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mzP4dfEGd9", "reviewer": "Reviewer_cWd8", "summary": "The paper establishes a novel method for estimating the efficient influence functions under mild assumptions, called Monte Carlo Efficient Influence Functions(MC-EIF). The method is easy to apply and flexible in many cases, where it can seamlessly equip an existing/popular efficient estimator.", "questions": "It draws my attention that, from Assumption 3.7 and Theorem 3.8, as the model size $p$ becomes larger, the constant of the bound of error will increase as $\\sqrt{p \\log p}$. So, when dealing with the high-dimensional problem, it may require people to use a really large number of samples ($M$) to get a desirable accuracy and the cost will be tremendous, which could be a potential drawback of this method, what do you think?", "limitations": "I am not sure what is the $p$ tested here, but from Figure 2, it seems like $p_{\\max}$ is only $1000$, which is not a very large model. People should test on much higher dimensional problems to defend their methods. Moreover, it would be worth looking at more challenging problems, other than well-defined Gaussian problems.", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "It is a really well-written paper with complete model establishment, theoretical results and fair comparisons between their MC-EIF method and other existing methods. Details are listed, like the different ways to generate the estimator and how it is sensitive to their methods and also the limitations on the dimension of model size are discussed. This MC-EIF is easy and flexible to use in many scenarios, so it gets good prospects in the application area.", "weaknesses": "When applying new methods in practice, especially in high-dimensional cases, the time cost is also a necessary aspect that needs to be considered, so it would be good to show the time cost for your methods and how it compares to other existing methods.\n\nSome settings in the experiments are not listed clearly, like what is the value of $D$ and $p$ for experiments in Figure 1,2,3 or provided the value of $F$ instead?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are very grateful to receive such positive feedback on our submission’s exposition, the quality of our theoretical and empirical evidence, and MC-EIF’s generality and ease-of-use. Thank you also for helping us to improve our work with actionable feedback.\n\n**(1) Time comparison relative to other methods**. Before MC-EIF, the only general-purpose method for approximating efficient influence functions was provided by JWZ22a, who use a finite-differencing approximation. This runtime of this method is exponential with respect to dimension p with a large constant. The primary benefit of MC-EIF over these existing techniques is its substantially faster runtime. In fact, in our experiments, we could not even compare against the method in JWZ22a because the runtime was too slow for p larger than 2. That is why we only compare against JWZ22a in experiment 1 and not the other experiments where p is in the hundreds or a thousand. In Figure 7 in the Appendix we provide the runtime of our method as a function of p. We will emphasize how MC-EIF is faster than existing approaches much more prominently in the revision.\n\n**(2) Missing labels**. We will add these missing labels in the revised submission and make the choices clearer in the main body. In Figure 1(a) D=1 and p=1 (only unknown mean parameter). In Figure 1(b), D=1 and p=2 (unknown mean and variance parameter). In Figure 2, F=200, D=202 and p=402. In Figure 3, D varies from 1 to 502 and p is shown on the y-axis.\n\n**(3) Approximation quality for large p**. The reviewer is correct that the error bound grows like $\\sqrt{p \\log p}$. To ensure an accurate error bound, that means the number of Monte Carlo samples $M$ must scale linearly with $p \\log p$. In practice, the runtime is not prohibitively slow. For example, in Figure 7 in the Appendix, we compare the runtime of fitting the model relative to computing MC-EIF as a function of model size. We see that when $p$ is less than 600, MC-EIF is faster than the time to fit the model. When $p$ is 1,000 then our implementation of MC-EIF takes 20 seconds to run on a standard M2 laptop. \n\nThe computational bottleneck of MC-EIF is computing the Fisher information matrix. Fortunately, given the importance of the Fisher information matrix in statistics, there are many ways to speed up computations here. See, for example, “A Kronecker-factored approximate Fisher matrix for convolution layers”. We will include a discussion of how these approximations can be used with MC-EIF in the paper. \n\n**(4) Larger p problems / real data**. Relative to existing work, we evaluated our method on much larger p problems. For example, in JWZ22a, the authors only evaluated on settings with p=1, as their method’s runtime scales exponentially in p. Nevertheless, we can certainly evaluate our method on a larger p problem in the revised submission. Since submission, we have evaluated MC-EIF on real, non-Gaussian data and found similar empirical results on synthetic data in Section 5 of our paper. We will include these results in the revised submission.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uQVnO9IHVe", "reviewer": "Reviewer_hhJt", "summary": "Efficient influence functions (EIFs) for nonparametric estimands are used to construct debiased estimators. Existing methods are primarily estimand-specific and require intricate analytic derivations; existing automated methods don't scale well. This paper proposes general Monte-Carlo estimators of the efficient influence function (MC-EIF) that to be used within a probabilistic programming workflow, shows their convergence to the true efficient influence function, as well as convergence of some estimates based on MC-EIF to the true value.\n\nDisclaimer: I had little familiarity with the use of influence functions for constructing estimators before reading this paper.", "questions": "* The maximum eigenvalue term in Theorem 3.8 may grow in N. Is there an assumptions making sure it does not?", "limitations": "* Authors do not discuss if there is value for lifting the assumptions of the method.", "rating": 7, "confidence": 1, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The motivation for the paper is clear, and the developed mechanism is sound, and the convergence results are convincing. The extensions of the use of EIF for estimation that are opened with the proposal of the estimator are intriguing.", "weaknesses": "* I found the paper was harder to understand than was necessary due to unclear notation. In the Problem Statement, P and Q are used interchangably to mean the pdf or the probability measure (e.g. in Definition 2, it's a measure with respect to which the L2 space is defined, and a function in said L2 space).\n* The Assumptions (esp 3.1-3.3) are not easily interpretable and are not discussed; as the paper proposes a practical method, it would be made stronger by a discussion of the assumptions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are very grateful to receive such positive feedback that our submission is well motivated, theoretically sound, the evidence is convincing, and that MC-EIF opens up future interesting directions. We also appreciate the actionable feedback.\n\n**(1) Notation**. We thank the reviewer for flagging this issue. To improve clarity, we will carefully clean up notation, including not using P and Q interchangeably to mean the pdf or the probability measure.\n\n**(2) Clarity of Assumptions**. We thank the reviewer for this honest feedback. We will include a more extensive discussion of assumptions in the Appendix, and clarify Assumptions 3.1-3.3 in the main body. In summary, Assumption 3.1 states that the parametric model’s probability density is continuous and differentiable with respect to its parameters, Assumption 3.2 states that the pushforward of the parametric model through the target functional is continuous and differentiable with respect to the model parameters, and Assumption 3.3 states that the Fisher information is invertible. Without these assumptions, the terms in Equation (1) do not have any meaning.\n\n**(3) Maximum eigenvalue growth**. The maximum eigenvalue of $\\Sigma^{-1}$ does not grow with $N$ (the number of datapoints); $\\Sigma = cov(\\tilde{x})$, where $\\tilde{x}$ is the normalized score vector associated with $P_{\\phi}$ (defined in Assumption 3.5) which is independent of $N$. This term could, however, increase with the model dimension $p$. We could add an assumption that $\\Sigma^{-1}$ is uniformly bounded but we proved the theorem for the more general case when $\\Sigma^{-1}$ is not. There are of course many instances when the maximum eigenvalue of $\\Sigma^{-1}$ is indeed uniformly bounded in $p$ (in which case our approximation error bound gets better). For example, if  $\\Sigma$ is the identity matrix, then the maximum eigenvalue is 1 for all p.\n\n**(4) Discussion of Limitations**. While the assumptions we make throughout the paper permit a very broad class of practical models and functionals, as we discussed in Section 3, there are certainly some circumstances where relaxing the assumptions and requirements would be of practical value. For example, one may wish to use MC-EIF on functionals in models that only contain an implicit likelihood, such as Bayesian variants of off-the-shelf physics emulators (See https://www.pnas.org/doi/10.1073/pnas.1912789117). Extending MC-EIF to these models, or similarly implicit functionals, is an exciting area of future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper develops a general Monte-Carlo approach for approximating efficient influence functions. It aims to provide an automated procedure within an existing probabilistic programming workflow. It also provides non-asymptotic error bounds on the quality of the Monte-Carlo approximation and examines three specific estimators: von Mises one step estimator, debiased/double ML, targeted minimum loss estimation. The paper is interesting and fits well with the current developments in the literature. It would be useful to revise the paper for the following aspects: (i) real-data experiments, which the authors indicate that they already carried out; (ii) an emphasis on the advantage in terms of computational speed; (iii) a more serious discussion of limitations (e.g., computation of the Fisher information matrix) as well as the underlying assumptions and requirements.
Decision: Accept (spotlight)