Doubly-Robust Estimation of Counterfactual Policy Mean Embeddings
Authors: Houssam Zenati
Bariscan Bozkurt
Arthur Gretton
Venue: NeurIPS 2025 poster ()
Keywords: off-policy evaluation; causal inference; doubly robust; kernel mean embeddings; kernel two sample tests;
Primary area: probabilistic_methods

Abstract: Estimating the distribution of outcomes under counterfactual policies is critical for decision-making in domains such as recommendation, advertising, and healthcare. We propose and analyze a novel framework—Counterfactual Policy Mean Embedding (CPME)—that represents the entire counterfactual outcome distribution in a reproducing kernel Hilbert space (RKHS), enabling flexible and nonparametric distributional off-policy evaluation. We introduce both a plug-in estimator and a doubly robust estimator; the latter enjoys improved convergence rates by correcting for bias in both the outcome embedding and propensity models. Building on this, we develop a doubly robust kernel test statistic for hypothesis testing, which achieves asymptotic normality and thus enables computationally efficient testing and straightforward construction of confidence intervals. Our framework also supports sampling from the counterfactual distribution. Numerical simulations illustrate the practical benefits of CPME over existing methods.

Doubly-Robust Estimation of Counterfactual Policy Mean Embeddings
Houssam Zenati
Bariscan Bozkurt
Gatsby Computational Neuroscience Unit University College London h. zenati @ucl. ac.
Gatsby Computational Neuroscience Unit
University College London
bariscan bozkurt 23@ucl. ac uk
Arthur Gretton
Gatsby Computational Neuroscience Unit University College London
Google Deepmind
arthur www.tggrreeeeaaccooooo
Abstract
Estimating the distribution of outcomes under counterfactual policies is critical for eecision-making in domains such as recommendation, advertising, and healtheare. We propose and analyze novel Tamewoork-CCuttffaccuuuu Policy Mean Embed- ding (CPME)-thhe -tht represents the entire counterfactual outcome distribution reproducing kernel Hilbert space (RKHS), enabling flexible and nonparametric distributional off-policy evaluation. We introduce both plug-in estimator and doubly robust estimator; the latter enjoys improved convergence rates by cor- recting for bias in both the outcome embedding and propensity models. Building this, develop doubly robust kernel test statistic for hypothesis testing, which achieves asymptotic normality and thus enables computationally efficient testing and straightforward construction of confidence intervals. Our framework also supports sampling from the counterfactual distribution. Numerical simulations illustrate the practical benefits of CPME over existing methods.
Introduction
Effective drcision-making requires anticipating the outcomes of actions driven by given policies [ This is especially critical when decisions rely on historical datt-whether experimentation limited infeasible (2). or under sequential designs [3]. For instance, doctors weigh drug effeets before prescribing [4], and businesses predict revenue impact from ads [5]. Off-Policy Evaluation (OPE) addresses this challenge by estimating the effect of target policy using data sampled under a different logging policy. Each logged record includes covariates (e.g., user or patient data), an action (e.g., recommendation treatment), and the resulting outcome (e.g., engagement health status). The goal is to evaluate the expected outcome under the target policy, which involves inferring counterfactual nutcomes-win would have happened under the alternative target policy.
Although many works have focused on estimating the mean of outcome distributions, for example, with the policy expected risk (payoff) [6] or the average treatment effects [7]- and their variants thereof- seminal works have considered inference on counterfactual distributions of outcomes [8] instead. The developing field of distributional reinforcement learning (RL) [9] and distributional
OPE [10 [1] provides insights on distribution-driven decision making, which goes beyond expected policy risks. Indeed, reasoning such distributions allows using alternative risk measure such as conditional value-at-rist (CVaR) [12, higher moments or quantiles of the distribution [3 [4]
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

However, most existing approaches leverage cumulative distribution functions (CDF) are not suited for inference on more complex and structured outcomes.
15 16 which
Conversely, counterfactual mean embeddings (CME) 17 represent the outcome distributions as
elements in reproducing kernel Hilbert space (RKHS) T8 19 and allow inference for distributions over complex outcomes such as images, sequences, and graphs 20]. Such embeddings leverage kernel mean embeddings [21]. a framework for representing distribution maintaining all of its information for sufficiently rich kernels 12 23] This framework allows to quantify distributional treatment
effects T7.. perform hypothesis testing 24 or even sample [2]] 26 from the counterfactual outcome
distribution. Recent works have employed counterfactual mean embeddings for causal inference in the context of distributional treatment effects [27 however these approaches have not been applied to OPE and limited mostly to binary treatments. Developing analogous distributional embeddings for counterfactual outcomes under target policies could enable a range of new applications, including principled evaluation, hypothesis testing, and efficient sampling from complex outcome distributions.
Our estimates will employ doubly robust methods, which have become central tool in causal inference due to the desirable property of consistency if either the outcome model or the propensity model is correctly specified 30 3T. DR estimators have since been studied under various functional estimation tasks, including treatment effects [32 and policy evaluation [33]. These estimators
originally leverage efficient influence functions 34 36 and sample-splitting techniques [37 38 to
achieve bias reduction and enable valid inference in nigh-dimensione and nonparametric settings Recently, doubly robust tests have been introduced for kernel treatment effects [28] 29].
39 40
Moreover, an extension of semiparametric efficiency theory and efficient influence functions has been proposed for differentiable Hilbert-space-vall valuld parameters [41] Leveraging these efficient influence functions to build doubly robust estimators of counterfactual mean embeddings would
therefore enable more theoretically grounded distributional OPE.
In this work, we propose novel approach to distributional OPE that embeds the counterfactual outcome distribution, which procedure term as Counterfactual Policy Mean Embedding (CPME). Our contributions are as follows: i) First, we define and formalize the CPME in the distributional OPE problem. We proposing a plug-in estimator, and analyze its consistency with convergence rate of up to (n14/) under standard regularity assumptions involving kernels and underlying distributions. ii) We then derive the Hilbert-space--aluee efficient influence function of the CPME to propose doubly robust estimator, and establish its convergence in the RKHS with an improved consistency rate of up O(n 1/2) robust and asymptotically normal statistic which allows a computationally efficient kernel test. iv) We demonstrate that our estimators enable sampling from the outcome distribution. v) Finally, provide numerical simulations on synthetic and semi-synthetic data, including structured outcomes, to support our claims in range of scenarios.
(n2) under the same assumptions. iii) Consequently, we propose efficient doubly
The remainder of the paper is organized as follows. Section 2 formalizes the CPME framework. Sections 3 and 4 introduce, respectively, the nonparametric plug-in estimator with consistency influence-funeee based estimator with improved convergence. Section
guarantees and an illustrates applications to hypothesis testing and sampling, 6 numerical results, and Section S concludes.
2 Counterfactual policy mean embeddings
We begin by formalizing the counterfactual policy mean embedding (CPME) framework, which provides a kernel-based foundation for distributional OPE.
2.1 Distributional off-policy evaluation setting
We are given an observational dataset generated from interactions between decision-making system and units with covariates Fg. For each instance {1....,,) n), a context z; was drawn i.i.d. from an unknown distribution Px. i.e., ~ Px. Given x,, an action was sampled from a logging policy To E II, such that d; ~ TOl. æz). Following the potential outcomes framework 77. we denote the set of potential outcomes by (Y(a)) and observe the realized outcome y; (a,) PYIx.A=z,a. The Gata-generating process is therefore characterized by the joint distribution Po PY|X.A X no Px. The dataset consists of n i.i.d. logged observations

---

{ as bi) Po. The action space A may be either finite or continuous. For notational purposes, we will also abbreviate the joint distribution Pa Py|x,A X π X Px.
Given only this logged data from Pa. the goal of distributional off-policy evaluation is to estimate v(r), the distribution of outcomes induced by a target policy belonging to the policy set IT:
( ExxXx (Y
(()) represents the marginal distribution of outcomes over Px. therefore when actions are taken from the target policy x E IT in counterfactual manner. Compared to "classical" OPE where only the average of the outcome distribution is considered, distributional RL and OPE [II allows defining further risk measures [16] depending for example on quantiles of the outcome distribution [42. In this work we focus on distributional OPE leveraging distributional embeddings.
2.2
Distributional embeddings
In this work, we employ kernel methods represent. compare, and estimate probability distributions. For both domains F E {AA x X,)), we associate an RKHS H of real-valued functions / F -> R, where the point evaluation functional bounded 4 Each RKHS is uniquely determined by its continuous, symmetric, and positive semi-definie kernel function ky F x F - IR. We denote the induced RKHS inner product and norm in Hz by (,r and || || respectively. Throughout the paper, we denote the feature maps ( ( (a,x)) (xx 2)) and ky(.,)) 1) = dy(v) for HAx and Hly the RKHSs over A x and V. See Appendix 9.1 for further background.
Building upon the framework of Muandet et al. embedding (CPME) as:
17 we define the counterfactual policy mean
x(=) Ep, [oy(Y)
(2)
which is the kernel mean embedding of the counterfactual distribution v(T). This causal embedding
allows to i) perform statistical tests (24), (ii) sample from the counterfactual distribution [25] 26 or
even (iii) recover the counterfactual distribution from the mean embedding (22). While Muandet
al. [17 introduced the counterfactual mean embedding (CME) of the distribution of the potential outcome ((a) under a single, hard intervention for binary treatments, we focus instead on counter- factual embeddings of stochastic interventions for more general policy action and sets II, A in the OPE problem. Next, we provide further assumptions for the identification of the causal CPME.
2.3 Identification
established sufficient conditions under which causal dunftions-definee in terms potential outcomes (aa)-can be identified from observable quantities such as the outcome Y. treatment A, and covariates X. These conditions commonly referred to as selection on observables.
In seminal works, Rosenbaum and Rubin 44 and Robins 45
Assumption 1. (Selection on Observables). Assume i) Consistency: Y = Y(a)) when A = ii) Conditional exchangeability> Y(a) A X, iï) Strong positivity: PEP ess infoe.A.zxx (o
σ) > 0, where the essential infinum is under Px.
Assumption (i), combined with the no- no-interference assumption (which rules out interference between units, ensuring that each individual's 's outcome depends only on their own treatment assignment) is also known as the stable unit treatment value assumption (SUTVA). Condition (ii) asserts that, conditional on covariates X, the treatment assignment is independent of the potential outcomes, implying that treatment is good as randomized once condition X X-thus ruling out unmeasured confounding. Finally, (iii) guarantees that all treatment levels have a nonzero probability of being assigned for any covariate value with positive density, preventing deterministic treatment allocation and ensuring overlap in the support of treatment assignment. Note that this mild condition on the essential infimum slightly stronger than the common positivity assumption [46]; in [I7] this will prove useful for the importance weighting in the counterfactual mean embedding. Now define the following conditional mean embedding [7 of the distribution PY|x,A:
MY|A. (0, 1
A a, X x].
DDeppite its name, the CPME represents the mean embedding of interventional (do-) distributions thus corresponding to the second rung of Pearl's ladder. The term "counterfactual" is retained for consistency with prior work [e.g. [1]. where potential outcomes Y Y() are colloquially called "counterfactuals..
3

---

Under the three conditions stated earlier, the CPME can be identified as follows.
Proposition 2. (Identified Counterfactual Policy Mean Embedding) Let us assume that Assumption 7 holds, then the counterfactual policy mean embedding can be written as:
x(x) ExxPx [uY]a, (a,
Further details on this Proposition are given in Appendix 993] We are now in position to use our RKHS assumption provide nonparametric estimator of the CPME in the next section.
3 A plug-in estimator
We further require some regularity conditions on the RKHS, which are commonly assumed 48 49
Assumption 3. (RKHS regularity conditions). Assume that i) KAN, and ky are continuous and bounded, i.e. SuPo,zEAxX Ax (a,,)))lHAx Kaa,: supyey (1o(y) & Kiy ii) ( and dy(y) are measurable; iii) ky is characteristic.
Let CYIA.X E S2(HAx. Hy) be the conditional mean operator, where Sa(HAA. Hy) denotes the Hilbert space of the Hilbert-Schmid operators (50) from HAx to Hy. Under the regularity condition that E[}(Y)]A that JA (a, = CyJA. (os.xxxNN ( x)). Moreover, define Pr. the joint policy-context mean embedding as:
X = I E HAx for all h E Hy, the operator CyJa, exist such
= Exxxs (axx(a, x)].
(5)
Importantly, note that [h denotes the joint embedding of actions under x and covariates under Px- We now state the following proposition, with its proof provided in Appendix [I.1]
Proposition 4, (Decoupling via joint policy-context mean embedding) Suppose Assumptions and hold. Then, the CPME can be expressed as:
x(x) CYJAXXP
(6)
This result suggests that estimator for the counterfactual policy mean embedding (Tr can be constructed by pluging-in in an estimate Cyja,x of the conditional mean operator and an estimate ür of the joint policy-contex. mean embedding. The resulting plug-in estimator Xpt writes:
X (T) = CYIAX PT.
Thus, we first require the estimation of the conditional mean embedding operator CyJA,x. x. To do so, given the regularization parameter >> O, we consider the following learning objective (53):
1 ni Σ boy (v:) CloA (a,, (o), Pay. +XICI (C(C)
C E S2 (HAx, Hy)
whose minimizer is denoted as, CYJA, mrg Hy (y) (C).. Given the observations
{as, Fi, Di ) the solution to this problem 53 is given by OrJA, = CYY(A.)) (Ōax + -1) where ČY.(A.))
1 E (1:) 8 ΦAx (4) and CA,X
" EE &AX (a;, 2r:) 6
dAx 4.). Since we work with infinit-ddimensional feature mappings, convenient ex- press the solution in terms of feature inner products (i.e., kernels), using the representer theorem [4]
(YIA,x(a, = Cy
Σ (a, = (y)((,
"The conditional mean operator formulation is valid under mild smoothness assumptions ensuring that the
conditional mean function (x) ElØ(Y) X x] belongs to Sobolev-type vector-valued RKHS, In particular, Li et al. (51) show that when the Matém kernel is used on the X -pscce and F, E H H"x (X;Hy) ), the induced operator Crix exists and acts boundedly from 7x to Hy. A regression-baste based alternative 52 can also be used, but the operator view is often more convenient for theoretical analysis.

---

where By = (v)())) kernel matrix over the [a),,;)]=11 and K aax,ar is the kernel vector between training points {(a;, z;)) and the target variable (d,z).
( (KaX,Ax + nXT) ax), KAX,AX is the
in Appendix 10 using the main result from Li et [48] This bound plays a key role in establishing the consisteney of both our plug-in and doubly robust estimators. The derivation relies on the widely adopted source condition (SRC) 55 56] and eigenvalue decay (EVD) assumptions, as formalized in Assumptions [5 and [6
We provide bound on the estimation error DYjA,x-CrJA.x
Second, we estimate the joint policy-context mean embedding P. which represents the joint embed- ding of the distribution which takes the following explicit form for discrete action spaces:
Px. We employ the empirical kernel mean embedding estimator (24).
Σ Σ nI aEA
1
n
Sax(a, (a)x)))))))
(8)
For continuous action spaces, we propose an empirical kernel mean embedding estimator combined with resampling strategy over actions (see Appendix TO.3). In OPE, the target policy is specified by the designer, making these estimators directly applicable. summary of the plug-in estimation procedure is provided in Appendix 10.3 (see pscudo-code in Algorithms
Importantly, our plug-in estimator of the CPME differs substantially from the approach of Muandet et al. [7]. First, they propose and analyze an importance-weigheee estimator for kernel treatment effects under the assumption of known propensities. Second, although they discuss an application to OPE, their method lacks formal analysis and is not evaluated beyond linear kemnlls.
Next, we arrive the theoretical guarantee for the plug-in estimator under the conditions we presented and the common Assumptions 15
16
-7ttteed in Appendix T0.2 for space considerations.
17
Theorem 5. (Consistency of the plug-in estimator). Suppose Assumptions /
15 and 17 hold.
Set À = n- -i//(==1/b), which is rate optimal regularization. Then, with high probubility, Xps defined
in Equation ( achieves the convergence rate with parameters E (0, 1] and e E (1,3)
1xxp((T) - -(()))),, = (ro((n,b,c))
= =0 ( (o-00-1//2(e41//0)
Here, (n b, c) bounds the error in estimating CY x. with and b denoting the source condition and spectral decay parameters (Assumptions 15 and [6). Appendix (oppendix(O.2pppppppp proviis a proof with explicit constants hidden in the (--) notation. Smaller values of 6 indicates slower eigenvalue decay of the correlation operator defined Assumption 15 as > oc the effective dimension is finite. The parameter controls the smoothness of the conditional mean operator CYJA.X- The optimal rate is n -1/4, which can attained when e = 3 [51] The convergence rate is obtained by combining two minimax-optimal rates: n (e-1))/((c+11//) for the conditional mean operator CyJ4,x [ST Theorem
3.. and n -1/2 for kernel mean embedding /r [57] Theorem 1]. In the next section, we introduce a doubly robust estimator of the CPME that improves upon this rate.
4 An efficient influence function-based estimator
To design our estimator, we rely on semiparametric efficiency theory for Hilbert space-valued parameters 34 4T) As in the finiteddimensional setting, efficient influence functions (EIFs) 36 quantify the local sensitivity of target parameter to perturbations of the underlying distribution. When they exist, they enable the construction of one-step estimators 588 59 which correct the plug-in in bias and often exhibit doubly robust properties (32). Assuming the existence of an EIF sh" for the CPME x()), the one-step estimator takes the form
(ar( (pi(
3Th (o., F..
One-step estimators rely pathwise differentiabiliyy, which describes how the target parameter varies under infinitesimal perturbations of the data distribution [60] 61.. When this condition holds, the EIF
coincides with the Riesz representer of the pathwise derivative 41 in case, the unique RKHS
5

---

element whose inner product with any score function recovers the target parameter's directional derivative. This derivative captures the target parameter's 's first-order sensitivity distributional changes, and projecting it onto the model's tangent space yields the optimal linear correction that removes the plug-in estimator's 's leading bias (see Appendix
To define the corresponding one-step estimator, we assume the spaces A, N' y are Polish (Assump- tion 177. Under these conditions, we derive the result stated below and prove it in Appendix 11.2
Lemma 4.1. (Existence and form of the efficient influence funcciion). Suppose Assumptions i and 17 hold. Then, the CPME x(xz) admits an EIF which is P-Bochner square integrable and takes the form
TT z) (a x) {Φy(y) - -xIAx((,x)) + f pylAx ())((dd))daaa ( 1x)-x(f)) (10)
2/ah (y, a.
Note that the EIF defined in Equation 10 similar to the EIF of the expected policy risk in OPE depends on both the propensity score mu(a)z) and the conditional mean embedding MYJ.. Note that, since we consider stochastic interventions, our EIF remains valid for continuous treatments-unlikk the setting in (28), which would require a kernel localization argument (62-64)
33 32),
to handle continuity. Estimating f PYJA,X (a', (d) .r) corresponds to the plug-in estimation
procedure described previously, while estimating the importance weighted term sataley (by(y) aligns with the CME estimator for kernel treatment effects analyzed by [17, who, however, assume known propensities Wy (a (r). By contrast, our framework permits estimation of propensities no (a(aaz) with machine learning algorithms [40] Leveraging the EIF, we define the following one-step doubly robust Xdx estimator:
m(alz)
n
T(us xi) (dy (w:) - ( + + iixiax(a, mi) (11)
}
(
n2
Ru (a: x;)
Like all one-step estimators in OPE. our estimator enjoys doubly robust property: it remains consis- tent if either An or fxyX is correctly specified. We elaborate on this property in Appendix [1.2 Note that originally, Luedtke and Chung [4] proposed cross- fitted variant of the one-step estimator. In Appendix [T.44] we discuss this variant and show that, under a stochastic equicontinuity condition (65],
cross- fitting may be discarded thus improving statistical power. We now state consistency result.
Theorem 6. (Consistency of the doubly robust estimator). Suppose Assumptions 7
15
16 and 17
Set X =n-1//o+///, which is rate optimal regularization. Then, with high probabilit,
(m)r x O
n 1/2 + +ro (n)
Here, Tmo (n) denotes the error in estimating the propensity score au(a z). The proof of this consis- tency result, along with explicit constants hidden by the O(-) notation, is provided in Appendix [1.3] These rates approach n 1122 when the product Twp (n, 8) rc(n, 6, 6, c) scales as n 11/2_ for instance, when both the conditional mean embedding and propensity score estimators converge at rate n -1/4. This result constitutes a clear improvement over Theorem
5
Testing and sampling from the counterfactual outcome distribution
In this section, we now discuss important applications of the proposed CPME framework.
5.1 Testing
CPME enables to assess differences in counterfactual outcome distributions v(TT) and v(T'). Such difference in the two distributions can be formulated specifically, two-sample testing (24] Moreover, we want to perform that test while only being given acess to the logged data. The null hypothesis Ho and the alternative hypothesis H1 are thus defined as
problem of hypothesis testing, more
Ho u(T) v(x',, H1 ((T) # (TT').
Specifically, we equivalently test Ho Er, [k (. y)] - Er,, [k (-. y)] = 0 given the characteristic assumption kernel ky. Moreover, leveraging the HIF formulated in Section 4 we have:
Ep, [oy (v) - P,, [dy (v)) EEll xr (v. x)).
(12)

---

where we take the difference of EIFs of (()) and x(=²);
(y) {
<T(azz)) <o(aaz))
x (a(a))
(dy(y) priax((oxx)) (x)x) (x) (13)
No [o(a
and use the shorthand notation (x) = JY]A. (a', x x). Thus, we can equivalently test for Ho E8] W (y, a, x]] = O. With this goal in mind, and recalling that the MMD is degenerated
statistic (24). we define the following statistic using cross U-statistic as Kim and Ramdas [66]]
m f St
1
7 Σ
Tt
where f1, as;
(Gx, ais T4), Pn (Uj, aj. xzy)) j=m+1
n1 m
Importantly, above, m balances the two splits, Pr. (y, a, x) is an estimate of yf, (y, a, x) (using to and /AYJA,) on the first samples while S is an estimate of the same quantity on the remaining n m samples. Further, St... and St denote the empirical mean and standard error of fT.='
m
m Σ 2=1
FL
m Σ Sla (Ui, 21), st x,,
(Ui, as, xz) - -Ft,rr
(14)
m
Having defined this cross U-statistic, we are now in position to prove the following asymptotic normality result, in (28) for kernel treatment effects.
Theorem 7. (Asymptotic normality of the test statistic) Suppose that the conditions of Theo- rem hold, and that Ep, [|<P=,
< 0o. <0.. Assume the non-degeneracy condition EkPa (Z), ar (Z')) Ny 0, and that the product of nuisance convergence rates satisfies Tap (2) (e,n,,, = O( 1/)). Set = n 1/(o+1/b) and m = |n/2]. Then it follows that
T n. 4 N00,1
We provide a proof of this result in Appendix 12 Note here that while Hilbert space CLT would allow to show asymptotic normality the EIF of the CPME the RKHS [41], using cross-U U statistic here is necessary due the degeneracy of the MMD metric. Kim and Ramdas [66] show that m = [n/2] maximizes the power of the test. Moreover, the doubly robust estimator the CPME allows to obtain faster convergence rate which is instrumental for the asymptotic normality of the statistic. Based on the normal asymptotic behaviour of of n, we propose as 28 to the null
hypothesis Hp v(*)= v(x') given the p-value p 1 - where + is the CDF of standard normal. For a-level test, the test rejects the null if p S Algorithm beloow illustrates the full procedure the test, which we call DR-KPT (Doubly Robust Kernel Policy Test).
Algorithm DR-KPT
Require: Data D = (Ii, at, y: 9i) kernels ky, KA.X'
Ensure: The p-value of the test
1: Set m = [n/2] and estimate fY14, do on first in samples, My A.X, to on remaining n - m.
2: Define ((v.a,x)
m(ala) n' (alx)
( NYI x)) (x()) (x) and V
Pa(aaz)
Tola
3: Define f1.x (yi, ais x..) n-m
Σn m+ (6 (3i, 04 x2) . (y aj, m;)) for 1....,
4: Calculate Fr. and st N.. using Equation then T N, mm
vmft
5: return p-value p (TT m.
Note that as Martinez Taboada et al. 28 our test is computationally efficient compared to the permutation tests required in CME Muandet et al. which would require the dramatic fitting of the plug-in estimator for each iterations.
7

---

23 lomid
Drous DSEPT
ADC
DRaFT
Yotta
X12
3.00
Hecerkee
Figure 1: Illustration of 100 simulations of DR-KPT under the null: (A) Histogram with standard normal pdf for n = 400, (B) Normal Q-Q plot for n 400, (C) False positive rate across sample sizes. The results confirm the Gaussian behavior and good calibration of the test under the null.
5.2 Sampling
We now present a deterministic procedure that uses the estimated distribution embeddings {(=) to provide samples (ū, from the counterfactual outcome distribution. The procedure is a variant of kernel herding [25 17 and is given in Algorithm
Algorithm 2 Sampling from the counterfactual distribution
Require: Estimated CPME {TT) y + R, kernel ky y x y - R, and number of samples m E N
91 arg m (
2: for = 2 to m do
3: ÿr : arg max ((T)(V)- - Ei
4: end for
5: Output: 1 3m
Below, prove that these samples converge in distribution to the counterfactual distribution. We state additional regularity condition under which we can prove that the empirical distribution py of the herded samples (ūz))² calculated from the distribution embeddings, weakly converges to the desired distribution.
Assumption 8. (Additional regularity). Assume i) y is locally compact. ii) Hl C Co. where Co is the space of bounded, continuous, real valued functions that vanish infinity.
As discussed by Simon-Gabriel et al. [67], the combined assumptions that y is Polish and locally compact impose weak restrictions. In partieular, y is Banach space, then to satisfy both conditions it must be finite dimensional. Trivially, = dim((Y) satisfies both conditions.
Proposition 9. (Convergence of MMD of herded samples, weak convergence to the counter- factual outcome distribution) Suppose the conditions of Lemma 4.1 and Assumption X hold. Let (āar.) and pm (resp. (ppi.), Pmpp)) be generated from Xar(T) (resp. (pi(T)) via Al- gorithm 2 Then, with high probability, MMD(PN) Pepis v(T) = Op(ro(n,b, + m m-1) and MMD P....v())) v(x)) Op(n (n)rrenn b,c) + m m-1)). Moreover, (jars) v(T) and (Un.s) ((T).
The proof is provided in Appendix 13 This proposition shows that the DR estimator of CPME yields an empirical outcome distribution with improved MMD convergence, with weak convergence toward the counterfactual outcome distribution (67
6 Numerical experiments
In this section, we present numerical simulations for testing and sampling from the counterfactual distributions. Full experimental details, including additional simulations, are provided in Appendix [4] All code and simulation materials used in this study are publicly available at at/ttss:: ///::://ttttttt com/ aitassanzenntiicoutteeeeee polllcccttal-lllccy--oo mean mean-embedding

---

(V)
1.0
10
1.0
Tue postte nate
0.9
0.0
0.0
DA.07
36.17
271
100
150
200
250
300
350
400
100
150
200
250
300
350
600
100
150
200
250
300
350
4.0
Sumple
Sample 5:20
Sampoo
Figure 2: True positive rates of 100 simulations of the tests in Scenarios II, III, and IV. DR-KPT shows notable true positive rates in every scenario, unlike competitors.
6.1 Testing
We assess the empirical calibration and power of the proposed DR-KPT test in the standard observa- tional causal inference framework. We assume access to i.i.d. samples {(rr, ai y:) (X, A, Y). All hypothesis tests are conducted at a significance level of 0.05.
Synthetic experiments We synthetically generate covariates, continuous treatments, and outcomes
under four scenarios adapted from [17 28]. Scenario I (Null): π x', implying no distributional shift and v<x) = v(z'). Scenario II (Mean Shift): л and n' differ by small opposite shifts in their mean treatment assignments, changing the expected mean. Scenario III (Mixture): n is stochastic mixture two policies with the same mean as n, creating bimodal treatment distribution that alters outcomes without affecting the mean. Scenario IV (Shifted Mixture): same Scenario III but with an additional mean shift of n relative to π.
In all cases, treatments are drawn from logging policy No, while outcome and propensity models are unknown. Propensities ān(-) are estimated linear regression, and outcome regressions via conditional mean embeddings. We first assess the empirical calibration of DR- KPT and the Gaussian behavior of T .. under the null. Figure shows that DR-KPT achieves near-standard normal be- havior and proper calibration in Scenario I. Figure 2 reports results for Scenarios II-IV. As baselines, we adapt the KTE method of Muandet al. [7 into a Kernel Policy Test (KPT) with estimated propensities and include a linear-kernel variant (PT-linear) testing only mean shifts. DR-KPT consistently outperforms all methods, including under pure mean shifts, where KPT and PT-linear degrade due to propensity estimation. Overall, DR-KPT reliably detects distributional changes, exhibits strong power across scenarios, and remains computationally efficient (see Appendix 14).
Warfarin dataset We the publicly available dataset Warfarin dosage 68 which contains patient covariates and expert- prpeer-prescribed therapeutic doses. The treatment corresponds to a continuous dosage level, making this dataset well suited for off-policy evaluation of continuous treatment policies. Although the data are fully supervised, we simulate an off-policy bandit environment (see Appendix (1) by defining reward function that is maximal when the assigned dose a lies within +10% of the expert's 's prescription, following Kallus and Zhou [4, Zenati et al. [69] logging and target policies are modeled as Gaussian distributions.
We mirror the synthetic testing protocol of the previous experiment and evaluate four scenarios (I) Null, (II) Mean Shift, (III) Mixture, and (IV) Shifted Mixture each introducing distinct shifts in the treatment and outcome distributions. Both outcome models and propensity scores are learned from data. We compare our Doubly Robust Kernel Policy Test (DR-KPT) with baseline KPT estimators using linear, RBF, and polynomial kernels. The results in Thoo that DR-KPT is well-calibrated under the null (Scenario 1) with near-nominal rejection rates. Across all alternative scenarios (II-IV), DR-KPT consistently outperforms or matches the best baseline.
dSprites (Structured Outcomes). We perform experiments on the dSprites dataset 70 71, which enables evaluation on structured image outcomes. Unlike scalar outcomes in our other experiments, here the counterfactual effect policy is evaluated on rendered x44 images generated from latent variables. The structural causal model is defined by latent contexts x U([0, 1J²), actions a (- x), and outcomes y == g(x, a) E Re1x61 where g maps each context-action pair to an image via the fixed dSprites renderer. All other latent factors (shape, scale, and orientation) are held

---

Table Rejection rates for the Warfarin dataset across four scenarios.
Scenario
KPT-lineer
KPT-rbf
KPT-ppoly
DR-KPT-rbf
DR-KPTTpoly
0.00
0.00
0.00
0.02
0.06
II
0.77
0.01
0.29
0,80
0,66
III
1.00
0.00
0.66
0.99
0.95
IV
0.24
0.00
0.11
0.76
0.55
constant. As in previous experiments, the logging and target policies π. x' are contextual Gaussians (u(UU),021) where u(U) encodes rotated and shifted transformation of the context. We focus on two scenarios: (I) Null, where outcome distributions coincide, and (IV) Shifted Mixture, where they differ due to policy-inducee shifts. Both outcome models and propensity scores learned from data, and the evaluation follows the same procedure as in the Warfarin experiment.
Table 2: Rejection rates for the dSprites dataset under structured outcomes.
Scenario
KPT-lineer
KPT-rbf
KPT-poly
DR-KPT-rbf
DR- PR-Kly-ppoly
I IV
0.394
0,401 0.054
0.375
0.024
0.000
0.081
0.073
0.656
0.502
The results highlight the poor calibration of baseline methods under the null (Scenario I), with inflated rejection rates approaching 40%, while DR-KPT maintains near-nominal levels. In the alternative scenario (IV), DR-KPT achieves substantially higher power than all baselines, confirming its robustness and sensitivity in detecting structured distributional shifts with complex outcomes.
6.2
Sampling
We also perform an experiment in which we generate samples from Algorithm@winn with both the plug-in and DR estimators of the CPME under multiple scenarios which we vary the design of the logging policy (uniform and logistic) and the outcome function (quadratic and sinusoidal) see Appendix 14 Figure we illustrate an example the outcome distribution from logged samples, the oracle counterfactual outcome distribution and the empirical distribution obtained from two kernel herding algorithms. Appendix (ereiiixxxeepprreee reportss MMD and Wasserstein distances between the counterfactual and oracle distributions, illustrating that the DR variant generally attains lower distances in our synthetic setting.
Conkenabbll Outore Dude
Figure 3: Logistic logging policy, nonlinear outcome function.
Discussion
In this paper, we presented a method for estimating the Counterfactual Policy Mean Embedding (CPME), the outcome distribution mean embedding of counterfactual policies. We proposed nonparametric plug-in estimator together with a doubly robust, efficient influence canction-base variant enabling computationally efficient kernel test. Our framework also supports sampling from counterfactual outcome distributions. Recent advances suggest more scalable extensions based on MMD gradient flows 772 73], which we view as promising direction for future work. Finally, our analysis relies standard identification assumptions such as positivity and exchangeability; relaxing these toward weaker or partially identifiable settings is another important avenue for future research. settings is important direction for future work.
10

---

Acknowledgments and Disclosure of Funding
The authors thank Dimitri Meunier for his valuable discussions and insightful comments. The authors also thank Liyuan Xu for early discussions. All authors acknowledge support from the Gatsby Charitable Foundation. The authors are grateful for the constructive feedback and insightful discussion provided by the anonymous reviewers of NeurlPS 2025.
References
[1] Tor Lattimore and Csaba Szepesvári. Bandit Algorithuss. Cambridge University Press, 2020.
[2] Alex Strehl, John Langford, Lihong Li, and Sham Kakade. Learning from logged implicit exploration data. Advances in neural information processing systems, 23, 2010.
[3] Houssam Zenati, Eustache Diemert, Matthieu Martin, Julien Mairal, and Pierre Gaillard. Se- quential counterfactual risk minimization. In International Conference Machine Learning. volume 202, pages 40681 10781-40706, 2023.
[4] N. Kallus and A. Zhou. Policy evaluation and optimization with continuous treatments. In International Conference on Artificial Intelligence and Statistics (AISTATS), 2018.
[5] Léon Bottou, Jonas Peters, Joaquin Quiñonero Candela, Denis X. Charles, D. Max Chickering, Elon Portugaly, Dipankar Ray, Patrice Simard, and Ed Snelson. Counterfactual reasoning and learning systems: The example of computational advertising. Journal of Machine Learning Research (JMLR), 14(1):320733260, 2013.
[6] Lihong Li, Wei Chu, John Langford, and Xuanhui Wang. Unbiased offline evaluation of contextual- baii-bbnnnddaaaiieee news article recommendation algorithms. In International Conference on Web Search and Data Mining, page 297-306. Association for Computing Machinery, 2011.
[7] Guido W. Imbens and Donald B. Rubin. Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction. Cambridge University Press, 2015.
[8] Christoph Rothe. Nonparametric estimation of distributional policy effects. Journal of Econo- metries, 155(1):56-70, 2010.
[9] Mare G. Bellemare, Will Dabney, and Mark Rowland. Distributional Reinforcement Learning. MIT Press, 2023.
[10] Yash Chandak, Scott Nickum, Bruno da Silva, Erik Learned-Miller Emma Brunskill, and Philip S. Thomas. Universal off-policy evaluation. In Advances in Neural Information Process- ing Systems, volume 34, pages 17475-27490 2021.
[11] Runzhe Wu, Masatoshi Uchara, and Wen Sun. Distributional offline policy evaluation with predictive error guarantees. International Conference on Machine Learning, pages 37685- 37712, 2023.
[12] Tyrrell Rockafellar, Stanislav Uryasev, et al. Optimization of conditional value-at-risk. Journal of risk, 2:21 2000.
[13] William Sharpe. Mutual fund performance. The Journal of business, 39(1):119-138, 1966.
[14] Will Dabney, Georg Ostrovski, David Silver, and Rémi Munos. Implicit quantile networks for distributional reinforcement learning. In International Conference Machine Learning, pages 1096-1105, 2018
[15] Victor Chernozhukov, Iván Fernández- Val, and Blaise Melly. Inference counterfactual distributions. Econometrica, 81(6):2205-2268, 2013.
[16] Audrey Huang, Liu Leqi, Zachary Lipton, and Kamyar Azizzadenesheli. Off-policy risk assessment in contextual bandits. In Advances Neural Information Processing Systems, volume 34, pages 23714-23720 2021.
11

---

[17] Krikamol Muandet, Motonobu Kanagawa, Sorawit Saengkyongam, and Sanparith Marukatat, Counterfactual mean embeddings. Journal of Machine Learning Research, 22(162):1-71, 2021.
[18] Alain Berlinet and Christine Thomas-Agnan Reproducing kemnel Hilbert spaces in probability and statislics. Springer Science & Business Media, 2011.
[19] Arthur Gretton. Introduction to rkhs, and some simple kernel algorithms. Adv. Top. Mach. Learn Lecture Conducted from University College London, 16(5-3):2, 2013.
[20] Thomas Gärtner. A survey of kernels for structured data. ACM SIGKDD explorations newsletter, 5(1):49-58, 2003.
[21] Alex Smola, Arthur Gretton, Le Song, and Bernhard Schölkopf. A hilbert space embedding for distributions. In International conference on algorithmic learning theory, pages 13-31. Springer, 2007.
[22] Motonobu Kanagawa and Kenji Fukumizu. Recovering Distributions from Gaussian RKHS Embeddings. In International Conference on Artificial Intelligence and Statistics, volume 33, 2014.
[23] B. Sriperumbudur, A. Gretton, K. Fukumizu, B. Schölkopf, and G. Lanckriet. Hilbert space embeddings and metrics on probability measures. Journal of Machine Learning Research, 11: 1517-1561, 2010.
[24] Arthur Gretton, Karsten M. Borgwardt, Malte Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13(25):723-773, 2012.
[25] Max Welling. Herding dynamical weights to learn. In International Conference on Machine Learning, 2009.
[26] Yutian Chen, Max Welling, and Alex Smola. Super-samples from kernel herding. In Uncertainty Artificial Intelligence, page 109-116, 2010.
[27] Junhyung Park, Uri Shalit, Bernhard Schölkopf, and Krikamol Muandet. Conditional distribu- tional treatment effect with kernel conditional mean embeddings and u-statistic regression. In International conference on machine learning, pages 8401-8412, 2021.
[28] Diego Martinez Taboada, Aaditya Ramdas, and Edward Kennedy. An efficient doubly-robust test for the kernel treatment effect. In Advances in Neural Information Processing Systems, volume 36, pages
2023.
[29] Jake Fawkes, Robert Hu, Robin J. Evans, and Dino Sejdinovic. Doubly robust kernel statistics for testing distributional treatment effects. Transactions Machine Learning Research, 2024.
[30] James M. Robins and Andrea Rotnitzky. Comments on "inference for semiparametric models: some questions and an answer Statistica Sinica, 1 114))920-996, 2001.
[31] H. Bang and J. M. Robins. Doubly robust estimation in missing data and causal inference models. Biometrics, 6144):92,997, 2005.
[32] Edward H Kennedy. Semiparametric doubly robust targeted double machine learning: review. Handbook of statistical methods for precision medicine, pages 207-236, 2024.
[33] Miroslav Dudik, John Langford, and Lihong Li. Doubly robust policy evaluation and learning. In International Conference on Machine Learning (ICML), 2011.
[34] Peter Bickel, Chris AJ Klaassen, Peter Bickel, Ya' acov Ritov, J Klaassen, Jon A Wellner, and YA' Acov Ritov. Efficient and adaptive estimation for semiparametric models, volume 4. Johns Hopkins University Press Baltimore, 1993.
[35] Anastasios Tsiatis. Semiparametric theory and missing data, volume Springer, 2006.
[36] Aaron Fisher and Edward H Kennedy. Visually communicating and teaching intuition for influence functions. The American Statistician, 1622):16-117,, 2021.
12

---

[37] Chris AJ Klaassen. Consistent estimation of the influence function of locally asymptotically linear estimators. The Annals of Statistics, 15(4):1548- 1987.
[38] Anton Schick. On asymptotically efficient estimation in semiparametric models. The Annals of Statistics, pages 1139-1151, 1986.
[39] Mark J. van der Laan and Daniel Rubin. Targeted maximum likelihood learning, The Interna- tional Journal of Biostatistics, 2(1). 2006.
[40] Victor Chemozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21(1), 2018.
[41] Alex Luedtke and Incheoul Chung. One-step estimation of differentiable Hilbert-valued param- eters. The Annals of Statistics, 52(4):1534 1563, 2024.
[42] Josh Givens, Henry W Reeve, Song Liu, and Katarzyna Reluga. Conditional outcome equiva- lence: quantile alternative to cate. Advances in Neural Information Processing Systems, 37: 102634-102671, 2024.
[43] Alain Berlinet and Christine Thomas-Agnan Reproducing Kernel Hilbert Space in Probability and Statistics. Springer New York, NY, 2004. ISBN 471-4-7713- doi: 10. 1007/ 978- 1-44199999999-99
[44] Paul R Rosenbaum and Donald B Rubin. The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1):41-555 1983.
[45] JM Robins. A new approach to causal inference in mortality studies with sustained exposure periods application to control of the healthy worker survivor effect. Mathematical Modeling. 7:1393-1512, 1986.
[46] Kosuke Imai and David Van Dyk. Causal inference with general treatment regimes: General- izing the propensity score. Journal of the American Statistical Association, 99(467):854-88 866, 2004.
[47] Le Song, Jonathan Huang, Alex Smola, and Kenji Fukumizu. Hilbert space embeddings of conditional distributions with applications to dynamical systems. In International Conference on Machine Learning, pages 961-968, 2009.
[48] Zhu Li, Dimitri Meunier, Mattes Mollenhauer, and Arthur Gretton. Optimal rates for regularized conditional mean embedding learning. Advances in Neural Information Processing Systems, 35: 1333-4445,20 2022.
[49] Rahul Singh, Liyuan Xu, and Arthur Gretton. Kernel methods for causal functions: dose, heterogeneous and incremental response curves. Biometrika, 111(2):497-516. 2024.
[50] JP Aubin. Applied functional analysis, vol. 2011.
[51] Zhu Li, Dimitri Meunier, Mattes Mollenhauer, and Arthur Gretton. Towards optimal sobolev norm rates for the vector-valued regularized least-squares algorithm. Journal of Machine Learning Research, 25(181):1-51, 2024.
[52] Junhyung Park and Krikamol Muandet. A measure-theoretic approach to kernel conditional mean embeddings. Advances in Neural Information Processing Systems, 2020.
[53] Steffen Grunewalder, Guy Lever, Luca Baldassarre, Massi Pontil, and Arthur Gretton. Modelling transition dynamics in mdps with rkhs embeddings. International Conference Machine Learning, 2012.
[54] Bernhard Schölkopf, Ralf Herbrich, and Alex Smola. generalized representer theorem. In David Helmbold and Bob Williamson, editors, Computational Learning Theory, pages 416-426, Berlin, Heidelberg, 2001. Springer Berlin Heidelberg. ISBN 978-3-540-44581
[55] Andrea Caponnetto and Ernesto De Vito. Optimal rates for the regularized least-squares algorithm. Foundations of Computational Mathematics, 7:33 7-368, 2007.
13

---

[56] Simon Fischer and Ingo Steinwart. Sobolev norm learning rates for regularized least-squaree algorithms. Journal of Machine Learning Research, 21(205):1-38, 2020.
[57] Ilya Tolstikhin, Bharath K. Sriperumbudur, Krikamol Mu, and et. Minimax estimation of kernel mean embeddings. Journal of Machine Learning Research, 18(86), 2017.
[58] Whitney K Newey and Daniel McFadden. Large sample estimation and hypothesis testing. Handbook of econometrics, 4:2111-2245, 1994.
[59] Johann Pfanzagl and Wolfgang Wefelmeyer. Contributions to a general asymptotic statistical theory. Statistics & Risk Modeling, 3(3-4):379-338)) 1985.
[60] J Pfanzagl. Estimation in semiparametric models, some recent developments, volume 63 of lecture notes in statistics, 1990.
[61] Aad Van Der Vaart. On differentiable functionals. The Annals of Statistics, pages 178-204, 1991.
[62] Edward Kennedy, Zongming Ma, Matthew D. McHugh, and Dylan S. Small. Non- -or-maraettric Methods for Doubly Robust Estimation of Continuous Treatment Effects. Journal of the Royal Statistical Society Series B: Statistical Methodology, 1924):122- 09 2017.
[63] Kyle Colangelo and Ying-Ying Lee. Double debiased machine learning nonparametrie inference with continuous treatments. Journal of Business Economic Statistics, pages 1-26, 2025.
[64] Houssam Zenati, Judith Abécassis, Julie Josse, and Bertrand Thirion. Double debiased machine learning for mediation analysis with continuous treatments. In International Conference on Artificial Intelligence and Statistics, volume 258, pages 4150 2025.
[65] Junhyung Park and Krikamol Muandet. Towards empirical process theory for vector-valued func- tions: Metric entropy of smooth function classes. In International Conference on Algorithmic Learning Theory, volume 201, pages 1216-1260, 2023.
[66] Ilmun Kim and Aaditya Ramdas. Dimension-agnostic inference using cross u-statistics. Bernoulli, 30(1):683 711, 2024.
[67] Carl-Johann Simon- Gabriel, Alessandro Barp, Bernhard Schölkopf, and Lester Mackey. Metriz- ing weak convergence with maximum mean discrepancies. Journal of Machine Learning Research, 24(184): 2023.
[68] Estimation of the warfarin dose with clinical and pharmacogenetic data. New England Journal of Medicine, 360(8):553-64, 2009.
[69] Houssam Zenati, Alberto Bietti, Matthieu Martin, Eustache Diemert, Pierre Gaillard, and Julien Mairal. Counterfactual learning of stochastic policies with continuous actions. Transactions on Machine Learning Research, 2025, ISSN 2835-8856.
[70] Loic Matthey, Irina Higgins, Demis Hassabis, and Alexander Lerchner. dsprites: Disentangle- ment testing sprites dataset.
2017.
[71] Liyuan Xu and Arthur Gretton. Causal benchmark based on disentangled image dataset. 2023.
[72] Michael Arbel, Anna Korba, Adil Salim, and Arthur Gretton. Maximum mean discrepancy gradient flow. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
[73] Alexandre Galashov, Valentin De Bortoli, and Arthur Gretton. Deep mmd gradient flow without adversarial training. arXiv preprint 2055 2024.
[74] Joseph Diestel and B Faires. On vector measures. Transactions of the American Mathematical Society, 198:253-274 271, 1974.
[75] Nicolae Dinculeanu. Vector integration and stochastic integration Banach spaces. John Wiley & Sons, 2000.
14

---

[76] K. Fukumizu, A. Gretton, X. Sun, and B. Schölkopf. Kernel measures of conditional dependence. pages 489 Cambridge, MA, 2008. MIT Press.
[77] Bharath Sriperumbudur, Kenji Fukumizu, and Gert RG Lanckriet. Universality, characteristic kernels and rkhs embedding of measures. Journal of Machine Learning Research, 12(7), 2011.
[78] Karsten M Borgwardt, Arthur Gretton, Malte J Rasch, Hans-Peter Kriegel, Bernhard Schölkopf, and Alex J Smola. Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics, 22(14):49-<:7, 2006.
[79] A. Gretton, K. Borgwardt, M. Rasch, B. Schölkopf, and A. J. Smola. A kernel method for the two-sample problem. pages 513-520, Cambridge, MA, 2007. MIT Press.
[80] David Lopez-Paz, Krikamol Muandet, Bernhard Schölkopf, and llyy Tolstikhin. Towards a learning theory of cause-effect inference. In International Conference Machine Learning, pages 1452-1461. PM 2015.
[81] Ilya Tolstikhin, Bharath K Sriperumbudur, Krikamol Mu, et al. Minimax estimation of kernel mean embeddings. Journal of Machine Learning Research, 18(86):1-47, 2017.
[82] Steffen Grünewälder, Guy Lever, Luca Baldassarre, Sam Patterson, Arthur Gretton, and Mas- similano Pontil. Conditional mean embeddings as regressors. In International Conference on Machine Learningg, 2012.
[83] Iia Klebanov, Ingmar Schuster, and Timothy John Sullivan. rigorous theory of conditional mean embeddings, SIAM Journal on Mathematics of Data Science, 2(3):583-606, 2020.
[84] Ingo Steinwart and Andreas Christmann. Support Vector Machines. Springer Publishing Company, Incorporated, 1st edition, 2008 ISBN 0387772413.
[85] Felipe Cucker and Steve Smale. On the mathematical foundations of learning. Bulletin of the American mathematical society, 39(1): 1-1):):4,,,)) 2002.
[86] Ingo Steinwart and Clint Scovel. Mercer's 's theorem on general domains: On the interaction between measures, kernels, and rkhss. Constructive Approximation, 35:363-417, 2012.
[87] Marine Carrasco, Peerree Florens, and Eric Renault. Linear inverse problems in structural econometrics estimation based on spectral decomposition and regularization. Handbook of econometrics, 6:5633-5751, 2007.
[88] Steve Smale and Ding-Xuan Zhou. Learning theory estimates via integral operators and their approximations. Constructive approximation. 26(2):153-172 2007.
[89] Dimitri Meunier, Zhu Li, Arthur Gretton, and Samory Kpotufe. Nonlinear meta-leaming can guarantee faster rates. SIAM Journal on Mathematics of Data Science, 7(4): 1615, 2025.
[90] Daniel G Horvitz and Donovan J Thompson. A generalization of sampling without replacement from finite universe. Journal of the American statistical Association, 47(26)):663 685, 1952.
[91] Alex Luedtke and Incheoul Chung. One-step estimation of differentiable hilbert-valued parame- ters. The Annals of Statistics, 1334):15341156) 2024.
[92] Peter D. Lax. Functional Analysis. Pure and Applied Mathematics. John Wiley & Sons, 1st edition, 2002.
[93] Denis Bosq. Linear processes in function spaces: theory and applications, volume 149. Springer Science & Business Media, 2000.
[94] AW van der Vaart and Jon A Wellner. Weak convergence and empirical processes with applications to statistics. Journal of the Royal Statistical Society-Series A Statistics Society, 160(3):596-608 1997.
[95] Mark J van der Laan, Sherri Rose, Wenjing Zheng, and Mark J van der Laan. Cross-validated targeted siciiiuum--os-bbaae estimation. Targeted learning: causal inference for observational and experimental data, pages 459-474, 2011.
15

---

[96] Ulf Grenander. Probabilities on algebraic structures. Wiley, New York, 1963.
[97] Francis Bach, Simon Lacoste-Julien, and Guillaume Obozinski. On the equivalence between herding and conditional gradient algorithms. Proceedings of the 29th International Coference on International Conference on Machine Learning. ICML' 12, page 1355-1362, 2012.
[98] Bharath Sriperumbudur. On the optimal estimation of probability measures in weak and strong topologies. Bernoulli, 22(3):1839-188) 1893, 2016. ISSN 13507265.
[99] Hidetoshi Shimodaira. Improving predictive inference under covariate shift by weighting the -edlikkli function. Journal of statistical planning and inference, 90(2)227-244, 2000.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims are stated in the abstract and are clearly detailed at the end of the introduction along with the contributions of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We provide discussion section at the end of the main text in the paper and diseuss all assumptions which are made.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
17

---

Answer: [Yes]
Justification: The paper does provide theorems and propositions in which assumptions are clearly stated. Proofs and further assumptions are provided in the Appendix of the paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
The necessary details reproduce the experiments are provided in the main text and the Appendix. The code to do the experiments will be open-sourced upon acceptance of the manuscript.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
18

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: The code is provided in the supplementary material with Read.ME file with instructions to reproduce the results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/
publicc/guude///ddeatttm httpsi//annaaeeeedeeeetaaasasaaa oow.ssoboioiiooooooooooooo
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https for more details.
//nips //ips.cc/puut cttp://cccpbbbccccuiceeeeecs guides/ http:p//bb///.....ooo/o/////aaaa
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification:
Such details are provided in Appendix 14
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
The paper provides such error bars and confidence intervals accross random experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
19

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: The information is provided in Appendix 14
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: Our research does not involve human subjects, sensitive data, personally identifiable information. All experiments are conducted using synthetic or publicly available datasets in accordance with licensing terms, and no foreseeable societal environmental harm anticipated.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper discusses how the method could enhance decision-making in a range applications such as precision medicine, targeted advertising, etc. Improving decision making in these applications can provide positive broader impact in society.
20

---

Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out, For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI...
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper does not pose such anticipated risk.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
21

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the datasets.paperssiii paperswi thcode. www.com//aaaa datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
22

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy ottp:e///eerrrpo.cccoo/oeeeereee for what should or should not be described.
23

---

Appendix
This appendix organized as follows:
Appendix 8
summary of the notations used in the analysis.
Appendix
9
review of counterfactual mean embeddings that are instrumental in Section
Appendix
T0
proof for the asymptotic analysis of the plug-in in estimator presented i
Section
3
Appendix contains further details on the efficient influence function of our counterfactual
policy mean embedding and the associated estimator presented in Section 4
Appendix [2]
provides the analysis of the doubly robust kernel test of the distributional
policy effect presented in Section 5.1
Appendix 3 does the same for the sampling algorithm presented in Section
Appendix details on the implementation of the algorithms and additional experiment
details, discussions and results.
All the code to reproduce our numerical simulations is provided in the supplementary material and will be open-sourced upon acceptance of the manuscript.
8 Notations
In this appendix, recall for clarity some useful notations that are used throughout the paper.
Notations for distributional off-policy evaluation setting and finite samples
V/: Og, xy are realizations of the outcome, action, and context random variables Y, A, X for E {1,...n) nn). Potential outcomes are written (Y(a) (Y(a))aEA A-
The distribution on the context space is written Px the distribution on outcomes is condi- tional actions and contexts and is written PYIX.A- Distributions on actions A are policies т belonging to a set II. In the logged dataset, actions are drawn from logging policy mo- Resulting triplet distribution is written Ps PYI X π x Px.
The distribution v(TT represents the marginal distribution of outcomes
Px.
Notations related the kernel-based representations used embed counterfactual outcome distributions
Hr is generic RKHS associated with a domain F.
HAx: RKHS on x X with kernel Ax and feature map (a, KAXls (a.
Inner product: (., ) HAA
Hy: RKHS on y with kernel by and feature map Φy(y) = ky Inner product: (: Hy
Given distribution P over F. the kernel mean embedding is μæ = Eplor Hy.
For conditional PFia. the conditional mean embedding is #F]((g) = Elos(F) G 9l E
H.y.
The counterfactual policy mean embedding (CPME): X(T) = Ep, (Y
Kazz Ky: bounds kernels: supa,æ (x ar)l Ka,z; supy (l w Kig S2(HHAx Hy) denotes the Hilbert space of the Hilbert-Schmice operators from HAx to Hy.
Cr|J, S2(HAx Hly is the conditional mean operator.
Ux: the kernel policy embedding in H.A.
C, d: source condition and spectral decay parameters.
A: regularization parameter for learning CY
L: Kernel integral operator Lh == f k(,, (w) dp(w), mapping (²( + (p).
{ng1ı21: Eigenvalues of L, ordered decreasingly, assumed satisfy spectral decay assumption π7 S Cj-
{40) (2:) Orthonormal eigenfunctions of L in (p2(p), satisfying Lup; nj44z-
H Interpolation space of order defined as H ;= { E, hypp h3/ < co).
Notations related to estimators and asymptotic analysis
- (T) and (T): plug-in and doubly robust estimators of CPME. AY z); z): estimator of the conditional mean embedding.
24

---

Au(a/z): estimator of the logging policy.
[n,, b, c): convergence rate of Cy)
Txg (n): convergence rate of tt.
-,,(1, (,(1): standard probabilistic asymptotic notations.
Notations for differentiability and statistical models
P: statistical model on 2 = y A X.
(p): space of square-integrable real-valued functions w.r. measure p. (P; H): Bochner space of H-valued functions with norm ||
(f dP(=)) d)))))
(h W]: orthogonal projection of onto closed subspace W C H.
Pxx submodel of P with fixed treatment policy To.
Pp: tangent space at P.
(P, ((PPP,s). s): smooth submodels of d at P with score
8, 8x, SY SA|X: score functions.
x(T)(()) value the CPME at P.
Xp: local parameter of x(T) at P.
XP
adjoint (efficient influence operator).
H.P: image of T.*
Notations for efficient influence functions
efficient influence function (EIF) at P. Dp candidate EIF: ( a, x) XP (dy)(,,
Error decomposition of the one-step estimator
Pn: empirical distribution of the sample {
Pn: estimated distribution using nuisance estimators.
Sn (Pr P) empirical average term.
-Tn (Pn (1 ))"): empirical process term.
Rr = x(Px) + P - ( remainder term.
Notations for empirical processes and equicontinuity
Tx = -
(p):: empirical process acting on y.
class valuue functions (e.g. vir r)
Notations for hypothesis testing
Ho: null hypothesis - =(T)= u(T²).
Hi: alternative hypothesis- - ((T)) # u(T').
Pn,at difference of EIFs for policies and n'
Pa,a's Pn,a' estimates of Pr.a" over disjoint subsets.
(x) x (da I): estimated conditional policy mean.
-ft,x( a, x)): cross- Pross-U-statistee kernel.
fi P 5t ". empirical mean and std of ft.
T r normalized test statistic.
IHl: limiting Gaussian process in Hy.
(IHI, 11) Hy: projection onto direction h.
4: CDF of standard normal.
(TT ) p-value.
Notations for sampling from counterfactual distributions
(ū, ) deterministic samples generated via kernel herding.
Pmm: empirical distribution over the ūj-
Yddr, Pmy. empirical distributions generated from & (Tr) and (T)(T).
pm
25

---

Review of Counterfactual Mean Embeddings
In this appendix, we provide a background section on counterfactual mean embeddings 17 and distributional treatment effects.
9.1 Reproducing kernel hilbert spaces and kernel mean embeddings
scalar-valued RKHS Hw is a Hilbert space of functions W - R. The RKHS is fully characterized by its feature map, which takes a point w in the original space W and maps it to feature (w(w) in RKHS Hw. The closure (pan()w(w) wew RKHS Hw. In other words, (dw(w) heww can be viewed as the dictionary of basis functions for RKHS Hw. The kernel kw : W x W + IR is the inner product of features Φw(w) and ow (w").
kw (w, w') = (dw( thw (w ()))))
(15)
real-valued kernel & is continuous, symmetric and positive definite. The essential property of function & in an RKHS Hw is the eponymous reproducing property:
h(w) = (h, (w(ww)us
(16)
In other words, to evaluate h at u, we take the RKHS inner product between and the features éw(w) for Hw. The reproducing property, importantly, allows to separate function h from features (w(w) and thereby decouple the steps of nonparametric causal estimation. Notably, the RKHS is practical hypothesis space for nonparametrie regression.
Example 9.1. (Nonparametric regression) Consider the output E R. the input w E W and the goal of estimating the conditional expectation function h(w) IE(Y W w). kernel ridge regression estimator of h is
T Σ tu (h, dw (w)) 32 + AIAA
h arg min hEH
(17)
Ti
where X >> O is a hyperparameter on the ridge penalty [/ile which imposes smoothness in estimation. The solution the optimization problem has well-known closed form:
hw)=='² (Kww + -nAT) Kww-
(18)
The closed-form solution involves the kernel matrix Kww E Rn with (i,j) th entry low (wx, wy), and the kernel vector Kww E I. with i th entry low (wi, w).
In this work, we use kernels and RKHSs to represent, compare, and estimate probability distributions. This is enabled by the approach known kernel mean embedding (KMH) of distributions (21), which we briefly review here. Let Hw be a RKHS with kernel kw defined on space W, and assume that supwew (w w) < oc. Then, for a probability distribution over W, the kernel mean embedding defined as the Bochner integra
# P Hw, PH up :- (kw(+, dP(w).
The embedded element ptp,, also written pww when W ~ P. serves as a representation of P in Hyy If Hw is characteristic 776 23 77., this mapping is injective: μr luQ if and only if P = Q. Thus, HP uniquely identifies P. preserving all distributional information. Common examples of characteristic kernels on dd include Gaussian, Matérn, and Laplace kernels (23 77,, while linear and polynomial kernels are not characteristic due to their finiteddimensional RKHSs
The kernel mean embedding induces a popular distance between probability measures known as the maximum mean discrepancy (MMD) 78 79 24 For distributions and Q, it is defined by:
InaP-
- f had
MMD Hw, P. Q] := ||p Mollawe
sup hEHws|AAlss
See, e.g., 74 Chapter 2] and 775 Chapter for the definition of the Bochner integral.
26

---

The second equality follows from the reproducing property and the structure of RKHSs as vector spaces [24 Lemma 4]. If Hyy characteristic, then MMD Q] 0 implies P Q, so
MMD defines a proper metric on distributions.
Given an i.i.d. sample {we) from P, the kernel mean embedding can be estimated via the empirical average:
Ap Σ kw(:, ws).
This estimator is w /n-consistent: ||up prll Hw Op(n=1/2 under mild assumptions
24 80
81
Given a second i.dd sample } from Q, the squared empirical MMD
MMD 2 Hw, P.Q] = |lAp- A fi
n,m Σ k-=1
m Σ
kw(ws,
kw(ws, wS) + m k-=1
kw(ws,
n 1
TUT
This estimator is consistent and converges at the parametric rate pn but simple to compute; an unbiased version is also available [24] Eq. 3].
1/2 + m 1/3). It biased
The KME framework extends naturally to conditional distributions 47 random variable W x V with joint distribution Pwy. Using kernels low and by with RKHSs Hw, Hy, the conditional mean embedding of Pviw= is defined as:
Let (W, V) be
WV| / kp(s dP(olw) E Hy.
This representation preserves all information if Hy is characteristic. Given a sample {[ws, o)) the conditional embedding can be estimated as
Σ (,(w)b((...)
with weights
B(w) (K + (nd))
(w(w) = (kw(w, w1). ". kw w,))
Here, K is the n kernel matrix with entries Kij kw(w; wj); and À > 0 is regularization parameter. This estimator corresponds to kernel ridge regression from W into Hy, where the target functions are feature maps ky(:, v2). To guarantee convergence, À must decay appropriately as + 00 51).
Finally, we make of the Hilbert space S2 S2WW, Hw:) of Hilbert-Schmidt operators between RKHSs. The conditional expectation operator C: Hw, + Hw, given by h(-) + [B]h(W))[W,, = is assumed to lie in S2 (2H,, Hw,)) and is estimated via ridge regression, by regressing dw, (W1) on ow, (W2) in Hw.
9.2 Assumptions for consistency
To prove consistency of our estimator, we rely on two standard approximation assumptions from RKHS learning theory: smoothness of the target function and spectral decay of the kernel operator. These are naturally formulated through the eigendecomposition of an associated integral operator, which we introduce below. The results may be found in 84]
Kernel smoothing operator Let Hw be a reproducing kernel Hilbert space (RKHS) over a space W. with reprodueing kernel with kernel kw W x W + F consisting of functions of the form h: W + R. Let be any Borel measure on W. Let (p( be the space of square integrable functions with respect to measure p. We define the integral operator associated with the kernel ky and the measure p as:
L: L: (p) + (2) + kwl (w((
(19)
27

---

Intuitively, this operator smooths function h by averaging it with respect to the kernel kw and the distribution p.
Remark 10. (L as convolution). If the kernel kw is defined on W C I and shift invariant, then L a convolution of kw and h. If kw is smooth, then Lh is smoothed version of h.
Spectral properties of the kernel smoothing operator The operator compact, self-adjoint, and positive definte, Therefore, by the spectral theorem, I. admits orthonormal basis of eigenfunctions (py)p in (WW), with corresponding non-negative eigenvalues (nz)-
Assumption 11. (Nonzero eigenvalues). For simplicity, we assume (1)) in this discussion; see 185 Remark for the more general case.
Thus, for any h (p). write:
Lh Σ (o) 3=1
where each by is defined up p-almoos--eeerywheeee equivalence.
Feature map representation The following observations help to interpret this eigendecomposition.
Theorem 12. 186 Corollary 3.5] (Mercer's Theorem). The kernel kw can be expressed as
kw (w, w²) = Ej=1 NjPr(w))) y (w'), where (w, w') are in the support of p, Pj is a continuous element the equivalence class (pp)), and the convergence is absolute and uniform.
Since the kernel kw can be decomposed as:
kw(w, w') ())(())))(1)))
with absolute and uniform convergence on compact subsets of the support p, we can express the feature map (w(w) associated with the RKHS as:
(w((w) = (VTisp VTasp(uw)....
Thus, the inner product (ow(w), dw (w"))) reproduces the kernel value kw(w, w').
Both (p) and the RKHS H. can be described using the same orthonormal basis (4yy), but with different norms.
Remark 13. (Comparison between H and LZ(()). A function h E (2)p) has an expansion
= E3 hy9s, and:
Σ h3
j=1
A functio E H. has the same expansion, but the RKHS nomn is:
h2
|
4
This means that functions with large coefficients on eigenfunctions associated with small eigenvalues are heavily penalized in H. which enforces a notion of smoothness.
To summarize, the space L. contains all square-integrable functions with respect to the measure p. In contrast, the RKHS 76 is subspace of L consisting of smoother functions-ther whose spectral expansions put less weight on figh-freeuency eigenfunctions (i.e., those associated with small eigenvalues ))))
This motivates two classical assumptions from statistical learning theory: the smoothness assumption, which constrains the target function via its spectral decay profile, and the spectral decay assumption, which characterizes the approximation capacity of the RKHS.
Remark 14. The smoothness assumption governs the approximation error (bias), while the spectral decay controls the estimation error (variance). These assumptions together determine the learning rate of kernel methods.
28

---

Source condition To control the bias introduced by ridge regularization, we assume that the target function lies in smoother subspace of the RKHS. This is formalized by a source condition, common assumption in inverse problems and kernel learning theory 55 87 88
Assumption 15, (Source Condition) There exists c E [1,2] such that the functtnn belongs to the subspace
h
H
h
20 H.
==1
When = 1, this corresponds to assuming only that h E H. Larger values imply greater smooth- ness: the function h can be well-approximated using only the leading cigenfunctions. Intuitively, smoother targets lead to smaller bias and enable faster convergence of the estimator h.
Variance and spectral decay To control the variance of kernel ridge regression, we must also constrain the complexity of the RKHS. This done via spectral decay assumption, which controls the effective dimension of the RKHS by quantifying how quickly the eigenvalues f] of the kernel operator vanish.
Assumption 16. (Spectral Decay) We assume that there exists constant C' > 0 such that, for all j.
n S Cj for some 2 1.
This polynomial decay condition ensures that the contributions of high-frequency components decrease rapidly. A bounded kemel implies that b 1 [56] Lemma 10]. In the limit 6 - the RKHS becomes mnite-dimensional. Intermediate values of define how "large" or complex the RKHS is, relative to the underlying measure p. A larger b corresponds smaller effective dimension and thus a lower variance in estimation.
Space regularity We also require an additional assumption on the regularity of the domains. Assumption 17. (Original Space Regularity Conditions) Assume that A, X (and y) are Polish spaces.
A Polish space is separable, completely metrizable topological space. This assumption covers broad range of settings, including discrete, continuous, and infiniteddimensional cases. When the outcome Y is bounded, the moment condition is automatically satisfied.
9.3 Further details on Counterfactual Policy Mean Embeddings
To justify Proposition we rely on the classical identification strategy established by Rosenbaum and Rubin [4 and Robins 45 Recall that the counterfactual policy mean embedding is defined as
x(x) := ExxPx (Y))
which involves the unobserved potential outcome (a). Under Assumption I we proceed to express this quantity in terms of observed data.
First, by the consistency assumption, we have that for any realization where = a, the observed outcome satisfies Y = Y()). Second, by conditional exchangeability, we have that Y()) 1 A X, which implies that the conditional distribution of Y(a) given X x is equal to the conditional distribution of Y given A = X x. That is,
(( X
(Y) A= a, X= = x) (a,)
Finally, under the strong positivity assumption, the conditional density (a ()) is strictly bounded away from zero for all a E A, x E X, ensuring that the conditional expectation PY (a, x.) is identifiable throughout the support of Px. follows that
( = Exxxx [E(oy(Y X æ]] ExxPx [uyIa,x(a,
which completes the identification argument.
29

---

10
Details and Analysis of the Plug-in Estimator
In this appendix, provide further details on the analysis of the plug-in estimator proposed in Section
10.1
Decoupling
We propose plug-in estimator based on conditional mean operators for the nonparametric distribution of the outcome under policy target policy я. Due to decomposition property specific to the reproducing kernel Hilbert space, our plug-in estimator has a simple closed form solution. Proposition 4 (Decoupling via kernel mean embedding)). Suppose Assumptions the counterfactual policy mean embedding can be expressed as:
and 3 hold. Then,
X(T) = CyJAXPP
Proof. In Assumption 3 we impose that the scalar kernels are bounded. This assumption has several implications. First, the feature maps are Bochner integrable [84] see Definition A.5.20]. Bochner integrability permits us to interchange the expectation and inner product. Second, the mean embeddings exist. Third, the product kernel also bounded and hence the tensor product RKHS inherits these favorable properties. By Proposition and the linearity of expectation,
/ prlAx((a (dd(())))))
x(x)
-CIAX |alaa) a
-CIAAXHT
10.2 Analysis of the plug-in estimator
We will now present technical lemmas for kernel mean embeddings and conditional mean embeddings.
Kernel mean embedding For expositional purposes, summarize classic results for the kernel mean embedding estimator و. for μı = E{o(Z))
Lemma 10.1. (Bennett inequality; Lemma of Smale and Zhou (88)) Let (Ei) be i.i.d. random
variables drawn from the distribution P' taking values real separable Hilbert space K. Suppose there exists M such that ||Silkx
M < co almost surely and o² (E:) E (11:1112). Then for all
E N for all 10,
2M log(2/8)
(2) log(2/0) 1/27
pr
& B(E)
21-8
7
We next provide a convergence result for the mean embedding, following from the above. This included to make the paper self contained, however see [57] Proposition A.I] for improved constant and a proof that the rate minimax optimal.
Proposition 18. (Mean embedding Rate). Suppose Assumptions 3 and hold. Then with probability 1 6,
ر| - Mxllx I Tun (n,, 6) 4x, log(2/8)
n/2
Proof.
The result follows from Lemma 10.1 with & = = (Zi), since
2ng log (2/6) 2nd (2/8))
1/2
4ng log(2/8)
à (Zz) Ep, {o())
1/2
n
n
See [21 Thcorem 2] for alternative argument via Rademacher complexity.
30

---

Below, we restate Assumptions 15 and 16 for the RKHS HAX which are used to establish the convergence rate of learning the conditional mean operator CyJa,x- Our formulation of Assumption 15 differs slightly from the one in Appendix 9.2 but they are
Conditional mean embeddings
equivalent due to [55] Remark 2].
Assumption
15 (Source condition.). We define the (uncentered) covariance operator Eax El X x (A, x)). There exists a constant B < oo such that for a given E (1,3).
E AX 11220LAx. S B
In the above assumption, the smoothness parameter is allowed to range up to c < 3, in contrast to prior work on kernel ridge regression, which typically restricts c S [e.g. 56]. This extension is justified by Meunier et al. [89] Remark 7 and Proposition 7J, who showed that the saturation effect of Tikhonov regularization can be extended in Theorem 19 rather than the I,² norm.
S 3 when the error is measured in the RKHS norm, as
Assumption 16 (Figenvalue decay.). Lel (A1,))521 be the eigenvalues of EAxx. For some constant B > 0 and parameter b E (0, 1] and for all $21,
d1. C
Theorem 19. (Theorem 3 (5TV) Suppose Assumptions,
16 and hold and take X1
+ 65176 There is constant J1 independent of 1 and å (0, 1) such that
añito
CYJA. CY1A.X (s(Hax:HH) S J1 log(4/8)
=: rc (6, b.
is satisfied for sufficiently large with probability at least 1-8. 8.
We will now appeal to these previous lemmas to prove the consistency of the causal function. Theorem 5 (Consssstyyyyy of the plug-in estimator))). Suppose Assumptions 3
15
16
and
17
Set
n 1/c+1/b), which is rate optimal regularization. Then, with high probability,
[n (T) - x = O (w..0r0.,nbbooooo.m 8, O 0 (e-1)/(((++111))
Proof of Theorem 5 We note that
Xxi(T) - x(*) CrJA,XAx CYIA,XHa
= CYIAX (it - -(x=-)a)+ + (CYIAXX - CY1AX) Ur
= =(CrIA,-- -C1AA)) (px- + Crjax (pn Ux) (Oriux Cr(ax) pr.
Therefore we can write with Cauchy-Schwarte inequality:
(xm((T) x(==) < |rIIx CrIA.X
l1 M lx
S2(HAx Hyy
+ ||Cr|A, (CYIA.xIls,CHHA.P. l1 Mall
+ CrAA. - CrIA, (SS(HAxx.H))) || 1l
Therefore by Theorems 19 and 18 with probability 1 25,
(Rpi(T) x(x)) re(n,,,,, b, ()) + |[Ciaax (p( Ka,z (n, 5, c).
Using Assumption 15
we observe that ||Cr|a,x| (lsaxxllss((Ax S Bx As result, the above
bound readily gives
(pp(T)) -(Tr S -
31

---

10.3 Further details and Estimation strategies for the kernel policy mean embedding
Discrete Action Spaces. When the action space A is discrete, we can directly compute the kernel policy mean embedding by exploiting the known form of the target policy Ta r). For each logged context 24, we compute a convex combination of the feature maps ΦAX (a, æı), weighted by the policy (a 2z). This leads to the following empirical estimator:
t(a (0,
The plug-in in estimator for the counterfactual policy mean embedding then admits the following matrix expression:
{(T) = -YJAXA
-KAA & Kxx +
($x
"E> Σ
Ta ())
i=1 aEA
=(KA & XXX eXx +
( & 2 (
3 2x)1- 1-
④ Oxx
= 6 xxx HAT) (K₂ @ 1xx)
n2
where Kx(i,5] EoEd kala, a) 2z), and o denotes the seiiiye features.
Algorithm 3 Plug-in estimator of the CPME (Discrete actions)
Require:
Kernels kx,, A, ky, and regularization constant À >>
Input: Logged data (zi, ai, 34 3/) target policy Taa x))
1: Compute empirical kernel matrices KAA, Kxx Ruxa from the samples {(as, Ri)
2: Compute the kernel outcome matrix K ky(y,, w)...., ky(Un, v)]
3: Compute K₂ E Rax= with entries K2 [. = EnEA a(a xz)) KAX (ax((as,aa 2z), (a,
4: Set K = K2 1 .L.....1)
Output: An estimate (v) Kyy (Kar @ Kxx + mA
Continuous Actions via Resampling. When A is continuous and no closed- sum over actions available, we instead approximate the kernel policy mean embedding by resampling from r(: x;). Specifically, for each logged covariate H. we sample ä; ~ .(. x;), and form the empirical estimate:
Ar
Σ oxx i=1
n1
This leads to the following expression for the plug-in estimator:
x(=) = C AxKx
n = -KAA O Kxx + + (# 8 2x)
Σ
(xx((r
71
=(KA Kxx +
+ xx
xx)1
n
KΛAOKXX
= (KAs & Kxx + (K (KAi O
where K KAil.5] ka(as, āz), and ā, is drawn from x(- 2z).
32

---

Algorithm 4 Plug-in estimator of the CPME
Require: Kernels kx, KA, ky, and regularization constant À > O.
Input: Logged data (zy, ag 3t) the target policy n,
1: Compute empirical kernel matrices KAA, Kxx RRxT from the empirical samples
2: Compute the kernel outcome matrix Kyr = (y(11, yy. (y(Un v)]
3: Compute K with resampling, K = (K AĀ @ (xx).(1... 1) and A ~ T(-1)).
Output: An estimate (p) Kyy (Kaa Kxx + mAT) m K.
Importance Sampling This resampling procedure can be quite cumbersome however, and not appropriate for off off-polleee policy learning. When propensity scores are known, an optional alternative is to invoke an inverse propensity scoring method [90], which expresses the embedding under the target policy и as reweighting of the observational distribution:
To(a l dAX (a, .0)
Ta x))
Fx = E To
(20)
This formulation enables direct estimator of f,, from logged data ((x,, Os y:)) using the known logging policy To:
(aa xz) (a; xF4)
Ax
&
Ax
(21)
Let Wx E Rn be the vector of importance weights Wa a(ag)mr)) and let 4 (0xx(11,2);; ",x (x)] Then the estimator admits the vectorized form:
+h
Ax
W
(22)
Accordingly, the closed-form expression for the plug-in estimator becomes:
x(x) = CrJA,x
= -KAa O Kxx + HAT) ( 2 xx)
FAN
W
n
OKAA O Kxx + +AAI (KAA C Kxx) W.
This estimator leverages all observed samples without requiring resampling or external sampling procedures, and especially suited to settings where both the logging and target policies are known or estimable. However, its stability critically depends on the variance of the importance weights W which may require regularization or clipping in practice. Morcover, this estimator is not compatible with the doubly robust estimator proposed in the next section.
11 Details and Analysis of the Efficient Score Function based Estimator
In this appendix, provide background definitions and lemmas on the pathwise differentiability of RKHS-valued parameters [34] 41), followed by the derivation and analysis of a one-step estimator for the counterfactual policy mean embedding (CPME).
As stated in Assumption 17 we work on a Polish space (Z, with y A X' and consider collection of distributions P defined on (2, Let 21...., ~ Po be an i.i.d. sample from some Po E P. and denote by P,, the empirical distribution. Let P, P be an estimate of Po. For measure on (X', Σ), the space (p) denotes the Hilbert space of p-almost surely equivalence classes of real-valued square-integrable functions, equipped with the inner product (f, () (²()) = J {g dp. For any Hilbert space H. write LP;P; H) for the space of Bochner-measurable functions Z H with finite norm
| == (/ [1(())]
1/2
If Wi is closed subspace of H. we denote by 11 [/| W] the orthogonal projection of h onto W.
33

---

11.1
Background on pathwise differentiability of RKHS-valued parameters
We begin with brief review the formalism used to characterize the smoothness of RKHS- RKHS-valued valued statistical parameters A1I Let P be a model, i.e., collection of probability distributions on the Polish space (J' x A B) dominated by a common o-finite measure p.
Definition 11.1. (Quadratic mean differentiability) A submodel {Pe : e E [0,0)3 C P is said be quadratic mean differentiable at P if there exists score function 8 E (22(P) such that
p2!2 p212
e _sp1/2 12(0)
= =(())
2
dP and P. where p dp
dP. dp
We denote by (PP, P.s., .) the set of submodels at with score function s. The collection of such (P) for which (PP.P, P,,. Ø called the langent sel, and its closed linear span is the tangent space of P at denoted Pp.
We define (P) = {s E (P) f dP = 0}. the largest possible tangent space, and refer to models with Pp = (P) for all P E P as locally nonparametric.
The parameter of interest is the counterfactual policy mean embedding and can written the model P as x : + Hy. such that
x(=)(P) /f Br A = a, x)) (da 1 Px)
(23)
Definition 11.2. (Pathwise differentiability) The parameter x(x) is pathwise differentiable at P if there exists continuous linear map Xp Pp - Hy such that for all [P.} (P.P..))
(Ix(#) (P) cx o(c)
We refer to X as the local parameter of x(x) at P. and its Hermitian adjoint (xp)' Hy Pp as the efficient influence operator. Its image, denoted H.p, closed subspace of Hy known as the local parameter space.
Next, we go on defining the efficient influence function of the parameter X(x).
Definition 11.3. (Efficient influence function) We say that x(x) has an efficient influence function (EIF) up y x xxxx X + Hy if there exists a P-almost sure set such that
xp (h)(,, a, = (h, 77 a, x) 2)H for all E Hl.
By the Riesz representation theorem, x(x) admits an EIF if and only if Xp (-) a, x) defines a bounded linear functional almost surely. In that case, up a, x) equals its Riesz representation in Hy.
In our case, since Hy an RKHS over a space V/ the local parameter space Hp itself an RKHS over ), with associated feature map øy. Define
a() = Xp (by)),y,,,,
(24)
which serves as candidate representation of the EIF, The following result will serve to show that Ji both provides the form of the EIF of X. when it exists, and also sufficient condition that can be used to verify its existence.
Proposition 20. [47 Theorem 1J, Form of the efficient influence function Suppose X is pathwise differentiable P and Hp an RKHS. Then:
i) If an EIF vp exists, then vp T almost surely.
ii) If 11 < then x(π) admits an EIF P.
Prior to that, state below a result to show a sufficient condition for pathwise differentiabilityy
34

---

Lemma 11.1. (Sufficient condition for pathwise differentiability) 191 Lemma 2| The parameter X Hy pathwise differentiable P if:
i) Xp is bounded and linear. and there exists a dense set of scores S(P) such that for all
8 S(P), a submodel {Pa] (P,, satisfies
- =
=
ii) and is locally Lipschitz at P, i.e., there exist (c, 6) > 0 such that for all Pt. P2 E B,(P),
Ilx(Pi) x( cc (P,, P2),
where H(:-)) denotes the Hellinger distance and (s(P) is the ô-neighborhood of P in Hellinger distance.
Finally, we will show that under suitable conditions, an estimator of the form
(a(T) == ((x) P
achieves efficiency.
11.2 Derivation of the Efficient Influence Function
We now prove Lemma 4. which characterizes the existence and form of the efficient influence function (EIF) of the CPME. We begin by restating the lemma for convenience.
Lemma 4.1 (Existtence and form of the efficient influence function).). Suppose Assumptions and
17 hold. Then, the CPME (T) admits an EIF which is P-Bochner square integrable and takes the form
T(a x)) (dx(F) (x) + |YIIAX(d (t)nax(dx))<(ddd" x) x(x)- To(a x)
" (y,a,
The proof proceeds in two main steps. First, establish that is pathwise differentiable in Lemma Then, we derive the form of its EIF.
11.2
Lemma 11.2. X is pathwise differentiable relative to a locally nonparametric model P at any P E
Proof. Fix II. To prove this lemma, we apply Lemma 11.1 to establish the pathwise differ- entiability of x relative to restricted model PR.. This model consists of all distributions P' such that xp- To, and for which there exists P E with P = PYIA.X and P = Px. Since the functional (TT) does not depend on the treatment assignment mechanism, we may then extend pathwise differentiability from Pro to the full, locally nonparametric model P.
Following the construction in Luedtke and Chung [41], we assume that for any P P and fixed 5 > 0, the model P contains submodels of the form {P. e (0, where the perturbations act only on the marginal of X and the conditional of A, X. Specifically,
d.P. X (x) 1+ cx)
dP d.e.Aix
d..YIA.X (y) a, = c8y1,,x) a,x),
x) =1,
dPx
dPAix
dPYjA,X
where x and 8 x are measurable functions bounded in (-5-1, 8 1,11),, satisfying
(x)) = 0 and Ep[sy]л.x( A, X] = a.s..
Step 1: Boundedness and quadratic mean differentiability of the local parameter Let my be such that mo Tp, for some fixed P' E P. The local parameter (s)s) can be expressed
(a x) To(a x)
(s)
dy(v) [SYJAXX a, a, + sx(x)] P(dz).
(25)
Boundedness. We first verify that X a bounded operator. This will establish the first part of condition (i) of Lemma [I. for the model P P.
35

---

Take any score function in the tangent space Pp. Define
SY a. == s(2, a. y) Er(s(X,, A, A = x = =]},
sx(x) = Pp Br(s) A, X =x].
It is straightforward to verify that (p(s(X,, A, A] - (p)s(X,A,Y)) X] = 0 P-almost
surely. Therefore, we have the decomposition $ = Sy |A.X + =x. Since E L (P), it follows that both A, and sx are (P) as well.
Now, under the strong positivity assumption and the boundedness of the kernel K, the integrand
(x, a, v) + Ta x)
Φy(v) (SY]A,X(V) a, sx(2]
To(a x)
belongs to ((P:Hy). Hence, the local parameter (s(s) is well-defined in Hy.
To establish boundedness of the local parameter Xp, compute its squared RKHS norm:
/I T(a x) x(a' |x') mo(a x)) no(a' xx) (y. [sy a, + sx (z)]
(
[SVIA,X(V (V' la',x') + sx(')) (dz,,
(26)
// T(a x) T(a' xx)
mola x) mola' x') V ka (ky( () kgl. a. + sx(x))
|8y (1/ d',x') + ax(x') (dzz,
(27)
no 37) V ky(w.v) sY a) sx(xa] P(az)
<aa x))
(28)
(a (K m) P(de)
// (sy1ax(yla (a +x P(dz)
(29)
Tp(a x))
suPa.r (a (+) suppey 1k (v.(v.))
/ (sriax(llaa) (xxxxx(llxx++x((())
+ sx(x)) P(dz) (30)
inf prep ess inf x Tp (a
x)
supa,p (a w) supyey Iky (v.3)) 11012=(PP)
(31)
inf ess infa. x Tp (a x))
Here: the first inequality applies Jensen's inequality to pull absolute values inside, and Cauchy-Sehwer the kernel hy. The second applies Cauchy-Schwar to split the integrals. The third uses Hölder's inequality with exponents o0). The final inequality follows from decomposing
SyJA.X 8x SAA, where
sa x) Erls(Z) A= a, X Er(s(()) X z).
We then use
|| || ||,ve,, + sx || (2a()) ||sr|J, + sxll2A())
Since the kernel ky is bounded and mo is uniformly bounded away from zero by the strong positivity assumption, the bound (3T) is finite. Therefore, X a bounded linear operator.
Quadratic mean differentiability. We now establish that x(*)) is quadratic mean differentiable P with respect to the restricted model P.o. assuming To = Tp.
As in Luedtke and Chung [11] we consider smooth submodel (Pe E (0,5)} C Pao of the form:
dPe.x (x:) 1+))x(x), dPx
dPc.A dPAix
d...YIA,X
(a x)) =1
(y a, = 1 1+SSYJA, (y (a dPriA.X
where sx and SyJA,x are bounded in 1::5-1/2,81111 satisfy Ep(sx (X) = and Ep(sy|A,X(
A, X) A, = almost surely. 'he score of this submodel at = 0 is given by ((x,,,y)
sx (z) s,)A, (yy a,x), and its ('(osuuul spans the tangent space of Pao at P.
36

---

Letting (s)s) be defined as in Equation 25
we compute
(P.) x(=)(P) ())(s))
= |// (1) + +sy11xx a, x) + esx(x)) PriA.x(dy) a,z))(da x) (dz)
-/lʃ xy(() PrIA (dy a xx)
2
[]]
a(a z))
by()) [SYIAx(V a. + +x((x)] Priaxx(y a x) =)Px(dx))
To(a x)
Hy
et |/ ( (y())sr(4xx(1 ( ())))((()) (dy laddyllaxxxnn a, x))(a 2))
i4y
e / T(a 1x)
To(a| x) by (,,))x(())) P(dz)
Nyy
This is o(E?) provided that the last Hy- Hy-norm is finite. To verify this, observe that the integrand
n (a x) ta x)
(x, v)
(y(y)sr11x (x)
belongs (P;P; assumption. Indeedm if we compute its squared norm:
((PPHy)) Hl), since ky, SYIA.X, and 8x are bounded and To satisfies the strong positivity
xaa x) |/
a. Tola x) dy( (x)))s(()) P(dz)
Hy
/f T(a x) T(a' x')
ky 3') 8YIAX(1 a, (x)))x()))) (dd, P(d=))(ds)
xo(a x)) aa x²) < 00.
Thus, x(T) is quadratic mean differentiable at P relative to Prp-
Step 2: Local Lipschitzness, Let To Tp' for some fixed P' P. We now verify that x(T) is locally Lipschitz over the restricted model P....
Fix any P, P E PTo. Define the --reweighted distributions:
Ta x) To(a x) P(z),
(T)()) = 7(a x)
(=(2)
P(z),
Tola x)
where = (.) Then:
lf -PP))))))) (P" pd=)
Ix(T)(P) (P)
/f ky (v(
T(a x)) (p- (P-P) P) (dz) T(a' x2)
To(a x²) (p- P) (dz')
To(a x)
II kolu.v')) VdP(s) VaP(s) VaP(z) - -d)())-)aP()))
VdP(z) + v dd(())
VdP(ot) + vdP(21)
(aa x)) ataa' x²)
To x)) To (a' x²)"
37

---

Applying the Cauchy-Schward Schwarz inequality yields:
lxx())(P) -(())()) (If (3(
Ta x) ataa [x') 72 To(a x)) Fo(a' xx)
VdP(z) + v vd((2)
[Vaple] + vapran) )
1/2
ddp()) vaP(s) (iP(e)-VaP(zo) 2)
(A) (P)
Where A (y,
s(ala)) ((a'(x)) T)(a|z) )o(a|>>
VdP(z) v dd(())
VaP(z) + + ddp(z))
Using the inequality (b c)² s 2(12 + c²) and applying Hölder's inequality:
A 2 /ʃ 13 (v.
TTaal x))
x(aa x²)
(P (P
To(a' x²) 2 supa,a (a x)) sSppy. k3,00))
ao(a .x)
inf e' inf 77 (a x)
This upper bound is finite under the strong positivity assumption and the boundedness the kernel ky. Therefore, x()) is locally Lipschitz over Pm.. This establishes part (ii) of Lemma 11.1 and therefore finishes the proof.
Now that we have proved Lemma 11.2 influence function.
we establish Lemma 4.1 and derive the form of the efficient
Proof. To prove Lemma
4.1 first recall that the local parameter takes the form, for E Pp
xp(x) /Iʃ dy(v) [ a, + saix(a æ)) + xx(()) (yl x) xx (dx)
Therefore, the efficient influence operator takes the form for E Fly
X (h),,,,,,, T(alx) (h(y) Bx(h(M))())))) A a, a, = x1}
(32)
To(a x)
+ Eplh(Y) A = d' = x))
(33)
// Er (b(Y) A d',x x 2' x Px (dr')
(34)
By Proposition the EIF is given by evaluating the efficient influence operator at the representer
by (3"), that is
т(a|x) Tola x) (Qv((') - Er(ov(1/) A -0 x x1}
()(() (V') = "ox((x(()))
+ Er(dy(1)) A x x x))
// Ep (0 11 d' = = (da' x) xx (dx')
38

---

Indeed this function belongs to (2(P: Recalling the definition of the conditional mean embed- ding (Y)A,((a, 2) in (3) and noting that Ep [UY]A,X (a, x)] = x())((),, we can rewrite the above as follows:
[a x) (z)2) (a x)
[d)(v) (a, + uylax((d, x) x(T)(()
Finally, since the kemel ky is bounded and To is bounded away from zero by Assumption I follows that vp E (Py:))y).
11.3 Analysis of the one-step estimator
In this section we provide the analysis of the one-step estimator. We start by restating Theorem Theorem 6 (CConsistency of the doubly robust estimator).). Suppose Assumptions 7
L5
16 and
17 Set X = n -1//ct2//b), which is rate optimal regularization. Then, with high probability,
(T) x -0 nn +.o (n)
For this Theorem, we will begin by decomposing the error terms.
Kdr ( = =x(p,) + Pri x(P) (Pr Pmbn + x((x) + Pttn x(P)
(35)
=Pp, p). (Pn - PP - 1²²²)) + x(pm) + P x(TT)
(36)
=S, To + Rn
(37)
where S, (Pn P)L*, R Tn = (Pr P (d) and R., = x(Pm) + -((). S, is sample average of a fixed function. We call R. the remainder terms and T, the empirical process term. The remainder terms R.,, quantify the error in the approximation of the one-step estimator across the samples. The following result provides a reasonable condition under which the drift terms will be negligible.
11.3.1 Bounding the empirical process term
As explained Appendix [1.4. Luedtke and Chung 41 proposed a cross-fitted version of the one- step estimator. However, splitting the data may lead to a loss in power. We are therefore interested in identifying a sufficient condition under which the empirical term T, becomes asymptotically negligible without sample splitting.
In the scalar-valued case, Donsker class assumption ensures the empirical process term asymp- totically negligible [32]. However, directly extending this notion to Hy-valued functions is not straightforward, since standard entropy-baat based arguments rely on the total ordering of R (65]. Fortu- nately, Park and Muandet (65] introduces notion of asymptotic equicontinuity adapted to Banach- or Hilbert-space valued empirical processes, which we adopt in this setting.
We say that the empirical process (Tx())) v (Pn - P up: E 9} with values in and indexed by G is asymptotic equicontinuous at :P E G if. for every sequence { C G with ll
Definition 11.4. (Asymptotic equicontinuity).
Poll E3 O, we have
|T. (G.) Tn (fo)) & 0.
(38)
Note that (38 is equivalent to T, = (P,, - () P) - +f²² or (ir Park and Muandet 665 gives
sufficient conditions for asymptotic equicontinuity hold that will leverage show asymptotic cquicontinuity. First we state the following result on the convergence of the efficient influence function estimator.
Assumption 21. (Estimated Positivity) There exists constant v) > O such that, with high probubility as nl XC,
to(a x) 2 n. for all (a. Ax X.
39

---

Lemma 11.3. (Influence Function Error). Suppose that the conditions of Lemma 4.1 hold, as well as Assumptions | 21 Then the following bound holds:
(1
- ||14aJ1X - Arra AAraxxlle ))
1
11 200 (P(P Op
no no 1P(xPX)
Proof. We expand the difference between the estimated and oracle influence functions:
<f(() - (=) = <aa x) T(a x)) (ór(v) (XIAAx(a,
(a x)) Ta a:)
क (a x)
+
(FYIA.X(a, - (x,
#a (a (x)
+ / (iYIAx((a',)) -MYlaxi x)) x x(da' x))
Taking the (P: norm and applying the triangle inequality yields:
1 v/0 (( S (1) + (II) + (III),
where:
(I) ( x(a x) a(a x))
xa x) xx) ) (Ør(u) - p)(xx((,z) L
Ao(a x)
Ta | x) Ro(a x)
(II)
(Fr1ax((ax) (o1(aa))))_1Axaaaa)))))))_ L'(PWte))"
(II) / (Axlaxidda))-mmxx(dd.a
(r1xx(dd,x)) x(da' xx) (2(P))))))
First, consider the term
uo x) mo(a|z)) ) (dy(v) - mmxx((a,))) L3((b:Ht))
(a x)) x(a
Let A(a, x) Ro(aja) mo(a|z)
x(al=) s(cl=) and M(a, v,
dy pr)A.x(a,)) E Hy. Then.
(1)² / (a( Ma, x. M) dPola, = / Aa(a,)) aI,a,,, dPo
Applying the Cauchy-Schward inequality gives:
(I) < (f A (a, x)) dr.)
1/2
1/2 (fllor()) (1oxx())-iiaxx(ax0)) Hy dPo) aPo)1!
Noting that Po (da, de) Ta(al Px (dx) and using the change of measure:
2 f (a, / (o(a x))(( Ro(a x) no(a ( ))
T(a Px(()))
we obtain:
(1-1)
no 2)
( - (0 dPA
1/2
To
no
(x))))
Using that the kernel is bounded in Assumption then the second factor is finite, and:
(I) = Op (1(6) no
2
no
40

---

for some constant depending on the kernel and outcome variance.
Second, analyze the term
(II) - sa x) aa ( x))
- (AYA)))))
(P(P6:H6))
By definition of the (P; Hy) norm, have:
(11) /
<(a x))
aa(a (UvIA,x(a,x) ( -Araxx(ax)))
dPo(a,
Hr
= T(a x)
|VYIA.((a,)) - -Mrxaaaaaa))-xxxxxxxxaa to(a x)) Px
aa x:)
Changing the measure to (aa xx) (dr) and bounding the weight by positivity assumptions yields:
(II) = f ||l114,((,,2) x) - Pay (a, x) 1)) w(a, x) xaa | 2x)
where w(a, x) = [ofal=)s(ala) xg(o]z)
If to 2 0, because of Assumption 21 then
(II) = Op ( ( :
for constant depending on the inverse propensity bound. Eventually, we bound the term
(II) - |/ (prisx((d',x) - Hyja,x(a',x) xdda' |x)
LP(Po:Hy)
Simply, the interm does Using Jensen's 's inequality in the Hilbert space Hy 92 Chapter 6], for each fixed x, we have:
// (Mld) [(nld..)) - pld' x)) π (dd')
20)
$ [1((d,,)) - mla', ( Ily n (dda' x))
Hy
Now square both sides and integrate over Po(a, = vola Px(x). Since the integrand is independent of a, this is equivalent to integrating over Px with the density no(a x)) marginalized out:
(II)) / /// (M(d' - p(d(,))) x)) o (da' |x)
To(a (x
Mr
-/l (a(a',2) p(d' x)) ilda' xx)
Px(dx)
My
$ f ( -)) x(da' xt) Px (dx)
-
( ulaa, x)) rdaa' xx) Px)
Therefore, using that x is bounded
(II) < |ssp x))l (oyllyyxxxxxxxxxxxxxx
2,a
Combining the bounds yields the desired result.
Then, we are now in position to state:
Lemma 11.4. (Asymptotic equicontinuity of the empirical process term) Suppose that Assump- tions
73
17 27 hold. Moreover, assume ky is a O Mercer kernel. Then the empirical process
term satisfies
Tm Hy
-1/22.
41

---

Proof. Under Assumptions 13
15 17 21 the functions u (v, a.z) - (1, a, x) lie in a finite and
shrinking ball of the RKHS Hly therefore if ky a C Mercer kernel, we can apply [85] Theorem DJ on the class G == [v "² c (P; Hy) to verify the conditions of Theorem of Park and Muandet (65).
Then, by Lemma 11.3 and with consistency of the nuisance parameters, || - - (1E + 0, and by their stochastic equicontinuity result in Corollary 8, [65] we readily have:
II(P, - - / ) + in probability.
Hence, [T (pp(n1/3) completing the proof.
11.3.2 Bounding the remainder term
Lemma 11.5. (Remainder term bound). Assumptions
3
15
16
17
21
then
| lly Op (rc(n, ó, b, c)r (n)).
Proof. From the definitions, the remainder term can be written as
Rr = x(Pm)
+ Pari. - X(T)
T(a x)
(dy(v) )Y)Axx(a,x)) + / (y)A.x(d,x)) xfr((dd xx)
= E.p
Ao(a x
Taa x)
Er
(dy(1) - (Y))
x:)
To(a x))
T(a x))
Ep.
(B(dy(v) A a = -
#a x.)
+ Ero
(MYIAXX(d' - 1Y|A, (d' x)) π (dd'' x))
Ta x)
E.P
(E(oy(v)
To(a x)
(a Aa
(HYIAXX(a,2) - (x)1Ax(a..))) + / (ixIAxx(, (x x)) <idd' (x)
= EEPo
x:)
We can expand the expectation into the following:
T(a x)) Ao(a x)
R₂
By ax((a,a)) -iy To(da xx) (dx)
(ivIA.((a',)) - -MM1xxx())) <(da' m))o(daa I Px()))))))
//
(a x.) (PYIAX((a,)) (11Ax((,))-Axxxx(00=)))) ((a 1 (dx)
#o(a x))
// (NYIA.X(UL (dl, - (o rdda' (x) Px)x)
// [ola | x))
-1) (uVIAx((a, - ( Tdda x)) Px (dx)
Aa xx)
/f [a (a (x)
1
(UYIAx((a ( 7 xx (dx)
(a [))
Ta x))
By the Cauchy-Schwarz Schwarz inequality, we have 1
|/ To(a x) ã x)) ao x) ) (HVIA,x(a,2) (xlAxx(a,2)) z(da x) p))x
Hy
42

---

1/2
1
nā(a x)
x(da x
|1XIAX ixIAx
aa(a
x)
To xx)
1
1
| - -AI--,AI,x
T0
no
To
L2(xPx)
If write rmo (n) sity scores, and noting that by Theorem 19 the regression error | Br14,xl12 is Op(rc(n, e)), and we conclude the proof.
error bound the estimation of the inverse propen-
Ru
xu
(xx))
11.3.3 Consistency proof
We now in position to prove Theorem 6
Proof.
The decomposition in Eq (37 provides:
X x (Po)ll S |Taall + ||Sull + 1|Rnll
(39)
The sample average Sn converges to 0 by the central limit theorem for Hilbert-valued random variables (see 193, see also Examples 1.4.7 and 1.8.5 in 94] that is ||S,lby Op(nn (12)..
Then by combining the results Lemma 11.4 (or Lemma 11.6 and Lemma 11.5 we obtain readily that:
ll - x (P6)ll = Op (n + 6,,
11.4 Additional details on the cross-fitted estimator
We now describe how cross-fitting 40 37 38 95,, can be used for our one-step estimator, following Luedtke and Chung AT. Let P denote the empirical distribution on the j-th fold of the samples and let P E P denote an estimate of Po based on the remaining folds. The cross- fitted one-step estimator takes the form
k
Xdr (T) =
p
+
PM
(40)
3=1
Using similar decomposition as in Eq.
37
obtain:
-
Σ (P)
p)6"
1 +
T: Σ (P - () -*)
(41)
K
Σ (x(P)) -
(42)
Then, to prove the consistency of the estimator, we use the following triangular inequality,
(T) -
S max TH 1xx + max ||s/ + -max || 1xx
(43)
where S = (P) - P) Tá = (P () (b R/ = =x(P.) + Puft. - x(17) We call
R/, the remainder terms and T the empirical process terms, {1,
43

---

Lemma 11.6. [4] Lemma 3][Sufficient condition for negligible empirical process terms). Suppose that x is pathwise differentiable Po with EIF Vn. For each j E {1,k], U. vo (oll (1))
implies that ||
7t Up (n=1/2)
Luedtke and Chung [91 proves this lemma via conditioning argument that makes use of Cheby- shev's 's inequality for Hilbert-valued random variables [96] and the dominated convergence theorem.
Then, to prove the sufficient condition, we recall the result of Lemma 11.3] which now allows to show that the cross-fitted CPME is consistent.
12 Details and Analysis of the Doubly-Robust Test for the Distributional Policy Effect
Theorem (AAsymptotic normality of the test statistic).). Suppose that the con- ditions of Theorem o hold. Suppose that Er [1 o (y: a, x))l is finite, that Ep ( (U, a, x)] and Ep, If (v, a) Pn (y a', x²) Tmo (n, 6) (e, (n,
O. Suppose also that
0(2-1/2), Set X -1/ce11/00 and [n/2]. then it follows that
T n, N0..1).
The proof uses the steps of Kim and Ramdas [66] and Martinez Taboada al. [28], but is restated as it leverage the theorems and assumptions relevant to CPME. Specifically we provide result similar on asymptotic normality that of Luedtke and Chung Theorem 2], which holds for the non-cross fitted estimator.
Lemma 12.1. (Asymptotic linearity and weak convergence of the one-step estimator). Suppose that the conditions of Theorem 6 hold. Suppose also that Two (n, 6) ro (n, b, O(n 1/2). Set n 1/(cc+1/b) Under these conditions,
n/2 [Rdr(T) x(=)] H,
where HI is tight H-valued Gaussian random variable that is such that, for each h E H. the marginal distribution (HI, h)H is N (0, Eo (vS (v, a), 11 1)
This lemma can obtained following the arguments of Luedtke and Chung 41 where the cross-
fitted estimator essentially requires for 21. R₁ Op (n 1/2) and T op (n 11/2) apply Slutksy's lemma and a central limit theorem for Hilbert-valued random variables [94].
Proof. We split the dataset { ai, 3:) into two disjoint parts:
D (x Oj Mm
D {(zz, aj 3,)1f
Further,
Vnfam ST.
(y, n m
Σ j=m+1
(e (y, a,,,,,0 a) Pr (g)))) Te.
where fn.x and $2 the empirical mean and variance respectively:
Jr n
S (Vi, (s, 1s),
$2
ST
(Sr,r (yi, as 2.)) )r,m()2
n
We define the test statistic using the doubly robust estimators Px. and Pa,w, which are computed respectively from D and D:
(2r ai Izi) = m j=++1
(ex (Wh, ais I:), Pa (1i, I,))
44

---

F,
(Zi),
(S ( =
(Z:)
-St,
i=1
==1
vm n. S K,
T n,,
As 28 66
the asymptotic normality results in four steps:
Consistency of the mean:
m mfs,x + 0p(1)
2. Consistency of the variance:
m(S m² = =m(S op()
3. Bounded variance under conditional law: 1mm. Op(1)
4. Conclude with asymptotic normality:
T n, 4 N(0,1)
Consistency of the mean We follow the same outline Martinez Taboada al. (88 did, using Lemma 12.1 for the asymptotic normality of Pa,x'
Consistency of the variance We follow the same outline as Martinez Taboada et did.
Bounded variance We now show that the denominator in the normalization T! a is bounded away from zero probability:
Op(1).
E m (f
D
For compactness, define:
1
Pain d;,);;
== vn m j=m+1
Px (yj,
v m
and
m
n Σ jmm+1
f
G... (Yi, ag, 2g) 3 Vn m
8 2g)
m
so that:
mf = (T,7), mft = (t,5),
(VmS
2z) (p (yis (yi, xz4, 7)²² ml mnfr...))?
(v (vmst.
( (Vis ai, (.), mi(t...)?
Recall that (Z) (f (Z), 7), and that 7 va-m Ejenti Px. (Z,) E H is random element measurable with respect to D. Conditional on D, the variance of the test statistic is:
E [mf2,r
D (C7,7),
where El(((Z) 69 ((Z) the covariance operator over H, which compact, self-adjoint and positive semi-definite.
45

---

Using the eigendecomposition of C' (see Section 9.1 we write:
C=
Ajos 2 U. 7 Σ Bzug
so that
m (2) = Σ 1j3
From Assumption 16 know that the eigenvalues satisfy ' S Cj-b for some b > 1. This decay implies that the kernel is not degenerate and the operator C' has at least one strictly positive eigenvalue: A O.
Moreover, by Lemma 12.1 and the Central Limit Theorem in separable Hilbert spaces 93 the limiting distribution of is Gaussian:
7
Nyy, where N, ~N(,
Hence,
81 v1) 4 VIN,
11 4 N2N?
Therefore, the conditional variance is lower bounded:
E m (
2)
E X1 N
This shows that the variance remains bounded away from zero in probability. More formally, for any >> O, we can find M >> such that:
P
M
<c.
E
m
f2 x
(Z)
D
Hence,
= =0p(1).
E mJ?
D
Asymptotic normality We now conclude the asymptotic normality of Ti T following Mar- tinez Taboada et al. (28). Suppose that Ep, all (y, a, I is finite, that Ep, [P (1, x)] = 0 and Ep, 16 (y, a, x), (x. (V', a', x'))] > O, from Kim and Ramdas (66), we have:
Vmff.
S2 S ElS2 f (Z)
N(O,,)), 1),
4 1.
D
Using the previous steps, we have:
mft mfx. or(1), mst2 T. ms2 f/ + (r()),
which implies:
mf n...
m ffx
00p(1) 4 A( 1).
Elmsz 2 (2) (
Elmm f 22, (2) D
Moreover,
i2
E 1.
T3. m f2 x,
(Z)
D
46

---

Taking square roots on both sides (which preserves convergence in probability by the continuous mapping theorem), we obtain:
Blmfa nxx (2)
D
2 1.
Vmst n,
By Slutsky's 's theorem by combining the last two:
Vmfa. 4 N),
T
n,5.
S 2as
13 Details and Analysis of the sampling from the counterfactual distribution
Proposition 9 (CConvergence of MMD of herded samples, weak convergence to the counter- factual outcome distribution).). Suppose the conditions of Lemma 4.7 and Assumption X hold. Let (Warz) and Praar (resp. (Ūpi)). Prppi) be generated from (dr(w) (resp. (pi( via Al- gorithm 2 Then, with high probability, MM v(T)) Op(rc(n,b, b, + m 11/2 and MMD Pm V.drs v(x)) Op(n 1/2 Two (n)re(n, b, + 1/2). Moreover, (Wars) v(T) and Y,dro
(fm.))
(TT).
Proof. Fix T E II. By Theorem 6 the estimated embedding Xar(T) satisfies:
[Xar(T) x(x))l =Op (n (n11122 1/2 + rsa (n) (n(n,,,))
Let {Te 1m be the herded samples generated from (x) using Algorithm According Bach et al. 197 Section 4.2], the empirical mean embedding of these samples approximates X (T) at rate:
Edr(T))
(ū.)
0m 1/2).
Hy
By the triangle inequality:
1 m
x(T)
= - (n) m m1/2)
m7
Hy
By definition of MMD and the reproducing property, we have:
(UID(P,P(())) =
m Σ (v.)
1
x(=)
Hy
so the same rate applies.
For the plug-in estimator (T), which does not involve nuisance estimation, we obtain:
(T)x - x(T))las Op(rc(n,,, b,c)),
yielding, with the same arguments
PPT b. - mm
Finally, weak convergence of the empirical measures im to v(x)) follows from convergence in MMD
norm with characteristic kernel; see Aimon-Gabriel et al. 67 Theorem 1.1] and Sriperumbudur 198).
47

---

14
Experiment details
In this Appendix we provide additional details on the simulated settings as well as additional experiment results.
14.1 Testing experiments
We are given a logged dataset mitt [( a4 1121))= ~ Po. collected under logging policy no- For two target policies т and I², the objective is to test the null hypothesis:
Ho (()) (TT)),
H1 : :)) * v(T'),
where (TT) and ((=') denote the counterfactual distributions of outcomes under π and n', respectively.
14.1.1 Baseline
We use baselines to evaluate the ability of our framework detect differences in counterfactual outcome distributions induced by different target policies, compared to alternative approaches.
Kernel Policy Test (KPT). An adaptation of the kernel treatment effect test of Muandet et al. 17 extended to the OPE setting. It tests whether the counterfactual distributions v(T) and 1T(T)) differ by comparing reweighted outcome samples using the maximum mean discrepancy (MMD). The key idea is to view both outcome distributions as being implicitly represented by importance-weighted samples from the logging distribution.
Given two importance weight vectors Wy and Wn corresponding to the target policies π and n', respectively, the test computes the unbiased squared MMD statistic:
MMD 2
1
Σ E
wo wz k(yi, Uj) + w wo k(ys, Mz) 2wf wo k(y; wi)
nnn
where k(yi, Uÿ) is positive definite kernel on the outcome space (typically RBF). To obtain p-value, KPT uses permutation-base null distribution. It repeatedly permutes the correspondence between samples and their importance weights (thus preserving the outcome data while randomizing their "assignment") and recomputes the MMD statistic under each permutation. The p-value is estimated as the proportion of permuted statistics that exceed the observed MMD. As Muandet et al. [17], use 10000 permutations.
Average Treatment Effect Test (PT-linear). A simple variant of KPT using linear kernels, testing only for differences in means. It serves as reference for detecting average treatment differences.
Doubly Robust Kernel Policy Test (DR-KPT). on the difference of efficient influence functions:
We construct a doubly robust test statistic based
T
S!
where ft is the empirical mean of pairwise inner products of influence function differences across data splits, and St the empirical standard deviation. The null is rejected when T म. exceeds standard normal threshold.
14.1.2 Model selection and tuning
We repeat each experiment 100 times and report test powers with 95% confidence intervals. For DR-KPT and KPT, the kernel ky is RBF. For DR-KPT the regularization parameter À is selected via 3-fold vross-validation in the range {10-1... 110-....., 10"), as done 17]. We use the median heuristic for the lengthscales of the kernel K, hx and ky.
48

---

14.1.3 Simulated Synthetic Setting
The experiments conducted in synthetic continuous treatment setting. Covariates x; Rd are sampled independently from a multivariate standard normal distribution (0, Ia). Treatments a, E R are drawn from Gaussian logging policy To x) = NNx" w, 1), where the weight vector is fixed as valud ld. Outcomes generated according to linear outcome model with additive noise:
= 22 + 70, CG, 62 ~(0,0))
where B E Fd is linearly increasing vector and y R controls the treatment effect strength.
We evaluate four distinct scenarios, each specifying different relationship between the target policies π, x', and the logging policy xg- These scenarios are designed to induce progressively more complex shifts in the treatment distribution, affecting the downstream outcome distribution. We set the covariate dimension to d = 5, ? 1 and evaluate / in the grid 3 = j0.1, 0.2, 0.3, 0.4, 0.5]. B taken at different values across samples to reflect heterogeneity in user features and outcome interactions.
Scenario I (Null). This is the calibration setting in which n'. The two policies generate treat- ments from the same Gaussian distribution with shared mean and variance, ensuring no counterfactual distributional shift. Under the null hypothesis, expect all tests to maintain the nominal Type I error rate.
Scenario II (Mean Shift). Here, the target policy remains identical to the logging policy, while the alternative policy n/ is a Gaussian with the same variance but a shifted mean. Specifically, x' uses weight vector w' = + 8, with d 2 1a. This results in systematic mean shift in treatment assignment, causing a change in the marginal distribution of outcomes through the linear outcome model. This tests whether the methods can detect simple, mean-level differences in counterfactual outcomes.
Scenario III (Mixture). In this case, the policy remains a standard Gaussian as in previous scenarios, while the alternative n' is 50/50 mixture of two Gaussian policies with opposing shifts in their means: wa = w + 1a.
w 1.- Although the resulting treatment distribution is bimodal, its overall mean matches that of स. This scenario introduces a change higher-order structure (e.g.. variance, modality) without altering the first moment, allowing us to test whether the methods detect distributional differences beyond the mean.
Scenario IV (Shifted Mixture). This is the most complex scenario. As in Scenario III, the alternative policy x' is mixture of two Gaussian components, but this time only one component is shifted: wi = w + 2- 1a, w2 w. The resulting treatment distribution under x' differs from x in both mean and higher-order moments. This scenario combines characteristies of Scenarios II and III and evaluates whether the tests remain sensitive to subtle and structured counterfactual shifts.
1000 samples per and estimate importance weights for and T' using fitted models based on the observed data. Specifically, we fit a linear regression model to the logged treatments T as function of the covariates X estimate the mean of the logging policy mo, and evaluate its Gaussian density to obtain estimated propensities. This experimental design enables evaluation of the calibration and power of distributional tests under range of realistic divergences.
Across all scenarios, we generate
In all scenarios (Tables 3 DR-KPT consistently demonstrates the best computational effi- ciency, with runtimes typically two orders of magnitude lower than both KPT and PT-linear. This efficiency stems from the closed-form structure of its test statistic, which avoids repeated resampling or kernel matrix permutations. In contrast, KPT relies on costly permutation- based MMD calculations, and PT-linear, while simpler, still requires repeated reweighting. For readability and to emphasize this computational advantage, we reorder the tables so that DR-KPT appears in the last row of each scenario.
We provide an additional Table 7 below with larger sample sizes and two kernels (RBF and polyno- mial).
49

---

Table 3: Average runtime (in seconds) for Scenario Values are reported as mean + std over 100
runs.
Method
100
150
200
250
300
350
400
KPT
0.495 + 0.070
0.740 + 0.039
1.134 + 0.081
1.623 = 0.075
2.257 = 0.074
3.204 : 0.118
4.180 + 0.136
PT-lincar 0.592 0.061
0.774 0.038
1.060
0.051
1.353
0.076
2.373
0.202
3.384
0.160
4.358 0.251
DR DR-KPT
0.004 1: 0.005
0.007 1 0.004
0.010 1: 0.009
0.008 0.002
0.013 :: 0.007
0.025 0.023
0.019 4: 0.007
Table 4: Average runtime (in seconds) for Scenario II. Values are reported as mean + std over 100
runs.
Method
100
150
200
250
300
350
400
KPT
0.559 0.044
0.794
+: 0.063
1.764
:: 0.085
4.204 +: 0.182
PT-lineur
0.486 + 0.035
0.767
0.037
1.071 + 0.030
1.630
0.062
2.405 + 0.182
3.738
0.251
4.767 + 0.228
DR-KPT 0.004 + 0.003
0.007 0.005
0.012 + 0.006
0.014 0.008
0.023 = 0.012
0.022 0.009
0.027 = 0.031
Next, to empirically illustrate the benefits of sample-splitting in the test statistic provided in Section 5.I we provide below Figure the same histograms given in Figure [ Concretly, instead of splitting the samples in m and m., we use all the samples the definition of T x, f (y/, ag. 24,) and in the test statistics in Eq. (14). As
see, the resulting distribution is not normal, the QQ plot does not conclude and the test is not at all calibrated.
14.1.4 Warfarin Semi-Synthetic Setting
We build a semi-synthetic evaluation based on the publicly available Warfarin dosing data, following the spirit of Kallus and Zhou [4) Zenati al. [69 and our distributional setup. Starting from the raw table from [68], we first (i) keep only subjects with a recorded stable therapeutic dose and stable observed INR (columns 38-39 NA and stability flag column 37 equal to 1). (ii) construct a covariate matrix X comprising demographics (gender, race, ethnicity, age group), anthropomet- rics (height, weight), BMI, clinical indications (8 binary indicators), selected comorbidities and concomitant medications (aspirin, acetaminophen including high dose, statins, amiodarone, carba- mazepine, phenytoin, rifampin, antibiotics, antifungals, herbals), smoking, and pharmacogenetic markers (CYP2C9 and VKORCI genotypes), and (iii) remove near-constant columns (empirical standard deviation < 0.05) and patients with missing/degenerate BMI (post-filter BMI 10 10-3). Let denote the resulting sample size.
Outcome construction (semi-synthetic). Let TherDose; be the recorded stable therapeutic dose and let a denote candidate weekly dose. We define an expert-motivated cesolute-tolerann cost
y(a, x) = max (In (eerrDsDess(()) - 0.1 TherDose(z), 0):
and add a small observation noise //0 0.12). For each patient i, the observed outcome is Y = yTT, X2) +
0.1.0.1
Logging policy (data-generatiin mechanism). Write μjj = 1 Σ, TherDose, and o its empirical standard deviation. Let ZEMI BMI- be the standardized BMI. The logged treatment is generated by contextual Gaussian policy with BMI-driven mean and homoskedastic variance:
OBMI
T = μ + of(vo ZBMI + vI-0e).
E e~ (0,,1,
0 = 0.5.
Equivalently, T|X ~ Ni + o7 10 ZBMI, (0 - -8)), i.e., continuous normal density over
T - μh. 07,00 Zumi
ojvi-6
Propensity estimation. form importance weights for target policies, we fit linear regression of T' on X (no intercept in the BMIlonly fit, standard scikit-learn linear model in the full fit) to obtain
50

---

Table Average runtime (in seconds) for Scenario III. Values are reported as mean std over 100
runs.
Method
100
150
200
250
300
350
400
KPT 0.523 + 0.063
0.836 + 0.025
1.161 + 0.018
1,396 = 0.008
2.157 = 0.042
3.174 : 0.014
4,044 = 0.021
PT-lincar
0.505 +: 0.052
0.802 0.015
1.134
0.013
1.577
0.014
2.142
0.043
3.181 0.041
4,051 +: 0.024
DR KPT
0.004 1: 0.003
0.008 0.009
0.011 1: 0.005
0.015 0.009
0.020 :: 0.010
0.025 0.013
0.025 : 0.014
Table Average runtime (in seconds) for Scenario IV. Values are reported as mean + std over 100
runs.
Method
100
150
200
250
300
350
400
KPT
0.548 + 0,065
0.839 0.014
1.171 + 0.012
1.611
0.013
2.176 - 0.042
3.239
0.032
4.142 : 0.032
PT-lincur
0.523 + 0.062
0.831
0.008
1.160 + 0.014
1.626
0.058
2.385 0.127
3.282
0.115
4.153 +: 0.043
DR-KPT
0.004 + 0.005
0.009 + 0.007
0.015 + 0.008
0.015 = 0.010
0.018 = 0.010
0.023 : 0.011
0.025 : 0.015
(X) and assume Gaussian residuals with variance fixed to (or)). The estimated logging density is modeled as Gaussian with mean 7o X) and variance (azj)²:
T Tox())1²) To X) x exp 2(az)
and importance weights for candidate policy w/ (T x) (T X), clipped at 108 as in the code.
Policies under comparison and scenarios. We obtain a baseline linear score Whuse by regressing T on X and (optionally) adding small Gaussian jitter to the coefficients. Let scale or and A of (the intercept shift unit). We evaluate four scenarios by specifying target policy æ and an alternative policy that generate normal treatments with means linear in X and common scale o¹. Mixtures are implemented as equal-weight mixtures of two Gaussians via intercept shifts.
(X Wosse T. and л. No counterfactual shift; tests should control Type I error.
Scenario I (Null).
Scenario II (Mean Shift). т N(X Wasee o7 and n' is the same Gaussian with an intercept increased by A (mean shift with unchanged variance). This probes sensitivity to first-moment shifts.
Scenario III (Mixture, mean preserved). N(X Wheses o1 and n' is 50/50 mixture of two Gaussians with intercepts shifted by +A. The overall mean matches π while the treatment distribution becomes bimodal, altering higher moments only.
Scenario IV (Shifted Mixture). = -V Whase: 2) and T' is a 50/50 mixture where one component is intercept-shifted by +A and the other unshifted. Both mean and higher-order structure differ from
Experimental protocol. For each scenario, we use n patients after preprocessing and repeat over 100 independent seeds. For kernel choices on outcomes, we consider linear, polynomial, and RBF kernels; the RBF bandwidth uses the median heuristic on {Yi when stable. We compare KPT (reweighted two-sample tests) and DR-KPT (doubly robust sample-split statistic) using the same weights W,, Wz', with the DR regularization set À 102 in the kernel ridge step. We report empirical rejection rates at a = 0.05 across seeds for Scenarios I-IV, thereby assessing calibration (I) and power to detect mean-only (II), shifts in clinically meaningful cost outcomes.
(III), and combined (IV) counterfactual
We provide in Table runtime of our tests on the Warfarin data.
14.1.5 dSprites structured-outcome semi-synthetie data
We evaluate distributional tests on structured outcomes using the dSprites dataset 77 71. Each outcome is 64 64 grayscale image obtained from a fixed renderer that decodes spatial latents while holding shape, scale, orientation, and color constant. Let latent contexts be ~ 2/( 1|²) and treatments A E TR. For each context-action pair, the renderer deterministically outputs an image
Y := A) 644664
51

---

Table Runtime (in seconds) of DR-KPT and KPT variants on the synthetic dataset.
Method
100
200
1000
2000
KPT-RBF
1.712 = 0.170
4.786 = 0.157
104,256 + 16.576
366. 166.104: = 99.063
306.406 = 44.161
KPT-Poly
1.824 1 0.260
4.587 1 0.246
106.186
1 55.371
354,062 1: 13,737
334.608 1 81.867
KPT-Lincar
1.801 + 0.191
4.573 + 0.465
84,999
14.167
387.368 + 262.004
285.999 + 71.099
DR-KPT-RBF
0.135 + 0.022
0.147
+ 0.016
0.325 0.019
1.196 + 0.158
1.126 + 0.135
DR-KPT-Poly
0.118 = 0.011
0.140
= 0.021
0.314
0.020
1.155 = 0.172
1.119 + 0.126
20
Thewetool qparn
50urh
Figure 4: Illustration of 100 simulations of the non-sample-spliteed DR-KPT under the null: (A) Histogram of DR-KPT alongside the pdf of standard normal for n 400, (B) Normal Q-Q plot of DR-KPT for n 400, (C) False positive rate of DR-KPT against different sample sizes.
where g maps the spatial latents pixel intensities through the dSprites generative process.
Policies and logging data. We define contextual Gaussian policies in IR2 with diagonal covariance I2 For parameters (0, B, o),
X1 cos + 9 A A|X~ 02) 0²12).
(x(X)
X2 0 +
Logged data are generated from a Gaussian logging policy with = 0.5. We compute analytical propensities for the target and alternative I and form importance weights
Wx = π (A X) (A XY
('AA X)
We
( X)
clipped at 105.
Scenarios. We consider two scenarios that parallel our continuous-treatmen experiments, now in structured image setting:
Scenario (Null). æ and n' share the same (0, a), hence produce identical treatment and outcome distributions.
Scenario IV (Policy Shift), π and n' share and a but differ by an intercept shift 3 + 0.3, inducing mean shift in A X and corresponding differences in the rendered image outcomes.
All other latent generative factors are fixed, ensuring that observed shifts arise purely from policy changes.
Experimental protocol. We generate n = 3000 samples per seed. Images are flattened into vectors 4096 for kernel computations. We compare KPT (reweighted kernel two-sample tests with linear, RBF, and polynomial kernels) and DR-KPT (doubly robust, cross-fitted) using the same weights Wr, Wy'. For RBF kernels, the bandwidth is set by the median heuristic; the DR regularization parameter is fixed to x 102. Each scenario is repeated over 100 random seeds, and we report empirical rejection rates at a 0.05, assessing calibration (I) and power under policy shifts (IV), consistent with our synthetic and semi-synthetic Warfarin setups.
52

---

Table 8: Runtime (in seconds) of DR-KPT and KPT variants on the Warfarin dataset.
Method
1000
2000
3000
4000
KPT-RBF
375.355 + 112.770
1338.653
262.921
2672.691 20.895
5657.573 + 413.196
KPT-Poly
331.831 48.219
1315.537
212.152
3308.014
1219.752
5165.057 667.347
KPT-Lineer
364.378 35.433
1302.173 300.546
2651.653 + 103.833
3623.222 + 373.766
DR-KPT-RBE
0.426 + 0.024
2.530 + 1.447
5.775 0.125
11.701 + 1.869
DR-KPT-Poly
0.485 + 0.076
1.862 + 0.030
6.660 3.180
11.184 + 0.170
14.2 Sampling experiments
We study whether estimated counterfactual policy mean embeddings (CPMES) can be used gen- erate samples that approximate the true counterfactual outcome distribution. Formally, given logged dataset iitt {24, as 3)) Po and target policy и,, we aim to generate samples {ū/ { such that their empirical distribution Pyr approximates the counterfactual outcome distribution v(T) under N.
14.2.1 Procedure
We employ kernel herding to deterministically sample from the estimated embedding X(x) in RKHS. The algorithm sequentially selects samples .1...., Fm that approximate the target embedding via greedy maximization:
y arg
{() yEY
where ky is universal kernel the outcome space.
Since no comparable baselines for counterfactual sampling are available the literature, we focus on comparing the quality of samples generated from two estimators of X(T): the plug-in estimator and doubly robust estimator. Both versions yield distinct herded samples, which we evaluate against ground truth samples generated under the target policy
14.2.2 Model selection and tuning
To report the distance metrics, we repeat each experiment 100 times and report the associated metric with 95% confidence intervals. For both plug-in and DR estimators, the kernel ky is RBF and the regularization parameter À is selected via 3-fold cross-validation in the range {10-4,.. ..... 10"). }. done in the sampling experiments of Muandet al. [7). We the median heuristic for the lengthscales of the kernel k,, xx and ky:
14.2.3 Simulated Setting
We simulate logged data under different outcome models and logging policies. Covariates x; R are sampled from a standard Gaussian distribution. Treatments a; E R are drawn either from uniform distribution or from a logistic policy whose parameters depend on x;. Outcomes yi are then generated via one of the following nonlinear functions:
Nonlinear: y sin(x 8) + a² +
Quadratic:
y y= (x B)² + a² +
where 3 is fixed coefficient vector and ~ W 1). For each synthetic setup. we generate logged data under the logging policy To and obtain oracle samples under the target policy π for evaluation. We set the covariate dimension to d = 5 and evaluate 3 in the grid 8 = [0.1, 0.2, 0.3, 0.4, 0.5]. taken different values across samples reflect heterogeneity in user features and outcome interactions.
Figure 5 illustrates the counterfactual outcome distributions recovered via kernel herding using both PI-CPME and DR-CPME estimators under different logging policies and outcome functions.
To assess the fidelity of the sampled distributions, we compare the empirical distribution pyn of herded samples to the true counterfactual distribution using two metries:
53

---

Counteffccuall Outcome Distriuttion via sernel Herling
Counterfactual Outcome Dearitution via Kornel Herd
Loged
611-
Logast
redeed CPMC
Herder PiCkI
0.22
033
833
(a) Logistic logging, nonlinear outcome
(b) Logistic logging, quadratic outcome
Lousterfatiad outuamee Uf si boiiiaiio via semal me meciing
Connttefootbal Gutueene Uistiiationn via Koreel Hurd by
531-
Tier lorded PICIIE
0.2
316-
035
(c) Uniform logging, nonlinear outcome
(d) Uniform logging, quadratic outcome
Figure 5: Counterfactual outcome distributions estimated via kernel herding from PI-CPME and DR-CPME samples, compared to the logged and true outcome distributions.
Wasserstein distance between the sampled and ground truth outcomes,
Maximum Mean Discrepancy (MMD) with a Gaussian kernel.
Table 9: Wasserstein distance between herded samples and samples from the oracle counterfactual distribution
Method
logistic-nonlinear
logistic-quadratic
uniform-nonlinew nonlinear
uniform-quadratte
Plug-in 1.29c-01 2.6e-01
1.41c-01 = 4.9c-02
9 9.08c-02 3.7e-01 5.00e-02 + 1.5e-02
6. 6.78c-02 = 1.9c-02 6.63e-02 + 1.6e-02
DR 8.60e-02 + 2.2e-02
1.36e-01 # 3.9e-02
Table 10: MMD distance between herded samples and samples from the oracle counterfactual distribution
Method
logistic-nonlinaar
logisti--quadratic
uniform-nonlincar
uniform-quadratic
Plug-in
1.1 + 5.9e-03
9.85e-04 + 6.0e-04
1.92e-04 + 1.2e-03
3.31e-04 # 2.5e-04
DR 4.38e-04 + 3.6e-04
9. 9.80e-04 = 6.0e-04
6.49e-05 4.4e-05
3.51e-04 = 2.5e-04
Results in Table 9 10 show that samples obtained from the doubly robust estimator exhibit lower discrepancy to the oracle distribution.
14.3 Off-policy evaluation
We are given dataset of n. i.i.d. logged observations {(x;, 0z ) Po. Given only this logged data from Po. the goal of off-policy evaluation to estimate R(T)), the expected outcomes induced by target policy belonging to the policy set II:
54

---

R(a) - Ep, [(Y()))
(44)
After identification, the risk of the policy simply boils down to R(T) = Ep. (aY)) and the CPME ((T) Ep, [dy(Y(a)) describes the risk when the feature map dy - y linear.
14.3.1 Baselines
We compare our method against the following baseline estimators on synthetic datasets.
Direct Method (DM). The direct method [33] fits regression model u A + R the logged dataset Din = {(),, 02 x,)]; and estimates the expected reward under a target policy л as
" f (( RDM((T) Σ
Since the evaluated policy differs from the logging policy To и, covariate shift is induced over the joint space A x X. is well known that under the covariate shift, a parametric regression model may produce significant bias [99]. To demonstrate this, a 3-layer feedforward neural network as the regressor and call DM-NN.
Weighted Inverse Propensity Score (wIPS). This estimator reweights logged rewards using inverse propensity scores 90
Di W;Ys
T(a; us) Ta u;)
RWIPS(T)
Di=1
wy
This estimator is unbiased when the true propensities are known.
Doubly Robust (DR). using:
The DR estimator
33 combines the two previous methods, that is f and wa
PDR - Σ Am1
(ʃ (ft + w; ~Zz; ai))
and remains consistent if either f or To is correctly specified. We use the same parametrization for do for the DM method and therefore call this doubly robust approach DR-NN.
Counterfactual Policy Mean Embeddings (CPME). We define a product kernel
kax((a,z)) x), (a', (a',x')) ka(a, a (x, with Gaussian kernels and The outcome kernel ky is linear.
Relation to DM. When ń is fit via kernel ridge regression (see Exemple 9.1. the DM estimate becomes:
(K (DM(TT) v Y (K + nAI) Σ KAX (ā;, x4)
1=1
where K.; = (axx((a I.), ) I,)), and à; T(- xz)) This matches the CME form proposed in 17], showing that CME/CPME as a nonparametric version of the DM. Because kernel methods mitigate covariate shift, CMPE is consistent and asymptotically unbiased. We will therefore refer to the plug-in (T) and the doubly robust (a)( estimators as DM-CPME and DR-CPME.
14.3.2 Model selection and tuning
Each estimator is tuned by 5-fold cross-validation procedure for OPE setting introduced in 17 Appendix B]: For the DM and DR-NN models, we vary the number of hidden units n4 50, 100, 200. 200. For CPME and DR-CPME, the regularization parameter À is selected from the range 810-8 10-33. We repeat each experiment 30 times and report mean squared error (MSE) with 95% confidence intervals. For CPME, the kernel ky is linear, and the regularization parameter selected via cross-validation. We use the median heuristic for the lengthscales of the kernel i and kx.
55

---

14.3.3 Simulated setting
We simulate the recommendation scenario of Muandet et al. 17 where users receive ordered lists of K items drawn from a catalog of M items. Each item m E [1....:Au M) is represented by a feature vector Um RR and each user j {I,.... Mj is assigned feature vector x; RR both sampled ii.d from (0, Ia). recommendation without replacement.
_ formed by sampling items
(Umi....,,
The user receives binary outcome based on whether they click on any item in the recommended list. Formally, given a recommendation ag and a user feature vector 2j, the probability of click is defined as
1
Oi- =
+ exp (-a, Ets)
where a; is the average of the K item vectors in the list a,, and egj N(0, 1) is independent noise. The binary reward is then sampled as Mij ~ Bernoulli(As)..
In our experiment, target policy (a x) generates a recommendation list (Um....... .......... Umx by sampling K items without replacement from the M-item catalog, where sampling is governed by multinomial distribution. For given user j. each item's selection probability is proportional to cxp(b, v1). where by is the user-specific parameter vector. we set bj = Ij, the policy optimal in the sense that aligns with user preferences.
To construct the policies for the experiment, we first generate user features ..,..., NN. The target policy uses b3 Pj @ Tzj. where Pj E {0, 1)d is a binary mask with i.i.d. Bernoulli(0.5) entries, zeroing out about half the dimensions of zz- The logging policy To then defined by scaling: recovers To
b, ab; with E 1-1, 1]. The parameter controls policy similarity: while a - results in maximal divergence.
We generate two datasets Dmm {(y, 0z, =;));= and Dunger = {(is, ā, 2;) )e using zo and n respectively, with shared user features z;. The target outcomes ÿi are reserved for evaluation.
We evaluate performance across five setting where we vary the the values of: (i) number of observa-
tions (n), (ii) number of recommendations (K), (iii) number of users (N), (iv) dimension of context
(d), (v) policy similarity (a). Results (log scale) are shown in Figure 6
We observe:
All estimators generally show improved performance as the number of observations increases, except for IPS, which exhibits slight decline between = 2000 and = 5000.
The performance of all estimators deteriorates as either the number of recommendations (K) or the context dimension (d) increases.
All estimators degrade a - 1, with IPS and OPME/DR-CPMAL demonstrating the better robustness.
CPME and DR-CPME consistently outperform the other estimators across most settings. Our proposed doubly robust method, DR-CPME, offers performance improvement over the CPME algorithm.
14.4 Computation infrastructure
We ran our experiments on local CPUs of desktops and on GPU-enabled node (in remote server) with the following specifications:
Operating System: Linux (kernel version 6.8.0-55-o (--5--eerrrrc)
GPU: NVIDIA RTX A4500
Driver Version: 560.35.05
CUUD Version: 12.6
Memory: 20 GB GDDR6
56

---

Number Observations MSL. 10.0--013 Recommendatiors
-000
MSE
Extimttor
CM
D6- MM
CA-
DK-CWS
100
Number Observations
Number Recommendations
Numbor of Use vs MSE. M 20, 2000. a 10, -0.3
Dimension of Contest MSE N 50, K'n 20, 2000
30-1
Fet meste
MSE
MIE
Edbruter DR RS.
100
150
200
250
300
Numbor Usars
Number Contoxt
(c)
MSE under domain shift, N 50, M 20, K-55. H. 2000, d
Extimete
10
DIT PNI
1098
10-3
-0.7
-0.3
(c)
Figure 6: Mean squared error results for the off-policy evaluation experiment described in Ap- pendix 14.3.3 reported across variations in: (a) the number of observations n, (b) the number of recommendations K, (c) the number of users N, (d) the context dimension d, and (c) the policy shift multiplier a.
57


Reviews:
[{"review_id": "ICNabqc6VA", "reviewer": "Reviewer_VKqz", "summary": "In this work, the authors propose Counterfactual Policy Mean Embedding (CPME), an approach that uses RKHS to represent the full counterfactual outcome distribution in OPE tasks. The authors propose plug-in and doubly-robust estimators leveraging this approach and characterize their consistency and convergence properties. The authors further validate the framework via semisynthetic experiments and illustrate that the approach demonstrates strong statistical properties.", "questions": "The authors argue that existing approaches leverage cumulative density functions, which are not suite for inference on more complex structured outcomes. Can the authors please clarify the limitations of CDF-based approaches in this setting? \n\nUnder the this work's formal setup, which specific technical challenges are specific to OPE beyond existing frameworks on treatment effect estimation? For example, fitting one-sided CATE can be viewed as an OPE problem. \n\nCan the authors provide further detail on the stochastic equicontinuity condition and when it is likely to hold? This detail is important for understanding whether the proposed approach is a viable alternative to cross-fitting.", "limitations": "The authors do not provide a discussion of framework limitations. While the identifying assumptions listed by the authors (Assumption 1) are standard, the work would benefit from clearly exploring the limitations of imposing these assumptions on the reliability of the estimation approach. Ideally, this discussion should also be supported by empirical or theoretical analysis.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "## Significance: \nOff-policy evaluation is a core problem relevant to a number of domains. The authors' approach targeting counterfactual distributions of outcomes enables recovering a richer space of target parameters than traditional approaches targeting means. The authors show clear applications of the framework in the form of hypothesis testing. The embedding-based approach is also consistent with a range of modern applications, including inference over graph, text, and image modalities. \n\n## Originality: \nThe authors clearly situate the proposed framework in related work. The application of counterfactual mean embeddings to OPE appears to be novel. However, the overlap between this OPE setting and prior applications to distributional treatment effects (e.g., 1,2) limits originality to an extent and affects my score. The author's note on how this work differs from Mundanet et al. (i.e., propensity is unknown in advance) is helpful, and the introduction of an alternative to crossfitting (line 215) to improve statistical power is also a benefit of the proposed approach. \n\n[1]  Conditional distributional treatment effect with kernel conditional mean embeddings and u-statistic regression. https://arxiv.org/abs/2102.08208\n[2]  An efficient doubly-robust test for the kernel treatment effect. https://arxiv.org/abs/2304.13237\n\n## Quality: \n\nThe work is generally well-supported by theoretical and empirical analysis. Theoretical details in the appendix supports claims made in the main text. Further, applications with testing (Section 5) illustrate why a distributional approach is useful in supporting downstream inference applications. \n\nHowever, while the fully synthetic experimental setup is sufficient for demonstrating that the proposed approach satisfies basic coverage properties, it could be strengthened. In particular, the work would benefit from a semi-synthetic or real-world data application illustrating how the framework can be applied to real-world OPE tasks. The fully synthetic setup is limiting. However, the additional synthetic results reported in the appendix do strengthen the empirical validation of the approach. \n\n\n## Clarity. \n\nThe writing and mathematical exposition are generally clear. However, the work has several typos and would benefit from a careful editing pass.", "was_revised": true, "final_justification": "I thank the authors for their detailed response to my questions. The additional experiments also provide further evidence for the practical applicability of the approach. My overall (favorable) assessment of the work remains unchanged. In particular, reviewer Gtoa gives an excellent summery of my assessment :  \n\n> This paper, while it does not provide anything groundbreaking, is a very solid paper which uses several existing methods and proof techniques in kernel mean embeddings, doubly robust estimation, cross U-statistics and kernel herding in the context of counterfactual policy. I think that showing that one can use existing methods and theory in different contexts is valuable.", "rebuttal": "We thank reviewer VKqz for their detailed and thoughtful review. Below, we aimed to address the reviewer’s concerns point by point. If any aspects remain unclear, we would be glad to provide further clarification. Should our responses satisfactorily resolve the reviewer’s concerns, we hope they will consider raising their score accordingly.\n\n**Related works**: \n\n> \"The application of counterfactual mean embeddings to OPE appears to be novel. However, the overlap between this OPE setting and prior applications to distributional treatment effects (e.g., 1,2) limits originality to an extent and affects my score.\"\n\nWhile our approach tackles indeed adjacent problems to [1] (Park et al, 2021) and [2] (Martinez-Tabaoda et al, 2023) we would like to emphasize the nontrivial technical novelty of CPME and its associated methods. Unlike [2] who focused their analysis exclusively on the asymptotic normality of the test statistic in kernel two sample tests, we provided a general framework but also a one-step estimator which convergence rate was rigorously proven using semi-parametric theory in RKHS. The latter estimator was compared to a plug-in estimator which is also our novel contribution. Our focus on semiparametric effiency of the estimation of that CPME allows us to derive guarantees and expand to more applications than [2], for example in Section 5.2 (sampling) and Appendix 14.3.3 (recommendation).\n\n**Semi-synthetic/real-world application**: \n\n> \" In particular, the work would benefit from a semi-synthetic or real-world data application illustrating how the framework can be applied to real-world OPE tasks.\"\n\nWe thank the reviewer for suggesting a semi-synthetic and real-world validation. In response, we conduct additional experiments using 1-the Warfarin dataset (PharmGKB, 2009), commonly used in continuous-treatment causal inference (Kallus & Zhou, 2018) and 2- on the d-sprite dataset (Matthey et al., 2017; Xu & Gretton, 2023) with structured outcomes, as requested by Reviewer 9xPR. \n\n1- **Warfarin:** The setup simulates treatment assignments based on standardized BMI and Gaussian noise, while outcomes are defined via a loss function penalizing predictions outside a ±10% margin of expert therapeutic doses $A_i^*$. Logging and target policies are modeled as stochastic perturbations around this process, enabling computation of importance weights in an off-policy evaluation setting.\n\nWe mirror the synthetic testing protocol of Section 6 by evaluating four scenarios: I (Null), II (Mean Shift), III (Mixture), and IV (Shifted Mixture), each introducing distinct shifts in the treatment and outcome distribution. As in the synthetic setup, both outcome models and propensity scores are unknown and learned from data. We include the same baselines using our adapted Kernel Policy Test (KPT) with estimated propensities and a linear-kernel variant (KPT-linear). Following Reviewer k5BY’s feedback, we also report results with RBF and polynomial kernels.\n\nThe results below demonstrate that DR-KPT is well-calibrated under the null (Scenario I), with near-nominal rejection rates. In all alternative scenarios (II–IV), DR-KPT outperforms the baselines or matches the top performance, showing strong power across complex distributional shifts.\n\n| Scenario | KPT-linear | KPT-rbf | KPT-poly | DR-KPT-rbf | DR-KPT-poly |\n|----------|------------|---------|----------|------------|-------------|\n| I        | 0.00       | 0.00    | 0.00     | **0.02**       | **0.06**        |\n| II       | 0.77       | 0.01    | 0.29     | **0.80**       | 0.66        |\n| III      | **1.00**       | 0.00    | 0.66     | **0.99**       | 0.95        |\n| IV       | 0.24       | 0.00    | 0.11     | **0.76**       | 0.55        |\n\nWe will include a detailed description of the Warfarin dataset, the experiment setting and the associated results in the manuscript.\n\n2- **d-Sprite:** We kindly refer reviewer VKqz to the answer given to reviewer 9xPR for a more detailed presentation of our experiments on the structured outcome d-sprite dataset.\n\n**Questions**:\n\n1) **Structured outcomes**\n\n>\"Can the authors please clarify the limitations of CDF-based approaches in this setting?\"\n\nCDF-based approaches are not applicable in the setting where structured outcomes (e.g., images, graphs) lack a natural total order, making notions like $\\mathbb{P}(Y \\leq y)$ ill-defined. In contrast, our CPME framework leverages kernel embeddings, which remain well-defined and effective on arbitrary outcome spaces.\n\n2) **Existing frameworks, CATE**: \n\n> \"Under the this work's formal setup, which specific technical challenges are specific to OPE beyond existing frameworks on treatment effect estimation?\"\n\nTo the best of our knowledge, kernel-based treatment effect literature has largely focused on RKHS variant of binary treatment ATE/ATT (Muandet et al., 2018; Martinez-Taboada et al., 2023; Fawkes et al., 2023), except [1] (Park et al, 2021). Our OPE setup introduces distinct challenges: (i) the estimand integrates the conditional outcome embedding over the policy $\\pi(a \\mid x)$, rather than evaluating it at a fixed $a$; this requires defining and estimating novel functionals of the conditional mean embedding over $\\pi \\times P_X$, rather than existing methods evaluating it at a single atomic $a$ (see Eq (8) and Appendix 10.3). (ii) we address stochastic interventions and continuous actions beyond binary treatment settings, whereas fitting a CATE estimate would require extra efforts with continous treatments, instead we use an embedding on $\\pi$ which directly integrates over the actions sampled from $\\pi$. (iii) the variance issue due to support mismatch between the two $\\pi$ and $\\pi_0$ in importance weighting can be particularly severe, this led us to introduce a DR correction anchored in RKHS semiparametric efficient theory unlike all previous works.  Eventually, (iv) should the reviewer refers to a scalar-valued CATE, we state that our target is an RKHS-valued mean embedding, making the analysis of the estimation and inference more technically demanding.\n\n3) **Stochastic equicontinuity**: \n> \"Can the authors provide further detail on the stochastic equicontinuity condition and when it is likely to hold?\"\n\nTo build more intuition, stochastic equicontinuity essentially means that the empirical process is \"smooth\" in a probabilistic sense—it does not fluctuate wildly when the functions it is applied to vary slightly. Formally, stochastic equicontinuity of the empirical process $\\mathcal{T}_n$ (as in Definition 11.4) is equivalent to showing\n$\\mathcal{T}_n = (P_n - P)(\\widehat{\\psi}_n^\\pi - \\psi^\\pi) = o_P(n^{-1/2})$.\n\nUnder Assumptions 1, 3, 15, 17, and 21, the functions $\\widehat{\\psi_n}^\\pi - \\psi^\\pi $ lie in a shrinking ball of the RKHS $\\mathcal{H}_\\mathcal{Y}$ (see Lemma 11.3).\n\nIf the output kernel $k_\\mathcal{Y}$ is a $C^\\infty$ Mercer kernel, then this ball has exponentially decaying covering numbers (see Cucker & Smale, 2002 \\[75, Thm D]).\nThus, we can apply Theorem 6 of Park & Muandet, 2023 \\[60] on the class\n$\\mathcal{G} = \\{ \\widehat{\\psi_n}^\\pi - \\psi^\\pi \\} \\subset L^2(P; \\mathcal{H}_\\mathcal{Y})$,\nwhich ensures that the empirical process $\\mathcal{T}_n$ is asymptotically equicontinuous.\n\nNote that more generally, stochastic equicontinuity holds when the output kernel $k_{\\mathcal{Y}}$ is smooth and the RKHS ball satisfies one of:\n- it is totally bounded and has finite Assouad dimension (Theorem 4, Park & Muandet, 2023), meaning local covering numbers scale uniformly across the set;\n- it has finite box-counting dimension (Theorem 5, Park & Muandet, 2023), capturing how the number of covering balls grows as their radius shrinks;\n- or it has exponential entropy (Theorem 6, Park & Muandet, 2023).\n\n\n**Limitations**: \n\nWe agree that the limitations of Assumption 1 merit further discussion. We will include a brief discussion on potential failures of these assumptions—particularly violations of exchangeability or positivity—and their implications for the reliability of our estimator. A more extensive empirical or theoretical analysis of these violations is an important direction we will leave for future work."}, {"review_id": "SLaXs8wX15", "reviewer": "Reviewer_9xPR", "summary": "The paper introduces a new method for distributional off-policy evaluation in causal inference. The method is based on so-called counterfactual mean embeddings, which allows to employ kernel-based methods. The authors combine the method with established concepts from causal inference such as doubly robust estimators and develop kernel-based statistical inference. The method is evaluated using synthetic data.", "questions": "- Could the authors elaborate on the novelty of their method, particularly compared to existing kernel-based methods for distributional causal inference. A list of technical novelties (method or theoretical results), ideally understandable without an extensive background in kernel-based methods would be highly appreciated.\n- What is the specific advantage of using kernel-based methods in the setting studied (as compared to using other ML-based approaches combined with influence-function based estimators)?", "limitations": "The discussion section (Sec. 7) would benefit from a more comprehensive discussion on limitations.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- Distributional off-policy evaluation is an important problem in various disciplines\n- The proposed method is theoretically sound\n- The experimental results seem promising\n\n\nWeaknesses:\n- From a causal inference point of view the novelty of the proposed method is limited: while an influence function-based estimator for the distribution of policy induced outcomes is proposed, its structure is similar to existing AIPTW-type estimators (see e.g., Kennedy et al., Semiparametric counterfactual density estimation). Thus the contribution seems to stem mostly for the kernel-based approach via counterfactual policy mean embeddings. Unfortunately, I am not very familiar with the kernel-based ML literature and thus unable to judge the novelty as compared to related literature (e.g., Muandet et al., Couterfactual mean embeddings). However, techniques for off-policy evaluation tend to be structurally very similar as compared to e.g., ATE estimation (in the world of expectations), which is why I suspect that the proposed method shares similarities with existing ones. \n- One main motivation as compared to existing methods (e.g., Chernozhukov et al.) is that the proposed methods allows for complex outcome types such as images. However, the synthetic experiments are based on simple DGPs and one-dimensional continuous outcomes. Hence, (i) existing baselines seem very well applicable and should be compared against, and (ii) I think the paper would benefit from an experiment using more complex outcome distirbutions. Additionally, a case study using real-world data could strengthen the motivation.\n- The method is restricted to a kernel-based approach. In contrast, many state-off-the-art causal inference methods are model-agnostic and allow for arbitrary estimation of nuisance functions. Would it be possible to instantiate the method using a different ML methodology (e.g., neural nets)?", "was_revised": true, "final_justification": "Technically solid and decently novel methodology for a relevant problem. In their rebuttal, the authors addressed my concerns regarding novelty and limited experimental evaluation. I recommend acceptance.", "rebuttal": "We thank reviewer 9xPR for their thoughtful and constructive comments. In what follows, we have done our best to address the key concerns raised in the review. If any aspect remains unclear or could benefit from further discussion, we would be happy to elaborate. If our responses satisfactorily resolve the concerns, we would greatly appreciate it if the reviewer considered updating their evaluation.\n\n**Novelty**: \n\n> \"From a causal inference point of view the novelty of the proposed method is limited: while an influence function-based estimator for the distribution of policy induced outcomes is proposed, its structure is similar to existing AIPTW-type estimators (see e.g., Kennedy et al., Semiparametric counterfactual density estimation). Thus the contribution seems to stem mostly for the kernel-based approach via counterfactual policy mean embeddings.\"\n\nOur contribution does not stem only from the kernel-based approach in counterfactual policy mean embedding (CPME). While our estimator shares a double robustness structure reminiscent of AIPTW, extending it to **Hilbert space-valued parameters** like the CPME is far from trivial. Unlike pointwise density estimation (e.g., Kennedy et al. 2021) which is a scalar-valued parameter, our method targets the entire counterfactual distribution as a functional object in an RKHS, requiring the following non-trivial and different from scalar-valued semiparametric effiency theory: \ni) formal pathwise differentiability analysis in Hilbert norm, \nii) derivation of a Hilbert-valued efficient influence function, and \niii) guarantees for asymptotic linearity in infinite-dimensional spaces. \n\nTo our knowledge, this is the first work to provide a one-step estimator and inference procedure for RKHS-valued causal estimands in OPE, making our contribution novel from both a statistical inference and kernel-based representation standpoint. We thank the reviewer for this feedback and we will include clarifications in our manuscript.\n\n> \"Could the authors elaborate on the novelty of their method, particularly compared to existing kernel-based methods for distributional causal inference. A list of technical novelties (method or theoretical results), ideally understandable without an extensive background in kernel-based methods would be highly appreciated.\"\n\nBelow we provide a list of technical points (methods, results) with a clear positionning with regard to existing kernel-based methods for distributional causal inference.\n\n- Novel CPME framework: The kernel-based treatment effect literature did not tackle OPE and has largely focused on distributional effect variants of binary treatment ATE/ATT (e.g., Muandet et al., 2018; Martinez-Taboada et al., 2023; Fawkes et al., 2023), with the exception of [1] (Park et al., 2021), which considers a CATE variant. In that sense, our distributional CPME embedding introduces and motivates novel objects.\n- Novel estimators and results. The plug-in and one-step estimators we propose—together with their theoretical analysis—are novel contributions of this work. The results in Theorems 5 and 6 establish consistency guarantees in RKHS that, to the best of our knowledge, are not available in prior literature. Previous works such as (Muandet et al., 2018) focused exclusively on IPW-type estimators, while (Fawkes et al., 2023) and (Martinez-Taboada et al., 2023) proposed DR-based test statistics for hypothesis testing but did not provide consistency results in the RKHS.\n- Existing t-statistic for hypothesis testing. Among these, Muandet et al., Park et al., and Fawkes et al. rely on computationally intensive permutation tests, whereas we use a doubly robust test statistic as in Martinez-Taboada et al. (2023), yielding improved efficiency and tractability. Our definition of t-statistic in OPE is novel itself but it shares similar elementary ideas and properties with that of Martinez-Taboada et al. (2023) and Kim and Randas (2024).\n- Novel EIF in RKHS, further applications. However, unlike Martinez-Taboada et al., who focus exclusively on hypothesis testing our CPME framework for OPE is grounded semiparametric efficiency theory of RKHS-valued parameters. As stated above, our analysis of the (EIF) efficient influence function in the RKHS is exclusive and novel. A consequence of these is that unlike Martinez-Taboada et al., our work improves the applications of Muandet et al, 2018 beyond hypothesis testing, such as policy sampling (Section 5.2) and recommendation (Section 14.3.3). \n\nWe will update our paper to better list its technical novelties. \n\n**Structured outcomes**\n\n> \"One main motivation as compared to existing methods (e.g., Chernozhukov et al.) is that the proposed methods allows for complex outcome types such as images. However, the synthetic experiments are based on simple DGPs and one-dimensional continuous outcomes. Hence, (i) existing baselines seem very well applicable and should be compared against, and (ii) I think the paper would benefit from an experiment using more complex outcome distirbutions. Additionally, a case study using real-world data could strengthen the motivation.\"\n\nWe agree entirely with the reviewer’s suggestion regarding the need to evaluate more complex outcome distributions. To this end, we include additional experiments on the dSprites dataset (Matthey et al., 2017; Xu & Gretton, 2023), which enables structured image outcomes. Unlike scalar outcomes in our other simulations, here the counterfactual effect of a policy is assessed through 2D images rendered from latent state variables. The structural causal model is defined by latent contexts $U \\sim \\mathcal{U}([0,1]^2)$, actions $A \\sim \\pi(\\cdot \\mid U)$, and outcomes $Y := g(U, A) \\in \\mathbb{R}^{64 \\times 64}$, where $g$ maps each context–action pair to an image by decoding spatial latents (X, Y) in the dSprites generative process, with all other latent factors (shape, scale, orientation) held fixed. As in previous experiments, the logging and target policies $\\pi, \\pi'$ are contextual Gaussians $\\mathcal{N}(\\mu(U), \\sigma^2 I)$, where $\\mu(U)$ encodes a rotated and shifted transformation of the context. In this semi-synthetic setup, the covariates $U$ and treatments $A$ are synthetic, and the outcome $Y$ is deterministically retrieved from the  image database via the fixed renderer $g$. Since we are considering distributions over images, scenarios I and IV are relevant, where outcome distributions are either identical (null case) or differ due to policy shifts, enabling controlled evaluation under limited outcome flexibility. We follow the same evaluation procedure as for the synthetic and semi-synthetic Warfarin (we kindly refer to the answer to Reviewer VKqz) experiments.\n\n| Scenario | KPT-linear | KPT-rbf | KPT-poly | DR-KPT-rbf | DR-KPT-poly |\n|----------|------------|---------|----------|------------|-------------|\n| I        | 0.394   | 0.401| 0.375 | **0.024**   | **0.000**    |\n| IV       | 0.081   | 0.054| 0.073 | **0.656**   | **0.502**    |\n\nThe results above highlight the poor calibration of baseline methods under the null (Scenario I), with inflated rejection rates approaching 40%, while DR-KPT maintains near-nominal levels. In the alternative scenario (IV), DR-KPT achieves substantially higher power than all baselines, confirming its robustness and sensitivity in detecting structured distributional shifts with complex outcomes. \n\nWe thank the reviewer for this suggestions; in the revised manuscript, we will include those and additionally add an illustration of Section 5.2 by sampling images from the estimated counterfactual outcome distributions, offering visual insight into the detected differences.\n\n\n**Alternative to kernel-based methods**\n\n> \"The method is restricted to a kernel-based approach. In contrast, many state-off-the-art causal inference methods are model-agnostic and allow for arbitrary estimation of nuisance functions. Would it be possible to instantiate the method using a different ML methodology (e.g., neural nets)?\" \n\nWe use kernel methods because our purpose is to estimate a kernel embedding of the outcome distribution. Tackling distributional causal inference by embedding such distributions in a RKHS is at the core of the literature on kernel treatment effects. Indeed, kernel mean embeddings (KMEs) provide the main practical and theoretically grounded approach for distribution embeddings in causal inference and statistical learning. That being said, the reviewer raises an important point: distribution representations could also be learned via neural mean embeddings, as explored by Xu & Gretton (2022) in \"A Neural Mean Embedding Approach for Back-door and Front-door Adjustment.\" We agree this is a compelling direction and view it as a promising avenue for future work.\n\n> \"What is the specific advantage of using kernel-based methods in the setting studied (as compared to using other ML-based approaches combined with influence-function based estimators)?\"\n\nThe influence function in our setting is inherently tied to a reproducing kernel Hilbert space (RKHS) due to the structure of the counterfactual policy mean embedding (CPME), which is an RKHS-valued quantity. This naturally leads to the use of kernel methods, particularly for estimating the conditional mean embedding $\\mu_{Y|X,A}$, which admits a closed-form solution via kernel ridge regression. Other nuisance components, such as the propensity score, are not RKHS-valued and can be estimated using any suitable machine learning method.\n\n**Limitations** \n\nWe thank the reviewer for their feedback on Section 7, we will include further limitations discussed above and from other reviews."}, {"review_id": "JnUBKKHxcm", "reviewer": "Reviewer_k5BY", "summary": "This paper proposes a novel framework for distributional off-policy evaluation based on Counterfactual Policy Mean Embeddings (CPME), extending kernel mean embedding methods to estimate the full distribution of counterfactual outcomes under stochastic policies. The authors introduce both a plug-in estimator and a doubly robust (DR) estimator leveraging efficient influence functions, achieving improved convergence rates. Furthermore, the work provides a doubly robust kernel test for hypothesis testing and a sampling procedure from the counterfactual distribution. The paper presents theoretical guarantees, including asymptotic normality of the test statistic and convergence rates for the estimators, and supports claims with numerical experiments.", "questions": "- Have you considered applying the method to more datasets, e.g., semi-synthetic benchmarks?\n- How sensitive is the performance to kernel choice, particularly for structured outcomes?\n- Can you provide empirical insights into computational cost and scalability on larger datasets?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\nI think the theoretical contribution of this paper is fairly strong, including \n\n- A novel generalization of CME, from binary interventions to general stochastic policies, significantly broadening applicability; \n- Introduction of a DR estimator for the CPME is technically sound, leveraging efficient influence functions to improve convergence that's approaching parametric rates;\n- Development of the DR-KPT test for comparing counterfactual distributions avoids expensive resampling techniques, improving computational efficiency.\n\nThe paper is also very well-motivated and clearly written. \n\n**Weaknesses**\n- Experiments are rather limited, currently only restricted to synthetic settings. \n\n- Evaluation of sampling quality is limited to standard metrics (MMD, Wasserstein distance) but lacks application-driven insights (e.g., downstream policy performance or decision-making).\n\n- Scalability analysis - While the DR-KPT test is computationally efficient relative to permutation tests, kernel methods can still struggle with large datasets. Empirical results on scalability (e.g., runtime vs. dataset size) are missing.", "was_revised": true, "final_justification": "Maintain my score. See discussion above. Thank you.", "rebuttal": "We are grateful to Reviewer k5BY for their valuable comments. In what follows, we have done our best to respond to the main concerns raised. If any points remain unclear, we would be happy to clarify further. Should our responses address the reviewer’s concerns, we kindly hope they will consider revising their evaluation favorably.\n\n**Semi-synthetic experiments**\n\n> \"Experiments are rather limited, currently only restricted to synthetic settings.\" \"Have you considered applying the method to more datasets, e.g., semi-synthetic benchmarks?\"\n\nWe thank reviewer VKqz for this valuable suggestion. In response to similar feedback from reviewers k5BY and 9xPR, we have extended our experiments to include two semi-synthetic benchmarks. The first is based on the Warfarin dataset, using real-world patient covariates and simulated treatment policies to evaluate OPE under realistic conditions. The second leverages the d-Sprites image dataset, where outcomes are high-dimensional images rendered from latent action–context pairs. These setups allow us to assess our method on both real covariates and structured outcomes, and we observe that DR-KPT consistently outperforms baselines in both settings. We kindly refer the reviewer to answers to reviewers k5BY and 9xPR.\n\nFull details and results will be included in the revised manuscript. \n\n\n**Sampling and application driven insights**\n> \"Evaluation of sampling quality is limited to standard metrics (MMD, Wasserstein distance) but lacks application-driven insights (e.g., downstream policy performance or decision-making).\"\n\n\nWe emphasize that sampling from the counterfactual outcome distribution has broader practical relevance—particularly when outcomes are structured (e.g., images or graphs). In such settings, visualizing samples from $\\nu(\\pi)$ can offer intuitive and interpretable insights into the qualitative effects of deploying different policies, which are not captured by scalar metrics like MMD or Wasserstein distance. Beyond visualization, access to samples enables application-driven evaluations, such as measuring diversity, safety, or coverage of outcomes under a given policy, and supports expert-in-the-loop assessments that are critical in high-stakes domains.\n\nMoreover, working with the entire outcome distribution embeddings allows practitioners to reason about distributional properties that directly affect decision-making—such as the presence of rare but critical outcomes (e.g., adverse drug effects), the tail behavior of policies (e.g., in finance), or the modality of outcomes. For instance, when outcome distributions are multimodal, a decision-maker may prefer policies that induce more predictable, unimodal behaviors. These considerations go beyond the expectation of the outcome distribution and are central to robust and informed policy selection.\n\nWe thank the reviewer for this feedback, we will clarify and emphasize these broader perspectives in the revised manuscript.\n\n**Structured outcomes, choice of kernel**\n\n> \"How sensitive is the performance to kernel choice, particularly for structured outcomes?\"\n\nWe thank the reviewer for raising this point. To assess sensitivity to kernel choice, we compare RBF and polynomial kernels in both the Warfarin and dSprites experiments (Tables in answers to k5BY and 9xPR). In both settings, DR-KPT achieves strong performance with both kernels, consistently outperforming baselines. While some differences appear across scenarios, the method remains robust, indicating limited sensitivity to the specific kernel choice in practice.\n\n\n**Scalability analysis** \n\n> \"Scalability analysis - While the DR-KPT test is computationally efficient relative to permutation tests, kernel methods can still struggle with large datasets. Empirical results on scalability (e.g., runtime vs. dataset size) are missing.\" \"Can you provide empirical insights into computational cost and scalability on larger datasets?\"\n\nWe thank the reviewer for raising this important point. We would like to clarify that Tables 1–4 in Appendix 14.1.3 report empirical results on scalability, showing how performance varies with dataset size (sample sizes ranging from 100 to 400). We acknowlege that this could have been made clearer, and we will revise the table captions and update the main text to highlight these results more explicitly. \n\nAdditionally, we provide further runtime tables (in seconds) below to strengthen our empirical support in this direction.\n\n**Synthetic data**\n| Method        | 100                | 200                | 500                 | 1000                | 2000                |\n|---------------|--------------------|--------------------|---------------------|---------------------|---------------------|\n| DR-KPT-RBF    | **0.135 ± 0.022**      | **0.147 ± 0.016**      | **0.325 ± 0.019**       | **1.196 ± 0.158**       | **1.126 ± 0.135**       |\n| DR-KPT-Poly   | **0.118 ± 0.011**      | **0.140 ± 0.021**      | **0.314 ± 0.020**       | **1.155 ± 0.172**       | **1.119 ± 0.126**       |\n| KPT-RBF       | 1.712 ± 0.170      | 4.786 ± 0.157      | 104.256 ± 16.576    | 366.104 ± 99.063    | 306.406 ± 44.161    |\n| KPT-Poly      | 1.824 ± 0.260      | 4.587 ± 0.246      | 106.186 ± 55.371    | 354.062 ± 13.737    | 334.608 ± 81.867    |\n| KPT-Linear    | 1.801 ± 0.191      | 4.573 ± 0.465      | 84.999 ± 14.167     | 387.368 ± 262.004   | 285.999 ± 71.099    |\n\n**Warfarin data**\n| Method        | 1000                 | 2000                  | 3000                  | 4000                  |\n|---------------|----------------------|------------------------|------------------------|------------------------|\n| DR-KPT-RBF    | **0.426 ± 0.024**        | **2.530 ± 1.447**          | **5.775 ± 0.125**          | **11.701 ± 1.869**         |\n| DR-KPT-Poly   | **0.485 ± 0.076**        | **1.862 ± 0.030**          | **6.660 ± 3.180**          | **11.184 ± 0.170**         |\n| KPT-RBF       | 375.355 ± 112.770    | 1338.653 ± 262.921     | 2672.691 ± 20.895      | 5657.573 ± 413.196     |\n| KPT-Poly      | 331.831 ± 48.219     | 1315.537 ± 212.152     | 3308.014 ± 1219.752    | 5165.057 ± 667.347     |\n| KPT-Linear    | 364.378 ± 35.433     | 1302.173 ± 300.546     | 2651.653 ± 103.833     | 3623.222 ± 373.766     |\n\nAs the tables show, DR-KPT achieves significantly lower runtimes than all KPT variants—often by several orders of magnitude—while maintaining efficiency at scale. This underscores its clear computational advantage. We will include these additional runtime experiments in the updated manuscript."}, {"review_id": "U6SXTLacS4", "reviewer": "Reviewer_Gtoa", "summary": "This paper proposes counterfactual policy mean embedding (CPME), using kernel mean embedding of counterfactual distributions. Both a plug in estimator and a doubly robust estimator are proposed. Some theoretical analysis of these estimators is performed, and the authors show that hypothesis testing and sampling can be performed using this embedding.", "questions": "On line 142, what decomposition are you referring to? Also, $\\mu_\\pi$ can be defined without the need for the operators to exist. \n\nIn (8), does this mean that you are using the uniform distribution over the action space $\\mathcal{A}$? And shouldn't you divide by the cardinality of $\\mathcal{A}$ here?", "limitations": "There is no discussion of limitations. Societal impact is not relevant for this paper.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\nThis paper, while it does not provide anything groundbreaking, is a very solid paper which uses several existing methods and proof techniques in kernel mean embeddings, doubly robust estimation, cross U-statistics and kernel herding in the context of counterfactual policy. I think that showing that one can use existing methods and theory in different contexts is valuable. \n\nIt is in general a very well-written paper, it was a pleasure to read. Thank you very much. \n\n**Weaknesses**\n\nHowever, there are some drawbacks that make me hesitate to give a solid accept recommendation for this paper. Some specific points are listed below and in the questions section, but in general, among many things one can do for counterfactual policy analysis, hypothesis testing of whether there is *any* difference and sampling seem like small subsets. I think it is a bit of a shame that only these were studied, and I have a feeling that, in particular, the hypothesis testing was chosen because of the choice of kernel embeddings, and what they allow one to do. For sampling, I do not kernel herding is perceived as the best way to sample from distributions, unless I'm mistaken?\n\nIn Assumption 1 and below, the authors say that the SUTVA assumption and the no-interference assumption are the same, but this is not true. Actually, what is written in maths in Assumption 1(i) is the *consistency* assumption, whereas its description below on lines 117-119 is the *no interference* assumption. SUTVA is usually considered as the combination of the *consistency* and *no interference* assumptions, but there is no maths given for the no-interference assumption. In fact, on line 1124 in the appendix, the authors write that the consistency assumption is the same as SUTVA, which is also not true. \n\nLines 140-141: The authors brush aside the assumption for the existence of the conditional mean operator as a *regularity condition* but this is actually a severe restriction that is violated with most common kernels and domains. Moreover, the authors miss out another severe assumption, that the covariance operator $\\mathcal{C}_{A,X}$ is invertible. This is another assumption that is very often violated. Proposition 4 relies on these assumptions, and therefore will not hold in most common situations. \n\nAll of this and the subsequent development can go through without operators, and the severe entailed assumptions. The conditional mean embeddings can be viewed as simply conditional expectations (which they are), and then estimation can be done by regression. \n\nThe theory in Section 3 is solid of course, but nothing new, as they are pretty much lifted from existing works. \n\n**Minor Points**\n\nLine 38: over -> or\n\nEquations (8), (11) and (14) need full stops. So does the displayed equation after line 230. \n\nAlgorithm 3 is in the Appendix but referred to without any reference in Theorem 5. I think it would be better to put it in the main body. \n\nIn (13) and the line below, the hats should be removed right? This discussion is all about population statistics. \n\nAlgorithm 1: For consistency of notation, the first line should have lowercase data $x_i,a_i,y_i$.", "was_revised": true, "final_justification": "I've read the authors' responses, and see no reason to change my positive (but not overly positive) evaluation of the paper.", "rebuttal": "We thank reviewer Gtoa for their encouraging and attentive review. We have carefully considered the points raised and provide our responses below. If any issues remain ambiguous, we would be pleased to elaborate further. We hope these clarifications will support a more favorable evaluation and lead the reviewer to consider raising their score.\n\n**Weaknesses** \n\n> \"among many things one can do for counterfactual policy analysis, hypothesis testing of whether there is any difference and sampling seem like small subsets\"\n\nWe agree that hypothesis testing and sampling cover only a subset of what can be done in counterfactual policy analysis. We highlight hypothesis testing here because it plays a key role in off-policy model selection, complementing evaluation and optimization: for instance, a learned policy may optimize expected reward, but selection may hinge on distributional properties like variance or CVaR. Moreover, sampling from the counterfactual distribution is similarly useful for policy simulation, safety checks, or data augmentation, especially in structured outcome settings. Regarding kernel herding, we considered it because it is a well-established and principled method for generating low-discrepancy samples from RKHS embeddings. That said, recent advances in MMD-based gradient flows provide powerful alternatives. In particular, \"Maximum Mean Discrepancy Gradient Flow\" (Arbel et al., NeurIPS 2019) and \"Deep MMD Gradient Flow without Adversarial Training\" (Galashov et al., ICLR 2025) propose to sample from distributions by transporting particles along a Wasserstein gradient of the MMD functional, enabling scalable generation in complex domains. Incorporating such approaches in our counterfactual framework is a promising direction for future work. We will discuss this in our paper.\n\n\n> \"In Assumption 1 and below, the authors say that the SUTVA assumption and the no-interference assumption are the same, but this is not true\"\n\nThank you for this precise and helpful comment. The reviewer is right that our current presentation conflates the *consistency* assumption with the full Stable Unit Treatment Value Assumption (SUTVA). The expression $Y = Y(a)$ when $A = a$ captures *consistency*, meaning the observed outcome equals the potential outcome under the treatment actually received. However, *SUTVA* consists of both consistency and *no interference*, the latter meaning that one unit’s potential outcome does not depend on the treatment assignments of other units—formally, $Y_i(\\mathbf{a}) = Y_i(a_i)$, where $\\mathbf{a}$ is the treatment vector for all units. We thank the reviewer for highlighting this imprecision and will revise the main text and appendix accordingly.\n\n> \" The authors brush aside the assumption for the existence of the conditional mean operator as a regularity condition but this is actually a severe restriction that is violated with most common kernels and domains.\"\n\nWe thank the reviewer for pointing out this important issue. We agree that the existence of the Hilbert–Schmidt conditional mean operator $C_{Y \\mid A,X} \\in S_2(H_{A,X}, H_Y)$\nis a nontrivial assumption and, as the reviewer correctly notes, can be violated with certain combinations of unbounded domains and common kernels (e.g., Gaussian RBF on $\\mathbb{R}^d$). However, we kindly disagree with the statement that this assumption is violated with *most* common kernels and domains. In fact, as shown in the recent JMLR paper by Li et al. (2024), *Towards Optimal Sobolev Norm Rates for the Vector-Valued Regularized Least-Squares Algorithm*, when the widely used Matérn kernel is employed on the $X$-space, the conditional mean function $F_\\star(x) := \\mathbb{E}[\\phi(Y) \\mid X = x]$ can be written as $C_{Y \\mid X} \\phi(x)$ (where $C_{Y \\mid X} \\in \\mathcal{L}(H_X, H_Y)$ is the conditional mean operator) as long as $F_*$ is smooth enough in a traditional sense.\nSpecifically, Proposition 1 in that work establishes that such a representation exists as long as $F_\\star \\in H^m(X; H_Y)$, the Sobolev-type vector-valued RKHS associated with a Matérn kernel on $\\mathbb{R}^d$ with smoothness $m$. Thus, the existence of the conditional mean operator $C_{Y \\mid X}$ is not assumed universally—it is *induced* as a representation whenever $F_\\star \\in H^m(X; H_Y)$, as is typical in the kernel ridge regression literature for vector-valued targets.\nIn addition, the membership $F_\\star \\in H^m(X; H_Y)$ can be established under a mild regularity assumption on the conditional distribution $p(y \\mid x)$. Concretely, by differentiating under the integral sign, one obtains\n$$\nD^\\alpha F_\\star(x) = \\int \\phi(y) D^\\alpha p(y \\mid x) dy,\n$$\nand thus\n$$\n\\Vert F_\\star\\Vert_{H^m}^2 = \\sum_{|\\alpha| \\le m} \\int_X \\left\\Vert \\int \\phi(y) D^\\alpha p(y \\mid x) dy \\right\\Vert_{H_Y}^2 d\\pi(x).\n$$\nAssuming the feature map is bounded $\\Vert \\phi(y)\\Vert_{H_Y} \\le \\kappa$ (which is the case when the kernel is bounded), this norm is finite as soon as\n$$\n\\max_{|\\alpha| \\leq m} \\sup_{x \\in \\mathcal{X}} |D^\\alpha p(y \\mid x)| < \\infty \\quad \\text{for all } |\\alpha| \\le m.\n$$\nThis condition corresponds to a smoothness assumption on the conditional density—an assumption that is mild and often satisfied in practice. Therefore, under such conditions, $F_\\star \\in H^m(X; H_Y)$, which in turn implies that the conditional mean embedding operator $C_{Y \\mid X}$ exists and acts boundedly from $H_X$ into $H_Y$.\nWe will clarify these points in the revised manuscript and would be happy to provide additional technical details if needed.\n\n> \"Moreover, the authors miss out another severe assumption, that the covariance operator $C_{A,X}$ is invertible.\"\n\nWe respectfully disagree with the claim that our framework relies on the invertibility of the joint covariance operator $C_{A,X}$. Throughout our work, we do not make such an assumption, nor is it required for the validity of our results or the results we invoked from the literature. The conditional mean embedding operator $C_{Y \\mid A,X}$ is defined as the solution to the vector-valued regression problem $\\mu_{Y|A,X}(a, x) := \\mathbb{E}[\\phi(Y) \\mid A = a, X = x]$, but can be estimated via regularized kernel regression without requiring the operator $C_{A,X} \\in \\mathcal{L}(H_{A,X})$ to be invertible. \n\nFor further discussion on the invertibility of kernel covariance operators in conditional mean embeddings, we refer the reviewer to Remark 2 of the NeurIPS 2022 paper by Li et al. \"Optimal Learning Rates for Regularized Conditional Mean Embedding\", which clarifies that operators like $C_{XX}^{-1}$ are not globally defined in infinite-dimensional RKHSs and highlights how well-specified formulations (with CME operators) circumvent this limitation.\n\n> \"All of this and the subsequent development can go through without operators, and the severe entailed assumptions. The conditional mean embeddings can be viewed as simply conditional expectations (which they are), and then estimation can be done by regression.\"\n\nWe thank the reviewer for the suggestion. Indeed, it is possible to develop an alternative formulation that bypasses the explicit use of operators and relies purely on the regression view of conditional expectations, as done in \"A Measure-Theoretic Approach to Kernel Conditional Mean Embeddings\" (Park & Muandet, 2020).\n\nNevertheless, in the well-specified case, as explained above, there exists an isomorphism between the space of conditional mean embeddings and that of conditional mean operators. This correspondence enables the use of the operator-theoretic viewpoint, which is particularly valuable for deriving convergence guarantees and facilitating theoretical analysis.\n\n\n> \"The theory in Section 3 is solid of course, but nothing new, as they are pretty much lifted from existing works.\n\"\n\nWhile the theory in Section 3 builds on established foundations, it supports a plug-in estimator that, to our knowledge, has not been previously studied in the context of distributional kernel treatment effects. Prior works—such as Muandet et al. (2018), Park et al. (2021), Fawkes et al. (2023), and Martinez-Taboada et al. (2023)—focused on IPW or DR-type estimators, and their proposed DR statistics were not analyzed through the lens of semiparametric efficiency theory.\n\n\n**Minor points**: we thank the reviewer for pointing out these corrections l.38, in Eq (8), (11), (14) and Algorithm 1. In Eq. (13) indeed, there should be hats. We will implement them and do our best given space constraints to put Algorithm 3 (and even a one-step estimator Algorithm!) in the main text and/or add references in Theorem 5, 6.\n\n**Questions**:\n1. Decomposition\n> On line 142, what decomposition are you referring to? Also, \n can be defined without the need for the operators to exist. \n \nThe \"decomposition\" mentioned on line 142 prematurely refers to the decoupled representation of the counterfactual policy mean embedding as a composition of the conditional mean operator and the kernel policy mean embeddin. We aknowledge a typo, this formulation should appear after the proper definition of $\\mu_\\pi$ to avoid confusion. Importantly indeed, the definition of $\\mu_\\pi$ itself does not rely on the existence of the conditional mean operator or this decomposition. \n\n2. Equation (8) \n> \"does this mean that you are using the uniform distribution over the action space $\\mathcal{A}$ ? And shouldn't you divide by the cardinality of $\\mathcal{A}$ here?\"\n \nWe are not assuming a uniform distribution over the action space, and the expression does not require normalization by the cardinality of $\\mathcal{A}$. It simply provides the analytical form of the expectation implied by definition of the kernel policy mean embedding for discrete actions under a general target policy $\\pi(a \\mid x)$. We will revise the text to make this clearer.\n\n**Limitations**:\n\nWe will include a discussion on the limitations of the methods, including these remarks above and feedbacks from all reviewers."}, {"review_id": "FdYJVXkeoX", "reviewer": "Reviewer_Q3YK", "summary": "The paper describes a representation of the distribution of counterfactual outcomes in reproducing kernel Hilbert spaces (RKHS). It is computed via a vector being the Hilbert space embedding of the joint distribution of actions and covariates, and an operator acting on that vector. \nFurther, it elaborates on the optimal estimation of these objects in detail.", "questions": "-\tPlease provide more intuition about (10). \n\n-\t“cumulative density functions”? Do you mean cumulative distribution functions?\n\n-\tRegarding “counterfactual mean embedding”. Actually, we are still talking about interventional distributions, which is rung 2 in Pearl and Mackenzies’s ladder of causation, while the term “counterfactual” suggests rung 3. Please clarify which notion of counterfactual you are referring to. \n\n-\tLine 84: “we will also abbreviate the joint distributional”  check grammar\n\n-\tLine 169: Please explain resampling strategy in a few words\n\n-\tAs stated above, Section 4 is getting technically heavy and doesn’t feel self-consistent: can you provide some sentences in your rebuttal that explain eq. (9). “Riesz representer of the pathwise derivative”, all this would require some reading of the references, unless one is an expert on that specific topic. \n-\t\n\n-\tExplain statement “which are often doubly robust for many causal parameters [31]” which causal parameters tdo you refer to and specify definition of “doubly robust”. \n\n-\tx_i instead of x in eq (11)?", "limitations": "the paper does not explicitly discuss limitations", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength: \n\nThis work defines, to the best of my knowledge, an interesting step in the direction of non-parametric confounder correction. It allows analyzing outcome distributions beyond means. Although it is not the first work to leverage the RKHS framework for this purpose, it clearly states which parts are novel and which are not. \n \nWeakness: \n\n-\tSection 4 is hard to read. While the sections before mainly require some familarity with RKHS, Section 4  requires knowledge from double robust ML, one-step estimators, effective influence functions, pathwise derivates. All these concepts occur simulateously within a short section before Lemma 4.1. This way, the paper stays a bit in the tradition of a lot of works in the field of “doubly robust ML”, where one gets easily lost in formulas without intuition. The RKHS framework is very elegant. Therefore, the authors should be more ambitious to leverage its elegance also for a presentation of dr ML that is more didactic than most of the dr ML literature out there.  I’m not saying that this is easy, but it would be worth an additional effort.  \n\nPlease use the rebuttal to suggest some formulations that render Section 4 more readable for readers who have a good math background, but do not know all the details of dr ML and do not want to read all the appendix. \n\n-\tThe paper uses the term “embedding” in  slightly inconsistent way. Eq. (5) is called a policy mean embedding, although it is actually an embedding of the joint distribution of actions and covariates, while the policy is defined as a conditional distribution.  Unfortunately, similar confusions already occur in the literature when people talk about “Hilbert space embedding of conditional distributions” in which conditional distributions are actually Hilbert space operators, and not vectors. In case these operators are Hilbert Schmidt class, they can be considered as vectors in the tensor product, but conditionals are often not Hilbert Schmidt, e.g., the identity.", "was_revised": true, "final_justification": "This is a technically challenging paper which appears to be very solid work. Some concerns remain that it is not accessible to a broad audience.", "rebuttal": "We thank reviewer Q3YK for their helpful and constructive feedback. In what follows, we address the main concerns raised to the best of our ability. If any points remain unclear, we would be happy to clarify further. Should our responses resolve the reviewer’s concerns, we kindly hope they will consider revising their evaluation favorably.\n\n**Clarification of Section 4**:\n> \"Please use the rebuttal to suggest some formulations that render Section 4 more readable\".\n\nWe thank the reviewer for highlighting that Section 4 could benefit from a more didactic presentation, especially for readers less familiar with the doubly robust (DR) literature. Below are reformulations focused on specific terms/concepts of the DR literature.\n\n1. Revision l. 191, first sentence of Section 4. \n> \"To design our estimator, we rely on the recent semiparametric efficiency theory of Hilbert-valued parameters...\"\n\n\"To design our estimator, we build on semiparametric efficiency theory extended to Hilbert space-valued parameters, which allows us to characterize the sensitivity of the functional object of interest -the counterfactual policy mean embedding- to small changes in the data distribution $P_\\pi$.\"\n\n2. Remove l. 194\n> \", which are often doubly robust for many causal parameters [31].\"\n\nDoubly robustness is explained in l. 212, there is no need to mention this property here, especially because it is hard to explain it without priorly giving the form in Eq. (10).\n\n3. Revision l. 194, \n\n> Assuming the existence of an EIF $\\psi^\\pi$ for the CPME\n$\\chi(\\pi)$, the one-step correction of the plug-in estimator requires an estimate $\\hat \\psi^\\pi$ and takes the form:\n\n\"Assuming the existence of an EIF $\\psi^\\pi$ for the CPME $\\chi(\\pi)$, we apply a one-step correction to the plug-in estimator. Recall the plug-in estimator computes the target quantity by substituting estimates of a nuisance component—the conditional mean embedding $\\mu_{Y|X,A}$— into its defining formula. However, when these components are estimated from data, the resulting estimator inherits their bias. The one-step estimator eliminates this leading bias by using the EIF, which captures the local sensitivity of the parameter to perturbations in the data distribution. Such an EIF provides the optimal linear adjustment to restore the statistical efficiency even when one of the nuisance estimators (outcome regression or propensity score) is imperfect.\n\"\n\n4. Revision l.196\n> \"One-step estimators rely on pathwise differentiability, which captures the smoothness of a parameter along regular univariate submodels [58, 59]. When this condition holds, the efficient influence function (EIF) exists and coincides with the Riesz representer of the pathwise derivative [40].\"\n\n\"One-step estimators rely on *pathwise differentiability*, a smoothness condition that characterizes how a parameter changes under infinitesimal perturbations of the data-generating distribution along regular univariate submodels \\citep{pfanzagl1990estimation, van1991differentiable}. When this condition holds, the *efficient influence function* (EIF) exists and serves as the best linear approximation to the parameter’s variation under such perturbations. In the Hilbert space setting, the EIF is formally the *Riesz representer* of the pathwise derivative \\citep{luedtke2024}, meaning it is the unique element in the RKHS whose inner product with any score function recovers the directional derivative of the parameter.\"\n\n\n5. Revision l. 205 \n\n> \"Note that the EIF defined in Eq. (10), similar to the EIF of the expected policy risk in OPE...\"\n\n\"The structure of Eq. (10) mirrors classical doubly robust EIFs: the first term adjusts for distribution shift via importance weighting, while the second term re-centers the estimate. Together, they form a Hilbert-valued function that acts as the optimal correction to the naïve plug-in embedding.\"\n\n6. Revision l. 212\n\n\n> \"Like all one-step estimators in OPE, our estimator enjoys a doubly robust property: it remains consistent if either $\\hat \\pi_0$ or $\\hat \\mu_{Y|X,A}$ is correctly specified.\"\n> \n\"Like other one-step estimators in OPE, our estimator enjoys doubly robustness: i) it remains consistent even when one of the propensity score $\\hat{\\pi_0}$ or the conditional mean embedding $\\hat{\\mu_{Y \\mid X,A}}$ is not consistently estimated. ii) As shown in Theorem 6 below and as most DR estimators, the estimation error of $\\hat{\\chi}_{dr}$ involves a product of the errors from these two nuisance components, which will improve the convergence rate.\"\n\n**Embedding terminology**\n\n> \"The paper uses the term “embedding” in slightly inconsistent way. Eq. (5) is called a policy mean embedding, although it is actually an embedding of the joint distribution of actions and covariates, while the policy is defined as a conditional distribution.\"\n\nWe thank the reviewer for pointing out the ambiguity in the use of the term “embedding.” Throughout the paper, we use *policy mean embedding* to refer to the RKHS embedding of the counterfactual marginal outcome distribution induced by the policy $\\pi$, namely $\\mu_\\pi = \\mathbb{E}_{\\pi \\times P_X}\\left[\\phi(a, x)\\right]$. Indeed this object is formally an embedding of the joint distribution $P_X \\times \\pi( \\cdot \\mid X)$, not only the conditional distribution $\\pi$. We will revise the terminology to reflect this more accurately (e.g., by referring to it as the *mean embedding under the policy-induced joint distribution*) to avoid confusion.\n\n\n**Questions:**\n1. Intuition on EIF in (10): the EIF consists of two terms. The first is an importance-weighted residual $\\frac{\\pi(a \\mid x)}{\\pi_0(a \\mid x)}\\left( \\phi_Y(y) - \\mu_{Y \\mid A,X}(a,x) \\right)$, which adjusts for discrepancy between the logging and target policies. The second term, $\\int \\mu_{Y \\mid A,X}(a', x) \\pi(da' \\mid x) - \\chi(\\pi)$, recenters the EIF to ensure it has zero mean. This decomposition mirrors classical EIFs for doubly robust estimators and makes clear how the RKHS structure naturally accommodates corrections for distribution shift between $\\pi$ and $\\pi_0$.\n\n2. CDF: yes, the correct term is “cumulative distribution functions (CDFs)”, we will revise the manuscript accordingly.\n3. \"Counterfactual\": our estimand corresponds indeed to interventional distributions (rung 2 in Pearl & Mackenzie's ladder), not counterfactuals in the strict sense (rung 3). We use the term \"counterfactual\" in line with prior work (e.g., Muandet et al., 2018; Park et al., 2021), where it refers to the distribution of potential outcomes under a hypothetical target policy $\\pi$. We will clarify this usage explicitly in the revision to avoid confusion.\n4. \"joint distributional\": we will change into \"joint distribution\".\n5. \"resampling\": For continuous actions, we approximate the policy mean embedding by resampling actions from the target policy $\\pi(\\cdot \\mid x_i)$ at each logged covariate $x_i$, and then compute the empirical average of the corresponding kernel features. This avoids costly integration and enables a practical plug-in estimator even in continuous action spaces. We will clarify this in the main text accordingly.\n\n6. Riesz representer of the pathwise derivative. The target parameter $\\chi(\\pi)(P)$ lies in an RKHS and varies with the data distribution $P$. Its pathwise derivative describes this variation and defines a bounded linear functional over valid score functions. By the Riesz representation theorem, this derivative has a unique representer called the efficient influence function (EIF), which provides the most variance-efficient correction term for asymptotically linear estimators. This is because the EIF is the element of $L^2(P; H_{Y})$ that both (i) satisfies the required linearity condition imposed by the pathwise derivative, and (ii) has the smallest possible norm among all such elements. Since asymptotic variance is given by the squared norm of the influence function, the EIF minimizes the variance by construction. Consequently, among all regular estimators, using the EIF $\\psi^*_\\pi$ ensures minimal asymptotic variance in the RKHS norm.\n\n7. \"Doubly robust\" we refer to the property in l. 212 that our one-step estimator remains consistent if either the propensity score $\\pi_0(a \\mid x)$ or the conditional mean embedding $\\mu_{Y \\mid A,X}(a,x)$ is correctly specified. We will clarify this.\n\n8. x_i, yes there should only be $x_i$ in Eq. 11, we will correct this.\n\n**Limitations**:\n\nWe will include a discussion of the method’s limitations in the revised manuscript, incorporating the points raised above as well as feedback from all reviewers."}]

Author rebuttal:

Meta-review: I concur with the reviewers’ positive view of the submission and recommend acceptance. The rebuttal and discussion added semi-synthetic and structured-outcome experiments that strengthen the empirical support. I ask the authors to incorporate these results, clarify the role of assumptions and limitations, and polish some technical exposition (e.g., SUTVA, operator assumptions, embedding terminology) in the final version.
Decision: Accept (poster)