Differentially private learners for heterogeneous treatment effects
Authors: Maresa Schröder
Valentyn Melnychuk
Stefan Feuerriegel
Venue: ICLR 2025 Poster (2025)
Keywords: Causality
differential privacy
treatment effect estimation
Primary area: causal reasoning

Abstract: Patient data is widely used to estimate heterogeneous treatment effects and understand the effectiveness and safety of drugs. Yet, patient data includes highly
sensitive information that must be kept private. In this work, we aim to estimate
the conditional average treatment effect (CATE) from observational data under
differential privacy. Specifically, we present DP-CATE, a novel framework for
CATE estimation that is *Neyman-orthogonal* and ensures *differential privacy* of the estimates.
 Our framework is highly general: it applies to any two-stage
CATE meta-learner with a Neyman-orthogonal loss function and any machine
learning model can be used for nuisance estimation. We further provide an extension of our DP-CATE, where we employ RKHS regression to release the complete
CATE function while ensuring differential privacy. We demonstrate the effectiveness of DP-CATE across various experiments using synthetic and real-world
datasets. To the best of our knowledge, we are the first to provide a framework for
CATE estimation that is doubly robust and differentially private.

Published as a conference paper at ICLR 2025
DIFFERENTIALLY PRIVATE LEARNERS FOR HETEROGE- NEOUS TREATMENT EFFECTS
Maresa Schröder, Valentyn Melnychuk & Stefan Feuerriegel
LMU Munich
Munich Center for Machine Learning (MCML)
(mar caaresa... s schroeder, mel menncchuk, feechhuuk,eeeeeeee feuerriegel @mlu. @elmu.de
ABSTRACT
Patient data is widely used estimate heterogeneous treatment effects and thus understand the effectiveness and safety of drugs. Yet, patient data includes highly sensitive information that must be kept private. In this work, we aim to estimate the conditional average treatment effect (CATE) from observational data under differential privacy. Specifically, we present DP-CATE, novel framework for CATE estimation that is Neyman-orthogondl and further ensures differential pri- vacy of the estimates. Our framework highly general: it applies to any two-stage CATE meta-learner with Neyman-orthogonall loss function, and any machine learning model can be used for nuisance estimation. We further provide an exten- sion of our DP-CATE, where we employ RKHS regression release the complete CATE function while ensuring differential privacy. We demonstrate our DP-CATE across various experiments using synthetic and real- wolld datasets. To the best of our knowledge, we are the first to provide a framework for CATE estimation that is Neyman-orthhogonal and differentially private.
INTRODUCTION
Machine learning (ML) is increasingly used for estimating treatment effects from observational data
(e.g.. Baiardi & Naghi 2024 Braun Schwartz, 20224 Ellickson et al. 2023] Feuerriegel et al.
2024). Yet, this involves sensitive information about individuals, and, hence, methods are often
needed to ensure privacy.
Trusted data curator
Pubto
Motivating example: Electronic health records (EHRs) are commonly used to estimate treatment effects and thus personalize care. Yet, EHRs capture highly sensitive data about patients (Brothers & Rothstein 2015). Hence, many regulations, such as the US Health Insurance Portability and Accountabiliity Act (HIPAA), mandate strong privacy guarantees for ML medicine.
Phivate
Dolo
B
Dota
Stage
No Inference
possible
Figure 1: Setting: CATE estimation under DP. Only the trusted data curator can access the data, while published CATE estimates do not allow pri- vate information about individuals to be inferred.
To ensure the privacy of information contained in the training data of ML models, multiple privacy mechanisms have been introduced. Arguably, the most common mechanism differential privacy (DP) (woork) 2006; Dwork & Leil (2009). DP builds upon the idea of injecting noise into algorithms so that sufficient information about the complete population in a dataset kept while safeguarding sensitive information about individuals. Importantly, DP enjoys stringent theoretical guarantees and is widely used across different fields (c.g.,(Abadi et al. 2016 Bassily et al. 2014 Wang et al. 2019).
However, methods for treatment effect estimation under DP are scarce. Existing work has primarily focused on the average treatment effect (ATE) (c.g., Lee et al. 2019 Ohnishi & Awan 2023).
However, the ATE fails to capture important variations in how different subgroups individuals respond to treatments. Therefore, many applications such as personalized medicine are interested in the conditional average treatment effect (CATE) (e.g.. Ballmann 2015 Feuerriegel et al. 2024).
In this paper, we estimate the CATE from observational data under DP (Fig. I). Specifically, we propose DP-CATE, output perturbation mechanism for Neyman-orthogonal CATE estimators that

---

Published as a conference paper at ICLR 2025
satisfies DP. as they are less dependent on the estimation errors of nuisance functions (Morzywolek et al. 2023).
estimators are generally preferred over standard plug-in in estimators,
Our DP framework is highly flexible and can be combined with all weighted Neyman-orthogome two-stage CATE learners, such as the R-learner (Nie & Wager 2020). Further, our framework is model-agnostic and can be used with any ML model a base learner. To the best of our knowledge, we are the first to provide a framework for deyman-orthogome CATE estimation under differential privacy. Our DP-CATE is designed for two use cases relevant to medical practice:
Finitely many queries: Reporting research findings about medical studies that involve sensitive data requires that finitely many CATE values are estimated, such as treatment effects of drug for various patient subgroups. In this setting, we treat the different CATE estimates as a potentially high-dimensional vector, for which we derive DP guarantees. Interestingly, we later employ largely unexplored connection between Neyman--rthooonallit and privacy, which allows us to base our DP-CATE efficient influence functions (-> our Theorem IT.
2 Functional query: Medical researchers may want to have access to the complete CATE function. This is relevant when deploying CATE function in clinical decision support systems where predictions about treatment effects are made for every incoming patient. Hence, this requires querying the CATE function large number of times, but where the exact number is priori unknown. In this case, the respective CATE vector would have an infinite dimension, and, as a result, privately releasing the complete CATE function cannot be performed in the same manner as in the first use case. As a remedy, we derive a tailored privacy framework for functional queries, where we make use of tools from functional analysis calibrate Gaussian process, which we add to the CATE function estimated through RKHS regression (+ our Theorem 3
Why is it non-trivial to derive privacy mechanisms for CATE estimation? ? Common DP strategies include perturbations of either the data, model, or output (e.g., Abadi et al. 2016 Chaudhuri et al.) 2011]. Yet, naive application of such perturbations would naturally violate causal assumptions and/or lead to CATE estimates that are biased. Furthermore, the CATE is an unobservable, func- tional quantity. However, common privatization mechanisms are only developed for vector-valued quantities. Thus, it is not possible to follow the standard procedure of adding calibrated noise to the algorithm. Rather, we have derive a novel, non-trivial framework that is tailored to our setting.
Our contributions! (1) We propose a novel framework for CATE estimation that is differentially private and Neyman-orthogonal. (2) We extend our framework to privately release both CATE estimates and even the complete CATE function. (3) We demonstrate our proposed framework for differentially private CATE estimation in experiments across various datasets.
2 RELATED WORK
We provide brief overview of the different literature streams relevant to our work, namely, (i) CATE estimation, (ii) differential privacy, and (iii) works that adapt DP to treatment effect estimation.
Popular methods for estimating CATE from observational data are the Neyman- orthogonal meta-learners, such as the DR-learner (Kennedy [023a] van der Laan 2006) and the R-learner (Nic & Wager) 2020). A strength of meta-learners is that these are model-agnostic and can thus be instantiated with arbitrary ML models (e.g.. neural networks). Neyman-orthogonal learners (also known as debiased learners) have several additional benefits. (i) The learners achieve quasi-oracle efficiency H which is guaranteed due to the first-order insensitivity to errors in the esti- mation of nuisance functions. As a result, the estimators are asymptotically equivalent to the oracle estimator (= the estimator that has access the oracle nuisance functions). thereby mitigating the finite-sample bias arising from the misspecification of the nuisance functions (Mackey et al. 2018]] Morzywolek et al., 2023). (ii) The DR-Iearner further doubly robust: it yields consistent esti- mation even when either the outcome or the propensity model is not correctly specified. (iii) The
CATE estimation:
R-learnrr less sensitive to overlap violations due to its overlap weighting of the loss
In this paper, we focus on DP for Heyman--rrhogonm meta-learners. We derive framework for the Relearner in the main paper and provide an extension to the DR-learner in Supplement B
1 'The source code is available at our GitHub repository. Informally quasi-oracle efficiency means that the target model is learned almost equally well using either the estimated nuisance functions or the ground truth.
2

---

Published as a conference paper at ICLR 2025
Differential privacy: DP ensures that the release of aggregated results does not reveal information about individual data samples, typically with strict theoretical results (Dwork) 2006: Dwork & Lei 2009). As result, DP has been employed in various fields of machine learning (e.g.. Abadi et al. 2016 Bassily et al. 2014; Wang et al., 2019), but typically outside of CATE estimation. We discuss different strategies how DP can be achieved in Supplement|A] For example, one strategy is output perturbation, where one adds calibrated random noise to the non-private model prediction prior to its release in order to ensure DP (e.g., Chaudhuri et al. 2011 Zhang et al. 2022
In our setting, we later adopt output perturbation to CATE estimation. Output perturbation has two clear advantages in our task: (i) it can be applied to any ML model after training, which naturally fits the idea of meta-learnrrs from above as model-agnostic approaches; and (ii) it leaves the original objective and the data unchanged. The latter crucial because changes to the input or the objective (i.e., the estimand) arguably could violate causal assumptions and lead to biased results.
DP in treatment effect estimation: The ex- isting literature DP methods for treatment effect estimation overview in Fig. 2 where we group prior works by the underlying estimand: Average trear- ment effect (ATE). Many works focus pri- vately estimating the ATE (e.g.. Javanmard eet al. 2024 Lee et al. 2019 Yao et al.) 2024). Yet, the ATE is a much simpler causal quan- tity than the CATE, It makes population-wide estimates and, thus, unlike the CATE, does not allow make individualized predictions about treatment effects. Conditional average treatment effect (CATE). The few works aimed at DP for CATE estimation have clear limitations (Fig. 2 they (i) are either restricted to interventional data from RCTs and thus not applicable to observational data (Betlei et al. [021]; (ii) are only applicable to binary outcomes (Guha & Reiter 2024); or (iii) require special private base learners and are thus not model-agnostic (Niu al. (022). In particular, the latter work (Niu et al. 2022) is restricted to explainable boosting machines. Therefore, it is not applicable to other models such as neural networks. In sum, none of the above works provide a method for CATE estimation under DP where both observational data and different types of ML models can be used.
ATE
cg. Javanmard (2024), ct (2019)
sparse. We provide an
References
Observational
Model/ data agnostic"
Bettel (2017)
CATE
Guha & Heiler (2024)
X
Nu (2019)
X
Ours
"Model agnostic methods applied ML modele, Data
rostrictions the dala (o.g.. binary outoomes).
Figure 2: Comparison of relevant literature.
So far, a DP framework for CATE estimation from observational data with meta- learners is missing. We are thus the first to propose a Neyman-orthogonal framework for CATE estimation that fulfills DP.
Research gap:
3 PROBLEM FORMULATION
Notation: We write random variables as capital letters X with realizations x. We denote the prob- ability distribution over X by Px. where we omit the subscript whenever it is apparent from the context. We denote the probability mass function / density function by P(xx) = P(X = x). We rely on the potential outcomes framework (Rubin 2005) and denote the outcome under intervention by (a). Finally, ||0||a- = v + + 2 is an l/ norm for z E 2d: |[],,,, = (E (2) is an La norm; S b means there exists C' 2 0 such that S C- b; and X,, E or((r,) means X,,r,, & 0.
Setting: We consider a dataset D : {(X,, AG, Y2) )=1...... 2n. consisting of observed confounders X in bounded domain X' C RR, binary treatment A E 1]. J. and bounded discrete or continuous
outcome Y E ), where Z; := (X,, Y;) ~ P ii.dd., Z, E Z. Let π (Tx) == P(A = 1 X = x) define the propensity score and μa a) == E|y X xr, A a) the outcome function.
Estimand: Our objective is to estimate the conditional average treatment effect (CATE) r(x) for individuals with covariates X x. We make the standard assumptions for causal treatment effect estimation: positivity, consistency, and unconfoundedness (e.g.. Curth & van der Schaar 2021 Then, the CATE is identifiable as
Rubin 2005
(2) = (1) Y()) x= x| p(x 1) ul 0).
(1)
We give more details on the standard causal assumptions and CATE estimation in Supplement A.4
3

---

Published as a conference paper at ICLR 2025
In our work, we aim to estimate (x) by (i) using -othhoo-tt meta-leamers (Sec. 3.1 while
(ii) ensuring differential privacy (Sec. 3.2 which we briefly review in the following.
3.1 NEYMAN-ORTTIOGONAL META-LEARNERS FOR CATE ESTIMATION
To estimate the CATE from Eq. ①. the common approach is to regress the difference between the potential outcomes (i.e., (1) (0)) on the covariates X. Thus, one considers the population risk
for a working model g E G: X + E via
(2) where 7 (J, are nuisance functions, X(-) > a weight function, and A(g) is regularization term. We denote population minimizer of Rp (g.gg. g.n))(TT) with g g²(:)) =
Rp ((T)) = (X)(+(X)) (()(X, ( - + Agg.
arg mingeg (p(g. 77, {(=)) (Hirano et al. 2003 Morzywolek et al., 2023). However, Rp cannot be directly estimated and subsequently minimized given the data D. as depends on the unknown nuisance functions and μr. One could employ the estimated nuisance functions (i.e., TD and nD))
here, but then, their estimation errors propagate into the minimization of Rp.
A popular approach to circumvent the above problem is to use Neyman-orthogome Formally, such meta-lcarners operate in two stages (e.g.. Keenedy 2023b Nie & Wager 2020). Let the dataset D a disjoint union of the two subsets and D size nI, i.e., D = D U D. In the first stage, the meta-lcarners estimate nuisance functioon î = (f PD) on dataset D, and, in the second stage, minimize the adapted Neyman-orthooonad population risk function
meta-learnent
Rp(g. n, {(=)) = E [p( (X)) (o(,,, (6T2) ((*(X))) - (X)) + A(g)
(3)
with
p(a, X(T(z))
(x)) + ((T())) and
(4)
a-
(())
p(a, (2)) (1- xxx) (u ((x, a)) p(x, 1) p1z 0)
(5)
on dataset D, where î is used in place of 1) (Mozyywllek et al. 2023). The use of this loss function allows for quasi-oracle efficiency of the final estimator. the tollowing, we denote the estimated population risk Rp via loss RD that is dependent on the data D, namely
Rolg th ((T) - E p(A, x P(A,
(0(Z,, (1)) M. - Mgg-
(6)
Later, we use the Relerrner (Nee Wager 2020), which is given by (B( (1- due to its theoretical advantages (e.g., Neymmn-orthogonality and oracle-efficiency) Importantly, Neyman-orthogonn
meta-learners such as the R-learner achieve state-of-the-an performance, and the orthogonality property makes the models less sensitive to the misspecification of the nuisance functions (e.g.. Curth & van der Schaar 2021 Kennedy 2023b Melnychuk et al. 2025).
3.2 DIFFERENTIAL PRIVACY
Differential privacy (DP) ensures that the inclusion exclusion of data from any individual does not significantly affect the estimated outcome (Dwork) 2006 Dwork & Leil 2009). Specifically, for a given privacy budget e, DP ensures that the probability density of any outcome y on dataset D E 2" is e-indistinguishable from the probability density of the same outcome y stemming from neighboring dataset D' E 2" with probability of at least 1 5. 5. The datasets D and D are called neighbors, denoted as D ~ D', if their Hamming distance equals one, i.e., du(DD, (D, D') 1.
Definition 1 (Differential privacy (Dwork & Lei 2009)). A function fD(x) 2d - TR.d trained on dataset D is (e, d)-differentially private if. for all neighboring datasets D, D' nd and all measurable S C Rd, it holds that
((fn(x) E S) exp(e) P( S for all all x E x
One common strategy to ensure DP is output perturbation (Chaudhuri et al., 2011 Zhang et al.) 2022), which we later tailor to CATE estimation as part of our framework. Intuitively, one perturbs the prediction in way that the predictions resulting from two neighboring databases cannot be differentiated. It has been shown (e.g., Dwork & Roth 2014) that adding appropriately calibrated zero-centered noise (e.g., Gaussian noise) to the prediction is sufficient to ensure differential privacy for traditional, supervised machine learning tasks (but not for CATE estimation, as we discuss later). This is stated in the following Gaussian noise privacy mechanism.
4

---

Published as a conference paper at ICLR 2025
Definition 2 (Gaussian noise privacy mechanism (wwork & Roth 2014 Let fD(x): x + R
be function on dataset D with l2-sensitivity A2(f) sup D-D xexx lxf - fn' (x)ll2 and
U NN for 2 (1.25/8)) (2) Then, the output perturbation mechanism returns M (x)(()) = fx()) U that preserves (e, 8) -diffeerrltial privacy.
Definition 2 describes how to ensure DP for a given prediction. However, this requires estimating the training sensitivity (2( of the employed model f. which, for general function classes such as neural networks, is infeasible. Hence, this motivates our custom framework later.
3.3 PROBLEM STATEMENT
In our work, we aim at Jeyman-orthogonn CATE estimation under differential privacy. Specifically, we aim to derive an (s, -ddifferrential private version of (n(::)) = arg minges RD(g, A(T) of the form
9DP(X: n) (x(xx)) + 0...0, U,
(8)
where (x; function. Importantly, we consider arbitrary working model classes G.
..0(....... U ~ NN Ia), and where r(-) is calibration
Our problem statement and thus our framework - is intentionally flexible. (I) We assume that the propensity score not known and, instead, estimated from observational data. (2) focus on deyman-orthogonal meta-learners because these model-agnostic and can thus be seamlessly instantiated with any machine learning model, including neural networks.
(3) Our derivations are general and, therefore, apply any orthogonal loss of the form in Eq. (3) Below, we present our DP-CATE framework for the R-learner due to its state-of-the-arr performance. We additionally provide extension of our framework for the DR-learner in Supplement
The above task is highly non-trivial as the identification of CATE relies on the standard causal as- sumptions of positivity, consistency, and unconfoundedness [Rubin| 2005). Yet, privacy mechanisms perturb different parts of the data (or the model), which could arguably violate the causal assump- tions (in the case of the input perturbation) or lead to biased estimates (in the case of the model perturbation). Thus, instead of blindly introducing noise to the estimation setup to guarantee DP, we must cautiously calibrate the noise in targeted setup to retain the consistency and quasi-oracle efficiency of the privatized CATE estimators.
OUR FRAMEWORK: DP-CATE
Overview: We now present our DP-CATE framework aimed at CATE estimation for Neyman- orthogonal meta-learners under (e, didiffeerett privacy. To address the challenges from above, we employ output perturbation, which is highly suitable to our purpose for two reasons. First, it allows us ensure that causal assumptions are fulfilled even after perturbation. Second, we retain the favorable Neyman- orthogonality Neyman-orthogonalll
properties of the existing CATE estimation methods.
Use cases: Our framework comes in two variants, relevant for different use cases in medical practice:
① DP-CATE for finite queries (-> Sec. 4.1):
We aim to report a number of CATE estimates (e.g., treatment effects across different age groups). = How do solve this? We draw upon the shared property of robustness and privacy of ML models in terms of insensitivity to outliers and small measurement errors (Dwork & Lei 2009). We then propose calibrating the noise added with a function r(-) that depends on the meta-leamner's second-star stage influence function. Importantly, our model-agnostic approach preserves the quasi-oracle guarantees of the non- private model stemming from the Neyman Neyman-orthogee orthogonal loss function.
2 DP-CATE for functional queries (-> Sec. : We release an estimate åDP of the complete CATE function which can then be queried arbitrarily often (e.g, as in clinical decision support systems). Existing output perturbation mechanisms only apply to scalar or finiteddimensional vector-valued outputs. Therefore, the above is only valid if the overall number of queries made to the algorithm both finite and known before the perturbation. = How do we solve this? We derive output perturbation method based on Gaussian processes that is valid for all functional CATE estimates solving Eq. (3), as long as the estimation in the second stage is performed through Gaussian kernel regression.
5

---

Published as a conference paper at ICLR 2025
4.1 DP-CATE FOR FIXED NUMBER OF QUERIES
Here, total number of d CATE estimates should be released. The number of queries, d, to the CATE function is known a priori. For notational simplicity, we thus rewrite the d separate CATE estimates as d-dimensional vector. We employ bold letters in the following to emphasize that we are interested a vectorized version the CATE meta-learner output (n( 77), i.e., ()(())(()) 7) E Rd.
We now derive calibration function r(-) that is applicable to any Neyman-orthogonad CATE meta- learner. We employ r(-) to calibrate a noise vector with respect to the privacy budget (s, 6) and the model sensitivity. Finally, we perturb (D(x;)) to fulfill DP through (DP(x; = ÅD (x:1)) + r(e, 6, 9D. 77 U.
For this, we borrow ideas from the literature that observed similarities between differential privacy and robust statistics Avella-
2021 Dwork & Lei 2009) but which we carefully tailor to setting in the following. Our idea is to employ the so-called influence function (IF) of the second- stage CATE model to calibrate the noise. The IF allows us to quantify how a single observation in D influences CATE estimation and the model output. Intuitively, the IF describes the effect of an infinitesimally small perturbation of the input on the model output.
Definition Let T' be a functional of distribution that defines the parameter of interest, T T(P) E Rd. The gross-error sensitivity of at z under P is given by the supremum of the l2 norm of the influence function, i.e.,
YT, = sup IF P)l2,,
ZEZ
where -
is an influence function (IF) of T at under distribution with the density/probabiiiiyy mass function P, and 8g denotes the Dirac-delta function.
- t)P ( t8=)
Next, we derive the IF of the second-stage models for CATE meta-learners. Observe that T(P) depends the data distribution directly through P and indirectly through the first-stage estimation. Specifically, now T(P) = 9" (xx: and T(D) ID (x; 7)) the nuisance functions were known; and T(P) 9" (x; MD)) and T(D) (x: nT)) if the nuisance functions estimated.
We now state our main theorem for differentially private CATE estimation with a known number d of queries. The intuition behind constructing the calibration function r(-) builds upon result in Nissim et al. (2077) in which (e, -diffferrnnt privacy is achieved through calibrating noise with respect the smooth sensitivity of the prediction model in comparison the commonly employed global sensitivity from Definition 2 However, calculating the smooth sensitivity is still difficult or even infeasible for general function classes. Nevertheless, we show that the smooth sensitivity of the second-stage model can be upper bounded by the gross-error sensitivity ? of the second-stage regression
Theorem 1 (DP-CATE for finite queries). Let z := (a,x,y) define a data sample following the joint distribution Z and fiPP = (ADr, DD) the nuisance functions estimated on dataset in a (e/2, 8/2))diffferentilll private manner of choice 5 Furthermore, lei D be the training dataset for the second-stage model with D = n. For = (a, [G, E Z. we define
ÿDP (x; (DDD) = _x; ) + 7 (T. c(E,6,n).U
T(P) g6 (x; f)DP),
(10)
<(.5..n..0D))
T(T,P) sup |(gg.. x, (pr) pla, Apr (x) for, ((FDr(x))) g" (a; (z: (orr)) =E2
(11)
where ((T,D) is sample gross-error sensitivity with T(D) instead of T(P) in Eq. 19.
U N(), c(E,6,n) == 2m/(n(()) In (2/8)/(en). and where hig", xx 2, 1DP) Rd and (" (Dr) depend the machine learning model employed for the second-stage regression. Then, 0Dr(x; (x; 1 is (e, -)ddifffrettially private.
Proof. We prove Theorem I in Supplement F To do so, we first state the IF of the second-stage model that minimizes weighted loss. Then, we calculate the gross-error sensitivity to show that the sample-size-ue sieigh-e sensitivity upper-bounds the smooth sensitivity of the respective learner. D
*See Lemma 5 in Supplement
"A need to query the nuisance functions multiple times, the privatization method needs to be suitable for functions, such as gradient perturbation through DP-SGD (Ahadi al. 2016).

---

Published as a conference paper at ICLR 2025
In the case of the R-learner, Theorem yields the following gross sensitivity 7 (T, P): + App(z, 1) (op(a, -g (2: fioe)) f |1 (12)
(DP( u)
7 (T. P) sup =e2
his" x, fDP) (a #
(D)(a)
pla.ápp)
ste.9pp.X(ppp))
In the following, we present two corollaries stating the form of the function higg, ,x, 2, frD) for parametric models such neural networks, well the non-parametric kernel ridge regression estimator.
Corollary (Parametric second-stage regression). If the second-siage regression is smooth para- metric model, namely 9 {g(2; 0): X' + IR, E G RP), then, in Theorem |
we have
9" (pD) = g(:00) and h(g" x, x, (DD) Ja (9 (x; iprr]] H Volg (z; fppr)] (13)
where H" arg min Ry(g, fpr, {) Jo Raxe is Jacobian matrix wrt. 0: Ho AEB
V Rp(g", fIP A(fDP))) E RexP is Hessian matrix; and V is a graddient.
Proof. We prove Corollary in Supplement
note on applicability: Corollary requires access and invertibility of Hg. This might restrict the applicability of certain deep neural networks for the second-stage stane regression. However, we note that, in Theorem[] [ we are only interested in predicting the CATE certain values x. This motivates the local CATE estimation through kernel weighting. We present our alternative approach below.
Corollary (Non-parametric second-stage regression). If the second-stage regression is kernel ridge regression with Agg) = Allg where G = H is reproducing kernel Hilbert space (RKHS) induced by a kernel K (:,-) : :''s x X Rt, then, in Theorem ! we have
())) = (Lp + - and hig", x, x, ADP) = (Ip + AI) (x),,
(14)
where Lp : H + H is weighted covariance operator L-f(()) = E[p(A, (X)) K(; (xx)) ATf(.) 2f()) is scaling operator; S E H is cross-covariance functional S())
(
E (X)) K(,, X) AD, (()DP(X)))]
Proof. We prove Corollary 2 in Supplement F
Scalability: The complexity of calculating r(-) is independent of the size of the dataset once the second-stage model ( r)rr) is fitted.
Theorem (Neyman-orthogomee orthogonality and quasi-oracle efficiency of DP-CATE). The privatization of the second-stage model asymptotically preserves the property of Neyman-orthogonaing namely
llg" (=)) - (D II 2 Rp (g (: fDp),, fp,,, A(FDF) - RR (sn)) hie,, A(+DP)) + R2(fpr. m)
+ l1g (:: fIDr) - 9o(:; ) Ml + op(n
(15)
dependr the model class G
cuiport perturbution
Furthermore, under additional regularity conditions on the privatization of the nuisance functions (e.g. gradient perturbation), our CP-CATE achieves quasi-oracle efficiency. Specifically, if the orig- inal estimation of the nuisance functions is at rate of at least op(n -1/4), then the privatized estima- tion preserves this rate.
Proof. We prove Theorem 2 in Supplement F
4.2 DP-CATE FOR COMPLETE CATE FUNCTIONS
In this variant of DP-CATE, we seek to privately release an estimate (n(+))) of the complete CATE function 7(-). Note that cannot leverage Theorem i because is only applicable to CATE estimates but not to complete functions. Instead, we must now derive a tailored approach.
Intuitively, we need to find (I) type of noise and (II) calibration function that does not depend on d. More precisely, the added noise should be a function itself to guarantee the privacy of the CATE function. For (I), we propose to add a calibrated Gaussian process (GP) the predicted CATE,
We refer Rasmussen & Williams 2006) for in-depth introduction Gaussian processes.

---

Published as a conference paper at ICLR 2025
Definition 4 (Gaussian process). A family of random variables {Xeher is Gaussian process if for any subset E T, {X₂ hes has a Gaussian distribution. The process is entirely determined by its mean function m(t) == [[] and covariance function (ss,t) == Cov(X,,X.))
We are left with answering question (II) from above: how calibrate the GP noise fulfill DP? We make the following important observation: If 0D( 77) lies in a reproducing kernel Hilbert space (RKHS), we can calibrate the GP noise with respect the RKHS norm using results from functional analysis Hall et al. 2013). To ensure that ( 17) indeed lies in an RKHS, we can later follow prior research (e.g.. Kennedy 2023a Singh al.] (024) and model the second-stage estimation in DP- CATE framework as Gaussian kernel regression.
We now want to bound the difference of CATE functions trained on neighboring datasets with re- spect to the norm of the Hilbert space. Recall that differential privacy of function I under the Gaussian mechanism in Def. requires knowledge about aboutsuppae nup xexx [1fb(x) fn (x)ll2 to cal- ibrate the Gaussian noise variable. Similarly, we now require knowledge of p-p IlfD - f l|x where fD specifies RKHS regression to calibrate the Gaussian process. However, to the best of our knowledge, no closed- form solution for this quantity exists. We thus derive the following lemma extension of Hall et al. (2013) in setting.
Lemma 1. Let H denote the RKHS induced by the Gaussian kernel ((x,x') (V2th) -9 exp(-Ilz= x' =2'11/(222) for x, x' the RKHS regression
X S R2, and let fo be the optimal solution to
1 Σ (X.)) ((f(X). A1ffl
fo(.) arg min JEH n
(16)
where w(-)) > O is a weight function, D is a dataset with |D| n, and {ii, y) is a convex and Lipschitz loss function in I with Lipschitz constant L. Then, for D ~ D', we have
Ilfo - 1 s suplw(x)] L
An (V(2T)a) -9
(17)
#Ex
Proof. We prove Lemma] in Supplement
We now use the above results and present our DP framework for CATE functions: (i) Stage I: We estimate the (s/2, 8/2 /iifeliffrr private nuisance functions ÅDP and Hp through any paramet- ric or non-parametric machine learning method and perturbation method. (ii) Stage 2: We perform a Gaussian kernel regression to minimize Eq. (3). (iii) We calibrate suitably chosen Gaussian pro- cess based on Lemma [ and add the resulting GP to the CATE function. In the case is a squared loss, the quasi-oracle efficiency of our framework directly follows from Theorem 2 and (Foster & Syrgkanis 2019). We present the pseudo-code for DP-CATE in Supplement C
Theorem (DP-CATE for functional queries). Let (DP and pr denote the -2))///)iiferrentalll private nuisance estimators trained in stage 1 on D. Let and
(a, v. be a data sample from X R1. Let H denote the RKHS induced by the kernel x' /13/2h22 and let f(:,) .) be convex and Lipschitz loss function with Lipxchitz constant L. We define gD(: (TD) the second-stage regression solving Eq. (3) via
dataset D with |D|
(<x,x²)= 2xth)= -9
(ecphi--exppppppepee exp(-
1
(D(:tfpP) arg min = gEH n i=1
Σ
(DA, (X.)) (x.))))((x))))).....))))) (()( Tpp. ((xx + Alll|l (18)
Furthermore, let U.-) E H be the sample path of a zero-centered Gaussian process with covariance function K(x, ).). Then, (e, diffferrenti privacy is guaranteed by
4L 4L (2/8)) -U(-)
TD TDP) := (=
sup (0.
[p(a, (x))
(19)
(V2th) "nn
r(e,6.gn.fin))
Proof. We prove Theorem 3 in Supplement
8

---

Published as a conference paper at ICLR 2025
Importantly, for both R- and DR-learner, ((.(..z))(..1)xx [p(a, ADp (z)) < 1. Also, the above theorem requires convex Lipschitz loss ((:--)- There many suitable loss functions (e.g., the squared loss on bounded domains, a trimmed squared loss, or the Huber loss). For many losses, the Lipschitz constant is data-independent and directly computable from the loss function ? We further require the second-stage in the meta- meta-learrer to be Gaussian kernel regression, which is widely used in causal inference (e.g., Kennedy 2023a Singh et al. 2024). Nevertheless, our DP-CATE still fairly flexible in that any Neyman orthogonall meta- meta-learner can be used (e.g., R-learner DR-learner) and that any machine learning model can be used for nuisance estimation. In Supplement C
we
present an algorithm for releasing private outputs of the function ÿDP (s3 (IDP).
5 EXPERIMENTS
Implementation: Our DP-CATE is model-agnostic and highly flexible. Therefore, we instantiate our DP-CATE with multiple versions of the R-leaner (Nie et al. 202T) where we vary the underlying base learners. Hence, we implement the pseudo-outcome regression in the second stage via both a neural network (NN) and the Kernel regression estimator (KR). We estimate the nuisance func- tions through neural networks. This is recommended as one typically allows for flexibility in the nuisance functions (Curth & van der Schaar 202I). Details on implementation and training are in Supplement G We emphasize again that there no suitable baselines for task.
As explained in Sec. [ there are no flexible CATE meta-leamers ensuring DP. Thus, there is no suitable baseline for our task. Hence, we perform experiments to primarily show the applicability of our DP-CATE across different privacy budgets. We expect that the pre- diction performance will increase with increasing privacy budget and then approach the prediction performance of a non-private CATE learner. We measure the performance via the precision in esti- mation of heterogeneous effects (PEHE) with regard the true CATE (e.g., 2011).
Performance metrics:
5.1 EVALUATION ON SYNTHETIC DATASETS
Synthetic datasets: Due to the fundamen- tal problem of causal inference, counterfactual outcomes are never observed in real-world data. Thus, follow common practice and evaluate DP-CATE on synthetic data, which allows us to access the ground-truth CATE and compute the
PEHE (c.g., Kennedy 2023a Oprescu et al.
2019 We evaluate OP-CATE on 300 queries.
Figure 3: Dataset 1 (finite queries). Predictions under different base learners and privacy budgets.
We consider two settings with different treat- ment effect complexities following Oprescu et al.] (2019). Dataset 1 contains two observed con-
founders from which only one influences the CATE. This allows us to visualize the CATE function and the effect of privatization on the prediction for varying covariate values. Dataset 2 contains 30 observed confounders and multiple dimensions influence the CATE. Details are in Supplement G
Results for finite queries: Dataset 1: Fig. 3
burket +
bugeet BB
budget
shows the predictions for different base learn-
ers and different privacy budgets. We make the following observations: (1) Our DP-CATE performs as expected: with increasing privacy budget, the predictions become less 'noisy" and converge to those of the non-private estimator. (2) Our CP-CATE shows consistent patterns for different base learners. For example, the predictions under both KR and NN are almost identical, showing the flexibility and robustness of our framework. Dataset Fig. shows the PEHE Note that we directly compare DP-CATE for finite queries (combined with KR and NN) and DP-CATE for functional queries. Again, we find that our DP-CATE performs as expected: the PEHE decreases with increasing privacy budget and converges towards that of the non-private learner.
0.5 Covarate
0.
0.5
0.0
85
14
Omci
Morpphiat
OP-CATE
Figure 4: Dataset 1 (functional queries). Predic- tions under different privacy budgets.
For example, for the 1 loss, the Lipschitz constant T. equals I; for the Huber loss, L equals the loss parameter 8; and, for the truncated lg loss, the constant equals the gradient the truncation value.

---

Published as a conference paper at ICLR 2025
Results for functional queries: e Dataset 1: Fig. shows the predictions of DP-CATE for fune- tional queries actoss different privacy budgets. We observe similar behavior as in the case of finite queries: With increasing privacy budget, the predictions converge to those of the non learner. Dataset 2: Fig. shows the results for more complex dataset. As before, we observe that DP-CATE for functions behaves in the same way as DP-CATE for finite queries. Overall, our findings are robust across datasets and base learner specifications.
NN
Fundion
7.5
0
5.2 EVALUATION ON MEDICAL DATASETS
10
5.0
0.0310
Medical datasets: We demonstrate the appli- cability of DP-CATE medical datasets by using the MIMIC-III dataset (Johnson et al. 2016) and the TCGA dataset (Weinstein et al. 2013). MIMIC-III contains real-world health records from patients admitted intensive care units at large hospitals. We aim to predict patient's red blood cell count after being treated with mechanical ventilation. The Can- cer Genome Atlas (TCGA) dataset contains a large collection of gene expression data from patients with different cancer types. We assign treatment indicator based on the gene expression level and aim to predict a constant effect across all expression levels. Details are in Supplement G
0.905
60
0,0300
0.1
0.
0.1
Privacy budge:
Privacy budget
Privacy budget
Non-phaae
CIN-CATEE (ours)
Figure 5: Dataset 2. Prediction emrors under dif- ferent privacy budgets of DP-CATE (finite) on KR and NN and DP-CATE (functional) over 10 runs. Plots centered at the PEHE of the original learner.
Results: MIMIC-III: Fig. Dreports the predictions of the CATE against different levels of hema- tocrit and different privacy budgets. Here, we have = 1312 queries (i.e., the size of the test set). Our DP-CATE framework works as desired: for smaller privacy budgets, more noise should be added, which is also reflected in a larger variation of the predictions. TCGA: Fig. shows again that our DP-CATE is effective on large number of queries (i.e., d 2659). Overall, the loss in precision due to DP (i.e., when comparing DP-CATE to the non-private learner) is fairly small.
budget
budget
budget
budget
budget 0.1
budget
1.7
CATE
10
CATE
-10
40
30 Hematocrtt
30 40
10 Covariate
20
30
10
30
10 Covariate
Hematocri
Hematoork
Covariate
Non-private
DP CATE (eurs)
Non Noiverivate
CA-C leurs)
Figure 6: MIMIC-III (finite queries). Our DP- CATE generates private estimates of the effect
Figure 7: TCGA (finite queries). DP-CATE consistently estimates the constant treatment effect across the sum of all covariate values.
of ventilation for different levels of hematocrit. Takeaways: The prediction error of DP-CATE decreases with larger privacy budgets desired and converges to the non-private error, confirming that our framework makes precise CATE predictions.
6 DISCUSSION
Applicability: We provide a general framework for differentially private and Neyman-ortthgooaal CATE estimation from observational data. First, our DP-CATE is carefully designed for observa- tional data, which are common in medical applications (Feuerriegel et al., 2024). Second, DP-CATE applicable to various meta-learners (e.g.. R-learner, DR-lcarner), which are widely used in prac- tice. Third, DP-CATE allows different use cases: one can release either a certain number of CATE estimates or even the complete CATE function (e.g., as in clinical decision support systems).
Extension to the DR-learner: Our derivations focus on the popular R-learnr due to its favorable theoretical properties. Nevertheless, our DP-CATE can be applied to any other deyman-orthogome meta-learner. In Supplement B we thus provide an extension to the DR-learner. Therein, we also provide additional numerical experiments. At a technical level, the weight function for the DR- learner simplifies to X (xx()) = 1 in contrast the weight function of the R-learner X (x)))) T (2) (1 - π (x)). However, ABR (T<()) is less suitable for DP because it needs a larger noise term during the perturbation (see Supplement B for detailed discussion), which hinders downstream performance. Hence, we recommend employing DP-CATE with the R-learner in practice.
Conclusion:
Ensuring the privacy of sensitive information in treatment effect estimation is man- dated for ethical and legal reasons. Here, we provide the first framework for differentially private CATE estimation from observational data using meta-learners.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
We thank Dennis Frauen and Lars van der Laan for their helpful feedback on our manuscript. Our research was supported by the DAAD program Konrad Zuse Schools of Excellence in Artificial Intelligence, sponsored by the Federal Ministry Education and Research.
REFERENCES
Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Iyya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. Conference on Computer and Communica- fions Security, 2016.
Anish Agarwal and Rahul Singh. Causal inference with corrupted data: Measurement error, missing values, discretization, and differential privacy. arXiv preprint, 2107.02780 2024.
Marco Avella-Medina. Frivacy-rreserving parametric inference: A case for robust statistics. Journal of the American Statistical Association, 116(534):969999)) 983, 2021.
Anna Baiardi and Andrea A. Naghi. The value added of machine learning to causal inference: evidence from revisited studies. The Econometrics Journal, 27(2):213-234, 2024.
Karla V. Ballmann. Biomarker: Predictive prognostic? Journal of Clinical Oncology, 33(33): 3968-3971, 2015.
Raef Bassily, Adam Smith, and Abhradeep Thakurta. Private empirical risk minimization: Efficient algorithms and tight error bounds. In IEEE Annual Symposium on Foundations of Computer Science, 2014.
Alain Berlinet and Christine Thomas-Agnan. Reproducing Kernel Hilbert Spaces in Probability and Statistics. Springer, New York, NY, 2004.
Artem Betlei, Théophane Gregoir, Thibaud Rahier, Alois Bissuel, Eustache Diemert, and Massih- Reza Amini. Differentially private individual treatment effect estimation from aggregated data. hal, 2021.
Michael Braun and Eric M. Schwartz. Where A-B testing goes wrong: How divergent delivery affects what online experiments cannot (and can) tell you about how customers respond to adver- tising. arXiv preprint, 2024.
Kyle B. Brothers and Mark A. Rothstein. Ethical, legal and social implications of incorporating personalized medicine into healthcare. Personalized Medicine, 12(1):43-51, 2015.
Kamalika Chaudhuri, Claire Monteleoni, and Anand D. Sarwate. Differentially private empirical risk minimization. Journal of Machine Learning Research, 12:109911009 2011.
Xiangyi Chen, Zhiwei Steven Wu, and Mingyi Hong. Understanding gradient clipping in pri-
vate sgd: geometric perspective. Conference on Neural Information Processing Systems (NeurIPS), 2020.
Alicia Curth and Mihaela van der Schaar. Nonparametric estimation of heterogeneous treatment effects: From theory to learning algorithms. In Conference on Artificial Intelligence and Statisties (AISTATS), 2021.
Cynthia Dwork. Differential privacy. International Colloquium on Automata, Languages, and Programming. 2006.
Cynthia Dwork and Jing Lei. Differential privacy and robust statisties. In ACM Symposium on Theory of Computing, 2009.
Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Foundations
and Trends in Theoretical Computer Science, 9(344)2111440 2014.
11

---

Published as a conference paper at ICLR 2025
Paul B. Ellickson, Wreetabrata Kar, and James C. Reeder, III. Estimating marketing component effects: Double machine learning from targeted digital promotions. Marketing Science, 42(4): 704-728, 2023.
Stefan Feuerriegel, Dennis Frauen, Valentyn Melnychuk, Jonas Schweisthal, Konstantin Hess, Ali- cia Curth, Stefan Bauer, Niki Kilbertus, Isaac S. Kohane, and Mihaela van der Schaar. Causal machine learning for predicting treatment outcomes. Nature Medicine, 30(4):958-968, 2024.
Dylan J. Foster and Vasilis Syrgkanis. Orthogonal statistical learning, arXiv preprint, 1901.09036v4, 2019.
Dennis Frauen, Valentyn Melnychuk, and Stefan Feuerriegel. Fair off-policy learning from obser- vational data. In International Conference on Machine Learning (ICML), 2024.
Dennis Frauen, Konstantin Hess, and Stefan Feuerriegel. Model-agnostic meta-leamers for es- timating heterogeneous treatment effects over time. In International Conference on Learning Representations (ICLR), 2025.
Kazuto Fukuchi, Quang Khai Tran, and Jun Sakuma. Differentially private empirical risk minimiza- tion with input perturbation. In Discovery Science, 2017.
Sharmistha Guha and Jerome P. Reiter. Differentially private estimation of weighted average treat- ment effects for binary outcomes. arXiv preprint, 14766, 2024.
Rob Hall, Alessandro Rinaldo, and Larry Wasserman. Differential privacy for functions and func- tional data. Journal of Machine Learning Research, 14:703-727, 2013.
Jennifer L. Hill. Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphical Statistics, 20(1):217-240, 2011.
Keisuke Hirano, Guido W. Imbens, and Geert Ridder. Efficient estimation of average treatment
effects using the estimated propensity score. Econometrica, 71(4):1161-1189 2003.
Ta-Wei Huang and Eva Ascara. Debiasing treatment effect estimation for privacy data: A model auditing and calibration approach. SSRN, 2023.
Roger Iyengar, Joseph P. Near, Dawn Song, Om Thakkar, Abhradeep Thakurta, and Lun Wang. Towards practical differentially private convex optimization. In IEEE Symposium on Security and Privacy, 2019.
Adel Javanmard, Vahab Mirrokni, and Jean Pouget-Abadie. Causal inference with differentially private (clustered) outcomes. arXiv preprint. 2308.00957, 2024.
Andrew Jesson, Sören Mindermann, Uri Shalit, and Yarin Gal. Identifying causal-effect inference failure with uncertainty-awart models. In Conference on Neural Information Processing Systems (NeurIPS), 2020.
Alistair E. W. Johnson, Tom J. Pollard, Lu Shen, Li-Wei Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G. Mark. MIMIC-III, a freely accessible critical care database. Scientific Data, 3:050,35, 2016.
Edward H. Kennedy. Semiparametrie doubly robust targeted double machine learning: a review. arXiv preprint, 2203.06469, 2023a.
Edward H. Kennedy. Towards optimal doubly robust estimation of heterogeneous causal effects. Electronic Journal of Statistics, (122)3008-3049 2023b.
Daniel Kifer, Adam Smith, and Thakurta Abhradeep. Private convex empirical risk minimization and dimensionl regression. In Conference Learning Theory (COLT), 2012.
Si Kai Lee, Luigi Gresele, Mijung Park, and Krikamol Muandet. presery- causal inference via inverse probability weighting. arXiv preprint, 1905.12592. 2019.
Yuchen Ma, Dennis Frauen, Valentyn Melnychuk, and Stefan Feuerriegel. Counterfactual fairness for predictions using generative adversarial networks. arXiv preprint, 2310. 17687v1, 2023.
12

---

Published as a conference paper at ICLR 2025
Lester Mackey, Vasilis Syrgkanis, and Ilias Zadik. Orthogonal machine learning: Power and limita- tions. In International Conference on Machine Learning (ICML), 2018.
James Martens. Deep learning via hessian-frre optimization. In International Conference on Ma- chine Learning (ICML), 2010.
Valentyn Melnychuk, Stefan Feuerriegel, and Mihacla der Schaar. Quantifying alcatoric uncer- tainty of the treatment effect: A novel orthogonal learner. In Conference on Neural Information Processing Systems (NeurIPS), 2024.
Valentyn Melnychuk, Dennis Frauen, Jonas Schweisthal, and Stefan Feuerriegel. Orthogonal repre- sentation learning for estimating causal quantities. arXiv preprint, 2502.04274, 2025.
Pawel Morzywolek, Johan Decruyenaere, and Stijn Vansteelandt. On weighted orthogonal learners for heterogeneous treatment effects. arXiv preprint, 2303. 2023.
Lizhen Nie, Mao Yc, Qiang Liu, and Dan Nicolac. VCNet and functional targeted regularization for learning causal effects continuous treatments. In International Conference on Learning Representations (ICLR), 2021.
Xinkun Nie and Stefan Wager. Quasi-oracle estimation of heterogeneous treatment effeets. Biometrika, 108(2):299-319, 2020.
Kobbi Nissim, Sofya Raskhodnikova, and Adam Smith. Smooth sensitivity and sampling private data analysis. In ACM Symposium Theory of Computing, 2007.
Fengshi Niu, Harsha Nori, Brian Quistorff, Rich Caruana, Donald Ngwe, and Aadharsh Kannan. Differentially private estimation of heterogeneous causal effects. Conference on Causal Learn- ing and Reasoning (CLeaR), 2022.
Yuki Ohnishi and Jordan Awan. Locally private causal inference for randomized experiments. arXiv preprint, 2301.00 1016166 2023.
Miruna Oprescu, Vasilis Syrgkanis, and Zhiwei Steven Wu. Orthogonal random forest for causal inference. In International Conference on Machine Learning (ICML), 2019.
Miruna Oprescu, Jacob Dorn, Marah Ghoummaid, Andrew Jesson, Nathan Kallus, and Uri Shalit. B-learner: Quasi-oracle bounds on heterogeneous causal effects under hidden confounding. In International Conference Machine Learning (ICML), 2023.
Gianluigi Pillonetto, Tianshi Chen, Alessandro Chiuso, Giuseppe de Nicolao, and Lennart Ljung. Regularized system identification; Learning dynamic models from data. Springer eBook Collee- tion. Springer, Cham, Switzerland, 2022.
Drago Plecko and Elias Bareinboim. Causal faimess analysis: A causal toolkit for fair machine learning. Foundations and Trends in Machine Learning, 17(3):004-58, 2024.
Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian process for machine learning. Adaptive computation and machine learning. The MIT Press, London, England, 3. print edition, 2006.
Rachel Redberg, Antti Koskela, and Yu-Xiang Wang. Improving the privacy and practicality of ob- jective perturbation for differentially private linear learners. In Conference on Neural Information Processing Systems (NeurIPS), 2023.
Donald. B. Rubin. Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association, 100(469):322 321-3331, 2005.
Maresa Schröder, Dennis Frauen, and Stefan Feuerriegel. Causal fairesss under unobserved con- founding: a neural sensitivity framework. In International Conference on Learning Representa- fions (ICLR), 2024a.
Maresa Schröder, Dennis Frauen, Jonas Schweisthal, Konstantin Heß, Valentyn Melnychuk, and Stefan Feuerriegel. Conformal prediction for causal effects of continuous treatments. arXiv preprint, 2024b.
13

---

Published as a conference paper at ICLR 2025
Jonas Schweisthal, Dennis Frauen, Mihaela van der Schaar, and Stefan Feuerriegel. Meta-leamers for ieiialll-identifieee treatment effects across multiple environments. In International Confer- on Machine Learning (ICML), 2024.
Rahul Singh, Liyuan Xu, and Arthur Gretton. Kernll methods for causal functions: dose, heteroge-
neous and incremental response curves. Biometrika, 111(2):477-516, 2024.
Cuong Tran, My H. Dinh, and Ferdinando Fioretto. Differentially empirical risk minimization under the fairness lens. In Conference on Neural Information Processing Systems (NeurIPS). 2021.
Mark van der Laan. Statistical inference for variable importance. The International Journal of Biostatistics, 2(1):2, 2006.
Kyle Wang, Michael J. Eblan, Allison M. Deal, Matthew Lipner, Timothy M. Zagar, Yue Wang. Panayiotis Mavroidis, Carrie B. Lee, Brian C. Jensen, Julian G. Rosenman, Mark A. Socinski, Thomas E. Stinchcombe, and Lawrence B. Marks. Cardiac toxicity after radiotherapy for stage III non-small-cell lung cancer: Pooled analysis of dose-escalation trials delivering 70 to 90 gy. Journal of Clinical Oncology, 35(13): 2017.
Puyu Wang, Yunwen Lei, Yiming Ying, and Ding-Xuae Xuan Zhou. Differentially private stochastic gradient descent with low-noisc. Neurocomputing, 585(C):125557, 2024.
Shirly Wang. Matthew B. A. McDermott, Geeticka Chauhan, Marzyeh Ghassemi, Michael C. Hughes, and Tristan Naumann. MIMIC-Extract. In Conference on Health, Inference, and Learn- ing (CHIIL), 2020.
Yue Wang, Daniel Kifer, and Jaewoo Lee. Differentially private confidence intervals for empirical risk minimization. Journal of Privacy and Confidentiality, 9(1, 2019.
John N. Weinstein, Eric A. Collisson, Gordon B. Mills, Kenna R. Mills Shaw, Brad A. Ozenberger, Kyle Ellrott, Ilya Shmulevich, Chris Sander, and Joshua M. Stuart. The cancer genome atlas pan-cancer analysis project. Nature Genetics, 45(10):1113-11333 2013.
Leon Yao, Paul Yiming Li, and Jiannan Lu. Privacy-preserving quantile treatment effect estimation for randomized controlled trials. arXiv preprint, 2401.14549, 2024.
Liang Zhang, Kiran Koshy Thekumparampil, Sewoong Oh, and Niao He. Bring your own algorithm for optimal differentially private stochastic minimax optimization. Conference on Neural In- formation Processing Systems (NeurIPS), 2022.
14

---

Published as a conference paper at ICLR 2025
A ADDITIONAL BACKGROUND MATERIAL
A.1 DIFFERENTIAL PRIVACY
Data
Objective
Model
Prediction
There are four main strategies of DP mechanisms (see Fig. 8) (i) Input perturbation independently randomizes each data sample before model training (e.g.. Fukuchi al. 2017). (ii) Objective perturbation adds random term to the objective and releases the respec- tive minimizer (e.g., [yengar et al.] 2019 Kifer al. 2012, Redberg al. 2023). The mechanisms in this field commonly make strong assumptions on the smoothness or convexity the objective. (iii) Gra- dient perturbation clips, aggregates, and adds noise to the gradient updates in each step of gradient descent methods during model training G.g...Abad et al. 2016 Wang et al., 2017 2019). (iv) Out- put perturbation adds noise the non-private model prediction before its release (c.g.. Chaudhuri
DP mechanisms:
min Ro(A) + A
((D) +
D+ A
Input perlurbation
Objective perturbation
Gradient perturbation
Output perturbation
Figure 8: Privacy mechanisms in the machine learning workflow.
et al. 2011 Zhhan et al. 2022). All stated mechanisms are general strategies that must be carefully
adapted CATE estimation setting. In work, we employ output perturbation for the reasons explained below.
Choice of DP mechanism: Our DP-CATE framework employs output perlurbation to achieve DP. Output perturbation is highly suitable for our setting since (i) it ensures that causal assumptions are fulfilled even after perturbation, and (ii) it retains the power of existing CATE estimation methods for addressing the fundamental problem of causal inference. The other DP strategies discussed above fail to fulfill the requirements.
In contrast, input perturbation might introduce confounding bias violate the consistency assump- tion. For gradient and objective perturbation, the convergence of the model might be unclear. Fur- thermore, objective perturbation might fail to achieve the targeted privacy guarantee if the model does not converge to the exact global minimum in finite time (Iyengar et al., 2019). Gradient per- turbation results in non-trivial privacy overhead and does not align with our goal of providing model-agnostic meta-lcarning framework (Redberg et al. 2023).
A.2 EXTENDED RELATED WORK
The only existing method designed for our setting was proposed by Niu et al. (022). The au- thors provide algorithm for differentially private CATE estimation through existing CATE meta- learners. However, the method necessitates special private base learners for the separate sub- algorithms in each stage of the meta-learner. It is thus not agnostic to the choice of ML method for the first- and second-stage regressions, meaning that arbitrary choices are not supported. Fur- thermore, has been shown that privatizing different parts of causal estimators separately can result in biased causal estimates OOhiihhi & Awan 2023).
different line of work proposes locally differentially private (LDP) algorithms (Agarwal & Singh 2024 Huang & Ascara 2023) Ohnishi Ohnishhi & Awan) (203).. the central data curator cannot be trusted. During data collection, calibrated noise added to each sample before adding it to the database. However, the perturbed data might violate the assumptions to identify causal treatment effects. Furthermore, this notion of privacy significantly reduces the predictive accuracy of the estimators (Huang & Ascara 2023). Thus, whenever the data curator is a trusted party (as assumed in our work), global differential privacy sufficient and should be the notion of choice it is less accuracy comprroici. than its local counterpart.
2023). This notion of privacy becomes necessary if
A.3 FUTURE RESEARCH DIRECTIONS FOR PRIVACY IN CAUSAL ML
Applying causal machine learning methods to real- world problems requires the methods to adhere to guidelines on ML safety and ethical ML. The privacy of predictions is only one aspect of the former. Of note, privacy has direct effect on other ethical and related aspects, such as the
15

---

Published as a conference paper at ICLR 2025
uncertainty or the faimess in the predictions. Hence, there many impactful directions for future research.
Uncertainty in causal ML, including sppsalit types of uncertainty such as unobserved confounding, have been studied in the literature (c.g. Jesson et al., 2020; MeInychuk et al.) [024] Schröder al. 2024a). However, far, there is method that quantifies the uncertainty added through the privatization of the causal model or the predictions. Developing such methods is an important step for future research.
Ethical causal ML has been mostly studied through the lens of causal fairness (e.g.. Frauen et al. 2024
Ma et al. 2023 Plecko & Bareinboim 2024 [Schröder al. (022b). However, the trade-off between fair and private causal ML is unexplored. Research outside the field of causal ML suggests opposing effects of privatization and fairness (c.g.. [Tran al. 2021). Therefore, investigating this relationship is an interesting avenue for future research and may help in providing ethical, causal treatment effect estimation.
A.4 THEORY ON CATE ESTIMATION
The estimation of causal quantities, such as the conditional average treatment effect 7(x) (1) - individual can be observed. Here, (a) is the potential outcome that would hypothetically be ob- served a decision a is taken.
z/. involves counterfactual quantities ((a), since only one outcome per
Due to the above, identification of causal effects from observational data necessitates the following three assumptions that are common in the literature (e.g.. Curth & van der Schaar 2021 Feuerriegel et al. 2024):
Consistency: The potential outcome ((a) equals the observed factual outcome Y when the individual was assigned treatment A
2. Positivity/overlap:: The treatment assignment is not deterministic. Specifically, there exists a positive probability for each possible combination of features to be assigned both the treated and the untreated group, i.e., there exists >> 0 such that <(x) < 1 - k for all x E
3. Unconfoundedness: Conditioned on the observed covariates, the treatment assignment independent of the potential outcomes, i.c.. (0), (1) TL A X. Specifically, there no unobserved variables (confounders) influencing both the treatment assignment and the outcome.
Importantly, the above assumptions are standard in the literature. Further, the assumptions are nec- essary for consistent causal effect estimation for any machine learning model. Then, CATE is iden- tifiable as
(xx = E (0) X = x] p(æz u1 0),
(20)
where p(x,,) a) E[Y X A a). To estimate т. one could thus train a machine learning model that estimates the aforementioned conditional expectation and then calculates the difference in conditional expectations for a given X ær This is commonly referred as plug-in method, yet which has several drawbacks, as we outline below. Rather, the preferred way to estimate the CATE through meta-learners.
Meta-learmers define model-agnostic algorithms, which can be implemented with arbitrary machine learning algorithms. Therefore, meta-learners are flexible and commonly employed in practice. CATE meta-learners be classified into four different categories, depending the ways they leverage the data: (1) one-step plug-in learners, (2) two-stage regression-adjusee learners, (3) two- stage -obeenntty-wiigeedd learners, and (4) two-stage Neyman-orthogonal learners (Curth & van der Schaar 2021), Meta-leamers for specialized tasks have also been proposed recently, such as partial identification or treatment effects over time (e.g.. Frauen et al.] 2025 Oprescu et al.) 2023
Schheessthl et al. (2024)
We now discuss each type of meta-leamer and their potential drawbacks in more detail:
16

---

Published as a conference paper at ICLR 2025
1. One-step plug-in learner: Here, ML models are trained to predict px a), either single model for both treatment values or two different models, i.e,, fi(x 1) and paa,, 0). Then, the CATE is estimated directly f(x) f(zz, 1) f((x 0).
2. Two-stage regression-aljisted learner: In the observed data, the difference between fac- mal and counterfactual outcomes is never present. Therefore, two-stage learners construct pseudo-outcomes surrogates, which equal the CATE in expectation. The regression- adjusted learner designs the pseudo-outcome through a reweighting based on the function μ., which is estimated by in the first stage. A misspecification of results in biased CATE estimator, where the error of j propagates with the same order into the final estima- tor F.
3. Two-stage propensity-weighhedd weighted learner: Here, the pscudo-outcome is constructed based on the Horvitz- Horvitz-Thompsse T'hompson transformation. Only the propensity function и needs to be estimated in the first step by it A misspecification of results in biased CATE estimator. Here, again, the error of propagates with the same order into the final estimator
4. Two-stage Neyman-orthogond learners: Different from the above two-stage learners, these learners have lower error if either the propensity function or the outcome regressions are correctly specified. Specifically, the final estimation first-order insensitive to small er- in the nuisance functions (known quasi-oracle efficiency). This property is achieved through Neyman-orthogonal losses such as Eq. (3)
Hence, we focus on the two-stage Neyman-orthogonal learners throughout our work due the clear advantages.
17

---

Published as a conference paper at ICLR 2025
B EXTENSION TO THE DR-LEARNER
In our main paper, we focused on the R-learner for the derivations and the experiments. However, DP-CATE is applicable any weighted two-stage Neyman-orthogonal CATE meta-learner. Here, we now provide an extension to the DR-learner.
For the DR-leamner, the weight function simplifies to (PR(T())) 1. Then, the d-dimensional (e, 8) -ififerrrentill private CATE estimated through the DR-learner is given by
9DP rDP) = (x fIP)) + 7 DR (T, c(e,,,)) U,
(21)
where yDR (T, D) is a sample version of the gross-error sensitivity 7 (T, P). c(e, = (n(2))()) In (2/0)/(an)) (en), and U (O, Ia). Specifically, 7D P) is given by
PR (T, sup hig (DF))
(a (w)) a)) ADP( (z (a; (or))
Spp (x) -
Observation: The noise calibration necessary for the privatization requires maximizing the influ- ence function. For the DR-learner, the IF includes the inverse of <1x) (x))). Although assume π (z) to be bounded away from zero and one (due to the overlap assumption), maximizing over this term can lead to very large calibration factor. This might limit the applicability of the DR-learner for differentially private CATE estimation.
Below, evaluate the private DR-learner in the manner the R-learner in the main paper. We observe the expected behavior in Fig. g
10.
19
3.
35
58
21
Countorn
Figure 9: Evaluation of DP-CATE for finite queries the on DR-learner for different base-learner specifications on dataset 1.
18

---

Published as a conference paper at ICLR 2025
C SIMILARITIES AND DIFFERENCES OF THE FINITE AND FUNCTIONAL DP-CATE FRAMEWORK
C.I DISCUSSION
If one only wants to release private CATE estimates once, both approaches 1 and 2 are applicable. Nevertheless, the second approach. which we call the "functional approach" can also be employed for iteratively querying the function, which is especially of interest to medical practitioners aiming assess the treatment effect of a drug for various patients with different characteristics. Put simply, the functional approach is relevant when companies want to release decision support system to guide treatment decisions of individual patients. Since such treatment decisions are made based on the entire CATE model, the functional approach is preferred. In contrast, the first approach (which we call "finite-query approach") is preferred whenever only a few CATE values should be released. This is relevant for researchers (or practitioners) who may want share the treatment effectiveness for a certain number of subgroups (but not for individual patients).
The functional approach requires sampling from Gaussian process. Depending whether one aims to report finitely many queries once through this approach or iteratively query the function, the sampling procedure from the Gaussian process U-) differs. We highlight the differences in the type of queries in the following:
Simultaneous finitely many queries; When querying the function only once with a finite number of queries, sampling from Gaussian process implies sampling from the prior dis- tribution the Gaussian process. In empirical applications, this means that one samples from multivariate normal distribution. Therefore, the noise added in the functional ap- proach is similar to the finite-query approach. However, the approaches ① and ? are not the same, as the noise added in the functional approach is correlated, whereas the noise variables in the finite-query approach are independent. Therefore, the functional approach might result in a consistent under- or overestimation of the target. Still, both approaches guarantee privacy.
Iteratively querying the function: In this setting, sampling from a Gaussian process im- plies sampling from the posterior distribution of the process. Specifically, if query has been made to the private function yet, the finite-query approach proceeds by providing the first private CATE estimate of query T1. Observe that the privatization of every further iterative query needs to account for the information leakage through answering for- mer queries. Thus, sampling from a Gaussian process now relates to sampling from the posterior distribution. To do so, it is necessary to keep track of and store former queries Ii,... -i-1 and the privatized outputs. This setting is entirely different from our finite setting approach, in which we propose adding Gaussian noise scaled by gross-error sensi- tivity.
C.2 ALGORIHHM FOR DP-CATE FUNCTIONS
Private outputs of the function ÔDP( îDDP) in Theorem 3 can be released in two ways: (i) the standard batch setting presented in Algorithm| T in which the private function outputs a private vector of CATE estimates once and (ii) the iterative (or online) setting, in which the function queried iteratively, outputting one private CATE estimate at time. Below, we provide an alternative algorithm to apply Theorem 3 in an iterative way.
19

---

Published as a conference paper at ICLR 2025
Algorithm 1: Pseudo-code of out DP-CATE for functions
Input: CATE meta-learer ğD (5 (IDP) trained dataset D with [D] ns, Gaussian kernel matrix
(K(=,,z;)) query Xquery X² privacy budget 6, Lipschitz const. I,, ridge
regularization
Output: Privatized CATE function (Dp(;i<pp)
/+ Calculate calibration term
r + (0.))a)) [p(a, fD)(())] (412222lg(2/0))((2zzr)) Ane);
/* Sample from Gaussian process
U ~ (O0,,, (K (I,, (1)),,)))
/+ Return private estimates
+/
GDP (Xquery; f(p)) + (Xquary ((P)) +r U:
Iterative approach: If no query has been made the private function yet, can employ Al- gorithm 2 to provide private CATE estimate (DD)),)D)), where denotes the first query. Specifically. Algorithm [ samples from a Gaussian Process prior by sampling suitable multi- variate Gaussian noise variable. Observe that the privatization of every further iterative query Jy needs account for the information leakage by answering former queries. Thus, sampling from a Gaussian process now relates to sampling from the posterior distribution. To do so, is necessary to keep track and store former queries Ti...., and the privatized outputs G (ĀDP (ppp)........................ 90
Algorithm 2: Pseudo-code of DP-CATE for functions (iterative setting)
Input: CATE meta-leamer ÔD( (:; fjrr) trained on dataset D with [D] = n, Gaussian kernel matrix
conditioned on the former queries 21,-.., Ti former private outputs
G (ğDe (I1; fjnP),,... .... 9DP (x<-1;)p)))) new query z4. privacy budget e, 6, Lipschitz const. ridge regularization À Output: Privatized new prediction (D:(
if i=/ then
/ Apply Algorithm
end else
/- Calculate pairwise kernel vector
Vi + (K).,, K())=,,=))";;
/* Sample from Gaussian process posterior
C. + (1)
N(V G. K(a,,)) - VTC:'V);
Return private estimate
(2P(2.;0p)) f
end
A note on complexity: Algorithm 2 requires storing and iterative updating the outcome vector G; and the inverse matrix C, The computational complexity of Alg. 2 will thus grow with an increasing number of queries. This poses a limitation of the above approach for settings with many iterative queries.
20

---

Published as a conference paper at ICLR 2025
D THEORETICAL EVALUATION OF EXCESS RISK
Let ( fiDr) and (o(:://r) denote the private and non-private CATE estimates, respectively. Also, let g" (-: hDp) denote the second-stage population minimizer. We follow the existing literature (e.g.. |Bassily et al. 2014) and measure success of our method through the worst-case expected excess empirical risk with respect to dataset D, defined as
EDLYDP. ΠDP) == EAA | (:: MDP) - g" fDP) [tz |ó îDP) - -" (:: [12.]
(23)
where the expectation is taken over the randomness in the second-stage privatization algorithm.
First, observe that, for a suitable calibration factor r(c, 6, åD: vDDr) and noise variable U specified in Theorems T [ we can bound the general formula of the excess risk by
(g(Dr, fDP) Ev |100 fDP)) r(c, 9D: MDP) U -U- (;
(24)
-I 9" (15
S rr(,, 6, ÅD. ) Bu [1U111
(25)
Finitely many queries: From Theorem I we have U U N/ Ia) and, by setting d = yield
r(e, 6, ÅD, (Dr) (T, D) 5v 2 21n(n) In (2/8)
(26)
En
where n is the sample size the training data D. As Ey || = Eg [U: = 1, the excess risk can be bounded by
EDlYDe, /DP) S D). 50d In()) In (2/0)
O(n (n(=))((n)) o(n
(27)
(en)²
Functional queries: From Theorem
we have
4T 21n (2/8) (V2th) Ans
r(c, o, 9.,9 1pp)
sup (0.x){(0.1)xx''
[p(a (DF()))
(28)
where h denotes the kernel bandwidth and the dimension of the covariates. Instead of per- turbing random variable U as above, Theorem 3 perturbs the functional output by sample path U = U(- E H. of a zero-centered Gaussian process with covariance function K ((22x') (v2mh) exp(- 22/3)))) First, we note that
Bu [x (U
Ex (U()))) [x([(XXX)) (V2nh)
(29)
Therefore, get
[p(a =no(()))) 32L" In (2/6) = O(n E oon
(D(rp,, (DP) 5
sup (0.2))(0.1)xx'
(v2Th)
(30)
21

---

Published as a conference paper at ICLR 2025
E SUPPORTING LEMMAS AND DEFINITIONS
Our proof's are based on important properties of differentially private algorithms, which we introduce
below. For proof's and further details, (Avella-Medina 2021 Dwork & Roth 2014 Nissim al. 2007).
Lemma (Post-processing theorem). Let the mechanism M satisfy (c, Le, For any function f, it then holds that (.V satisfies 6DD well.
Lemma 3 (Sequential composition property). Let the set of privacy mechanisms Mj. j == -......
satisfy (Ej, DDP When applying the mechanisms Mj on the same dataset, the resulting overall mechanism (i.e., the concatenation of all the M2) guarantees ( E
Definition 5 (g-smooth and local sensitivities). Let fo be learned function dataset D with [D] and let Xd. Then, local sensitivity of f is defined as
LS(F. D) = sup D'~D,
[1fp(x)- - (x)22)
(31)
where the supremum is taken wrt. D' (unlike D2, where the supremum is taken wrt. to both and D'). Furthermore, a & sensitivity of f is defined as
D) := sup D' E
[exp( (exp(EEda(D.D'))LSLL L')
(32)
where 2d denotes the data domain.
Lemma (Differential privacy via smoohh sensitivity (Avella-Medine Nissimissi
Nissim et al. 2007
Let fo be learned function on dataset D with D = and let E x Then, fDr(x) fulfills (e, diifffeernntt privacy:
f = fo(x) 5. 2 log(2/0) SSe D). U.
(33)
where U NO, Ia) and { 3,d+2 (og(/)))
Finally, we show that in our case, the E-smooth sensitivity can be upper-bounded by the (appropri- ately scaled) gross-error sensitivity.
Lemma 5, (Aveila-Medina 2021 Let fo be a learned function on dataset with D = n and let Xd, Furthermore, let (T, D) denote the sample gross error sensitivity of the parameter the interest T(D) fD(x). Under minimal regularity and boundedness conditions for the E-moooth sensitivity (f (f,(fD) with g = 4(d42 1og(2/63) it holds that
SS D) < (log(n)
(T.D).
(34)
n
22

---

Published as a conference paper at ICLR 2025
F PROOFS
F.1 PROOFS OF THEOREM II COROLLARIES 2 THEOREM 2
Theorem 1 (DP-CATE for finite queries). Let 2 == (a, x. y) define a data sample following the joint distribution Z and fpP = (ĀDr, TDP) the nuisance functions estimated on dataset D in (e/2, / private manner of choice Furthermore, let D be the training dataset for the second-stage model with |D| For 2 (a, T, v) E Z, we define
ÿDP (x; (DP) = 9x ()pe) + (T, d
T(P)= = 9x; QDP),
(10)
r(e,5..D..DP)
P(T, sup hig" x, (DP) pla, ADP (x)) fip,, A(MD) - gg (a; foo)) 2e2
(11)
where (T, is a sample gross-error sensitivity with T(D) instead of T(P) Eq. 1.
U ~ N(0, Ia), c(c,ô,n) := 5 (2/6)/(cn) and where hlg", ; x,,,, fDP) E Rd and (- (DP) depend on the machine learning model employed for the second-stage regression. Then, @DP x: ) is (E, ( diffferrentl private.
Proof. First, observe that by Lemma 3 the first-stage nuisance estimation îpP overall fulfills 5c, DP. By Lemma 2 the privacy of the nuisances not affected by the second-stage -ttage regression. Note that we estimate both stages on disjoint subsets of data, i.e., D Dul D.
If we had access to the f-smooth sensitivity SSe(g. D) of the meta-learner with & H(d+2lox(28)) the estimator 9DP(X; (x; dDP) with
(Dr ipr) = (D fDP) + 5v 2 (2g(2/6)
SS (S, U,
(35)
where U La), x Id would fulfill (e, diifferrentl privacy by Lemma and the parallel composition property of DP.
However, the E-smooth sensitivity highly difficult, even impossible, compute for general function classes such as neural networks. Therefore, we seek an upper bound on SSç D) to ensure that the privacy guarantees stay valid while making feasible to compute the calibration function r(s, 6, åD. (DP). By Lemma we find such an upper bound through
( D) < Vlog(n)
(T, Vlo((n)
sup 1F (2; D) 2EZ
(36)
n
where (T, D) denotes the sample gross-error sensitivity with the parameter of interest T defined T(P)= = 9" (x; înP) (population version) or T(D) = (x; îpp) (sample version).
Now, the influence function of T(P)
g" (DP) where g²(: (or)))
arg mingeg Rp(g, iDP A(ADP)) is follows:
IP( de
(arg min 0 (g, ine (tte))) PEG
(arg min {(1 (1--)) t) for M(AD)) + tpla, Apr())) for, ((to)) (xx
dr
PeG
(38)
Here, the arg min is achieved at off (:3 )DD), and, for smooth models 9, the following holds:
[(1 R) fnr, ((tn)) lp(a, ĀPp AmP (1)) fop, A(#Dp) -9(x))))
0,
(39)
dg
y= (:rie))
and only if (1 (1-t) dg [Rrlg ypp, ((ADP))) - 2tp(a, (op, - ((x)) dg(x)
d
0.
dg
9mm9 (wiDe)
F(g.4)
(40)
As need to query the nuisance functions multiple times, the privatization method needs to be suitable for functions, such as gradient perturbation through DP-SGD (Ahadi al. 2016).
23

---

Published as a conference paper at ICLR 2025
Therefore, by the implicit function theorem:
di ge (x:; (xxs)pe)]
d
-(arte (d)(g.t)
df
(41)
dg
1=0: 0-0:0=00 0=000=0000
dl
g=0:
d² d=g
Rglg. fDD, {(for))
(g for, ((DD)))
=
(x:1()))
dg
10 (NIPDD))
dg(x)
- 2p (DF(z))(((z, for, A(#Dr)) -g" (a; fir)))
(42)
dg
(a:
Now, by regrouping and setting h(g", x, (DD) as
d² d2g
dg(Ø)
h(g", x, iDP) =
[R MDP A(FD)))
(43)
dg
g= 9=9" (x:NDe)
g= (w:ADe)
we obtain the desired influence function:
IF(2,T;P) = -h(g". , x, n) p(a, (DP(()) MDp, A(ADP)) g" (x; fiDP))
(44)
Here, h(g", z, f)PP) Fdd depends the machine learning model employed for the second- stage regression (see Corollaries and 2). Finally, taking the supremum over the data space and evaluating (F T, P) at the empirical distribution D with trained functions îpp and (pp)) states the desired result.
Corollary (Parametric second-stage regression). the second-stage regression is smooth para- metric model, namely G = (g(2; 0) X' + IR, 0 E E C RP), then, in Theorem] 7
we have
g" (((p))) 9(::0'))) and higg ĀDP) Jo (x; fIpP)] He Volg" (I:
(13)
where 0* arg min Rp(g, îDp, A(ADP)); Jq kaxp is Jacobian matrix wrt. 0: Ho BEB V Ro("" (itt,l"""nnnn... ((FDP))] E RpxP is Hessian mutrix; and Vo is gradient.
Proof. First, we note, that 8 is an example of the M-estimator. Therefore, 1F(2,T:P) yields an influence function of the M-estimator. Specifically, it can be shown that
d2
Rr(g, ipr, A(fDD))) d2g
Jo [g" (x; (DD))]
Hg* and
(45)
"=:"(xxmr>"
(g(x))
(x;
(46)
dg
gg=g (anftoo))
Therefore, we yield the desired result
Mggg: x, îDP) = 2 Ja / [9" (x: (mp)]] HT² Volg" (1;"(2;;)))) (z;
(47)
Remark 1. Importantly, for the sample gross error sensitivity, (T,D) sup [1F D) to be finite, we need additional regularity conditions. Specifically, we denote the score function of Eq. by
(22, fpp) == pla, x(x)) (0(2, fIP,, M(TD)) - g(z)).
(48)
First, note that 112.9 f(D)) is differentiable w.r.. g(x) for all E Z. Since we assume that our data stems from a bounded domain, there exist constants K and L such that 1022, g, âpp) and the derivative vú' (z, g, fIDe) are uniformly bounded in by and L. Additionally, we assume that \VoğD fDpP)) < K. The constant K can be regulated through, e.g., gradient clipping. Furthermore, the Hessian Hg of the loss w.E.f. 0 is positive semi-definite by assumption.
If Hg is further restricted to be positive definite, we can employ Lemmas and 2 (02T)) to show the desired upper bound of the smooth sensitivity for a sufficiently large sample size
Meeiaa
24

---

Published as a conference paper at ICLR 2025
9 Specifically, the minimum sample size depends inversely the privacy budget e and ö and grows in the bounds L, K, K and the ratio of the maximum and minimum eigenvalue of Hg.
Note: the Hessian does not fulfill the above criteria and has negative eigenvalues, one can employ the "damping" trick (Martens 2010) by approximating Ha by Ha == Hg + al for a > The parameter defines the conservalivenes of the approximation.
Corollary 2 (Non- -on-parrmettric second-stage regression). If the second-stage regression is a kernel ridge regression with Agg) Al[g|l where G = H. is reproducing kernel Hilbert space (RKHS) we have
induced by a kernel K (.+,) X' x X' + Rt, then, in Theorem] |
9º (:()))) (L, + A1) s() and h(g", x, (DP) = (Lp + XI) (x),)(x),
(14)
where ppp H + ' is weighted covariance operator [.p = [[p( (x)( K(, ()xx)) XIf(-) Xf(+) is a scaling operator; S E H is cross-covariance functional S(.)
[[p(A, ((p(A,TDP(XX))) (X)) K(, X) 6lZ ((tnp(X)))]
Proof. By the representer theorem (population version), the optimal solution of the weighted kernel ridge regression is given by (; = (L, + XL) S(-), where L.f(.) [[[(A, +DP (X)) ((, X))()) and S(.) = E[p( A, (x)( K(, X) of (Z iDp, X(AD))]] For
a description, see, e.g., Berlinet & Thomas thomas-Agnan (2004); Pillonetto et al. (2022).
We now turn to deriving the influence function of y (x; /pP) at g" (xx; fipP;; P), to then receive the explicit form of higg.. ,x, x, f)DP).
(,, IF T
Consider the point-mass perturbation of the distribution P in the direction of denoted as P. (1 (1-t) P)P + td.. Then, the solution to the kernel ridge regression at Pt is given by
gi (x; ) = (L.r, + XTI) Sr. (x),
(49)
where Lp, (1 Lp + tLs. and Sp. = (1 t)S + USs with Ls, [xx) p(a, (z) x(x,x))(( and Ss, (x) p(a, #DP (z)) ((x,2) 6(2, dpp A(RDP)).
=
Then we get
d di "0 (x; fmp))
[F T= = (x: (np); P)
(50)
(Lp AT) -1 Ls, (x) (L, + An) 5(x) (IL, + An) - So, (x)
(51)
=-lla ADF(z)) ((Lp + AL) -1 (((, ()) (xx)
(o: (or))))
(52)
+ pla #DP())) (II, + AT) K(,, (x) 0(2, hop, A(ADP)).
(53)
With h(gg, x, 2, (DP) = ((Lp + Al) A1) K(: (((xx))))xx we get
[F (x; (np); P) = h(g"", x, (pP) pla, ADP (0(z, inp, A(FDP)) g" (æ; fpp))) (54)
which shows the desired result.
Theorem 2 (Neyman-orthogonalit and quasi-oracle efficiency of DP-CATE). The privatization of the second-stage model asymptotically preserves the property of Neyman-orthogonalin namely
llg" (:1) - (= (fp)) & Re(g (g(g"((:) (s (ipppp ((tD)) - Re(g" fip, + Re(sppe n))
l1 (:: (pp)) 9D(: ADPP + (n
(15)
depends on the model class G
ouiput pertusbation
Furthermore, under additional regularity conditions on the privatization of the nuisance functionn (e.g., gradient perturbation), our DP-CATE achieves quasi-oracle efficiency. Specifically, if the orig- inal estimation of the nuisance functions is at rate of at least Op(n -1/4), then the privatized estima- tion preserves this rate.
For details, we refer to Avella-Medina 2021
25

---

Published as a conference paper at ICLR 2025
Proof. First, show that the privatization at the second stage of learning, namely the output per-
turbation, asymptotically does not affect the Neyman- syymam-ortoooonality. Specifically, we show that the L2 error can be upper-bounded
l|g (+:n) 9DP(: (DD):
(55)
= (::m) 9D(-)DP)) r(e, 6, ÅD. MDPP) U1
(56)
<lgg" (:m)) (D(::/D)) 1نi + r(c, 9D .DP) ||
(57)
Elgg((mm) 9" (+ (DP) l12 119" (:: hDP) (- dDp 11i +
op(n
(58)
depends on the model class ş
outpul perturbution
where the last inequality holds due to the excess risk of the output perturbation (see Appendix D Now, we also employ the result from Morzywolek al. (2023):
(g - g" (x & Rr(g (fr),, for A(fDr) Ry (g" (sn), fpr, A(#Dr) + Ra (fpr, 7),
(59)
where R2(fDDP 77) is the higher-order error of misspecifying the nuisance functions. Specifically, the (2DDP 17) takes a different form for different learners:
De-meer R2 7)
Σ (l((;a)) ADP( a)12,, || - # aE(0.1}
(60)
R-learner: (eDiD,,)))
Σ aE(0,1)
(lm(;a) - ADD(: a))? 1| Air-#prll.. + 1| I1..
(61)
Second, we want to demonstrate when the quasi-oracle efficiency holds for our DP-CATE. Specifi- cally, given that the non-pn non-private estimators ü and t are estimated the rate of at least op 1/4), we need to show when the privatized nuisance functions ř)рp can also achive this rate.
We focus on the smooth parametric models for ñ and basic gradient perturbation method for privatization, namely DP-SGD (Bassily et al. 2014 Abadi et al. (016). Let m denote the number of model parameters. From the literature, (e.g., Chen al. [2020), we know that the convergence rate for (E.-)DDPSSGD for convex and Lipschitz losses holds
m
DDD.SGD = =Op
(62)
where n denotes the sample size.
Recall that in our setup we only allocate the budget (s/2, to the privatization of each # and
AD We yield the following result by straightforward calculation: For and with m S vn DP-SGG (1-1)).
For other losses (e.g., non-Lipschitz, non-convex) as well as advanced versions of DP-SGD, an /olog(//8) has been derived in the literature. For an overview, see Waan et al. 2024). In this case, we achieve
upper bound of Op( Vn ns
DPSSGD Eop(n > m log(1/6) S ne 1
(63)
Overall, we thus face a trade-off between the model size in terms of the number of parameters, privacy budget, and sample size. For appropriately chosen factors, the first-stage nuisances achieve rate op(n -1/4), rendering DP-CATE oracle-efficient.
F.2 PROOFS OF LEMMA I THEOREM 3
Lemma 1. Let H denote the RKHS induced by the Gaussian kernel K(x, x') (V2th) -9 exp(-I1x x' dd11/(2h2) for x, X R2, and let fo be the optimal solution to the RKHS regression
26

---

Published as a conference paper at ICLR 2025
fD()) arg min JEH TI oe (X.) (()))) XIlsI
(16)
where w()) > O is weight function, D is dataset with |D] and {(ú, y) is convex and Lipschitz loss function in y with Lipschitz constant L. Then, for D', we have
Ilfo - - I sup [up[(())]
L An
(V(2T)/)
(17)
=EX
Proof. Observe that, for all R2, the Gaussian kernel norm is given by ((x,, x)) (v2wh)o Since
the loss 1 is convex and Lipschitz with constant L, the w--e--eighteed loss is (suprex [w((a) L)- Lipschitz. Thus, the overall loss in Eq. 16 is (supre (x) -ddamiiisbe (see Hall et al,
2013). Therefore, we can employ result from Hall et al. (013)) stating that the RKHS norm of minimizers of neighboring datasets can be bounded as
L An
IlfD - fprll S suppw(a))
sup K (x,x).
(64)
=EX
x
With our observation above, the result follows.
Theorem 3 (DP-CATE for functional queries), Let jAP and TDP denote the private nuisance estimators trained in stage 1 on D. Let = (a,z,y) y) be a data sample from dataset with |D] and x X S R7. Let H denote the RKHS induced by the kernel (²x,²)= = (v2zh) exp(=_1e===1B/2233 and let f(-,)) be a convex and Lipschitz loss function with Lipschitz constant L. We define ğD fIDP) as the second-stage regression solving Eq. 3 via
/
n
(D(::fDP) = arg min gEH n
P(A,, (x.)) ADr,, (()(X.), oZ MDp. ((FDe(X))))) + Algll (18)
im1
Furthermore, let U.) E H be the sample path of a zero-centered Gaussian process with covariance function K x²). Then, (e, diifferrentl privacy is guaranteed by
4L 2n (2/0) U(.)
OD (Dp) ( (DP) +
sup (0.1)
[pla, (D)(
(19)
(V2th) 'nn
re,,
Proof. Let U-) be the sample path of zero-centered Gaussian process with covariance function K(x, x.). For our proof, we make use of Corollary 9 from Hall et al. (2033) for f E H. where H the RKHS corresponding to the kernel K, the release of
A c(6)
fo(.) = fo(-) +
U(-
(65)
(e, 8) differrrerlyl private for
c(0) 2 V 2 2
(66)
and
A > sup 1lIo for 1l D~D1'
(67)
Therefore, for Au = supp llá (; (DP) - فD (-: (DP |
@DP (DP) = gD(+ -DPp) - Au /2 2/g/ U(.)
(68)
is (e, differreettill private. Note that here, again, we implicitly make use of Lemma and Lemma in the same manner as in the proof of Theorem
Finally, from Lemma we know that
L sup 0ل hpp) 9p' (:3 )l S (V((T))))
(69)
D~DD'
An
Thus, the desired result follows.
27

---

Published as a conference paper at ICLR 2025
G EXPERIMENTS
G.1 SYNTHETIC DATASET GENERATION
We consider two different data-generation settings with different complexity. Both settings follow the mechanism described in (Oprescu et al. 2019):
X, ~ 21 1]",
(70)
4, = 1((x 3), 2 n/
(71)
Y, = 0(x,),,+++ (X" +
(72)
where l/. E 1[ and 3,7 y have support with values drawn from (4[0, and 2400 1]. The
dimension of the covariates is set to = 2 for Dataset I and 30 for Dataset 2. In Dataset 1, the conditional treatment effect ((x) defined as
(z) = exp(2ro) + ssin(4x))
(73)
In Dataset ((x) defined as
0 (z) = exp(22o) + sin(4)))
(74)
For each setting, we draw 3000 samples, which we split into train (90%) and test (10%) sets.
G.2 MEDICAL DATASETS
We showcase DP-CATE on the MIMIC-III dataset (Johnson et al. 2016), which in- cludes electronic health records (EHRs) from patients admitted to intensive care units. We extract 8 confounders (heart rate, sodium, red blood cell count, glucose, hematocrit, respiratory rate, age, gender) and a binary treatment (mechanical ventilation) using an open-source preprocessing pipeline [Wang et al. 2020 We define the outcome variable the red blood cell count after treatment which we adapt to be more responsive to the treatment ventilation. To extract features from the patient tra- jectories in the EHRs, we sample random time points and average the value of each variable over the ten hours prior to the sampled time point, All samples with missing values and outliers are removed from the dataset. We define samples with values smaller than the 0. Ist percentile or larger than the 99.99 9ttth percentile of the corresponding variable as outliers. Our final dataset contains 14719 samples, which we split into train (90%) and test (10%) sets.
MIMIC-III:
The Cancer Genome Atlas (TCGA) dataset (Weinstein et al. 2013) contains comprehen- sive and diverse collection of gene expression data collected from patients with different types of cancer. We consider the gene expression measurements of the 4,000 genes with the highest variabil- ity which we employ as our features X. The study cohort of consisted of 9659 patients. We model the binary treatment based on the sum of the 10 covariates with the highest variance and assign constant treatment effect in the sum of the covariates.
TCGA:
G.3 IMPLAMENTATION DETAILS
Our experiments are implemented in Python. We provide our code in our GitHub repository:
https://github. http:hhh..ccmmccce com/ m-oom/mmmae http://a....oomo.cccdeeeeee//eee Dpcchroder/DP-A..
Our DP-CATE is model-agnostic and highly flexible. Therefore, we implement multiple versions of the R- and the DR-leaner with varying base learner instantiations. For the outcome and the propen- sity estimation, we always employ a multilayer perceptron regression and classification model, re- spectively. The models consist of one layer of width 32 with ReLu activation function and were optimized via Adam at learning rate of 0.01 and batch size 128.
For our experiments with the finite-query DP-CATE, we implement the pseudo-outcome regression in the second stage as (a) a kernel ridge regression model with a Gaussian kernel and default pa- rameter specifications (KR) and (b) neural network (NN) with two hidden layers of width 32 with tanh activation function trained in the same manner as the nuisance models. In the experiments for functional DP-CATE, we employ Gaussian kernel ridge regression with tions and default regularization parameter X = 1. Furthermore, our functional DP-CATE requires the specification of the Lipschitz constant L. This constant is either known based the employed
50 basis func-
28

---

Published as a conference paper at ICLR 2025
loss, e.g., L = 1 for the l loss can be upper-bounded. In settings, employ the l2 loss on bounded domain. Therefore, although the l2 loss itself is not Lipschitz, we can calculate L numerically as the upper bound of the domain. We did not perform hyperparameter optimization, as our model-agnostic framework is applicable to any prediction model.
Our framework requires calculating the supremum of the influence functions. We implemented the maximization problem through mathematical optimization using the L-BFGS-B, a limited-memory - algorithm for solving nonlinear optimization problems with bounded variables. The solver was run with default parameters.
29


Reviews:
[{"review_id": "JsvgakhCqu", "reviewer": "Reviewer_ukCv", "summary": "This study proposes two new methods to address the problem of computing CATE estimates under differential privacy. Leveraging the often used assumption of number of queries being known a priori the authors develop a method that uses influence functions as an upper bound to the smooth sensitivity quantity. This is used in a traditional output perturbation algorithm of the CATE estimate to calibrate the Gaussian noise. The second method is for releasing the CATE function in its entirety under differential privacy. This is a much more difficult problem. The authors develop an algorithm that guarantees DP by using a calibrated Gaussian process to modify the output of the original CATE algorithm. They develop an algorithm for determining how to calibrate this process leveraging theory about RKHSs and Gaussian kernel regression. The efficacy of these algorithms are demonstrated on synthetic datasets where access to the ground truth CATE is available and observational medical datasets.", "questions": "1. How many queries were done for Dataset 1 (Figure 3 and 4)?\n2. Do the authors have a sense for how tight they think the upper bound for smooth sensitivity is using the gross error sensitivity? Is there room to make it tighter with more assumptions?\n3. In Figure 4, it seems like DP-CATE consistently underestimates the CATE at the 0.01 privacy budget. Is there any intuition and/or concrete results on whether this method tends to underestimate or overestimate?\n4. What does y-axis represent in Figure 6 and 7? Is it the actual CATE or is it the error?\n5. How was the Lipschitz constant computed for the empirical results?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- By and large this is a well-written paper and it was easy to understand what both algorithms were doing at a high-level.\n- The methods provide more flexibility than prior work in supporting all types of ML models\n- The separation between knowing the number of queries a priori and not is well taken as it allows them to build a stronger estimator in the fixed query setting\n- The experiments show some promise that the estimator is relatively close to the ground truth in the synthetic experiments", "weaknesses": "- The algorithm for releasing the CATE function assumes knowledge of the Lipschitz constant which unless I am mistaken seems like an unrealistic assumption (note I am not concerned with assuming Lipschitzness of the loss)\n- More details about the experiments are needed to help understand them. Right now the details are quite sparse so it’s difficult to contextualize them in each of the challenges described. I leave my questions related to this for the Questions section of the review.\n- Unless I am misunderstanding Figure 6 and 7, it seems like th CATE is constant along the ages / covariates for each task? If so, this is not as compelling for the success of this method. I think like the synthetic dataset, an observational task should be chosen where the CATE differs based on the covariate I will be adjusting. It’s important to understand how well the DP estimator can capture the variation in CATE as the condition covariate changes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mcxz2ta4Vo", "reviewer": "Reviewer_smF6", "summary": "This paper studies the problem of causal inference with sensitive observational data with DP, motivated by medical applications.  specifically, the authors study estimtaing the CATE function (conditional average treatment effect), which as the name implies, quantifies the effect of a treatment as a function of some covariate.  The authors propose a simple output perturbation mechanism to estimate the CATE with DP.", "questions": "* In the problem statement, do you have any constraints on the domain of X?  In practice is it usually a small domain (e.g., one feature) or large domain (many features)?  \n*  In the finitely many queries setting, can you be more precise about the typical characteristics of the setting?  How many queries is typical, and what do those queries look like?\n>* If the number of queries is small (e.g., quantify CATE for ages 0-20, 20-30, 30-40, ...,) then the problem is trivial.\n>* I think it would be good to give a motivating example to make the abstract problem formulation you have a bit more grounded.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "* The paper is technically sound and statistically rigorous.\n* The problem studied is novel and of practical interest.\n* The paper is clearly written and nicely polished.", "weaknesses": "* The paper uses some jargon that may not be familiar to the reader (e.g., doubly-robust). \n* There are no comparisons to baselines.  While the problem studied is new there are no published baselines on this approach, there are some simple baselines you could compare against.  \n>* Here's one: in 1D case you study in e.g., Fig 1, 6 just compute the CATE function non-parametrically.  That is, compute COUNT(Y=1, a <= X <= b) and COUNT(Y=0, a <= X <= b) for a variety of intervals in the domain, from which you can estimate CATE easily.\n>* To handle higher dimensional case, you could compute those counts for each covariate and then make some kind of conditional independence assumption.  In the p=2 case you consider in experiments, you could also just directly compute the full histogram and it should be pretty doable.   I would assume there are both qualitative and quantitative advantages to your approach, but I think it would be good to demonstrate that explicitly.  \n>* From the idea above, it seems this can be framed as a marginal-preservation problem, a problem that many synthetic data algorithms are pretty good at (e.g., PrivBayes).  That could be another baseline.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wPaNAsrhik", "reviewer": "Reviewer_Ucae", "summary": "This paper presents  a new framework (DP-CATE) for estimating conditional average treatment effects (CATE) under differential privacy while ensuring double robustness (see below my comment on robustness). DP-CATE is broadly applicable to two-stage CATE meta-learners with a Neyman-orthogonal loss. The framework can perform both pointwise estimation of the CATE function and direct functional estimation. The authors provide experimental results on synthetic and real data that demonstrate the effectiveness of DP-CATE.", "questions": "- How does the proposed approach for functional data compare to the techniques in Hall et al. (2013)? Highlighting these distinctions would provide valuable context for readers.\n- While the framework is general, how does its performance compare numerically to similar algorithms (e.g.,Betlei et al. (2017), Guha & Reiter (2024) and Niu et al. (2019)) in settings where those methods are applicable?\n- Could the authors provide insights on how functional estimation compares to pointwise estimation in practical scenarios?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The paper is exceptionally well-written and easy to follow, providing a clear presentation of complex concepts.\n- The authors present a well-founded methodology and provide rigorous mathematical analysis.\n- Experimental results on both synthetic and real data help to validate the proposed framework.\n- The framework is general and flexible, covering a wide range of learning algorithms without making unrealistic assumptions.\n- The discussion in line 314 about the relationship between doubly robust learners and smooth sensitivity offers an interesting insight that could have broader implications.", "weaknesses": "- While the authors discuss the novelty of their work on functional data privacy, they underplay previous work in this area. Notably, Hall et al. (2013) provides a mechanism for functional data, which the authors should highlight more explicitly to offer better context for readers.\n- The use of “robustness” could be misleading, as it carries different meanings across fields. Clarifying what robustness specifically entails here would be helpful, especially considering existing work on “privacy and robustness.”\n- Certain definitions, such as those on line 157, could be recalled for clarity. Additionally, the definition of Y ( ⋅ )    used on line 160 appears to be missing, which may cause confusion.\nOverall, while these issues do not significantly detract from the quality of the work, addressing them could improve clarity and reader understanding.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1H2QlWrzt6", "reviewer": "Reviewer_qbNM", "summary": "The paper introduces a novel, privacy-preserving approach for estimating the Conditional Average Treatment Effect (CATE), motivated by the need for privacy in electronic health records. The authors propose DP-CATE, a flexible framework that ensures differential privacy while maintaining double robustness in CATE estimation. The framework is offered in two versions: one for finite queries (e.g., treatment effects for specific patient groups) and another for functional queries (releasing the complete CATE function). A key technical innovation lies in calibrating noise using influence functions for finite queries and Gaussian processes for functional queries. The authors provide theoretical privacy guarantees and demonstrate the framework's effectiveness using both synthetic data and real-world medical datasets (MIMIC-III and TCGA).", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "- The presentation of this paper is clear and well-organized.\n- It addresses an important practical problem: ensuring privacy in treatment effect estimation from sensitive medical data.\n- The proposed DP-CATE framework is highly flexible and model-agnostic, compatible with any doubly robust meta-learner and machine learning model.\n- The authors provide theoretical guarantees for differential privacy while preserving the double robustness property.", "weaknesses": "- The paper does not sufficiently analyze the consistency of the proposed estimators, i.e., whether the estimators remain consistent.\n- The presentation of the identification condition (3) is unclear. The authors should clarify the assumptions (e.g., unconfoundedness) under which the optimizer of (3) represents the true conditional average treatment effect function.\n- Theorem 1 builds upon the work of Avella-Medina (2021). The authors seek an upper bound on $\\zeta$-smooth sensitivity to ensure privacy. However, is this bound tight, and might there be a more optimal bound for $\\zeta$-smooth sensitivity?\n- Could the authors comment on the inclusivity of the two proposed methods? For example, if one generates a functional query and then uses it to answer finite queries, what would be the potential advantages or disadvantages of this approach?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)