Targeted Sequential Indirect Experiment Design
Authors: Elisabeth Ailer
Niclas Dern
Jason Hartford
Niki Kilbertus
Venue: NeurIPS 2024 poster (2024)
Keywords: causality
experiment design
instrumental variables
indirect experiments
Primary area: causal_inference
TL;DR: We adaptively learn optimal indirect experiments to narrow the bounds on a functional of f in high-dimensional, non-linear settings with unobserved confounding.

Abstract: Scientific hypotheses typically concern specific aspects of complex, imperfectly understood or entirely unknown mechanisms, such as the effect of gene expression levels on phenotypes or how microbial communities influence environmental health. Such queries are inherently causal (rather than purely associational), but in many settings, experiments can not be conducted directly on the target variables of interest, but are indirect. Therefore, they perturb the target variable, but do not remove potential confounding factors. If, additionally, the resulting experimental measurements are high-dimensional and the studied mechanisms nonlinear, the query of interest is generally not identified. We develop an adaptive strategy to design indirect experiments that optimally inform a targeted query about the ground truth mechanism in terms of sequentially narrowing the gap between an upper and lower bound on the query. While the general formulation consists of a bi-level optimization procedure, we derive an efficiently estimable analytical kernel-based estimator of the bounds for the causal effect, a query of key interest, and demonstrate the efficacy of our approach in confounded, multivariate, nonlinear synthetic settings.

Targeted Sequential Indirect Experiment Design
Elisabeth Ailer
Niclas Dern
Technical University of Munich Helmholtz Munich
Technical University of Munich
Munich Center for Machine Learning (MCML)
Jason Hartford
Niki Kilbertus
Valence Labs
Technical University of Munich
Helmholtz Munich
Munich Center for Machine Learning (MCML)
Abstract
Scientific hypotheses typically concern specific aspects of complex, imperfectly understood or entirely unknown mechanisms, such the effect of gene expression levels on phenotypes or how microbial communities influence environmental health. Such queries are inherently causal (rather than purely associational), but in many settings, experiments can not be conducted directly on the target variables interest, but indirect. Therefore, they perturb the target variable, but do not remove potential confounding factors. If, additionally, the resulting experimental measurements multi-dimension and the studied mechanisms nonlinear, the query of interest generally not identified. We develop an adaptive strategy design indirect experiments that optimally inform a targeted query about the ground truth mechanism in terms of sequentially narrowing the gap between an upper and lower bound on the query. While the general formulation consists of a bi-level optimization procedure, we derive an efficiently estimable analytical kernel-based estimator of the bounds for the causal effect, query of key interest, and demonstrate the efficacy of our approach in confounded, multivariate, nonlinear synthetic settings.
Introduction
Experimentation is the ultimate arbiter of scientific discovery. While advances in machine learning (ML) and ever increasing amounts of observational data promise accelerate scientific discovery in virtually all scientific disciplines, all hypotheses ultimately have to be supported or falsified by experiments. But the space of possible experiments is combinatorial, and as a result, experimentation in the physical world may become major bottleneck of the scientific process and critically inhibit, for example, fast turnaround in drug discovery loops even in fully automated labs. Effciient adaptive data-driven strategies to propose the most useful experiments are thus of vital importance.
Targeted experimentation requires well-formed hypothesis about the underlying system. We focus on hypotheses that are typically expressed in terms of causal effects or properties of functional relationships. For example, we may posit that there is some ground truth mechanism/function f that describes how a given phenotype depends on gene expression levels in a cell, or how the composition of microbes affects certain environmental health indicators. In the natural sciences, such mechanisms are typically (a) nonlinear, (b) dependent on multi-variate inputs, and (c) confounded by additional, unobserved quantities. For example, the way which the gut microbiome composition affects energy metabolism in humans is likely confounded by various environmental and lifestyle choices. Due to
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

these factors, f is generally unidentifiable, i.e., we cannot hope to infer f from (even infinite amounts of) observational data alone.
Scientific queries are typically more narrow: 'Can expect changes in body mass index when I increase the relative abundance of Lactobacillus bacteria in the gut? How does the susceptibility of breast cancer depend on BRCAI expression levels?' Such targeted scientific queries do not require full knowledge of the underlying mechanism f. but only concern a specific aspect of f. which we can express mathematically as functional Q of f. Due to potential unobserved confounding, even targeted queries Q[ may not be identified from observational data alone (Bennett et al., 2022) experimentation is required. In these examples, the inputs to f of interest which we call treatments- are multi-varial measurements such as gene expression levels or microbiome abundances. Randomizing these treatment variables is typically thought of as the gold standard for removing confounding effects, but perfect randomization is typically impossible: we can seldom perform hard interventions that set distribution over these variables' outcomes independently of confounding factors. Instead, experimental access in the natural sciences often amounts to limited possible perturbations of the system that induce distribution shifts of the treatments. For example, while we can administer antibiotics with very predictable effects on certain microbes, such an intervention does not remove confounding effects from lifestyle choices such as nutrition and exercise or environmental factors. Therefore, we think of experimentation as a source of indirect interventions, which strongly perturb the distribution over the treatment variables of interest.
Our primary goal is to design experiments that maximally inform the query of interest, Q[f]. within a fixed budget of experimentation. For multi-variate treatments and nonlinear f the query Q[ may not be identifiable all, or only be identified after an infeasible number of experiments. We address this by maintaining an upper and lower bound on the query, and sequentially selecting experiments that minimize the gap between these bounds. We show that by treating experiments as instrumental variables (IV) we can estimate the bounds by building on existing techniques in (underspecified) nonlinear IV estimation. Our procedure involves bi-level optimization, where an inner optimization estimates the bounds, and an outer optimization seeks to minimize the gap between the bounds. We show that we are prepared to assume that f lies within a reproducing kernel Hilbert space (RKHS), then there are queries Q[ for which we can solve the inner estimation in closed form. In summary, we make the following contributions:
We formalize the problem of indirect adaptive experiments in nonlinear, multi-variate, confounded settings as sequential underspecified instrumental variable estimation, where we seek to adaptively tighten estimated bounds on the target query.
We derive closed form expressions for the bound estimates when assuming f lie within reproducing kernel Hilbert space (RKHS) for linear Q
We develop adaptive strategies for the outer optimization to iteratively tighten these bounds and demonstrate empirically that our method robustly selects informative experiments leading to identification of Q[f] (collapsing bounds) when can be identified via allowed experimentation.
2
Problem Setting and Related Work
2.1 Problem Setting
Data generating process. We assume the following data generating process
x h)),U,
Y= fo() + U.
where we have experimental access 2 e Z C ged=xe X c. IRd= are the actual inputs to the mechanism of interest, i.e., the 'treatments' (in the cause-effect estimation sense) or "targets' (in the indirect experimentation sense), Y E y IR the scalar outcome, e.g., the phenotype of interest, and U is an unobserved confounding variable with E[U] = 0. The function h determines how experiments (choices of Z) affect treatments X and can be arbitrary. The function fo is the mechanism of interest for our scientific query and can also be arbitrary. As in typical indirect experiment settings, we assume that Z consists of valid instrumental variables: (1) Z. X, i.e., we do perturb the distribution of X by experimenting on Z, (2) Z I1 U. i.e., our choice of experiments is not influenced by the confounding variables, and (3) 2 11 Y X,U, i.e., only affects Y via X and does not have a direct effect on Y. When X is multi-variate and fo, h are allowed to be non-linean fo is commonly not
2

---

identified form observational data. However, we argue that in such settings one should be aiming for more targeted query, i.e. does not aim to identify fo fully, but only a certain aspect of the mechanism.
Scientific queries. We represent the scientific query of interest by a functional Q F - IR where F C (x2(X) is the space of considered mechanisms f.' Generic functionals can measure both local as well as global properties of any f E F: simple average treatment effects via Q[ := J:when F is a Hilbert space, projections of f onto fixed basis function )
E|Y |do(X = x=)
===]== =
f) the local causal effect of an individual component X, on Y at x² via Q[ = (a.)(=*)? Here, x² could be the mean of the observed treatments, í.e., represent 'base gene expression level' and we are interested in how local change away from that base level affects the outcome. While our methodology applies to any functional, we focus our empirical experiments on causal effects of individual components as key scientific queries, such as how does up- or down-regulating a given gene affect the phenotype?
f via [[[][]]] =-
Learning experiments. Our goal is to sequentially learn a policy E P(Z)² such that data sampled from the joint distribution Px(X. Z) x (()) Z) in eq. (1) optimally informs Q[fu]. We highlight that depending on fo. h and the distribution of U, there may exist a policy π such that Q[ is identified from Px,X,, Z), but in general it may remain unidentified for all policies even in the infinite data limit. For non-optimal policy, i.e., non- informative experimentation, we should expect [[fo] to be partially identified from P(X, Y, Z) at best. Therefore, we aim to estimate upper and lower bounds Q (xr), ( of [[fo] and sequentially
IX) induced by the model
learn the policy π that minimizes A(T) == Q+( (TT) Q (T). In each round t E [T' := ,
we observe n i.i.d. samples from Pz, (X, Y. 2) using policy Te. which we use (potentially together
with data collected under previous policies Te) to estimate Q (Tz), Q (TT) and then aim to propose a new policy a++1 that yields smaller gap in bounds: A (1T+41) < <.e.).
Connections to other settings. Besides the 'indirect experimentation" lens, the setting in eq. (1) can be viewed from different perspectives. While they all share the same mathematical formulation, they start from different conceptions of Z. 'he literature on invariant causal learning (Peters et al., 201 6; Heinze-Deml et al., 2018) may interpret Z in eq. (1) as environment indicator and fo is invariant across these environments. Each policy corresponds to an environment (Z cannot be designed, but we can collect data for different 2) with its own distribution of X. The heterogeneity across environments can be leveraged to learn about the invariant mechanism f. In instrumental variables (Pearl, 2009; Angrist & Pischke, 2008), one typically assumes to encounter a situation as in eq. (I) with the assumptions (1)-(3) where X, Y, Z are observed 'in the wild'. Valid instruments Z then sometimes allow for (partial) identification and estimation of fo despite the unobserved confounder U. Again, there is typically no active experimentation in IV estimation, but Z is taken 'as is'. Finally, experimentation on Z in our setting can also be interpreted as soft interventions (Jaber al., 2020; Pearl, 2009) on X, where we intervene on X setting it to follow a given distribution instead of a fixed value. Instead of allowing arbitrary soft interventions, eq. (1) restricts us to distributions P(X Z Fz(( (Z) for feasible experimental policies T (Z). Importantly, unlike proper soft interventions, we do not get rid of unobserved confounding. Ultimately, we face a sequence of IV estimation tasks. Experimental access to Z justifies the instrumental variable assumption (2) Z JL U, and indirect experiments are setup by design such that (1) H X holds. Assumption (3) JL Y |X, U instead may be hard to justify in practice and limits the types of experimentation allowed in our framework.
2.2 Related Work
IV estimation. Instrumental variables are a cornerstone of cause-effect estimation in econometrics (Angrist & Pischke, 2008) at least since their use by Philip G. Wright in 1928 (Wright, 1928; Stock & Trebbi, 2003). Even though valid instruments are hard to come by in practice (Hernán & Robins, 2006), generalizations to settings with relaxed assumptions have sparked renewed interest in IV estimation from the machine learning community not least due to successful applications in Mendelian randomization (Sanderson et al., 2022; Didelez& & Shechan, 2007; Legault et al., 2024) or on microbiome data (Sohn& & Li, 2019; Wang et al., 2020; Ailer et al., 2021). 1). In particular, we build recent advances in nontinear/nonparanee & Syrgkanis, 2018; Singh et al., 2019; Hartford et al., 2017; Saengkyongam et al., 2022; Zhang
IV estimation (Newey & Powell, 2003; Lewis
(X) is the Tp-space of valued functions of X with respect the distribution of X. We write (a.))(=) ) for the partial derivative of f with respect to the i-th argument evaluated at x." Here, d( denotes the space of probability distributions over Rdz
3

---

et al., 2020; Xu et al., 2020), with a specific focus on the minimax formulation using the generalized method of moments (Liao et al.. 2020; Dikkala et al., 2020; Bennett et al., 2023; Zhang et al., 2023; Liao et al., 2020; Bennett et al., 2019; Muandet et al., 2019; Bennett et al., 2022). In addition, we do not assume identifiability in line with recent approaches to partial identification and bounding causal effects in IV settings relaxing various discreteness and additivity assumptions (Kilbertus et al., 2020; Padh et al., 2022; Frauen et al., 2024; Melnychuk et al., 2024; Wang & Tehetgen Tchetgen, 2018; Gunsilius, 2019; Hu et al., 2021; Zhang & Bareinboim, 2021).
Sequential experiment design. The work closest to ours is perhaps by Ailer et al. (2023). where they consider similar setting of sequential indirect experiment design via IV estimation. The key differences are that they only consider fully linear setting (h, fu linear), aim for full identification of fo (even though the setting is underspecified), and only provide non-adaptive strategies, i.e., the sequence of experiments does not depend on data collected throughout the experiments. Other work on indirect experimentation either assumes no unobserved confounding (Singh, 2023) or focuses primarily on sample efficiency and variance reduction when aiming for full identification of (Chandak et al., 2024). Adaptive learning has also been used in another context for cause effect estimation by 'deciding what to observe' Gupta et al. (2021), While clearly related, we highlight that most of the literature active experimentation for causality aims at learning the causal structure (instead of estimating properties of f) from access to interventions, i..., direct experiments (instead of more realistic indirect experiments), e.g., (von Kügelgen et al., 2019; Sverchkov & Craven, 2017; Agrawal et al., 2019; He & Geng, 2008; Gamella & Heinze-Dema & Miller, 2021). Additionally, we require strategy that is wath-dependent, while in active learning the objective function remains the same over the different iterations.
2020; Elahi et al., 2024; Zemplenyi
3 Methodology
3.1 Minimax Instrumental Variable Estimation
For the general instrumental variable setting in eq. (1) that we face in each round, we aim to solve - - f(X) Z] = O. To solve this, we follow the conditional moment formulation (Bennett et al., 2019; Dikkala et al., 2020; Bennett et al., 2022, 2023), which builds on the observation that we only want consider functions f. such that the residuals, U,- == Y f(X) are independent of the instrument Z. If two random variables are independent, U, 11 Z, then Flg(Z)U,} = Flg (Z) E[U,] for all test functions g. We search for f that minimize the residual while maintaining the independence constraint by solving a minimax optimization that minimizes f over a worst case g. The integral equation E /XX] = for f can be written as Tf = To. where T is a linear, bounded operator mapping f + R to Tf = (X)(() 2]: 2 - IR and ro- E[y 2). This is typically an ill-posed inverse problem where both T and ro are not known but have to be estimated from i..d. data D= = {(T,, V/s 2,) )/ Assuming f to come from a set of hypothesis functions F C (X) and c (Z) class of 'witness functions' or 'test functions', we can write the non-parametric IV problem as a minimax optimization problem of the form (Dikkala et al., 2020; Bennett et al., 2023)
arg inf sup L(f.g) FEF gEG
(2)
for some objective function L mapping tuples of hypotheses f and test functions g to R. While most existing approaches start by assuming that there exists a unique solution of TJ = ro (Lewis & Syrgkanis, 2018; Liao et al., 2020; Muandet al., 2019; Dikkala et al., 2020; Bennett et al., 2019), based on Lagrange multipliers Bennett et al. (2023) recently have shown that under a source condition on T and mild realizability assumptions regarding the capacity of the (statistically restricted) function classes FF and G (Bennett et al., 2023, Assumptions 2, 3, 4), the solution of TT = ro is given by
fo arg min max 2 (xxf (ro- -f. arg min max JEF gEG
JEF gEG
(3) This formulation of the objective is immediately amenable to empirical estimation from D by using empirical averages for the expectations. Eq. (3) can be viewed as penalized optimization, where (X)) penalizes the solution towards the minimum L2 norm.
The inner maximization over test functions ensures that we only consider hypotheses f E F compatible with the conditional moment restrictions that enforce the required independence of the residuals. While there may still be multiple hypotheses f e F compatible with observational data in

---

this way, Bennett et al. pick the ff with the least L2 norm, yielding a unique solution. In the following. we build on this intuition to estimate bounds on a chosen scientific query Q among all f compatible with the observed data and assumed structure (via the moment restrictions).
3.2 Sequential Policy Learning for Bounding Functionals
The source condition required for eq. (3) is that ro lies in the range of TT where T is the adjoint of T. As Bennett et al. point out, this is nontrivial restriction and paramount in ensuring uniqueness in eq. (3), Without this source condition the IV problem is still ill-posed in that there may be multiple solutions f E F compatible with the observed data D (even in the infinite data limit). One of our key realizations is that we can still meaningfully make use of the penalized optimization formulation in eq. (3) to compute bounds targeted queries about f-
Let Q: + IR be a functional on hypotheses F that captures the scientific query. For any given joint distribution (x Y, 2) induced by experimental policy to bound [[..]. we can compute
Q 2 (T) = inf sup +Q[1] + B[(Y - (z)),, fer gEG
where again the inner maximization aims at only considering hypotheses compatible with the conditional moment restrictions Fe F and the outer minimization aims at finding the f E Fr with the largest/smallest value of Q[/)- The realizability assumption that So E F then clearly implies that Jo Fe and the bounds Q* will contain Q[fo]- We note that specially when X is of higher dimension than Z. in practice neither ff nor the much more narrow query Q[f) are guaranteed to be identified. However, depending on æ,, we may shape Px such that Fe becomes restricted enough to obtain informative bounds Q* on Q[fo]. Bennett et al. (2022) work out in detail when a linear functional of fu. but not fo itself, identified. If P is such that T satisfies the source condition, certain queries Q (such as the Qif] = 211/12,, as canonical example) may actually be identified and the bounds Q# will coincide.
We can now formulate our main policy learning goal as
A(T), where A(T) = Q( (T), elll
inf
(5)
with (2) C P(2) being a subset of implementable experimentation policies. In practice, we aim at approximating this optimization via sequential updates of n, over T rounds, where observe
n i.i.d. samples of P=, at each round. This means that given Tt, we seek to choose Te+1 such that
A (TT+1 < (T.). The final output of our method is the interval [Q (TT), Q+ (TT)] Q[ A key advantage of this formulation is that we need not assume identifiability of nor of QV/.. will obtain valid (albeit potentially loose) bounds when the experimentation budget T is restricted, and that potential identifiability will be 'captured automatically' by the bounds coinciding. If after T rounds the bounds are still not informative, the experimenter can choose more expressive experiments (different II( (1()) and even different 2), a more specific query Q, or put stronger assumptions on the hypothesis class F.
3.3 Solving the Minimax Optimization Problem
Let us unpack the sequential policy learning problem. At each round E [T], we need to solve two minimax problems for (m()) where the objective is estimated from finite data and then take minimization step over the difference of the two solutions. To further complicate this 'tri-level' (min + minimax) optimization problem, we are optimizing over two function spaces (F, and a family of distributions (TT( (2). In the remainder of this section we develop techniques to render these optimizations feasible in practice for suitable choices of F, (T2).
Existing solutions for the minimax problems typically employ adversarial optimization (iterative gradient-based optimization over parametric function spaces) kernel-based techniques. Since the former are usually hard and expensive to optimize reliably even without the outer minimization, we focus on the latter approach using reproducing kernel Hilbert spaces (RKHS) for test functions g = Hz induced by a characteristic kernel kz 2 x Z + R. Then there exists a closed form expression for consistent estimate of the inner supremum.
5

---

Theorem (Zhang et al. (2020)), ff((YY - f(X)) <
(y =, ((xz))
(6)
consistently estimates su lall (ro- Tf 9) Hz from data D.
Hence, for n samples D jid Pr reduce the minimax optimizations in eq.
(4)
(=) = inf FQIA fEF
12a E 9 ((2))))((,, -
(7)
One option to solve eq. (7) efficiently is via gradient-based methods leveraging highly efficient auto-differentiation packages for parametric function families, such as choosing neural networks for F, i.e., we optimize framework may also benefit from other existing approaches to the minimax optimization from the literature, e-g., Muandet al. (2019); Dikkala al. (2020). Parametric functions may have the advantage that they render the computation of Q [[) feasible. For example, the causal effect of X; on Y at x= given by [[/o] (zz²) can be obtained directly from automatic differentiation as well. Global functionals such as [[ = fx (xx (x) dr for some function e may pose greater difficulties as the integral over X' is typically infeasible. Still, this approach ultimately requires bi-level optimization with substantial amount of tuning.
minee-dimensional real-vector 8 the weights of the neural network. Our
In order to also obtain closed-form estimate of the minimization over F, we have to specify the functional Q. To this end, we focus on bounded linear functionals. One example is the causal-effect of of the most common and relevant scientific
an individual input (x=f) because it is arguably queries: how does a small change of a specific variable affect the outcome? When following a fully non-parametric approach by also assuming hypotheses come from an RKHS J Hx induced by characteristic kernel kx : X x X + R, we can also estimate the minimization over F in closed form for the causal effect-a local, linear functional. The key realization that functionals of functions represented as linear combinations of RKHS functions can be written as linear functions of the coefficients.
Theorem 2. Assume functions in Hz, Hxx to have bounded variation, (w.Lo.g.) images contained 1], and for h E Hz also -h E Hzz Denote by K Kxx, Kzz E Raxx the empirical kernel matrices of D, let kx be continuously differentiable, and let Xg. >,, 2 O. Further, fix Q(f) to be bounded linear functional, and write fo(.) = -E= 0,k(zi, ). Then the solutions to the regularized minimax problems
arg min FQ + do sup (ro - Tf,g Ayl1/lllx Aallol1Hz gEHz
(8)
FEHx
are consistently estimated by S&x with
1 Q[(xx Ag Kxx)+ KxxXxzy
g+ (KxxXzzKxx
d
X
where (Q[Kx E R" is the vector ( (x1, these general bounded linear functionals be computed analytically for many common kernels such as linear, polynomial, or radial basis function (RBF) kernels.
(Q[kx(xxx )z()). Note that
One common example is the partial derivative at x":
Xg Kxx)+ (KxxKz2U 1 (O)xx)))))) de
x= =KxxxXXzzKi -4
(10)
Ne
where (0rKx)) E 12" is the vector (xxx) (0.kk)
We defer all proofs of our theoretical statements to Appendix
We can now substitute these closed-fom estimates for the minimax problems into our policy learning objective A(T) Qlf Qifa in eq. (5)
de (Kxx +4 Nods do
2
Kxx) (Xxx) "(o)""((o) (Q[xxx))(x')
A (T)
(11)

---

(In)validity of bounds. Our general policy learning formulation in eqs. (4) and (5) involves nonlinear, nonconvex optimizations over function spaces and families of distributions. Since we cannot provably obtain global optima to those, we cannot guarantee valid bounds. Even for the RKHS setting in Theorem 2 for linear Q. while the estimates 0= are consistent, for finite sample estimation we have to regularize both the hypotheses and witness functions (Ag, A > 0) to obtain numerically stable and reliable estimates. Clearly, those pose restrictions the search spaces for f. g and may thus lead to invalid bounds. We may expect real-world mechanisms fu to be relatively smooth such that these regularization terms do not render estimates invalid in practice. Moreover, since made no explicit assumptions about the functional Q, using Q[f] as a penalty (as compared to, e.g., 7 )l) may not yield unique solution to eq. (8) even in the infinite data limit and for X Xy = Ó. While uniqueness can be recovered for strictly convex, coercive, and lower semicontinuous Q. this implies non-trivial restrictions the allowed scientific queries, which we may not be willing to tolerate. Therefore, we introduce a 'penalization parameter' de (in front of the supremum term rather than Q for convenience) that allows us empirically trade off the scale/importance of Q[f] and the conditional moment restriction in practice. Enforcing conditional moment restrictions via a minimax formulation from finite samples typically exhibits high variance. Therefore, a lack of provably valid bounds due to practical regularization does not impair the usefulness of our method in a setting where we aim learn about potentially unidentified scientific query from limited experimental access. In particular, we argue that overly conservative bounds are still more useful than likely invalid point estimates based on one-size-fits-al assumptions required for identifiability.
Continuing with the closed-form expression in eq. (11) for the gap between maximally and minimally possible values of the scientific query here the causal effect of changing one treatment component around a base level X - over all hypotheses that are compatible with the observed data, we can now seek to represent the experimentation strategy π in a way that allows us sequentially improve it.
3.4 Sequential Experiment Selection
We now describe different approaches to sequentially update the experimentation policy. The experimentation budget and the number of samples per round n are fixed. We denote by ne the policy in round I and by D(t) (DIS0) the data collected in (up to) round l. We denote by Q and A, the bounds and gap between them, respectively, estimated in round l and explicitly mention which data was used for the estimates. Some strategies rely on a mitta-distribution over policies II(z). which we denote by I' A sample ~ I' is thus a policy in 11(Z) C P(Z). While the 'final result' of all strategies are just the final bounds Q7, we report O also for suitable < T for different strategies to compare efficiency.
Simple baseline. In modern methods for experiment design, random exploration is often surprisingly competitive (Ailer et al., 2023). We consider a simple entirely non adnaavee baseline called random. which independently samples n. I for t E [T] and estimates Q on D(St)
Locally guided strategies. Next, we look to simple adaptive strategies that leverage the locality of Q. If Q is determined in a small neighborhood around some x² E X, useful guiding principle for to aim at concentrating the mass of the marginal (x( around x=. The causal effect (ax)) or the average treatment effect B[(Y) do(X = xx") are examples of local queries. Estimating such local relationships in indirect experimentation without unobserved confounding has recently been studied from a thcoretical perspective by Singh (2023). They propose simple explore-then-ed the--eeehoe-etttttttteeeee strategy and prove favorable minimax convergence rates depending on the complexities of h and fo as well as the noise levels in the unconfounded setting, i.e., X and Y have independent noises and h does not depend on U. We highlight that these strategies do not use the intermediate estimated bounds as feedback to select the next policy Pseudocode for the different strategies is in Appendix G.
1. Explore then exploit (EE) (inspired by Singh (2023)): Split the budget into T = Ti + TT. Sample Te ~ P for / E [Ti]. Then, fit π as a (parametric) distribution on the Z values of the K < Ti samples in D(ST)) closest to x² (i.e., their X values have the smallest distance to x for some suitable distance measure on X)). Set ग. == x for t E {Ti + I,...,T) T) and estimate Q from U ().) The split between Ti and T3 may be informed by n to maximize exploration while retaining sufficiently many samples during exploitation for low variance estimate of Q Alternating explore exploit (AFE): Throughout all rounds, keep list of all observed samples sorted by the distance of their X values to x". For odd t E [T] \ (T} independently sample
7

---

Ti ~ r. For even t E [T] and t = T fit as (parametric) distribution to the min [K n t} nearest samples to x² for some K' E N and estimate OF on Ueven D)4)
There is some freedom in which data is used to estimate OF and using D(S is always a valid choice. For local queries can still be beneficial in practice to discard data far away from x². More broadly, the above strategies could potentially be further improved by weighing samples in DD4) inversely proportional to the distance of (the mean of) Te from x' when estimating QF from D(SO, We only report the vanilla versions described above in our experiments and leave further variance reduction techniques for future work. In all experiments, we use K = n and the Euclidean distance on X.
While these locally guided strategies appear rather limited which information from previous rounds is used in updating the policy, Singh et al. (2019, Sec. 7) make a convincing case for why the simple types of active learning EE and AEE greatly improve over the naive random strategy and may be competitive with fully adaptive experiments. For the fixed policy in ι and the meta-distribution I one should arguably err on the side of 'uninformative" priors, i.., covering all of Z mostly uniformly.
Targeted Adaptive Strategy. We now develop an adaptive strategy denoted by adaptive that actually uses the intermediate estimates of QF to update Tt. A natural choice for policy updates is via gradient descent on our objective A
T++1 = ग. a (T)
(12)
for step sizes Ot at round I. In practice we assume parametric policies म. := TT. with parameters Φe C Rdd With the regdivivvt trick (Williams, 1992) we then compute
V ) = Ex,,,,= (X,Y.2) | TT V log no)
(13)
and can update Te, akin to the REINFORCE algorithm with horizon one. Since the gradient in eq. (12) is evaluated at The from which we have samples available, we can directly use the empirical counterpart of the expectation in eq. (13). One caveat is that our objective A cannot be computed on
the 'instance level' for individual samples (Ia; yi, yi,, zi), but is already an estimate based on multiple samples. In practice, we therefore split the n observations per round into random batches, and average over the A(TO) estimate for each batch times the mean score function across the batch in eq. (13).
t is to reweigh the term within the expectation in eq. (13) by no. To, (Z) (Z) (and then average across rounds). While this allows us to use more data for the gradient estimates, this approach does not yield an unbiased estimator and typically also increases variance due to arbitrarily small reweighing terms. In practice, we may want to only consider data from a small number of previous rounds, where we expect the policy not have changed too much. Again, we believe our technique could benefit from further variance reduction techniques (Chandak al., 2024), but leave these for future work.
An extension use data D(T) from previous round
While our formulation in eqs. (12) and (13) can be efficiently implemented for any parametric choice of næ. in experiments choose multivariate Gaussian mixture model (GMM) (2 EM Nz; Pm, Em) with weights E [0, 1 with Em=1 M = 1, means Jm E Rd and covariances Σ E [Rd, for i E [M]. Hence, the parameters are á = (r, Mi...... HM, Ei,.., Em> ). With the score function V log T known analytically for GMMs, eq. (13) can be estimated efficiently (see Appendix B for details).
4 Experiments
Setup. We consider low-dimensional setting (for visualization purpose) with d,. dz 2
hy(2,U) a (sin(Z) f(x) = 3 exp(Xx) sixx)) U U~,(0,11),
(14)
where we use a = B 20. The local point of interest is x² 0 E Rdd and we easily check that a.f(*))= = 3. We use n = 250 samples in each round over a total of T = 16 experiments.
Method parameters. We use radial basis function (RBF) kernels k exp(pl<= - =2) with a fixed p = 1. Note, that the three hyperparameters are relative weights Thus we set \ := dgdi 0.01 and X 0.04, we refer to the Appendix C for further comparison of different
8

---

of Funetioni
Mighie Ahomanng Eoplo Rindem
Vakee of Functional
Mappie Ahtrating Poplo-Sappi Random
1000
200
100
-500
-100
200
10
12
10
14
Itention
Tieration
100
50
20
13
-50
230
-100
Random
Explore Exploit
Altemnoiing
Adaapive
Random
Explore Exploit
Alteraning
Adaptive
Figure 1: We compare the different strategies in our synthetic setting. Left and right only differ in the range of the y-axis. The black constant line represents the true value of Q[fo]- Top: Estimated upper and lower bounds OF over t E [T] for nseeds = 50 and two different "zoom levels' on the y-axis. Lines are means and shaded regions are (10, 90) percentiies. Bottom: The final estimated bounds Q at T = 16. The dotted line is y O. Both locally guided heuristic (explore-ther) cetoor--the-exploiit alternating explore exploit) confidently bound the target query away from zero with relatively narrow gap between them. Our targeted adaptive strategy is even better and essentially identifies the larget query Q[fo] 20 after T 16 rounds.
hyperparameter choices. For all strategies the variance of our policy is set at de = 0.001. For explore then exploit, we chose Ti = 10, T2 6 with कH = (N,,, o.Id2,) ) and independent 14, ~ ~N Ida.) for 1. E [Ti]. The Gaussian mixture of the adaptive strategy is initialized with M = 3 (1/a, 1/2, 1/3), Em Ida, and independent jmm N( Tda,). )- We use constant learning rate 0g = 0.01 for all f and restrict ourselves to learning only weights Tmm means (4 and the diagonal entries of the covariances Em.-
Results. We perform Rseeda = 50 runs for different random seeds and report means with the 10- and 00-percentiles in Figure 1. The simple non-adaptive random baseline starts out with poor performance as expected, and improves mildly over multiple rounds. We note that the ultimate performance of the random baseline depends primarily meta-distribution I') puts near x", i.e., whether we collect sufficiently many samples near x² over the T rounds. Explore then exploit performs quite well soon sufficient informative examples have been collected during the exploration stage depends on the choice of Т. However, explore then exploit does outperform random (with the same exploration distribution) indicating that the heuristic of focusing on the informative samples (the ones near xx) does provide substantial improvements (as also found by (Singh, 2023)). The alternating strategy AEE provides bounds across all rounds and clearly shows step-wise improvement from the first round on ultimately performing similar to EE. Finally. the adaptive strategy quickly narrows the bounds and achieves fairly narrow gap A already after few rounds. At T = 16 it has essentially identified the target query Q[fu] 20 with low variance across seeds. For completeness, we provide a runtime comparison of the different methods in Appendix D. The code to reproduce results is available at https://gtt tiuu.b.b..... www.hh..../......eeeeeeee wwtt//oleeeee/eee/te...ccaeaeeee www.lleeettarreeeeeeee weweeetddd---eeeeriae iv-experiments..
how much mass the random exploration (defined by the
start exploitation. Again, whether
5 Discussion and Conclusion
Summary. We formalized designing optimal experiments to learn about a scientific query as sequential instrument design with the goal of minimizing the gap between estimated upper and lower bounds of a target functional Q of the ground truth mechanism fo. We only assume indirect experimental access, allow for unobserved confounding, and consider nonlinear fo with multi-variate inputs such that both fo and Q[fo] may be unidentified. For broad set of queries, we derive closed form estimators for the bounds within each round of experimentation when / lies within an RKHS. Based on these estimates, we then develop adaptive strategies to sequentially narrow the bounds on the scientific query of interest and demonstrate the efficacy of our method in synthetic experiments. Given the increased amount of data collected in fully or partially automated labs, for example for drug

---

discovery, we believe that efficient, adaptive experiment design strategies will be a vital component of data-driven scientific discovery.
Limitations and future work. We have not validated our method in a real-world setting, as it would require access (and full control) over an actual experimental setup well the time and resources required to conduct these experiments. A key technical limitation of our work is that neither our general policy learning formulation in eqs. (4) and (5) nor the concrete closed form estimates in Theorem 2 obtain provably valid bounds for all queries Q from finite samples (see discussion right before Section 3.4). While our approach is reliable in synthetic experiments, we believe thorough theoretical analysis of necessary and sufficient conditions for (asymptotically) valid bounds (including ideally asymptotic normality with known asymptotic covariance or even finite sample guarantees) is useful direction for future work beyond the scope of our current manuscript. In practice, the applicability of our approach may also limited by the assumptions that the underlying system is well described via fixed, static function fo X' -> Y as opposed to, say, temporally evolving and interacting systems governed by a differential equation. Similarly, while IV assumptions (1) and (2) can arguably be justified in our setting, the third assumption Z TT Y |X, U limits the types of experimentation we can consider (see discussion at the end of Section 1).
Methodologically, we believe that our approach could benefit from advanced variance reduction techniques and be sped up by more efficient estimators (Chandak al., 2024). Along these lines, is worthwhile future work to analyze the optimal trade-off between exploration and retaining large number of samples for exploitation and thus lower variance estimates. Finally, our current empirical validation is limited to linear local functionals, rather simple parametric choices for policies (such as (mixtures of) Gaussians), and we have not optimized the kernel choices and hyperparameters. Hence, a validation where all these choices have been tailored to real-world application scenario is an important direction for future work.
Acknowledgments and Disclosure of Funding
EA is supported by the Helmholtz Association under the joint research school "Munich School for Data Science MUDS". This work has been supported by the Helmholtz Association's ''s Initiative and Networking Fund through the Helmholtz international lab "CausalCellDynamics" (grant # Interlabs-0029).
References
Agrawal, R., Squires, C., and Yang. K. Abed-strategy: Budgeted experimental design for targeted causal structure discovery. In Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics, pp. 2874 2882. PMLR, 2019. 4
Ailer, E., Müller, C. L., and Kilbertus, N. A causal view on compositional data. arXiv preprint arXiv:2106. 1234 20
Ailer, E., Hartford, J., and Kilbertus, N. Sequential underspecified instrument selection for cause- effect estimation. In Proceedings of the 40th International Conference on Machine Learning. ICML'23. JMLR.org, 2023. 4, 7.
Andy, Y., Morris, J., and Mark, G. Slurm; Simple linux utility for resource management. Workshop on job scheduling strategies for parallel processing, 2003. 24
Angrist, J. D. and Pischke, J.-S. Mostly harmless econometrics: An empiricist's 's companion. Princeton university press, 2008.
Bennett, A., Kallus, N., and Schnabel, T. Deep generalized method of moments for instrumental variable analysis. In Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 32. Cur- ran Associates, Inc., 2019. URL htwps: http://////pop///ee//eeeee wtw.oeeeeddiinnnnnnniieee. neurips cwturips.cc/pppeeeeee cc/paperr2019//ilee 1551885ea77755ee7777444444441113
Paper. pdf.
Bennett, A., Kallus, N., Mao, X., Newey, W., Syrgkanis, V., and Uehara, M. Inference on strongly identified functionals of weakly identified functions. arXiv preprint arXiv:2208 0829)), 2022. 2, 5
10

---

Bennett, A., Kallus, N., Mao, X., Newey, W., Syrgkanis, V., and Uchara, M. Minimax instrumental variable regression and 2_2 convergence guarantees without identification or closedness. In The Thirty Sixth Annual Conference on Learning Theory, pp. 2291-2318. PMLR, 2023. 4, 5
Bradbury, J., Frostig, R., Hawkins, P., Johnson, M. J., Leary, C., Maclaurin, D., Necula, G., Paszke, A., VanderPlas, J. Wanderman-Milne, S., and Zhang, Q. JAX: Composable transformations of Python+NumPy programs, 2018. 24
Chandak, Y., Shankar, S., Syrgkanis, V., and Brunskill, E. Adaptive instrument design for indirect experiments. In The Twelfth International Conference on Learning Representations, 2024. URL https://openrevver nett/forummddd47:550000
4, 8, 10
Didelez, V. and Sheehan, N. Mendelian randomization an instrumental variable approach to causal
inference. Statistical Methods Medical Research, 16(4):309-330, 2007. 3
Dikkala, N., Lewis, G., Mackey, L., and Syrgkanis, V. Minimax estimation of conditional moment
models. Advances in Neural Information Processing Systems, 01112248-12266 2020. 4. 6, 14
Elahi, M. Q., Wei, L.. Kocaoglu, M., and Ghasemi, M. Adaptive online experimental design for causal discovery, 2024.
Frauen, D., Melnychuk, V., and Feuerriegel. S. Sharp bounds for generalized causal sensitivity analysis. Advances in Neural Information Processing Systems, 36, 2024. 4
Gamella, J. and Heinze-Deml, C. Active invariant causal prediction: Experiment selection through stability. In Advances in Neural Information Processing Systems, 2020. 4
Gunsilius, F. A path-sampling method partially identify causal effects in instrumental variable models. arXiv preprint 1xXiv:1110.095000 2019,
Gupta, S., Lipton, Z. C., and Childers, D. Efficient Online Estimation of Causal Effects by Deciding What to Observe. Papers (.2..,9,,,, arXiv.org, August 2021. URL https://ideas repec. org/p p/ //aar//aapppae atx/papers//eeie 2208 09265 html.
Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J., Kern, R., Picus, M., Hoyer, S., van Kerkwijk, M. H., Brett, M., P., Sheppard, K., Reddy, T., Weckesser, W., Abbasi, H., Gohlke, C., and Oliphant, T. E. Array programming with NumPy. Nature, 2020. 24
Haldane, A., del Río, F., Wiebe, M., Peterson, P.,
Hartford, J., Lewis, G., Leyton-Brown, K., and Taddy, M. Deep iv: A flexible approach for counterfactual prediction. In International Conference on Machine Learning, pp. 1414-1423 2017. 3
He, Y. and Geng, Z. Active learning of causal networks with intervention experiments and optimal designs. Journal of Machine Learning Research, 9:2523-2547, 2008. 4
Seinze-Demll C., Peters, J., and Meinshausen, N. Invariant causal prediction for nonlinear models.
Journal of Causal Inference, 6(2):20170016, 2018. 3
Hernán, M. A. and Robins, J. M. Instruments for causal inference: epidemiologist's dream? Epidemiology, pp. 360-372, 2006. 3
Hu, Y., Wu, Y., Zhang, I.,, and Wu, X. A generative adversarial framework for bounding confounded causal effects. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 12104-12112, 2021. 4
Hunter, J. D. Matplotlib: A 2D graphics environment. Computing in Science & Engineering, 2007.
24
Jaber, A., Kocaoglu, M., Shannugam, K., and Bareinboim, E. Causal discovery from soft inter- ventions with unknown targets: Characterization and learning. Advances neural information processing systems, 33:9551-9561, 2020. 3

---

Kilbertus, N., Kusner, M. J., and Silva, R. A class of algorithms for general instrumental variable models. In Advances in Neural Information Processing Systems, volume 33, 2020. 4
Kluyver, T., Relln-Kelley, B., Pérez, F., Granger, B., Bussonnier, M.. Frederic, Kelley, K., Hamrick, J., Grout, J., Corlay, S., Ivanov, P., Avila, D., Abdalla, S., Willing, C., and Jupyter Development Team. Jupyter Notebooks a publishing format for reproducible computational workflows. In IOS Press, pp. 87-90. IOS Press, 2016. doi: 10.3233/978 1-61499-6660 1-61499-649---87 24
Legault, M.-A., Hartford, J., Arsenault, B. J., Yang, A. Y., and Pineau, J. A novel and efficient machine learning mendelian randomization estimator applied predict the safety and efficacy
of sclerostin inhibition. medRxiv, 2024. doi: 10.1 10.1101/002400.330.22221 wwtrriiivvogg/aeoo
URL https:
wwt.ooogo/oooo.oooooooooo.o ccontnnteaarllyy /tt/eeel/////////////////2/ 00.30. 31.....43300221...1 3
Lewis, G. and Syrgkanis, V. Adversarial generalized method of moments. arXiv preprint
1803.0 164, 2018. 3,4
Liao, L., Chen, Y.-L., Yang. Z., Dai, B., Kolar, M., and Wang, Z. Provably efficient neural estima- tion of structural equation models: An adversarial approach. Advances in Neural Information Processing Systems, 33:8947-8958, 2020.
Melnychuk, V., Frauen, D., and Feuerriegel, Partial counterfactual identification of continuous outcomes with curvature sensitivity model. Advances in Neural Information Processing Systems, 36, 2024. 4
Muandet, K., Mehrjou, A., Lee, S. K., and Raj, A. Dual instrumental variable regression, arXiv preprint arXiv: 12358, 2019. 4,
Newey, W. K. and Powell, J. L. Instrumental variable estimation of nonparametric models. Econo- metrica, 71(5):1565-155
2003.
Padh, K. Zeitler, J., Watson, D., Kusner, M., Silva, R., and Kilbertus, N. Stochastic causal pro-
gramming for bounding treatment effects, 2022. URL
pandas development team. Pandas, 2020. URL https//dat.orr//6000058///// 3509134. 24
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N.. Antiga, L., Desmaison, A., Köpf, A., Yang, E.. Vitooo, Z., Raison, M.. Tejani, A.. Chilamkurthy, S., Steiner, B., Fang, I.., Bai, J., and Chintala, S. Pytorch: An imperative style, high-performance deep learning library, 2019. 24
Pearl, J. Causality, Cambridge university press, 2009. 3
Pedregosa, F. Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, É. Scikit-learn: Machine learning in Python. JMLR, 2011. 24
Peters, J., Bühlmann, P., and Meinshausen, N. Causal inference by using invariant prediction: identification and confidence intervals. Journal of the Royal Statistical Society Series B: Statistical Methodology, 7855)9947-1112 2016.
Petersen, K. B. and Pedersen, M. S. The matrix cookbook. Technical University of Denmark, 7(15):
510, 2008. 15
Saengkyongam, S., Henckel, L., Pfister, N., and Peters, J. Exploiting independent instruments: Identification and distribution generalization. arXiv preprint arXiv:2202.018865 2022. 3
Sanderson, E., Glymour, M. M., Holmes, M. V., Kang, H., Morrison, J., Munafò, M. R., Palmer, T., Schooling, C. M., Wallace, C., Zhao, Q., and Davey Smith, G. Mendelian randomization. Nature Reviews Methods Primers, 2(1):6, 2022. 3
Severini, T. A. and Tripathi, G. Some identification issues in nonparametric linear models with endoge- nous regressors. Econometric Theory, 22(2):258-22 278, 2006. doi: 10.1077//0266446606666 17. 18
12

---

Severini, T. A. and Tripathi, G. Efficiency bounds for estimating linear functionals of nonparametric regression models with endogenous regressors, Journal of Économetrics, 170(2):491-499 2012. ISSN 0304-4076. doi: /ttp:///arearrcaaa////////000000 sciencedirect. cww/sc////////o comm/scince// of Generalized Method of Moments. 18
URL https: //www. Thirtieth Anniversary
Singh, R., Sahani, M., and Gretton, A. Kernel instrumental variable regression. In Advances in Neural Information Processing Systems, pp. 1593-4605, 2019.
Singh, S. Nonparametric indirect active learning. In Ruiz, F., Dy. J., and van de Meent, J.-W, (eds.), Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, volume 206 of Proceedings of Machine Learning Research, pp. 2515-2541. PMLR, 25-27 Apr 2023. URL https: ptttps://proceeddie http://pooeeeenn..../. mlr. www.eppressseee v206// singh23a. html. 4, 7,9
Sohn, M. B. and Li, H. Compositional mediation analysis for microbiome studies, Annals of Applied Statistics, 13(1)6661-68.) 2019. ISSN 19417330. doi: 10.1214/1--AOSS122 3
Stock, J. H. and Trebbi, F. Retrospectives: Who invented instrumental variable regression? Journal
of Economic Perspectives, 17(3):177-194, 2003. 3
Sverchkov, Y. and Craven, M. A review of active learning approaches to experimental design for uncovering biological networks. PLOS Computational Biology, 13(6):1-26 06 2017. doi: 10.1331/jjurnal.per 1005466. URL httph:::s//oooooooo 10. 13771/oorraal patuuraal.ppb........ 1005466.
Van Rossum, G. and Drake, F. Python 3 Reference Manual: (Python Documentation Manual Part 2). Documentation for Python. CreateSpace Independent Publishing Platform, 2009. ISBN 9781441412690. URL https: boctps://bat //books.../oo//ooo google...ooooooooooooggooog de/bookss Sitessidi=KKIIAAACC 24
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et Scipy 1.0: fundamental algorithms for scientific computing in python. Nature methods, 17(3):261-27,, 2020. 24
von Kügelgen, J., Rubenstein, P. K., Schölkopf, B., and Weller, A. Optimal experimental design via bayesian optimization: active causal structure learning for gaussian process networks. arXiv preprint ar Xtw::1111000996..2 2019. 4
Wang, C., Hu, J., Blaser, M. J., Li, H., and Birol, Estimating and testing the microbial causal mediation effect with high-dimensional and compositional microbiome data. Bioinformatics, 2020. ISSN 14602059. doi: 10110000.oinfmmmtt///555555... 3
Wang, L. and Tchetgen Tchetgen, E. Bounded, efficient and multiply robust estimation of average treatment effects using instrumental variables. Journal of the Royal Statistical Society Series B: Statistical Methodology, 80(3):531-5500 2018.
Williams, R. J. Simple statistical sradient-following algorithms for connectionist reinforcement learning. Machine learning, 8:229-256, 1992. 8
Wright, P. G. Tariff on animal and vegetable oils. Macmillan Company, New York, 1928.
Xu, L., Chen, Y., Srinivasan, S., de Freitas, N., Doucet, A., and Gretton, A. Learning deep features in instrumental variable regression. arXiv preprint arXiv:2010.07154 2020. 4
Zemplenyi, M. and Miller, J. W. Bayesian optimal experimental design for inferring causal structure. Bayesian Analysis, 2021. doi: 211214/22-bb13333 4
Zhang, J. and Bareinboim, F. Bounding causal effects on continuous outcome. In Proceedings of the
AAAI Conference Artificial Intelligence, volume 35, pp. 12207-12215, 2021. 4
Zhang, R., Imaizumi, M., Schölkopf, B.. and Muandet, K. Maximum moment restriction for instrumental variable regression. arXiv preprint arXiv:2010.06684, 2020. 3, 6
Zhang, R., Imaizumi, M., Schölkopf, B., and Muandet, K. Instrumental variable regression via kernel maximum moment loss. Journal of Causal Inference, 11(1):2020073, 2023. doi: doi: 10. 10.15115/cc
URL ttcttss://do. hwidddd.o//d..oooooooooooooooeoo org/10, 10. 1515/ 1tt.:..1111/111////1111111 -i22022-0073. 4
13

---

A Proofs
In this section we provide proofs of the statements in the main paper. Before proving Theorem we restate a result from the literature that provides closed form consistent estimates of the minimax problem with the L2 penalty term (instead of penalizing the functional).
Theorem 3 (Proposition 10 of (Dikkala et al., 2020)). Assume without loss of generality that functions in Hz, Hx have bounded variation and their images are contained in [-1,1]. Additionally, assume that for each h E Hz also -h E Hz. Denote by Xxx and Kzz the empirical kernel matrices from D. Then we can consistently estimate
f arg inf sup B[(Y f(X)))(()) - + Ayllf fEF DEg
(15)
via
f()) = Σ ())),
== (Kxx + 4 Mxx
(16)
with M = Kzz. If the function classes F,G 9 are already norm constrained, the estimator needs no penalization.
Proof.
Note that we have the optimum of the inner supremum
sup B[(Y - [())) A,llslle 9Eg
(17)
(Y ((x) M(Y - f(X)) 4
(18)
Therefore, we are left to solve the outer minimization
min - f(X)) (M()) - f(X)) + 42gAs1lf113
(19)
feF
f mYn - f(X)) - - f(x)) 42,
JEF
min (y - (xx0) M(y - Kxx0) + 42,AA11K22113
Replacing the function with the (empirical) kernels
BER"
min TJ -xx 22 42g420 K20 /EF
Writing out product
min JEF
OJ (Kxx + 44,,y1Kxx00- 2y MKxx0
Aggnegation terms wet 6
f (KxXMXxX + * KxxMy
Solution for minimization problems
Theorem 2. Assume functions in Hz, Hx to have bounded variation, (w.lo.g.) images contained
in 1], and for h E Hz also -h E Hz. Denote by Kxx Kzz E Raxn the empirical kernel matrices of D, let kx be continuously differentiable, and let Xg, Xy, de 2 0. Further, fix Q(f) to be bounded linear functional, and write fe(.) = Dm= 0.k(2 +). Then the solutions to the regularized minimax problems
8
arg min FQ[f] + de sup (ro Tf.ghuz Ayllflllx - Agllgllux BEHz
(8)
fEHx
are consistently estimated by fōa with
Ngds x (XxxXzzV 1 Q(Kxx((r))
_+ (Kxx BTxxX22XXxs +
(9)
dc
x
where (Q[Kx.))('')) E R. is the vector ((Q]kx(s), (.*),............ (Q ((n))(())))))))))) ()*).. Note that these general bounded linear functionals can be computed analytically for many common kernels such linear, polynomial, radial basis function (RBF) kernels.
14

---

Proof. The solution of the inner supremum remains the same as in Theorem 3. This leaves us to solve the outer minimization
min Qlf + 44gAA111ffl + Ao - -xx)) M Kxx0), SEHx
(20)
where M Zzz. We write out linear combination of kernel basis functions (at the observed data points) and use the linearity of the gradient functional, which applied to the kernel basis functions
is just (0xKx-)(x"). This yields for the overall functional Q[fol = (a,Kx.) )o. We are thus seeking
min (O=Kx))(=")0 + NO Xxx 2Ney MKxx0 + 42g2,0 Kxx0, BER"
(21)
which we rewrite as
min 0 (KxxMXxX + 8E
A xxx 0+ (10K - MXxx) de
(22)
Building on the arguments in the proof of Theorem where we replace the linear part in 0 by the partial derivative functional, obtain the minimizer
Mxxx + Ag21 A x
"(KxsMy- - 1 (((Kx.)(x*)) de
(23)
dc
as consistent estimator.
Proof. We re-iterate the same arguments to prove the theorem for general bounded linear functionals. We assume Q[f) to be linear bounded functional acting on f.
Theorem Riesz representation theorem Let Q[f) Hx T. be continuous linear functional on a separable Hilbert space Hx. Then there exists 9 E Hxx such that Qif) (f.q)Hx, Vf E Hxx.
Assuming Hxx via the Riesz-representer theorem for linear functionals, there exists unique element q & Hxx such that for any / E Hxx The bounded linear functionals on Hxx form themselves hilbert space called the dual space.
Q1f1 = (f, H)
(24)
We assume H to be an RKHS. Therefore, we can write out f(x) as linear combination of the kernel basis functions (at the observed data points):
We replace Q[f] = (f. q) = <Ei=1 0,k(2,, :), 9) = = 0gg(2:) as E Hx. Due to the RKHS, we can write the empirical version as g(-) = Ex Q1k(.,,)))
min 0J Q[k(2, 4hg2g0 T + Acly JeHx
Kxx0)
(25)
Thus obtain the minimizer
Agds kxx) (Kxxay- 1 ([(xx(10))
XxxMMXxx+ +
(26)
do
Ac
B Details on Adaptive Policies
For completeness, we here recall the score function of Gaussian mixture models, i.e., the gradients with respect to the mixture weights, means, and covariances of the log-likelihood function. We write for the GMM for brevity. First, we define the 'responsibilities' Çim. the probability that sample 2; comes from component m via (see, e.g., Petersen & Pedersen (2008))
Vn Vo: Mm Em))
Sim
(27)
Σ/ 7g:(2z: M4j, Eg)
15

---

50
40
Functional Value
30
20
10
(0 50000 109001100
(1000 59000 100001000
1000
(0) "te0 1010
(rop Too 10
tou "80n 1000
(1070 sero 1000
(0T "E'O 100!
(100 'oo 1oro
in s090 220000
5009 '2000
(y) "100 "200
(190 1880 "2000
10y 590 22000000
(100) S80 (2000
(1810 'O (E00)
(D "900 "5000
(1000 5080 5000
1or 1180 '0000
380 b000
(1000 9800 '0000
(1010 O '00000
mY 5000 'Snn
(n '900 (000
(1000 50800 S000
(100 10 '0000
fgx 5500 '000
(TO "900 'St0))
(1000 5900 $000
(6) 'E9 (5000
thoo TO '55000
nn 10900 11000
tb 00000 (200)
"Lnn
Ton do eoo cool soo
aoo see 2212 to to Food 1o00 to (200
(1e 500000
(1000 5000
380
101 "18V
for
(100
d)
Figure 2: Adaptive Method for different hyperparameter settings. The x-axis shows them in the following order: (Ae, Ag at)
Then, we have for sample 2; and mixture component E [M] that
Sim
7m log TT (z;)
(28)
7m
Hmm log (2:) = Sim Em (Zi Hmn)
(29)
mm (z;) = Sim (Em) Em (2s - pm (2i - um )² Em
(30)
C Hyperparameter Tuning
In Eq. (1I) end up with three hyperparameters. Note however, that those three mentioned hyperparameters are relative weights. Thus, we can set X, := Agde and only tune A and Ag- Intuitively, A, regularizes the smoothness of the function spaces and A weighs the functional Q relative the moment conditions. In Fig. 2, show the dependence of our bounds on these hyperparameters: very low values of X lead to conservative bounds whereas large values of X yield narrow bounds throughout, A, effectively controls the size of the search space via regularity restrictions. For the learning rate, a lot learning rates below a certain value actually work for the method. More gradient update steps (and therefore learning rates, very high learning rates result in drastic updates of the policy; something that is not very favorable in real-worle experiments.
experiments)
required for very small
D Runtimes
For completeness, we show wallclock runtimes the different methods in our empirical evaluation in Figure 3. All methods were run MacBook Pro with Intel CPU. The runtimes are per iteration and clearly increase for methods that accumulate data over previous rounds. Even the most expensive passive baseline is relatively fast to compute without specialized hardware and we generally imagine computational cost be negligible compared the actual cost of physical experimentation required in each round in most applications.
16

---

Adaptive - Alternating Explore-Exploit Random
Time in Minutes
80
60
40
20
0
2
4
6
8
10
12
14
Iteration
Figure 3: Wallclock runtimes of the different methods.
150
150
100
50
-50
150
-100
200
Random
Explore-Expoit
Alteroringg
Adepiie
Random
Explone-Exploit
Altemoning
Adaettive
Figure 4: We compare the different strategies our synthetic setting. The black constant line represents the true value of [[fo]- Both plots show the estimated upper and lower bounds QF at T = 16 for Asoods = 50. Lines are means and shaded regions are (10, 90) perceentils Left: d_ 5, dy 20, Right: d, dz 20.
E Additional Experiments
We assume the same setting as the low-dimensional one, only increasing the dimensions.
hz,), = a (r-sin((2)) - UI,
if S des if > dz.
(31)
U+
du
1(X) = 3 Σ
U, U~N(0.1)
(32)
#=1
with d, = 5, 20 and d, 20, d 20. The setting guarantees, that though we have mismatch in dimensions, i.c. d, < dz. we would in principle- still be able to identify the full causal effect, as the completeness property is fulfilled. We refer Fig. 4 for the results of the different methods. Moreover, we note that the increase in time is still manageable, c.f. Fig 5. The main driver for the computation time the samples within each experiment (T₁, T2) due the kernel approach and, therefore, the inversion of gram matrices based on the sample size.
For d: = 5, dg = 20, we used A = Audy = 0.04 and A = 0.1. For d, = dN = 20, we used A
A,Ar = 0.05 and A = 0.1. Intuitively, the optimization requires stronger hyperparameters in higher dimensional settings as with dimensions, also the function space increases.
X, = A.
F Underspecification
Additionally, we include some examples along the lines of underspecification. We refer to Ailer et al. (2023) for thorough discussion underspecifiiation, but give motivation for the examples in the following:
Since / is the solution to linear inverse problem, the problem is often ill-posed and thus is not identified. However, underspecification means something more explicit. It can be seen as violation of the completeness property.
17

---

Miglie Ahomanng Eoplo--Eoplot Kandem
Adgliie Aborving Poplre-Fippot
Raddrn
Time in Mimutes
120
80
10
14
Deration
Deration
Figure 5: We compare the wallclock time in a higher dimensional setting. The time increase is still mild. The main driver is the number of samples in each experiment, instead of the dimensionality itself. Left: d- = 5, da 20, Right: dz = dz = 20.
Lemma (Severini & Tripathi (2006)). The conditional distribution of p( (X () is complete if and only for each function f() such that []f(=)] 0 and Var(f(x)) > 0, there exists a function g(z) such that f()) and g(2) are correlated.
In most papers, the completeness property is trivially fulfilled by assuming exponential family for p( 2) with non-zero variance for all components. This means that both and T are injective which again guarantees the identifiability of f-
Now, we denote Py as the orthogonal projection space of the linear operator T Hx - Hz with Tf = (X)( ZJ. Following this notation, Severini & Tripathi (2012) argue that PA(TT f is the identifiable part of S. This leads to the following lemma:
the function space FF and (T)) as the null
Lemma (Severini & Tripathi (2012)). E[Q[f]] is identified if and only if E(T)
Examples. Let us introduce an example for an unidentified Q[í): Consider Z, R. R² and ((=) (z, 0), (x1, = 0.5z + 202 and Q[ 02 fx= In this fully linear setting, due the structure of / the instrument can only ever perturb the first component of X regardless of what policy we choose. Hence, it is impossible to identify e.g. Q[f] w.r.t. the second argument with 0zf(x²) 2vz". Even a policy with full support will not (even partially) identify Q[fo]. In contrast, let us discuss a fully identifiable scenario, which is identifiable for all Q[ff. but still depends on the policy: Consider similar setting with fx1 x2) xf + 22 and Q[f] fxf((=)) Assume we have
x² = (6. 1), then [[fol = 2 6 = 12. Any policy that has positive density in a neighborhood of 6 will be able fully identify Q[ However, any policy that puts no mass (or in practice, little
mass) near 2 = 6, will not identify Q @f This would be an uninformative policy.
G Algorithmic Boxes
We present pseudocode for the 'explore then exploit' (EE) strategy in Algorithm 1, for 'alternating explore exploit' (AEE) in Algorithm and for the adaptive strategy in Algorithm 3.
18

---

Algorithm Explore then exploit (EE)
1: Input: Budget T, T1, T2, n, distance measure d on X' target x:
2: Initialize: D(ST)) <
3: Split budget: = 7 72
4: f= to Ti do
1
5: Sample मe
6: Observe sample (Xe, Zz)
7: D(STi) D(ST)) U {(X,,
8: end for
9: Find K nearest samples to x² in D(STi) based on distance d
Fit as a parametric distribution on the Z values of these K samples
11: to + 7 do
12: Set = π
13: Observe sample (Xt; Ze)
14: D(1)_ - (Xe, Z()
15: end for
16: Estimate O from Ul D(
Algorithm 2 Alternating explore exploit (AEE)
Input: Budget T, K, n, distance measure on X, target x*
2: Initialize: Sorted list of observed samples L +- 0
3: for [T' do
4: if is odd and t + T then
5: Sample ne ~ r
6: Observe sample (X,, Zx)
7:
Lu [(X,, 2,)
8:
Sort C by distance x*)
9:
else t is even or t = T then
10:
Find the (i, n .t} nearest samples to x² in L
11:
Fit ne as a parametric distribution on the values of these samples
12:
Observe sample (X,, Z.)
13:
L + LU {(X,, Z.)
14:
Sort L by distance xx)
15:
end
16:
end for
17: Estimate Q on Uevent D
Algorithm Adaptive
1: Input: Budget T, Ti, T2,
2: Initialize: +,
3: Split budget: 7' = Ti + T2
4: for prrt= to Ti do
5: Sample Z, ~ T4.
6: Observe sample (X,, Zz)
7: Compute gradient update v(T.)A(TT,) by splitting observed samples
8: end for
9:
for t= Ti to T do
Set ग. := n
Observe sample (X,, Zz)
12: D(Ij_ < ((X,, Z+)
13: end for
14: Estimate Q from - D())
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract clearly states contributions, the additional Limitation block marks additionally all goals that are motivational and states which of those are further projects.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The conclusion includes an additional paragraph stating the limitations our work.
Moreover, acknowledge the open question and explain potential way how to address those. Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification,
asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't It acknowledged in the paper. The authors should use their best judgment and recognize that individual actions favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: (1) Assumptions: we state them directly in the main paper, moreover we provide references the papers on which our results are based on, including the section of their proofs. (2) Proofs: we sketch the proofs in the main paper and reference to the full proof in the supplementary.
Guidelines:
20

---

The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experi- mental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The code will be provided in public repository. The experiments are performed via a command line run file. The results are saved to npy-formated data file. One jupyter notebook does perform the visualization. Additionally, we listed the seeds that are relevant for the data generation to ensure full reproducibility. Guidelines:
Justification:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g.. to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access data and code
Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: Yes]
Justification: The data is based on a simulation. The code, as well as the exact command line arguments are github. Moreover, all code reproduce the results is openly available and commented.
Guidelines:
21

---

The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines httppp:///iisccc///ppppp guides/CodeSubbig SumeSSumissii oniolicyy for more details.
While encourage the release of code and data, understand that this might not be possible,
so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for
open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: http://iisssspssciccccccccc Sttps//CoeSuubbisseee wwwissssssiiiiyyic for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only a subset of experiments are reproducible, they should state which ones are omitted from the script and why. At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended. but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits. hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The code, as well as instructions are openly available. The code and results can be reproduced via a python file that run via the command line. Hyperparameters are listed and the choice justified in the Experimental section of the main paper and the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is
necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material. Experiment Statistical Significance
Question: Does the paper report eтror bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The experiments are run multiple times over Tiseeds 25 different seeds. We state barplots in the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors). It should be clear whether the enror bar is the standard deviation or the standard enror of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors verified.
For asymmetric distributions, the authors should be careful not show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
22

---

8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experi- ments?
Answer: [Yes]
Justification: The computing was done an internal academic cluster. We provide estimated time in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: //tpuri////uris... cc/ppubccc/thhieeeuuueel..l..lle
Answer: [Yes]
Our data is evaluated on simulated data, moreover, our algorithm tries to move away from correlational assumptions and tries to identify a part of the causal quantity. Even though our claim is to recover part of the causal effect, we state its limitations. We provide sufficient documentation to reproduce the results, but also to understand and comprehend our claims. In conclusion we are convinced that we compile to the NeurlPS Code of Conduct.
Justification:
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there special consideration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our paper focuses on foundational research.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deploy- ment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to par- ticular applications, let alone deployments. However, if there direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
23

---

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g.. pretrained language models, image generators. or scraped datasets)?
Question:
Answer: [NA]
Justification: Neither the simulated dataset, nor the algorithm have high risk of misuse. The algorithm only adds value as soon as it is used on real world experimental set-ups. Currently, is only based on simulation studies.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere usage guidelines or restrictions access the model implementing safety filters.
Datasets that have been seraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification: The theoretical contributions on which our work is based are correctly cited in the main paper, the code is written from scratch and solely showcases our proposed algorithm. As packages we are using:
Table 1: Overview of resources used in our work.
Name
Reference
License
Python
Van Rossum & Drake (2009)
PSF License
PyTorch
Paszke al. (2019)
BSD-style license
Numpy
Harris et al. (2020)
BSD-style license
Pandas
pandas development team (2020)
BSD-style license
Jupyter
Kluyver et al. (2016)
BSD-style license
Matplotlib
Hunter (2007)
modified PSF (BSD compatible)
Scikit-learn
Pedregosa et al. (2011)
BSD 3-Clause
SciPy
Virtanen et al. (2020)
BSD 3-Clause
JAX
Bradbury et al. (2018)
SLURM
Andy et al. (2003)
Apache 2.0
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi weerssiihhhhdee thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
For existing datasets that are re-packaged, both the original license and the license the derived asset (if it has changed) should be provided.
24

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: While our main contribution not the code that we provide alongside the paper, but only provided to ensure reproducibility, the code is still well documented and ready to use. The consent is given, as the authors the papers are owners of the code as well.
Guidelines:
The answer NA means that the paper does not release new assets. Researchers should communicate the details of the dataset/code/model as part of their sub- missions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve any crowdsourcing or research with human objects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human
subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Sub- jects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: The paper does involve any crowdsourcing research with human objects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applica- ble), such the institution conducting the review.
25


Reviews:
[{"review_id": "tzt7eMLghe", "reviewer": "Reviewer_GhAo", "summary": "The authors' primary goal is to design experiments that maximally inform a query of interest about the underlying causal mechanism, within a fixed experimentation budget. They address this by maintaining upper and lower bounds on the query and sequentially selecting experiments to minimize the gap between these bounds. They show that by treating experiments as instrumental variables, they can estimate these bounds using existing techniques in nonlinear instrumental variable estimation. Their procedure involves a bi-level optimization: an inner optimization estimates the bounds, while an outer optimization seeks to minimize the gap between them. For certain queries, when assuming the underlying function lies within a reproducing kernel Hilbert space (RKHS), they derive closed-form solutions for the inner estimation problem. The authors develop adaptive strategies for the outer optimization to iteratively tighten these bounds and demonstrate empirically that their method robustly selects informative experiments, leading to identification of the query of interest when possible within the allowed experimentation framework.", "questions": "No.", "limitations": "No.", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "S1. The presentation is generally clear, with a logical structure that guides readers through the problem formulation, methodology, and initial experimental results.\n\nS2. The authors present a novel approach to causal effect estimation in nonlinear systems with potential unobserved confounding. This framework addresses some challenges in scientific settings where direct experimentation is not possible, potentially aiding more targeted scientific discovery.", "weaknesses": "W1. The paper only demonstrates results on a low-dimensional synthetic setting (d_x = d_z = 2). While this provides a proof-of-concept, it's unclear how the method scales to higher dimensions or performs on real-world data. \nThe authors could:\n- Extend experiments to higher dimensions (e.g. d_x, d_z = 10 or 20) to show scalability.\n- Test on semi-synthetic data by using real covariates with simulated outcomes.\n- Discuss computational complexity as dimensionality increases.\n\nW2. The paper makes strong assumptions (e.g. RKHS, valid IVs) without thoroughly discussing their implications. The authors could:\n- Provide sensitivity analyses for key assumptions.\n- Discuss scenarios where assumptions may not hold in practice.\n- Clarify which parts of the method rely on which assumptions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their kind assessment and useful comments. We reply to the raised questions one by one.\n\n**Scalability:** Following the suggestion, we have worked on additional higher-dimensional experiments. We show all results in the main pdf of the rebuttal. In Fig. 1 and 2, we present results for higher-dimensional settings with $d_z = 5, d_x = 20$ as well as $d_z=20, d_x=20$. Overall, these demonstrate that the method can indeed scale beyond the proof-of-concept low-dimensional setting. Specifically, in the $d_z=5, d_x=20$ setting, our adaptive approach is the only one that essentially rules out the functional being negative (or zero). Similarly, the causal MSE is not only the smallest for our adaptive procedure on average but also has the smallest variance across runs.\nFinally, we report runtimes for these settings as well, highlighting that while runtime does increase in higher-dimensions, the increase is extremely mild. A key driver of the practical runtime is still the number of samples per experiment. As this may grow exceptionally large in certain applications, one may leverage existing techniques to efficiently handle the inversion of large kernel matrices, i.e. [1, 2] to speed up computations. Importantly, we note that computational cost is mostly a concern in our synthetic experiments, as we imagine the time and cost to run physical experiments in real-world applications dwarf computational cost and time, which are at most polynomial in their input size (like kernel methods). We will include these clarifications as well as the additional experimental results (in polished figures) to the final paper.\n\n**Strong Assumptions:** We agree that obtaining strong guarantees in causality typically requires strong assumptions and that our methodology may be restricted by those (as mentioned in l.125, 129). We have the following comments:\n\n1) *valid IVs*: Indeed “valid instruments are hard to come by in practice” (l.129, see also l.404-405). At the same time, we believe this problem to be less grave in our setting than it is in other applications, where one hopes to “accidentally find valid instruments naturally”. In our setting, we control/design the instrument. Therefore, the independence assumption $Z \\perp U$ actually holds (by simply randomizing the instrument). Similarly, it is reasonable that domain experts would not include instruments in the considered domain that may have no effect at all on $X$, i.e., we believe it is not outrageous to defer to practitioners to only consider fairly strong instruments for which $Z \\not \\perp X$ holds. For example, a biologist may know the gene that a CRISPR guide is targeting, or the protein to which a compound binds. Off-target effects remain a very real concern, but when IV’s are experimentally selected, we can hope to select IVs with the IV assumptions in mind to minimize  exclusion violations.\nAs mentioned in the paper (e.g., l.125), the exclusion restriction ($X \\perp Y \\mid X, U$) remains a potentially restricting factor. At the same time, the fact that we may consider multi-dimensional treatments can effectively make the exclusion restriction more realistic as we may add all known factors via which $Z$ may affect $Y$ into $X$. A useful example are orally administered antibiotics in so called “sub-therapeutic dosages”. This means that the antibiotics enter the stomach and gut, but cannot be detected in the blood stream. When using the gut microbiome for $X$, it is then reasonable to assume that the antibiotics only affect macroscopic health outcomes via the changes to the gut microbiome.\n\n2) *Sensitivity*: We agree that sensitivity analyses towards these assumptions are an important direction for further work. In terms of the exclusion restriction, there has been a line of works (recently summarized and extended into a usable framework by [4]) providing tools to analyze the sensitivity of IV estimates to violations of the exclusion restriction. However, these works are limited to the linear setting and thus do not apply to our setting. Rare extensions to non-linear systems (e.g., [5]) only apply to limited settings and in particular cannot handle multi-dimensional treatments or underspecified settings. Providing such sensitivity analyses not only for point estimates, but to the bounds in our underspecified setting with multi-dimensional treatments would likely be a substantial contribution in and of itself.\nThere is also an active area looking into other types of partial identification in the IV setting (e.g., [6], where partial identifiability stems from dropping the additive noise assumption). While additive (measurement) noise may actually be a defendable assumption in the settings we’re aiming at, major technical challenges remain in the multiply nested optimizations that would be required to perform sensitivity analysis (in the case of [6], requiring a nested optimization in and of itself) on top of our adaptive procedure.\n\n3) *Source condition*: The practical limitations of the source condition are not conclusively resolved or at least still debated. To the best of our knowledge, [3] are the first to relax this assumption. Some of the ideas there may transfer to our setting to relax the source condition, but we consider this a direction for future work that likely requires substantial novel theoretical developments.\n\n4) *RKHS*: We do not fully understand the restrictions the reviewer is pointing towards by assuming RKHSs. Broadly speaking, RKHS are in our experience generally viewed as a reasonable and flexible assumption for many practical purposes. For characteristic/universal kernels (like the RBF we use), they enjoy universal approximation properties and often have strong consistency and convergence guarantees (see [8] for an overview). Together with the empirical success of kernel-based methods in many domains and the efficient existing algorithms for practical estimation, we believe them to be a reasonable choice.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IBk7MxAebW", "reviewer": "Reviewer_pD3c", "summary": "The authors provide a procedure to use a sequence of encouragement designs to identify target functionals about a particular causal relationship.", "questions": "see above", "limitations": "see above", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This is a really cool problem setting. It isn't obvious to me that it's particularly common, but I think the larger idea of trying to think about _which_ experiments to run in order to gain knowledge about the world is a worthy line of inquiry.\n\nIn general, I think the approach of getting partial identification on a parameter and then determining a series of actions to take in order to reduce the width of that bound is a fantastic approach. This is a great way to conceptualize a variety of problems, I suspect.", "weaknesses": "It isn't clear to me why Equation 4 is a bound on Q[f_0]. Maybe this should be obvious to me, but I think if it isn't obvious to me, it probably won't be obvious to a lot of your readers, as I think I've read more into this literature than most people. This feels like your big contribution: after this bound is setup, the remainder isn't what I would call straightforward, but it feels more like standard machinery that I'm used to. Conditional on the bound, Theorem 1, 2 and the experiment selection procedures you provide all make a lot of sense. But I just don't see where this bound comes from. I would like to see (i) a clearer explanation of why this expression bounds Q[f_0], (ii) some clearer examples of the cases in which the bounds are equal and Q[f_0] is identified. I think it might also help to lay out when a non-optimal policy (what you call \"non-informative experimentation\") doesn't identify Q[f_0]. This is a bit confusing to me, as I'd expect that complete randomization as a policy would provide identification so long as there is positivity across the whole space: one could do something like off-policy evaluation to identify any particular \\pi(Z). I believe the problem comes with the restriction discussed at the start of Section 3.2, but I do not see why it is that the solution to ensure that r_0 lies in TT^* is to take the +\\- of Q[f] in the objective as from Eq 3. There's a connection here that you have not clearly spelled out to me.\n\nMaybe its possible that the problem is that I need to read Bennett et al 2023 much more carefully to see why this follows, but that isn't a fair ask for your readers who are reading _this_ paper rather than that one.\n\nUnfortunately, it's quite difficult for me to evaluate the overall novelty of this work without understanding this component. I think you have something very cool here, but I can't quite work that out based on the paper as it stands. I'd like to reiterate that I follow what you're doing at a high level (i.e. line 96-106 makes sense in general), but I don't follow when you get into specifics.\n\nSome other miscellaneous thoughts that aren't as important:\n- Do you _have_ to call this (\\partial_i f)(x^*) a local effect? Between LATEs and everything else, this feels like an absurdly overloaded term.\n- The experiments section makes it difficult to see clear quantitative comparisons between methods. I would like to see things like causal mean-squared error. I recognize that's a bit difficult when you're doing partial-ID, but demonstrating that the bounds collapse to the correct Q[f_0] is important. Even just doing something like showing the midpoint of the bounds quantitatively (with confidence interval based on replicates) and the width of the bound would be useful to make sure the process is behaving sensibly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their kind assessment and useful comments. We reply to the raised questions one by one.\n\n**Eq. 4 giving valid bounds**: Thanks for pointing out the missing steps here. Indeed, much of the machinery to see that eq. (4) yields valid bounds is in the Bennet et al. (2023) paper. Essentially, we follow the steps in their Section 3 replacing $\\tfrac{1}{2} \\langle h, h\\rangle_{L_2}$ with $Q[h]$. (They use $h$ instead of $f$, apologies for that!) Then, the first equation in their Section 3 makes it more obvious that the constrained optimization yields bounds on $Q[f_0]$ as they read: “Minimize/maximize $Q[f]$ subject to $f$ satisfying the moment conditions, i.e., being compatible with the IV assumptions and data.” The realizability assumption then ensures that the optima of these optimizations are valid bounds on $Q[f_0]$. From here on, the solution to this problem is shown to be equivalent to the solution of our eq. (4).\nWe follow Bennet et al. in (their) eq. (3), in using the method of Lagrange multipliers to obtain an alternative minimax formulation of the problem. Note that so far they do this in $L_2$, spaces. Analogous to their eq. (4), we can then replace the resulting estimator with a finite sample version and restrict ourselves to some function classes for $h$ (our $f$) and $g$. They then view the $L_2$ norm term as a penalization term (“so we call our estimator a penalized minimax estimator”) and we do the same for $Q[f]$. Up until this point, this distinction did not matter for the theory. As Bennet et al. note, the key distinction from previous work is to use the method of Lagrange multipliers. These steps demonstrate that – if we manage to show their eq. (5) for our modified Lagrangian – our minimax problem in (our) eq. (4) indeed yields bounds on $Q[f_0]$. In Section 4, Bennet et al. then go into showing that the problem can indeed be formulated as the minimax problem using the assumptions stated there (which we also had to carry over to our work). The results there (in particular Lemma 3) also directly apply to our modified Lagrangian. The key point in which their proof of the key identification Theorem 1 can fail in our modified setting is when they leverage the strong convexity of their Lagrangian in $h$ (our $f$), which comes from the $\\langle h, h, \\rangle_{L_2}$ term. Hence, as we discuss in “(In)validity of bounds” (specifically l.270 onwards), not every $Q$ will guarantee a unique optimum and valid bounds for eq. (4). First, valid bounds can be recovered for strictly convex functionals $Q$ (following the proof by Bennet et al. 2023). Second, we argue in the same paragraph why even for functionals that are not strictly convex, our method remains highly useful. We’re happy to discuss these arguments further.\n\nWe agree that this line of reasoning is not self-contained in our paper and we will add a section in the appendix cleanly working out the computations for our Lagrangian explaining that we closely follow the work of Bennet et al. 2023.\n\n**non-informative policies and identification**: Thanks for the suggestion. We will add the following trivial examples in the appendix.\n1) *Unidentified:* Consider $Z, Y \\in \\mathbb{R}$, $X \\in \\mathbb{R}^2$ and $h(z) = (z, 0)$, $f(x_1, x_2) = 0.5 * x_1 + 2 * x_2$ and $Q[f] = \\partial_2 f (x^*)$. In this fully linear setting, due to the structure of $h$ the instrument can only ever perturb the 1st component of $X$ regardless of what policy we choose. Hence, it is impossible to identify the derivative of $f$ w.r.t. the second argument, which is just constant 2 (regardless of $x^*$). Even a policy with full support will not (even partially) identify $Q[f_0]$.\n2) *Fully identifiable:* Consider a similar setting as above, but with $f(x_1, x_2) =  x_1^2 + x_2$ and $Q[f] = \\partial_1 f(x^*)$. Let’s say $x^* = (6, 1)$, then $Q[f_0] = 2 \\cdot 6 = 12$. Any policy $\\pi$ that has positive density in a neighborhood of $z=6$ will be able to fully identify $Q[f_0]$. However, any policy that puts no mass (or in practice, little mass) near $z=6$, will not identify $Q[f_0]$. This would be an uninformative policy.\n\n**”local” effect**: Indeed, this is not the best wording in this context. We’ll think of a more distinct alternative (not so easy actually).\n\n**experiments and metrics**: Thanks for this suggestion. We included the causal mean-squared error in the plots in the main rebuttal pdf as well as in the revised version of the paper. As expected, our adaptive method not only achieves the best average causal MSE, but also exhibits the least variance across runs. We are still working on – and plan to also include – the suggested plots with the midpoint between bounds + ranges as intervals to present results more clearly.\n\n\n[1] Luenberger, David G. Optimization by vector space methods. John Wiley & Sons, 1997.\n\n[2] Elisabeth Ailer, Jason Hartford, and Niki Kilbertus. 2023. Sequential underspecified instrument selection for cause-effect estimation. In Proceedings of the 40th International Conference on Machine Learning (ICML'23), Vol. 202. JMLR.org, Article 19, 408–420.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zgcM4a3mVB", "reviewer": "Reviewer_aNmJ", "summary": "This paper proposes a framework for designing sequential indirect experiments for estimating targeted scientific queries in complex, nonlinear environments with potential unobserved confounding when direct intervention is impractical or impossible. The authors formulate the problem as the sequential instrument design, using instrumental variables and minimax optimization to estimate the upper and lower bounds of targeted causal effects. The proposed method then tightens these bounds iteratively through adaptive strategies. Experiments on simulated data demonstrate the proposed method's efficacy compared to non-adaptive experimental design baselines.", "questions": "1. Please see the questions in the Weaknesses part.\n\n2. It seems that the proposed method is closely related to causal Bayesian Optimization [1]. I wonder if the authors could discuss the connection in detail. \n\n3. Experiments demonstrate that the proposed method performs well for local causal effect queries. However, it is usually unknown whether the target causal effect is local or more global (i.e., long-range) in real-world applications. I wonder if the proposed method could estimate long-range causal effects accurately.\n\n4. What is the difference/connection between the proposed method and targeted indirect experiment design in an active learning setting?\n\n[1] Aglietti, V., Lu, X., Paleyes, A., & González, J. (2020, June). Causal bayesian optimization. In International Conference on Artificial Intelligence and Statistics (pp. 3155-3164). PMLR.", "limitations": "The authors adequately addressed the limitations of their work.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper focuses on targeted sequential indirect experimental design, which is an important problem in scientific discovery where direct intervention is often impractical or impossible.\n\n2. The proposed method is more flexible as it considers nonlinear, multi-variate, and confounded settings, and formulating the problem as a sequential underspecified instrumental variable estimation is intuitive and sound.\n\n3. The authors develop closed-form estimators for the bounds given targeted queries when the mechanism is in an RKHS.\n\n4. The proposed method outperforms non-adaptive baselines in synthetic experiments.", "weaknesses": "1. Although the proposed method focuses on indirect experimental design, it still requires the causal structure between variables known, which is also one of the main challenges in scientific discovery. I am curious about the method's sensitivity to imperfections in the causal structure, assumptions regarding instrumental variables, and the presence of confounders. It would be great if the authors could discuss further about them.\n\n2. The authors only conduct synthetic experiments in a simple setting when the number of variables is small. I wonder if the authors could conduct more experiments in a more complex setting with a larger number of nodes, and also discuss the scalability of the proposed method (since kernel-based techniques are used for optimization as mentioned in the paper).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their kind assessment and useful comments. We reply to the raised questions one by one.\n\n**Causal Structure and Assumptions:** As long as the instrumental variable assumptions hold, we are agnostic to any confounding $U$ between the treatment variable $X$ and the outcome $Y$. We agree that the IV assumptions are rather strong and “valid instruments are hard to come by” (l.129, l.404-405). However, this issue is less severe in our setting than it is in other applications, as we control/design the instrument, e.g. the independence assumption $Z \\perp U$ holds by simply randomizing the instrument. Moreover, we think it is reasonable that domain experts would not include instruments that may have no effect at all on $X$, e.g. a biologist may know the gene that a CRISPR guide is targeting, or the protein to which a compound binds. The exclusion restriction ($Z \\perp Y \\mid X, U$) is a potential constraint (e.g., l.125). However, as we consider multi-dimensional treatments we can make the exclusion restriction more realistic as we may add all known factors via which $Z$ may affect $Y$ into $X$. A useful example are orally administered antibiotics in so-called “sub-therapeutic dosages”: with the gut microbiome as $X$, it is reasonable to assume the antibiotics only affect macroscopic health outcomes via changes to the gut microbiome.\n\n**Sensitivity**: Sensitivity analyses are an important direction for further work. Frameworks (e.g., [1]) analyze the sensitivity of IV estimates to exclusion restriction violations, but are limited to linear settings and do not apply here. Extensions to non-linear systems (e.g., [2]) are limited and, in particular, cannot handle multi-dimensional treatments or underspecified settings. Developing such sensitivity analyses for point estimates and especially for bounds as in our setting would be a significant contribution.\nThere is also an active area looking into other types of partial identification in the IV setting (e.g., [3], where partial identifiability stems from dropping the additive noise assumption). While additive (measurement) noise may actually be a defendable assumption in the settings we’re aiming at, major technical challenges remain in the multiply nested optimizations that are required to perform sensitivity analysis (in the case of [3], requiring a nested optimization in and of itself) on top of our adaptive procedure.\n\n**Scalability:**  We conducted higher-dimensional experiments as suggested, presenting results in the main rebuttal pdf. Fig. 1 and 2 show results for settings with $d_z = 5, d_x = 20$ and $d_z=20, d_x=20$, demonstrating the method's scalability beyond low-dimensional settings. In the $d_z=5, d_x=20$ setting, our adaptive approach effectively rules out the functional being negative or zero. The causal MSE is the smallest for our adaptive procedure on average, with the smallest variance across runs.\nWe also report runtimes, highlighting that while runtime does increase in higher-dimensions, this increase is extremely mild. A key driver of the runtime is still the number of samples per experiment. As this may grow exceptionally large in certain applications, one may leverage existing techniques to handle the inversion of large kernel matrices, i.e. [4, 5]. Importantly, we note that computational cost is mostly a concern in synthetic experiments, as the time and cost to run physical experiments in real-world applications dwarf computational cost, which are at most polynomial in their input size. We will include these clarifications as well as the additional experimental results.\n\n**Causal Bayesian Optimization**: While there are clearly parallels between our work and Causal Bayesian Optimization, we believe that the setup (and thus the resulting methodology) are still distinct:\n(a) Their main goal is to maximize a target outcome, i.e., to set variables to certain levels that maximize the expected value of another variable $Y$ in the causal graph. Our goal is to maximally inform a causal mechanism (the function that determines $Y$) with no regard for what value $Y$ takes.\n(b) They assume feasible interventions on all graph variables. In contrast, we note that such interventions are often unrealistic, and experimentation amounts to “perturbations” (instrumenting) rather than “intervening”. If interventions on any variable were possible, we would directly intervene on $X$ for a trivial solution. Note that direct intervention allows them to consider general graphs, while our methodology pertains to perturbing/instrumenting a single treatment that directly affects the outcome.\n(c) In contrast to their approach, we do have a path dependence in our setting, i.e. our objective function does depend on the experiments we have performed so far. Due to this difference, our adaptive algorithm is closer to reinforcement learning instead of bayesian optimization or active learning where the objective function does not change depending on previous trials.\nWe will add Causal Bayesian Learning to the related work section and clarify these differences.\n\n**global causal queries**: To identify global functionals, $h(z)$ with $z \\sim \\pi$ must put sufficient mass on all relevant treatment space regions for $Q$. If this region is large, more samples per experiment may be needed for reliable estimates. In practice, especially when $Z$ is much lower-dimensional than $X$, finding perturbations that cover the relevant $X$ regions can be difficult. In general, bounding nonlinear global functionals is possible, but still remains an important direction for future work, except for the linear case [6], where global properties can be inferred via linear extrapolation.\n\n**active learning**: See response to Causal Bayesian Optimization. One additional difference is the assumption of no unobserved confounding in active learning which we can relax via the use of instrumental variables. We will clarify these differences in the manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "66gSdgLgHr", "reviewer": "Reviewer_sEkT", "summary": "This paper designs comprehensive experiments that maximize the information gained about the query of interest within a fixed budget of experimentation, including nonlinear, multi-variate, confounded settings.", "questions": "1. Are there any crossing problems when minimizing the optimization problem in Equation (5)? How do we ensure that $Q^{+}(\\pi)$ is always larger than $Q^{-}(\\pi)$? \n2. Theoretically, are there any requirements for the dimensions of $Z$ and $U$? In your experiments, you only consider a low-dimensional setting. Can you add any experiments for high dimensions for $Z$ and $U$? \n3. For Theorem 2, the proposed method involves several tuning parameters, such as $λ_g,λ_f,λ_c$. This step can be quite worrying for the practical implementation of the proposed method. 1) it can be time-consuming and 2) setting the candidate values is likely quite subjective. The authors should conduct more thorough studies and provide more guidelines on how the choice of tuning parameters affects the method's performance. For example, the authors have selected $λ_g=0.1,λ_f=0.1,λ_c=0.1$, and the learning rate $α_t=0.01$. This selection process appears too subjective. \n4. The experiments are not sufficient for several reasons: (1) The number of replications used in the experiments is quite small (only 25), which raises concerns about the time efficiency of the proposed method. (2) The paper does not include a real-world data analysis to validate the effectiveness of the proposed algorithm from a practical perspective.", "limitations": "1. Theoretical requirements for the dimensions of $Z$ and $U$ are not discussed. Experiments only consider a low-dimensional setting, see questions\n2. The proposed method involves several tuning parameters, which can be problematic for practical implementation due to time consumption and the subjective setting of candidate values; see questions.\n3. Insufficient Experiments: see questions.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "This paper is well-motivated and relevant to the causal inference. Overall, this work is well-written and organized.", "weaknesses": "1. Are the theoretical results practically useful when addressing real-world problems?\n2. How does the proposed method handle non-convex optimization, and what are the assumptions on the loss function?\n3. Can the authors provide more intuitive theoretical explanations of the difference between convex and non-convex optimization in their settings?\n4. How can the effectiveness of the proposed method be guaranteed when the dimension p is significantly larger than the number of training samples?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their useful comments and address the mentioned concerns/questions one by one.\n\n## Weaknesses\n\n1 **Practical usefulness**: Yes, we do believe our contribution is practically useful. In particular, academia (e.g., the “A-Lab” at the Berkeley lab) and industry (big pharma and biotech startups) alike work feverishly on AI driven “automated labs” to accelerate drug, material, or molecule design and scientific discoveries. Efficient and targeted adaptive algorithms to inform subsequent experiments remain a major open challenge. Notably, our targeted formulation (aiming at a specific property of the mechanism) and allowing for non point-estimable properties (via bounding) are major steps towards this goal.\n\n2+3 **Convexity**: We are not entirely sure which (non-)convexity is being referred to, and appreciate a clarification if we are answering the wrong question.\nThere are multiple optimizations. First, there is the minimax problem (eq. (4)) required to obtain a single upper/lower bound on $Q[f_0]$ under a fixed policy. Thm 1 and Thm 2 show that both the inner supremum (eq. (7)) as well as the outer minimization (eq. (9)) can be solved analytically within RKHSs. Hence, we do not see any issues around (non-)convexity for those. The remaining optimization is to update the policy $\\pi$ from one round to the next to minimize $\\Delta (\\pi)$. This is an optimization over the space of distributions on $Z$ with the loss function being given by eq. (9) and estimated from finite data in practice. Depending on how we parameterize $\\pi$ in practice and what we choose for $Q$, this loss function need not be convex in the optimization parameters and we optimize it with a local gradient-based method with no guarantees of finding a global optimum. Importantly, the upper/lower bounds estimation at each step is consistent so our method always provides (narrowing) valid bounds. They may just not be as narrow as they could have been. In practice, we do achieve informative bounds throughout.\n\n4 **p > n:** We assume that $p$ refers to $d_x$, the treatment dimension? We are relying on kernel ridge regression for the estimation tasks from $X$ to $Y$, i.e., mapping $\\mathbb{R}^{d_x} \\to \\mathbb{R}$. Since intuitively this means that the $d_x$ features are embedded into an infinite-dimensional feature space, regularization is required in any case to deal with the “sparsity” (more features than samples), which is why we use kernel ridge. Increasing $d_x$ beyond the sample size $n$ is thus no issue in our framework and taken into account by the regularization.\n\n## Questions\n\n1 **Crossing:** The optimization problem in eq. 4 is the Lagrangian multiplier formulation of maximizing/minimizing $Q[f]$ over$\\mathcal{F}$ subject to the moment conditions ($\\mathbb{E}[(Y - f(X))g(Z)]$). Therefore, $Q^+$ and $Q^-$ are the maximum and minimum over the same set $\\mathcal{F}_c$. Therefore, $Q^+ \\ge Q^-$ (no crossing) is guaranteed. In practice, our estimators (inevitably) have non-zero finite sample variance, which can lead to crossing only when (a) the gap is already very small (we have essentially identified $Q[f_0]$ already), or (b) the sample size is very small (diagnosable from large variance in estimates across rounds).\n\n2 **Dimensionality of $U$ and $Z$:** In general, there are no theoretical requirements on the dimensionality of $U$ and $Z$. The IV setting allows for arbitrary confounding $U$ of any dimension between $X$ and $Y$ and we never explicitly need $U$ in our framework (as it is unobserved). Typically a large number of instruments $Z$ is desirable as having more instruments helps identification. We primarily focus on the more difficult (yet more realistic) setting where $d_z < d_x$ and thus underspecification is likely. However, our framework equally applies to $d_z \\ge d_x$, where identification is generally going to be easier. (The full theoretical conditions for identification are Sec 3.1.)\nFollowing the suggestion, Figs. 1 and 2 in the rebuttal pdf show results for higher-dimensional settings with $d_z = 5, d_x = 20$ and $d_z=20, d_x=20$, demonstrating that our method indeed scales to such settings while remaining informative. In the $d_z=5, d_x=20$ setting, our adaptive approach is the only one that essentially rules out negative values and the causal MSE is smallest both on average as well as in variance for our adaptive procedure.\nFig. 3 in the rebuttal pdf reports runtimes, demonstrating the computational feasibiilty. The number of samples per experiment is the key driver of computational cost, which can be easily reduced by deploying established techniques to handle the inversion of large kernel matrices [1, 2]. We note that computational cost and time are likely negligible compared to the cost and time of physical experiments in real-world applications.\n\n3 **Hyperparameter Tuning:** Since the three mentioned hyperparameters (the different $\\lambda_s$s) are relative weights, we can set $\\lambda_s := \\frac{\\lambda_f * \\lambda_g}{ \\lambda_c}$ and only tune $\\lambda_s$ and $\\lambda_c$. Intuitively, $\\lambda_s$ regularizes the smoothness of the function spaces and $\\lambda_c$ weighs the functional $Q$ relative to the moment conditions. In Fig.  4 in the pdf, we show the dependence of our bounds on these hyperparameters: very low values of $\\lambda_s$  lead to conservative bounds whereas large values of $\\lambda_s$ yield narrow bounds throughout, as $\\lambda_s$ effectively controls the size of the search space via regularity restrictions. We note that there are substantial ranges of reasonable hyperparameter choices where our bounds are fairly insensitive to the exact choice.\nIn practice, all learning rates below a certain value work. More gradient update steps are required for convergence for smaller learning rates and – as in most (stochastic) gradient based optimizations – one chooses a learning rate just below a value that yields reliable convergence.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank the reviewers for their valuable feedback. We included the pdf showcasing various additional experiments in response to specific questions and kindly refer to the individual responses. Due to the space restrictions for the individual rebuttals, we had to heavily compress our replies to the point where we could not express everything we planned to comment on and important details or nuance may have gotten lost. We are more than happy to elaborate on any of the answers in more detail.

Meta-review: All reviewers agree that this is a solid paper with novel and interesting contributions. There are certainly some weaknesses, such as the limited experimental validation involving only  synthetic and low-dimensional settings, or questions about the validity of some assumptions. On the other hand, the authors could address at least some of these critical comments in a rather convincing way, so that for me as an AC, in the end the positive aspects clearly outweigh the points of criticism. Therefore I recommend acceptance of this paper.
Decision: Accept (poster)