Targeted Sequential Indirect Experiment Design

Authors: elisabeth  ailer
niclas  dern
jason  hartford
niki  kilbertus
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 16:26:22

Abstract: Scientific hypotheses typically concern specific aspects of complex, imperfectly understood or entirely unknown mechanisms, such as the effect of gene expression levels on phenotypes or how microbial communities influence environmental health. Such queries are inherently causal (rather than purely associational), but in many settings, experiments can not be conducted directly on the target variables of interest, but are indirect. Therefore, they perturb the target variable, but do not remove potential confounding factors. If, additionally, the resulting experimental measurements are high-dimensional and the studied mechanisms nonlinear, the query of interest is generally not identified. We develop an adaptive strategy to design indirect experiments that optimally inform a targeted query about the ground truth mechanism in terms of sequentially narrowing the gap between an upper and lower bound on the query. While the general formulation consists of a bi-level optimization procedure, we derive an efficiently estimable analytical kernel-based estimator of the bounds for the causal effect, a query of key interest, and demonstrate the efficacy of our approach in confounded, multivariate, nonlinear synthetic settings.

Main paper: 
 introduction: 
 Experimentation is the ultimate arbiter of scientific discovery. While advances in machine learning (ML) and ever increasing amounts of observational data promise to accelerate scientific discovery in virtually all scientific disciplines, all hypotheses ultimately have to be supported or falsified by experiments. But the space of possible experiments is combinatorial, and as a result, experimentation in the physical world may become a major bottleneck of the scientific process and critically inhibit, for example, fast turnaround in drug discovery loops even in fully automated labs. Efficient adaptive data-driven strategies to propose the most useful experiments are thus of vital importance.Targeted experimentation requires a well-formed hypothesis about the underlying system. We focus on hypotheses that are typically expressed in terms of causal effects or properties of functional relationships. For example, we may posit that there is some ground truth mechanism/function f that describes how a given phenotype depends on gene expression levels in a cell, or how the composition of microbes affects certain environmental health indicators. In the natural sciences, such mechanisms are typically (a) nonlinear, (b) dependent on multi-variate inputs, and (c) confounded by additional, unobserved quantities. For example, the way in which the gut microbiome composition affects energy metabolism in humans is likely confounded by various environmental and lifestyle choices. Due to these factors, f is generally unidentifiable, i.e., we cannot hope to infer f from (even infinite amounts of) observational data alone.Scientific queries are typically more narrow: 'Can I expect changes in body mass index when I increase the relative abundance of Lactobacillus bacteria in the gut? How does the susceptibility of breast cancer depend on BRCA1 expression levels?' Such targeted scientific queries do not require full knowledge of the underlying mechanism f , but only concern a specific aspect of f , which we can express mathematically as a functional Q of f . Due to potential unobserved confounding, even targeted queries Q[f ] may not be identified from observational data alone (Bennett et al., 2022) -experimentation is required. In these examples, the inputs to f of interest -which we call treatments -are multi-variate measurements such as gene expression levels or microbiome abundances. Randomizing these treatment variables is typically thought of as the gold standard for removing confounding effects, but perfect randomization is typically impossible: we can seldom perform hard interventions that set a distribution over these variables' outcomes independently of confounding factors. Instead, experimental access in the natural sciences often amounts to limited possible perturbations of the system that induce distribution shifts of the treatments. For example, while we can administer antibiotics with very predictable effects on certain microbes, such an intervention does not remove confounding effects from lifestyle choices such as nutrition and exercise or environmental factors. Therefore, we think of experimentation as a source of indirect interventions, which strongly perturb the distribution over the treatment variables of interest.Our primary goal is to design experiments that maximally inform the query of interest, Q[f ], within a fixed budget of experimentation. For multi-variate treatments and nonlinear f the query Q[f ] may not be identifiable at all, or only be identified after an infeasible number of experiments. We address this by maintaining an upper and lower bound on the query, and sequentially selecting experiments that minimize the gap between these bounds. We show that by treating experiments as instrumental variables (IV) we can estimate the bounds by building on existing techniques in (underspecified) nonlinear IV estimation. Our procedure involves a bi-level optimization, where an inner optimization estimates the bounds, and an outer optimization seeks to minimize the gap between the bounds. We show that if we are prepared to assume that f lies within a reproducing kernel Hilbert space (RKHS), then there are queries Q[f ] for which we can solve the inner estimation in closed form. In summary, we make the following contributions:• We formalize the problem of indirect adaptive experiments in nonlinear, multi-variate, confounded settings as a sequential underspecified instrumental variable estimation, where we seek to adaptively tighten estimated bounds on the target query. • We derive closed form expressions for the bound estimates when assuming f to lie within a reproducing kernel Hilbert space (RKHS) for linear Q. • We develop adaptive strategies for the outer optimization to iteratively tighten these bounds and demonstrate empirically that our method robustly selects informative experiments leading to identification of Q[f ] (collapsing bounds) when it can be identified via allowed experimentation.2 Problem Setting and Related Work 
problem setting: 
 Data generating process. We assume the following data generating processEQUATIONwhere we have experimental access to Z ∈ Z ⊂ R dz , X ∈ X ⊂ R dx are the actual inputs to the mechanism of interest, i.e., the 'treatments' (in the cause-effect estimation sense) or 'targets' (in the indirect experimentation sense), Y ∈ Y ⊂ R is the scalar outcome, e.g., the phenotype of interest, and U is an unobserved confounding variable with E[U ] = 0. The function h determines how experiments (choices of Z) affect treatments X and can be arbitrary. The function f 0 is the mechanism of interest for our scientific query and can also be arbitrary. As in typical indirect experiment settings, we assume that Z consists of valid instrumental variables: (1) Z ̸⊥ ⊥ X, i.e., we do perturb the distribution of X by experimenting on Z, (2) Z ⊥ ⊥ U , i.e., our choice of experiments is not influenced by the confounding variables, and (3) Z ⊥ ⊥ Y | X, U , i.e., Z only affects Y via X and does not have a direct effect on Y . When X is multi-variate and f 0 , h are allowed to be non-linear, f 0 is commonly not identified form observational data. However, we argue that in such settings one should be aiming for a more targeted query, i.e., does not aim to identify f 0 fully, but only a certain aspect of the mechanism.Scientific queries. We represent the scientific query of interest by a functional Q : F → R where F ⊂ L 2 (X) is the space of considered mechanisms f .1 Generic functionals can measure both local as well as global properties of any f ∈ F: simple average treatment effects viaQ[f ] := E[Y |do(X = x * )] = f (x * );when F is a Hilbert space, projections of f onto a fixed basis function f * via Q[f ] := ⟨f * , f ⟩ F /∥f ∥ 2 F ; the local causal effect of an individual component X i on Y at x * via Q[f ] := (∂ i f )(x * ). 2Here, x * could be the mean of the observed treatments, i.e., represent a 'base gene expression level' and we are interested in how a local change away from that base level affects the outcome. While our methodology applies to any functional, we focus our empirical experiments on causal effects of individual components as key scientific queries, such as how does up-or down-regulating a given gene affect the phenotype?Learning experiments. Our goal is to sequentially learn a policy π ∈ P(Z)3 such that data sampled from the joint distribution P π (X, Y, Z) = π(Z)P (X | Z)P (Y | X) induced by the model in eq. ( 1) optimally informs Q[f 0 ]. We highlight that depending on f 0 , h and the distribution of U , there may exist a policy π such that Q[f 0 ] is identified from P π (X, Y, Z), but in general it may remain unidentified for all policies even in the infinite data limit. For a non-optimal policy, i.e., noninformative experimentation, we should expect Q[f 0 ] to be partially identified from P (X, Y, Z) at best. Therefore, we aim to estimate upper and lower boundsQ + (π), Q -(π) of Q[f 0 ] and sequentially learn the policy π that minimizes ∆(π) := Q + (π) -Q -(π).In each round t ∈ [T ] := {1, . . . , T }, we observe n i.i.d. samples from P πt (X, Y, Z) using policy π t , which we use (potentially together with data collected under previous policies π <t ) to estimate Q + (π t ), Q -(π t ) and then aim to propose a new policy π t+1 that yields a smaller gap in bounds: ∆(π t+1 ) < ∆(π t ).Connections to other settings. Besides the 'indirect experimentation' lens, the setting in eq. ( 1) can be viewed from different perspectives. While they all share the same mathematical formulation, they start from different conceptions of Z. The literature on invariant causal learning (Peters et al., 2016; Heinze-Deml et al., 2018) may interpret Z in eq. ( 1) as an environment indicator and f 0 is invariant across these environments. Each policy corresponds to an environment (Z cannot be designed, but we can collect data for different Z) with its own distribution of X. The heterogeneity across environments can be leveraged to learn about the invariant mechanism f . In instrumental variables (Pearl, 2009; Angrist & Pischke, 2008) , one typically assumes to encounter a situation as in eq. ( 1) with the assumptions (1)-( 3) where X, Y, Z are observed 'in the wild'. Valid instruments Z then sometimes allow for (partial) identification and estimation of f 0 despite the unobserved confounder U . Again, there is typically no active experimentation in IV estimation, but Z is taken 'as is'. Finally, experimentation on Z in our setting can also be interpreted as soft interventions (Jaber et al., 2020; Pearl, 2009) on X, where we intervene on X setting it to follow a given distribution instead of a fixed value. Instead of allowing arbitrary soft interventions, eq. (1) restricts us to distributions P (X | Z)π(Z) for feasible experimental policies π(Z). Importantly, unlike proper soft interventions, we do not get rid of unobserved confounding. Ultimately, we face a sequence of IV estimation tasks. Experimental access to Z justifies the instrumental variable assumption (2) Z ⊥ ⊥ U , and indirect experiments are setup by design such that (1) Z ̸⊥ ⊥ X holds. Assumption (3) Z ⊥ ⊥ Y | X, U instead may be hard to justify in practice and limits the types of experimentation allowed in our framework.IV estimation. Instrumental variables are a cornerstone of cause-effect estimation in econometrics (Angrist & Pischke, 2008) at least since their use by Philip G. Wright in 1928 (Wright, 1928; Stock & Trebbi, 2003) . Even though valid instruments are hard to come by in practice (Hernán & Robins, 2006) , generalizations to settings with relaxed assumptions have sparked renewed interest in IV estimation from the machine learning community not least due to successful applications in Mendelian randomization (Sanderson et al., 2022; Didelez & Sheehan, 2007; Legault et al., 2024) or on microbiome data (Sohn & Li, 2019; Wang et al., 2020; Ailer et al., 2021) . In particular, we build on recent advances in nonlinear/nonparametric IV estimation (Newey & Powell, 2003; Lewis & Syrgkanis, 2018; Singh et al., 2019; Hartford et al., 2017; Saengkyongam et al., 2022; Zhang et al., 2020; Xu et al., 2020) , with a specific focus on the minimax formulation using the generalized method of moments (Liao et al., 2020; Dikkala et al., 2020; Bennett et al., 2023; Zhang et al., 2023; Liao et al., 2020; Bennett et al., 2019; Muandet et al., 2019; Bennett et al., 2022) . In addition, we do not assume identifiability in line with recent approaches to partial identification and bounding causal effects in IV settings relaxing various discreteness and additivity assumptions (Kilbertus et al., 2020; Padh et al., 2022; Frauen et al., 2024; Melnychuk et al., 2024; Wang & Tchetgen Tchetgen, 2018; Gunsilius, 2019; Hu et al., 2021; Zhang & Bareinboim, 2021) .Sequential experiment design. The work closest to ours is perhaps by Ailer et al. (2023) , where they consider a similar setting of sequential indirect experiment design via IV estimation. The key differences are that they only consider a fully linear setting (h, f 0 linear), aim for full identification of f 0 (even though the setting is underspecified), and only provide non-adaptive strategies, i.e., the sequence of experiments does not depend on data collected throughout the experiments. Other work on indirect experimentation either assumes no unobserved confounding (Singh, 2023) or focuses primarily on sample efficiency and variance reduction when aiming for full identification of f (Chandak et al., 2024) . Adaptive learning has also been used in another context for cause effect estimation by 'deciding what to observe ' Gupta et al. (2021) . While clearly related, we highlight that most of the literature on active experimentation for causality aims at learning the causal structure (instead of estimating properties of f ) from access to interventions, i.e., direct experiments (instead of more realistic indirect experiments), e.g., (von Kügelgen et al., 2019; Sverchkov & Craven, 2017; Agrawal et al., 2019; He & Geng, 2008; Gamella & Heinze-Deml, 2020; Elahi et al., 2024; Zemplenyi & Miller, 2021) . Additionally, we require a strategy that is path-dependent, while in active learning the objective function remains the same over the different iterations.Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: The conclusion includes an additional paragraph stating the limitations of our work. Moreover, we acknowledge the open question and explain a potential way how to address those. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
minimax instrumental variable estimation: 
 For the general instrumental variable setting in eq. ( 1) that we face in each round, we aim to solveE[Y -f (X) | Z] = 0.To solve this, we follow the conditional moment formulation (Bennett et al., 2019; Dikkala et al., 2020; Bennett et al., 2022 Bennett et al., , 2023)) , which builds on the observation that we only want to consider functions f , such that the residuals, U f := Y -f (X) are independent of the instrument Z. If two random variables are independent,U f ⊥ ⊥ Z, then E[g(Z)U f ] = E[g(Z)]E[U f ]for all test functions g. We search for f that minimize the residual while maintaining the independence constraint by solving a minimax optimization that minimizes f over a worst case g. The integral equation E[Y -f (X) | Z] = 0 for f can be written as T f = r 0 , where T is a linear, bounded operator mapping f :X → R to T f := E[f (X) | Z] : Z → R and r 0 = E[Y | Z]. This is typically an ill-posed inverse problem where both T and r 0 are not known but have to be estimated from i.i.d. data D = {(x i , y i , z i )} n i=1 . Assuming f to come from a set of hypothesis functions F ⊂ L 2 (X) and G ⊂ L 2 (Z) a class of 'witness functions' or 'test functions', we can write the non-parametric IV problem as a minimax optimization problem of the form (Dikkala et al., 2020; Bennett et al., 2023) arg infEQUATIONfor some objective function L mapping tuples of hypotheses f and test functions g to R. While most existing approaches start by assuming that there exists a unique solution of T f = r 0 (Lewis & Syrgkanis, 2018; Liao et al., 2020; Muandet et al., 2019; Dikkala et al., 2020; Bennett et al., 2019) , based on Lagrange multipliers Bennett et al. (2023) recently have shown that under a source condition on T and mild realizability assumptions regarding the capacity of the (statistically restricted) function classes F and G (Bennett et al., 2023, Assumptions 2, 3, 4) , the solution of T f = r 0 is given byf 0 = arg min f ∈F max g∈G 1 2 ∥f ∥ 2 L2(X) +⟨r 0 -T f, g⟩ L2(Z) = arg min f ∈F max g∈G 1 2 E[f 2 (X)]+E[(Y -f (X))g(Z)].(3) This formulation of the objective is immediately amenable to empirical estimation from D by using empirical averages for the expectations. Eq. (3) can be viewed as a penalized optimization, where 1 2 E[f 2 (X)] penalizes the solution towards the minimum L 2 norm. The inner maximization over test functions ensures that we only consider hypotheses f ∈ F compatible with the conditional moment restrictions that enforce the required independence of the residuals. While there may still be multiple hypotheses f ∈ F compatible with observational data in this way, Bennett et al. pick the f with the least L 2 norm, yielding a unique solution. In the following, we build on this intuition to estimate bounds on a chosen scientific query Q among all f compatible with the observed data and assumed structure (via the moment restrictions).The source condition required for eq. ( 3) is that r 0 lies in the range of T T * , where T * is the adjoint of T . As Bennett et al. point out, this is a nontrivial restriction and paramount in ensuring uniqueness in eq. ( 3). Without this source condition the IV problem is still ill-posed in that there may be multiple solutions f ∈ F compatible with the observed data D (even in the infinite data limit). One of our key realizations is that we can still meaningfully make use of the penalized optimization formulation in eq. ( 3) to compute bounds on targeted queries about f . Let Q : F → R be a functional on hypotheses F that captures the scientific query. For any given joint distribution P π (X, Y, Z) induced by experimental policy π, to bound Q[f 0 ], we can computeEQUATIONwhere again the inner maximization aims at only considering hypotheses compatible with the conditional moment restrictions F c ⊂ F and the outer minimization aims at finding the f ∈ F c with the largest/smallest value of Q[f ]. The realizability assumption that f 0 ∈ F then clearly implies that f 0 ∈ F c and the bounds Q ± will contain Q[f 0 ]. We note that specially when X is of higher dimension than Z, in practice neither f nor the much more narrow query Q[f ] are guaranteed to be identified. However, depending on π, we may shape P π such that F c becomes restricted enough to obtain informative bounds Bennett et al. (2022) work out in detail when a linear functional of f 0 , but not f 0 itself, is identified. If P π is such that T satisfies the source condition, certain queries Q (such as the Q[f ] = 1 2 ∥f ∥ 2 L2 as a canonical example) may actually be identified and the bounds Q ± will coincide.Q ± on Q[f 0 ].We can now formulate our main policy learning goal asinf π∈Π(Z) ∆(π) ,whereEQUATIONwith Π(Z) ⊂ P(Z) being a subset of implementable experimentation policies. In practice, we aim at approximating this optimization via sequential updates of π t over T rounds, where we observe n i.i.d. samples of P πt at each round. This means that given π t , we seek to choose π t+1 such that ∆(π t+1 ) < ∆(π t ). The final output of our method is the interval[Q -(π T ), Q + (π T )] ∋ Q[f 0 ].A key advantage of this formulation is that we need not assume identifiability of f nor of Q[f ], will obtain valid (albeit potentially loose) bounds when the experimentation budget T is restricted, and that potential identifiability will be 'captured automatically' by the bounds coinciding. If after T rounds the bounds are still not informative, the experimenter can choose more expressive experiments (different Π(Z) and even different Z), a more specific query Q, or put stronger assumptions on the hypothesis class F.Let us unpack the sequential policy learning problem. At each round t ∈ [T ], we need to solve two minimax problems for Q ± (π t ) where the objective is estimated from finite data and then take a minimization step over the difference of the two solutions. To further complicate this 'tri-level' (min + minimax) optimization problem, we are optimizing over two function spaces (F, G) and a family of distributions (Π(Z)). In the remainder of this section we develop techniques to render these optimizations feasible in practice for suitable choices of F, G, Π(Z).Existing solutions for the minimax problems typically employ adversarial optimization (iterative gradient-based optimization over parametric function spaces) or kernel-based techniques. Since the former are usually hard and expensive to optimize reliably even without the outer minimization, we focus on the latter approach using reproducing kernel Hilbert spaces (RKHS) for test functionsG = H Z induced by a characteristic kernel k Z : Z × Z → R.Then there exists a closed form expression for a consistent estimate of the inner supremum.Theorem 1 (Zhang et al. (2020)) .If E[(Y -f (X)) 2 k Z (Z, Z)] < ∞, 1 n 2 n i=1 n j=1 (y i -f (x i ))k(z i , z j )(y j -f (x j )) 1 2 (6) consistently estimates sup g∈H Z ,∥g∥≤1 ⟨r 0 -T f, g⟩ H Z from data D.Hence, for n samples D iid ∼ P π we reduce the minimax optimizations in eq. ( 4) toEQUATION)One option to solve eq. ( 7) efficiently is via gradient-based methods leveraging highly efficient auto-differentiation packages for parametric function families, such as choosing neural networks for F, i.e., we optimize over a finite-dimensional real-vector θ-the weights of the neural network. Our framework may also benefit from other existing approaches to the minimax optimization from the literature, e.g., in Muandet et al. (2019) ; Dikkala et al. (2020) . Parametric functions may have the advantage that they render the computation ofQ[f θ ] feasible. For example, the causal effect of X i on Y at x * given by Q[f θ ] = (∂ i f θ )(x *) can be obtained directly from automatic differentiation as well. Global functionals such asQ[f θ ] = X ψ(x)f θ (x)dx for some function ψ may pose greater difficulties as the integral over X is typically infeasible. Still, this approach ultimately requires a bi-level optimization with a substantial amount of tuning.In order to also obtain a closed-form estimate of the minimization over F, we have to specify the functional Q. To this end, we focus on bounded linear functionals. One example is the causal-effect of an individual input (∂ i f )(x * ), because it is arguably one of the most common and relevant scientific queries: how does a small change of a specific variable affect the outcome? When following a fully non-parametric approach by also assuming hypotheses to come from an RKHS F = H X induced by a characteristic kernel k X : X × X → R, we can also estimate the minimization over F in closed form for the causal effect-a local, linear functional. The key realization is that functionals of functions represented as linear combinations of RKHS functions can be written as linear functions of the coefficients. Theorem 2. Assume functions in H Z , H X to have bounded variation, (w.l.o.g.) images contained in [-1, 1], and for h ∈ H Z also -h ∈ H Z . Denote by K XX , K ZZ ∈ R n×n the empirical kernel matrices of D, let k X be continuously differentiable, and let λ g , λ f , λ c ≥ 0. Further, fix Q[f ] to be a bounded linear functional, and writef θ (•) = n i=1 θ i k(x i , •).Then the solutions to the regularized minimax problemsf ± = arg min f ∈H X ∓Q[f ] + λ c sup g∈H Z ⟨r 0 -T f, g⟩ H Z + λ f ∥f ∥ H X -λ g ∥g∥ H Z (8)are consistently estimated by f θ± withEQUATIONwhere(Q[K X• ])(x * ) ∈ R n is the vector ((Q[k X (x 1 , •)])(x * ), . . . , (Q[k X (x n , •)])(x * ).Note that these general bounded linear functionals can be computed analytically for many common kernels such as linear, polynomial, or radial basis function (RBF) kernels.One common example is the partial derivative at x * :EQUATIONwhere(∂ i K X• )(x * ) ∈ R n is the vector ((∂ i k X (x 1 , •))(x * ), . . . , (∂ i k X (x n , •))(x * ).We defer all proofs of our theoretical statements to Appendix A.We can now substitute these closed-form estimates for the minimax problems into our policy learning objective 5)∆(π) = Q[f θ+ ] -Q[f θ-] in eq. (EQUATION(In)validity of bounds. Our general policy learning formulation in eqs. ( 4) and ( 5) involves nonlinear, nonconvex optimizations over function spaces and families of distributions. Since we cannot provably obtain global optima to those, we cannot guarantee valid bounds. Even for the RKHS setting in Theorem 2 for linear Q, while the estimates θ± are consistent, for finite sample estimation we have to regularize both the hypotheses and witness functions (λ g , λ f > 0) to obtain numerically stable and reliable estimates. Clearly, those pose restrictions on the search spaces for f, g and may thus lead to invalid bounds. We may expect real-world mechanisms f 0 to be relatively smooth such that these regularization terms do not render our estimates invalid in practice. Moreover, since we made no explicit assumptions about the functional Q, using Q[f ] as a penalty (as compared to, e.g., 1 2 ∥f ∥ L2 ) may not yield a unique solution to eq. ( 8) even in the infinite data limit and for λ g = λ f = 0. While uniqueness can be recovered for strictly convex, coercive, and lower semicontinuous Q, this implies non-trivial restrictions on the allowed scientific queries, which we may not be willing to tolerate. Therefore, we introduce a 'penalization parameter' λ c (in front of the supremum term rather than Q[f ] for convenience) that allows us to empirically trade off the scale/importance of Q[f ] and the conditional moment restriction in practice. Enforcing conditional moment restrictions via a minimax formulation from finite samples typically exhibits high variance. Therefore, a lack of provably valid bounds due to practical regularization does not impair the usefulness of our method in a setting where we aim to learn about a potentially unidentified scientific query from limited experimental access. In particular, we argue that overly conservative bounds are still more useful than likely invalid point estimates based on one-size-fits-all assumptions required for identifiability.Continuing with the closed-form expression in eq. ( 11) for the gap between maximally and minimally possible values of the scientific query -here the causal effect of changing one treatment component around a base level X * -over all hypotheses that are compatible with the observed data, we can now seek to represent the experimentation strategy π in a way that allows us to sequentially improve it.We now describe different approaches to sequentially update the experimentation policy. The experimentation budget T and the number of samples per round n are fixed. We denote by π t the policy in round t and by D (t) (D (≤t) ) the data collected in (up to) round t. We denote by Q ± t and ∆ t the bounds and gap between them, respectively, estimated in round t and explicitly mention which data was used for the estimates. Some strategies rely on a 'meta-distribution' over policies Π(Z), which we denote by Γ. A sample π ∼ Γ is thus a policy in Π(Z) ⊂ P(Z). While the 'final result' of all strategies are just the final bounds Q ± T , we report Q ± t also for suitable t < T for different strategies to compare efficiency.Simple baseline. In modern methods for experiment design, random exploration is often surprisingly competitive (Ailer et al., 2023) . We consider a simple entirely non-adaptive baseline called random, which independently samples π t ∼ Γ for t ∈ [T ] and estimates Q ± t on D (≤t) . Locally guided strategies. Next, we look to simple adaptive strategies that leverage the locality of Q. If Q is determined in a small neighborhood around some x * ∈ X , a useful guiding principle for π is to aim at concentrating the mass of the marginal P π (X) around x * . The causal effect(∂ i f )(x * ) or the average treatment effect E[Y | do(X = x * )]are examples of local queries. Estimating such local relationships in indirect experimentation without unobserved confounding has recently been studied from a theoretical perspective by Singh (2023) . They propose a simple explore-then-exploit strategy and prove favorable minimax convergence rates depending on the complexities of h and f 0 as well as the noise levels in the unconfounded setting, i.e., X and Y have independent noises and h does not depend on U . We highlight that these strategies do not use the intermediate estimated bounds as feedback to select the next policy π. Pseudocode for the different strategies is in Appendix G.1. Explore then exploit (EE) (inspired by Singh (2023) ): Split the budget into T = T 1 + T 2 . Sample π t ∼ Γ for t ∈ [T 1 ]. Then, fit π as a (parametric) distribution on the Z values of the K ≤ T 1 • n samples in D (≤T1) closest to x * (i.e., their X values have the smallest distance to x * for some suitable distance measure on X ). Set π t := π for t ∈ {T 1 + 1, . . . , T } and estimate t) . The split between T 1 and T 2 may be informed by n to maximize exploration while retaining sufficiently many samples during exploitation for a low variance estimate of Q ± T . 2. Alternating explore exploit (AEE): Throughout all rounds, keep a list of all observed samples sorted by the distance of their X values to x * . For odd t ∈ [T ] \ {T } independently sample π t ∼ Γ. For even t ∈ [T ] and t = T fit π t as a (parametric) distribution to the min{K, n • t} nearest samples to x * for some K ∈ N and estimate Q ± t on even t D (t) . There is some freedom in which data is used to estimate Q ± t and using D (≤t) is always a valid choice. For local queries it can still be beneficial in practice to discard data far away from x * . More broadly, the above strategies could potentially be further improved by weighing samples in D (t) inversely proportional to the distance of (the mean of) π t from x * when estimating Q ± t from D (≤t) . We only report the vanilla versions described above in our experiments and leave further variance reduction techniques for future work. In all experiments, we use K = n and the Euclidean distance on X . While these locally guided strategies appear rather limited in which information from previous rounds is used in updating the policy, Singh et al. (2019, Sec. 7 ) make a convincing case for why the simple types of active learning in EE and AEE greatly improve over the naive random strategy and may be competitive with fully adaptive experiments. For the fixed policy in π and the meta-distribution Γ one should arguably err on the side of 'uninformative' priors, i.e., covering all of Z mostly uniformly.Q ± t from T t=T1+1 D (Targeted Adaptive Strategy. We now develop an adaptive strategy denoted by adaptive that actually uses the intermediate estimates of Q ± t to update π t . A natural choice for policy updates is via gradient descent on our objective ∆EQUATIONfor step sizes α t > 0 at round t. In practice, we assume parametric policies π t := π ϕt with parametersϕ t ∈ Φ ⊂ R d .With the log-derivative trick (Williams, 1992) we then computeEQUATION)and can update π ϕt akin to the REINFORCE algorithm with horizon one. Since the gradient in eq. ( 12) is evaluated at π ϕt from which we have samples available, we can directly use the empirical counterpart of the expectation in eq. ( 13). One caveat is that our objective ∆ cannot be computed on the 'instance level' for individual samples (x i , y i , z i ), but is already an estimate based on multiple samples. In practice, we therefore split the n observations per round into random batches, and average over the ∆(π ϕ ) estimate for each batch times the mean score function across the batch in eq. ( 13).An extension to use data D (τ ) from a previous round τ < t is to reweigh the term within the expectation in eq. ( 13) by π ϕt (Z)/π ϕτ (Z) (and then average across rounds). While this allows us to use more data for the gradient estimates, this approach does not yield an unbiased estimator and typically also increases variance due to arbitrarily small reweighing terms. In practice, we may want to only consider data from a small number of previous rounds, where we expect the policy not to have changed too much. Again, we believe our technique could benefit from further variance reduction techniques (Chandak et al., 2024) , but leave these for future work.While our formulation in eqs. ( 12) and ( 13) can be efficiently implemented for any parametric choice of π ϕ , in our experiments we choose a multivariate Gaussian mixture model (GMM)π ϕ (z) = M m=1 γ m N (z; µ m , Σ m ) with weights γ ∈ [0, 1] M with m=1 M = 1, means µ m ∈ R dz , and covariances Σ ∈ R dz×dz for i ∈ [M ]. Hence, the parameters are ϕ = (γ, µ 1 , . . . , µ M , Σ 1 , . . . , Σ M ).With the score function ∇ ϕ log π ϕ known analytically for GMMs, eq. ( 13) can be estimated efficiently (see Appendix B for details).Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [Yes] Justification: (1) Assumptions: we state them directly in the main paper, moreover we provide references to the papers on which our results are based on, including the section of their proofs.(2) Proofs: we sketch the proofs in the main paper and reference to the full proof in the supplementary. Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.• All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 
experiments: 
 Setup. We consider a low-dimensional setting (for visualization purpose) with d x = d z = 2 andh j (Z, U ) = α • (sin(Z j )(1 + U )) , f (X) = β dx j=1 exp(X j ) sin(X j ) + U , U ∼ N (0, 1) , (14)where we use α = β = 20. The local point of interest is x * = 0 ∈ R dx and we easily check that ∂ i f (x * ) = β. We use n = 250 samples in each round over a total of T = 16 experiments.Method parameters. We use radial basis function (RBF) kernels k(x 1 , x 2 ) = exp(ρ∥x 1 -x 2 ∥ 2 ) with a fixed ρ = 1. Note, that the three hyperparameters are relative weights. Thus we set λ s := T at T = 16. The dotted line is y = 0. Both locally guided heuristic (explore-then-exploit, alternating explore exploit) confidently bound the target query away from zero with a relatively narrow gap between them. Our targeted adaptive strategy is even better and essentially identifies the target query Q[f 0 ] = 20 after T = 16 rounds. hyperparameter choices. For all strategies the variance of our policy is set at σ e = 0.001. For explore then exploit, we chose T 1 = 10, T 2 = 6 with π t = N (µ t , σ e Id dz ) and independent µ t ∼ N (0 dz , Id dz ) for t ∈ [T 1 ]. The Gaussian mixture of the adaptive strategy is initialized with M = 3, γ = ( 1 /3, 1 /3, 1 /3), Σ m = Id dz and independent µ m ∼ N (0 dz , Id dz ). We use a constant learning rate α t = 0.01 for all t and restrict ourselves to learning only weights γ m , means µ m and the diagonal entries of the covariances Σ m .Results. We perform n seeds = 50 runs for different random seeds and report means with the 10and 90-percentiles in Figure 1 . The simple non-adaptive random baseline starts out with poor performance as expected, and improves mildly over multiple rounds. We note that the ultimate performance of the random baseline depends primarily on how much mass the random exploration (defined by the meta-distribution Γ) puts near x * , i.e., whether we collect sufficiently many samples near x * over the T rounds. Explore then exploit performs quite well as soon as we start exploitation. Again, whether sufficient informative examples have been collected during the exploration stage depends on the choice of Γ. However, explore then exploit does outperform random (with the same exploration distribution) indicating that the heuristic of focusing on the informative samples (the ones near x * ) does provide substantial improvements (as also found by (Singh, 2023) ). The alternating strategy AEE provides bounds across all rounds and clearly shows step-wise improvement from the first round on-ultimately performing similar to EE. Finally, the adaptive strategy quickly narrows the bounds and achieves a fairly narrow gap ∆ already after few rounds. At T = 16 it has essentially identified the target query Q[f 0 ] = 20 with low variance across seeds. For completeness, we provide a runtime comparison of the different methods in Appendix D. The code to reproduce results is available at https://github.com/EAiler/targeted-iv-experiments.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?Answer: [Yes] Justification: The code will be provided in a public repository. The experiments are performed via a command line run file. The results are saved to a npy-formated data file. One jupyter notebook does perform the visualization. Additionally, we listed the seeds that are relevant for the data generation to ensure full reproducibility. Guidelines:• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 
discussion and conclusion: 
 Summary. We formalized designing optimal experiments to learn about a scientific query as sequential instrument design with the goal of minimizing the gap between estimated upper and lower bounds of a target functional Q of the ground truth mechanism f 0 . We only assume indirect experimental access, allow for unobserved confounding, and consider nonlinear f 0 with multi-variate inputs such that both f 0 and Q[f 0 ] may be unidentified. For a broad set of queries, we derive closed form estimators for the bounds within each round of experimentation when f lies within an RKHS. Based on these estimates, we then develop adaptive strategies to sequentially narrow the bounds on the scientific query of interest and demonstrate the efficacy of our method in synthetic experiments. Given the increased amount of data collected in fully or partially automated labs, for example for drug discovery, we believe that efficient, adaptive experiment design strategies will be a vital component of data-driven scientific discovery.Limitations and future work. We have not validated our method in a real-world setting, as it would require access (and full control) over an actual experimental setup as well as the time and resources required to conduct these experiments. A key technical limitation of our work is that neither our general policy learning formulation in eqs. ( 4) and ( 5) nor the concrete closed form estimates in Theorem 2 obtain provably valid bounds for all queries Q from finite samples (see discussion right before Section 3.4). While our approach is reliable in synthetic experiments, we believe thorough theoretical analysis of necessary and sufficient conditions for (asymptotically) valid bounds (including ideally asymptotic normality with known asymptotic covariance or even finite sample guarantees) is a useful direction for future work beyond the scope of our current manuscript. In practice, the applicability of our approach may also limited by the assumptions that the underlying system is well described via a fixed, static function f 0 : X → Y as opposed to, say, a temporally evolving and interacting systems governed by a differential equation. Similarly, while IV assumptions (1) and (2) can arguably be justified in our setting, the third assumption Z ⊥ ⊥ Y | X, U limits the types of experimentation we can consider (see discussion at the end of Section 1).Methodologically, we believe that our approach could benefit from advanced variance reduction techniques and be sped up by more efficient estimators (Chandak et al., 2024) . Along these lines, it is worthwhile future work to analyze the optimal trade-off between exploration and retaining a large number of samples for exploitation and thus lower variance estimates. Finally, our current empirical validation is limited to linear local functionals, rather simple parametric choices for policies (such as (mixtures of) Gaussians), and we have not optimized the kernel choices and hyperparameters. Hence, a validation where all these choices have been tailored to a real-world application scenario is an important direction for future work.Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification: The data is based on a simulation. The code, as well as the exact command line arguments are on github. Moreover, all code to reproduce the results is openly available and commented.Guidelines:• The answer NA means that paper does not include experiments requiring code. Justification: Our data is evaluated on simulated data, moreover, our algorithm tries to move away from correlational assumptions and tries to identify a part of the causal quantity. Even though our claim is to recover part of the causal effect, we state its limitations. We provide sufficient documentation to reproduce the results, but also to understand and comprehend our claims. In conclusion we are convinced that we compile to the NeurIPS Code of Conduct. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 10. Broader Impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [NA] Justification: Our paper focuses on foundational research. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.• If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 11. Safeguards Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: Neither the simulated dataset, nor the algorithm have a high risk of misuse. The algorithm only adds value as soon as it is used on real world experimental set-ups. Currently, it is only based on our simulation studies. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [NA] Justification: The theoretical contributions on which our work is based are correctly cited in the main paper, the code is written from scratch and solely showcases our proposed algorithm. As packages we are using: Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL.• The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.• If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [Yes] Justification: While our main contribution is not the code that we provide alongside the paper, but only provided to ensure reproducibility, the code is still well documented and ready to use. The consent is given, as the authors of the papers are owners of the code as well.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: In this section we provide proofs of the statements in the main paper. Before proving Theorem 2 we restate a result from the literature that provides closed form consistent estimates of the minimax problem with the L 2 penalty term (instead of penalizing the functional). Theorem 3 (Proposition 10 of (Dikkala et al., 2020) ). Assume without loss of generality that functions in H Z , H X have bounded variation and their images are contained in [-1, 1]. Additionally, assume that for each h ∈ H Z also -h ∈ H Z . Denote by K XX and K ZZ the empirical kernel matrices from D. Then we can consistently estimateEQUATIONwith M = K ZZ . If the function classes F, G are already norm constrained, the estimator needs no penalization.Proof. Note that we have the optimum of the inner supremumEQUATIONTherefore, we are left to solve the outer minimization off = min f ∈F (Y -f (X)) ⊤ M (Y -f (X)) + 4λ g λ f ∥f ∥ 2 F (19) f = min f ∈F (Y -f (X)) ⊤ M (Y -f (X)) + 4λ g λ f ∥f ∥ 2 F min θ∈R n (y -K XX θ) ⊤ M (y -K XX θ) + 4λ g λ f ∥K x θ∥ 2 2|Replacing the function with the (empirical) kernels min f ∈F θ ⊤ K XX M K XX θ -2y ⊤ M K XX θ + 4λ g λ f θ ⊤ K x θ |Writing out product min f ∈F θ ⊤ (K XX M K XX + 4λ g λ f K XX )θ -2y ⊤ M K XX θ |Aggregation of terms wrt θ f = (K XX M K XX + 4λ g λ f K XX ) + K XX M(•) = n i=1 θ i k(x i , •).Then the solutions to the regularized minimax problemsf ± = arg min f ∈H X ∓Q[f ] + λ c sup g∈H Z ⟨r 0 -T f, g⟩ H Z + λ f ∥f ∥ H X -λ g ∥g∥ H Z (8)are consistently estimated by f θ± withEQUATIONwhere(Q[K X• ])(x * ) ∈ R n is the vector ((Q[k X (x 1 , •)])(x * ), . . . , (Q[k X (x n , •)])(x * ).Note that these general bounded linear functionals can be computed analytically for many common kernels such as linear, polynomial, or radial basis function (RBF) kernels.Proof. The solution of the inner supremum remains the same as in Theorem 3. This leaves us to solve the outer minimizationEQUATIONwhere M = K ZZ . We write out f as a linear combination of kernel basis functions (at the observed data points) and use the linearity of the gradient functional, which applied to the kernel basis functions is just(∂ i K X• )(x *). This yields for the overall functionalQ[f θ ] = (∂ i K X• )(x * )θ.We are thus seekingEQUATIONwhich we rewrite asEQUATION)Building on the arguments in the proof of Theorem 3 where we replace the linear part in θ by the partial derivative functional, we obtain the minimizerEQUATIONas a consistent estimator.Proof. We re-iterate the same arguments to prove the theorem for general bounded linear functionals. We assume Q[f ] to be a linear bounded functional acting on f . Theorem 4. Riesz representation theorem Let Q[f ] : H X → R be a continuous linear functional on a separable Hilbert space H X . Then there exists a q ∈ H X such that Q[f ] = ⟨f, q⟩ H X , ∀f ∈ H X .Assuming f ∈ H X , via the Riesz-representer theorem for linear functionals, there exists a unique element q ∈ H X such that for any f ∈ H X . The bounded linear functionals on H X form themselves a hilbert space called the dual space.Q[f ] = ⟨f, q⟩ H X (24)We assume H X to be an RKHS. Therefore, we can write out f (x) as a linear combination of the kernel basis functions (at the observed data points):We replaceQ[f ] = ⟨f, q⟩ H X = ⟨ n i=1 θ i k(x i , •), q⟩ = n i=1 θ i q(x i ) as q ∈ H X .Due to the RKHS, we can write the empirical version as q(•)= n i=1 Q[k(x i , •)]. min f ∈H X θ ⊤ Q[k(x i , •)] + 4λ g λ f θ ⊤ K XX θ + λ c (y -K XX θ) ⊤ M (y -K XX θ) (25)Thus we obtain the minimizerEQUATIONFor completeness, we here recall the score function of Gaussian mixture models, i.e., the gradients with respect to the mixture weights, means, and covariances of the log-likelihood function. We write π for the GMM for brevity. First, we define the 'responsibilities' ζ im , the probability that a sample z i comes from component m via (see, e.g., Petersen & Pedersen (2008) ) EQUATIONEQUATIONEQUATIONEQUATION)In Eq. ( 11) we end up with three hyperparameters. Note however, that those three mentioned hyperparameters are relative weights. Thus, we can set λ s := λ f λg λc and only tune λ s and λ c . Intuitively, λ s regularizes the smoothness of the function spaces and λ c weighs the functional Q relative to the moment conditions. In Fig. 2 , we show the dependence of our bounds on these hyperparameters: very low values of λ s lead to conservative bounds whereas large values of λ s yield narrow bounds throughout, as λ s effectively controls the size of the search space via regularity restrictions. For the learning rate, a lot of learning rates below a certain value actually work for the method. More gradient update steps (and therefore more experiments) are required for very small learning rates, very high learning rates result in drastic updates of the policy; something that is not very favorable in real-world experiments.For completeness, we show wallclock runtimes of the different methods in our empirical evaluation in Figure 3 . All methods were run on a MacBook Pro with Intel CPU. The runtimes are per iteration and clearly increase for methods that accumulate data over previous rounds. Even the most expensive passive baseline is relatively fast to compute without specialized hardware and we generally imagine computational cost to be negligible compared to the actual cost of physical experimentation required in each round in most applications. We assume the same setting as the low-dimensional one, only increasing the dimensions.EQUATIONwith d z = 5, d x = 20 and d z = 20, d x = 20. The setting guarantees, that though we have a mismatch in dimensions, i.e. d z < d x , we would-in principle-still be able to identify the full causal effect, as the completeness property is fulfilled. We refer to Fig. 4 for the results of the different methods. Moreover, we note that the increase in time is still manageable, c.f. Fig 5 . The main driver for the computation time are the samples within each experiment (T 1 , T 2 ) due to the kernel approach and, therefore, the inversion of gram matrices based on the sample size. Additionally, we include some examples along the lines of underspecification. We refer to Ailer et al. (2023) for a thorough discussion on underspecification, but give some motivation for the examples in the following:Since f is the solution to a linear inverse problem, the problem is often ill-posed and thus f is not identified. However, underspecification means something more explicit. It can be seen as a violation of the completeness property. Lemma 5 (Severini & Tripathi (2006) ). The conditional distribution of p(X | Z) is complete if and only if for each function f (x) such that E[f (x)] = 0 and Var(f (x)) > 0, there exists a function g(z) such that f (x) and g(z) are correlated.In most papers, the completeness property is trivially fulfilled by assuming an exponential family for p(x | z) with non-zero variance for all components. This means that both T and T * are injective which again guarantees the identifiability of f . Now, we denote P F as the orthogonal projection onto the function space F and N (T ) as the null space of the linear operator T : Severini & Tripathi (2012) argue that P N (T ) ⊥ f is the identifiable part of f . This leads to the following lemma:H X → H Z with T f = T [f (X) | Z]. Following this notation,Lemma 6 (Severini & Tripathi (2012)). E[Q[f ]] is identified if and only if Q ∈ N (T ) ⊥ .Examples. Let us introduce an example for an unidentifiedQ[f ]: Consider Z, Y ∈ R, X ∈ R 2 and h(z) = (z, 0), f (x 1 , x 2 ) = 0.5x 1 + 2x 2 and Q[f ] = ∂ 2 f (x * ).In this fully linear setting, due to the structure of h the instrument can only ever perturb the first component of X regardless of what policy we choose. Hence, it is impossible to identify e.g. Q[f ] w.r.t. the second argument with ∂ 2 f (x * ) = 2∀x * . Even a policy with full support will not (even partially) identify Q[f 0 ]. In contrast, let us discuss a fully identifiable scenario, which is identifiable for all Q[f ], but still depends on the policy: Consider a similar setting with f(x 1 , x 2 ) = x 2 1 + x 2 and Q[f ] = ∂ 1 f (x *). Assume we have x * = (6, 1), then Q[f 0 ] = 2 • 6 = 12. Any policy π that has positive density in a neighborhood of z = 6 will be able to fully identify Q[f 0 ]. However, any policy that puts no mass (or in practice, little mass) near z = 6, will not identify Q[f 0 ]. This would be an uninformative policy.We present pseudocode for the 'explore then exploit' (EE) strategy in Algorithm 1, for 'alternating explore exploit' (AEE) in Algorithm 2, and for the adaptive strategy in Algorithm 3.Algorithm 1 Explore then exploit (EE) 1: Input: Budget T , T 1 , T 2 , n, distance measure d on X , target x * 2: Initialize: D (≤T1) ← ∅ 3: Split budget: T = T 1 + T 2 4: for t = 1 to T 1 do 5: Sample π t ∼ Γ 6:Observe sample (X t , Z t )7:D (≤T1) ← D (≤T1) ∪ {(X t , Z t )} 8: end for 9: Find K nearest samples to x * in D (≤T1) based on distance d 10: Fit π as a parametric distribution on the Z values of these K samples 11:for t = T 1 + 1 to T do 12: Set π t := π 13:Observe sample (X t , Z t ) 14:D (t) ← {(X t , Z t )} 15: end for 16: Estimate Q ± t from T t=T1+1 D (t)Algorithm 2 Alternating explore exploit (AEE) 1:Input: Budget T , K, n, distance measure d on X , target x * 2: Initialize: Sorted list of observed samples L ← ∅ 3: for t ∈ [T ] do 4: if t is odd and t ̸ = T then 5: Sample π t ∼ Γ 6:Observe sample (X t , Z t )7: L ← L ∪ {(X t , Z t )} 8: Sort L by distance d(X, x * ) 9:else if t is even or t = T then 10:Find the min{K, n • t} nearest samples to x * in L 11:Fit π t as a parametric distribution on the Z values of these samples 12:Observe sample (X t , Z t )13:L ← L ∪ {(X t , Z t )} 14: Sort L by distance d(X, x * ) 15: end if 16: end for 17: Estimate Q ± t on even t D (t) Algorithm 3 Adaptive 1: Input: Budget T , T 1 , T 2 , n 2: Initialize: Φ t 3: Split budget: T = T 1 + T 2 4: for t = 1 to T 1 do 5: Sample Z t ∼ π Φt 6:Observe sample (X t , Z t )7:Compute gradient update ∇(Φ t )∆(π Φt ) by splitting observed samples 8: end for 9: for t = T 1 + 1 to T do 10:Set π t := π Φ T 1 11:Observe sample (X t , Z t ) 12:D (t) ← {(X t , Z t )} 13: end for 14: Estimate Q ± t from T t=T1+1 D (t)NeurIPS Paper Checklist 1. Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? Answer: [Yes] Justification: The abstract clearly states our contributions, the additional Limitation block marks additionally all goals that are motivational and states which of those are further projects. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.38th Conference on Neural Information Processing Systems (NeurIPS 2024).L2(X) is the L2-space of scalar-valued functions of X with respect to the distribution of X.We write (∂if )(x * ) for the partial derivative of f with respect to the i-th argument evaluated at x * .Here, P(R dz ) denotes the space of probability distributions over R dz . 


Decision: All reviewers agree that this is a solid paper with novel and interesting contributions. There are certainly some weaknesses, such as the limited experimental validation involving only  synthetic and low-dimensional settings, or questions about the validity of some assumptions. On the other hand, the authors could address at least some of these critical comments in a rather convincing way, so that for me as an AC, in the end the positive aspects clearly outweigh the points of criticism. Therefore I recommend acceptance of this paper.

Reviews: review: paper_summary: paper_summary: The authors' primary goal is to design experiments that maximally inform a query of interest about the underlying causal mechanism, within a fixed experimentation budget. They address this by maintaining upper and lower bounds on the query and sequentially selecting experiments to minimize the gap between these bounds. They show that by treating experiments as instrumental variables, they can estimate these bounds using existing techniques in nonlinear instrumental variable estimation. Their procedure involves a bi-level optimization: an inner optimization estimates the bounds, while an outer optimization seeks to minimize the gap between them. For certain queries, when assuming the underlying function lies within a reproducing kernel Hilbert space (RKHS), they derive closed-form solutions for the inner estimation problem. The authors develop adaptive strategies for the outer optimization to iteratively tighten these bounds and demonstrate empirically that their method robustly selects informative experiments, leading to identification of the query of interest when possible within the allowed experimentation framework.
strength_weakness: strength_weakness: W1. The paper only demonstrates results on a low-dimensional synthetic setting (d_x = d_z = 2). While this provides a proof-of-concept, it's unclear how the method scales to higher dimensions or performs on real-world data. 
The authors could:
- Extend experiments to higher dimensions (e.g. d_x, d_z = 10 or 20) to show scalability.
- Test on semi-synthetic data by using real covariates with simulated outcomes.
- Discuss computational complexity as dimensionality increases.

W2. The paper makes strong assumptions (e.g. RKHS, valid IVs) without thoroughly discussing their implications. The authors could:
- Provide sensitivity analyses for key assumptions.
- Discuss scenarios where assumptions may not hold in practice.
- Clarify which parts of the method rely on which assumptions.
questions: questions: No.
limitations: limitations: No.
paper_summary: paper_summary: The authors provide a procedure to use a sequence of encouragement designs to identify target functionals about a particular causal relationship.
strength_weakness: strength_weakness: It isn't clear to me why Equation 4 is a bound on Q[f_0]. Maybe this should be obvious to me, but I think if it isn't obvious to me, it probably won't be obvious to a lot of your readers, as I think I've read more into this literature than most people. This feels like your big contribution: after this bound is setup, the remainder isn't what I would call straightforward, but it feels more like standard machinery that I'm used to. Conditional on the bound, Theorem 1, 2 and the experiment selection procedures you provide all make a lot of sense. But I just don't see where this bound comes from. I would like to see (i) a clearer explanation of why this expression bounds Q[f_0], (ii) some clearer examples of the cases in which the bounds are equal and Q[f_0] is identified. I think it might also help to lay out when a non-optimal policy (what you call "non-informative experimentation") doesn't identify Q[f_0]. This is a bit confusing to me, as I'd expect that complete randomization as a policy would provide identification so long as there is positivity across the whole space: one could do something like off-policy evaluation to identify any particular \pi(Z). I believe the problem comes with the restriction discussed at the start of Section 3.2, but I do not see why it is that the solution to ensure that r_0 lies in TT^* is to take the +\- of Q[f] in the objective as from Eq 3. There's a connection here that you have not clearly spelled out to me.

Maybe its possible that the problem is that I need to read Bennett et al 2023 much more carefully to see why this follows, but that isn't a fair ask for your readers who are reading _this_ paper rather than that one.

Unfortunately, it's quite difficult for me to evaluate the overall novelty of this work without understanding this component. I think you have something very cool here, but I can't quite work that out based on the paper as it stands. I'd like to reiterate that I follow what you're doing at a high level (i.e. line 96-106 makes sense in general), but I don't follow when you get into specifics.

Some other miscellaneous thoughts that aren't as important:
- Do you _have_ to call this (\partial_i f)(x^*) a local effect? Between LATEs and everything else, this feels like an absurdly overloaded term.
- The experiments section makes it difficult to see clear quantitative comparisons between methods. I would like to see things like causal mean-squared error. I recognize that's a bit difficult when you're doing partial-ID, but demonstrating that the bounds collapse to the correct Q[f_0] is important. Even just doing something like showing the midpoint of the bounds quantitatively (with confidence interval based on replicates) and the width of the bound would be useful to make sure the process is behaving sensibly.
questions: questions: see above
limitations: limitations: see above
paper_summary: paper_summary: This paper proposes a framework for designing sequential indirect experiments for estimating targeted scientific queries in complex, nonlinear environments with potential unobserved confounding when direct intervention is impractical or impossible. The authors formulate the problem as the sequential instrument design, using instrumental variables and minimax optimization to estimate the upper and lower bounds of targeted causal effects. The proposed method then tightens these bounds iteratively through adaptive strategies. Experiments on simulated data demonstrate the proposed method's efficacy compared to non-adaptive experimental design baselines.
strength_weakness: strength_weakness: 1. Although the proposed method focuses on indirect experimental design, it still requires the causal structure between variables known, which is also one of the main challenges in scientific discovery. I am curious about the method's sensitivity to imperfections in the causal structure, assumptions regarding instrumental variables, and the presence of confounders. It would be great if the authors could discuss further about them.

2. The authors only conduct synthetic experiments in a simple setting when the number of variables is small. I wonder if the authors could conduct more experiments in a more complex setting with a larger number of nodes, and also discuss the scalability of the proposed method (since kernel-based techniques are used for optimization as mentioned in the paper).
questions: questions: 1. Please see the questions in the Weaknesses part.

2. It seems that the proposed method is closely related to causal Bayesian Optimization [1]. I wonder if the authors could discuss the connection in detail. 

3. Experiments demonstrate that the proposed method performs well for local causal effect queries. However, it is usually unknown whether the target causal effect is local or more global (i.e., long-range) in real-world applications. I wonder if the proposed method could estimate long-range causal effects accurately.

4. What is the difference/connection between the proposed method and targeted indirect experiment design in an active learning setting?

[1] Aglietti, V., Lu, X., Paleyes, A., & González, J. (2020, June). Causal bayesian optimization. In International Conference on Artificial Intelligence and Statistics (pp. 3155-3164). PMLR.
limitations: limitations: The authors adequately addressed the limitations of their work.
paper_summary: paper_summary: This paper designs comprehensive experiments that maximize the information gained about the query of interest within a fixed budget of experimentation, including nonlinear, multi-variate, confounded settings.
strength_weakness: strength_weakness: 1. Are the theoretical results practically useful when addressing real-world problems?
2. How does the proposed method handle non-convex optimization, and what are the assumptions on the loss function?
3. Can the authors provide more intuitive theoretical explanations of the difference between convex and non-convex optimization in their settings?
4. How can the effectiveness of the proposed method be guaranteed when the dimension p is significantly larger than the number of training samples?
questions: questions: 1. Are there any crossing problems when minimizing the optimization problem in Equation (5)? How do we ensure that $Q^{+}(\pi)$ is always larger than $Q^{-}(\pi)$? 
2. Theoretically, are there any requirements for the dimensions of $Z$ and $U$? In your experiments, you only consider a low-dimensional setting. Can you add any experiments for high dimensions for $Z$ and $U$? 
3. For Theorem 2, the proposed method involves several tuning parameters, such as $λ_g,λ_f,λ_c$. This step can be quite worrying for the practical implementation of the proposed method. 1) it can be time-consuming and 2) setting the candidate values is likely quite subjective. The authors should conduct more thorough studies and provide more guidelines on how the choice of tuning parameters affects the method's performance. For example, the authors have selected $λ_g=0.1,λ_f=0.1,λ_c=0.1$, and the learning rate $α_t=0.01$. This selection process appears too subjective. 
4. The experiments are not sufficient for several reasons: (1) The number of replications used in the experiments is quite small (only 25), which raises concerns about the time efficiency of the proposed method. (2) The paper does not include a real-world data analysis to validate the effectiveness of the proposed algorithm from a practical perspective.
limitations: limitations: 1. Theoretical requirements for the dimensions of $Z$ and $U$ are not discussed. Experiments only consider a low-dimensional setting, see questions
2. The proposed method involves several tuning parameters, which can be problematic for practical implementation due to time consumption and the subjective setting of candidate values; see questions.
3. Insufficient Experiments: see questions.

Comments: comment: We thank the reviewer for the comment and increasing their score.
The connection to Mendelian Randomization can be drawn via the Instrumental Variable setup we use as experiments. MR is a special case of instrumental variables in which genetic variants are used as instruments. Therefore---in theory---our method would propose the next genetic variant which would inform the estimator the most. In practice, MR relies on existing genetic variation within a population and not so much on designing genetic variation. The difference therefore lies in the assumption of the instrument: in our setting we are able to adjust the instrument/experiment, MR looks at existing genetic variation. We will add this aspect in the manuscript as both approaches target similar questions. Thank you for the comment.
Thanks for the author's response, which addressed most of my concerns. I will maintain my score.
Thank you for your detailed explanation, which has addressed my concerns. I ask this question because recent research in social sciences has highlighted that the validity of instrumental variables remains a significant challenge [1]. I have increased my score accordingly.

Also, CRISPR example piqued my interest. I'm curious about how your work relates to or differs from research in biostatistics centered on Mendelian randomization (if any). 

[1] Rain, rain, go away: 194 potential exclusion-restriction violations for studies using weather as an instrumental variable. Jonathan Mellon.
We thank the reviewer for their comment and increasing their score. The intuition about the bounds is very well put. We highly appreciate the suggestion and will include the description of the CRISPR use-case into the revised version.
I thank the authors for their clarifications, I think I see more clearly where the bound is coming from. I'll raise my score given this.

To put it a bit more plain (to me), you're taking the minimum and maximum norm solutions (by changing the sign of Q[f] in Eq.4) from within the satisfying set of functions $f$ and $g$. Under full identification, those two would optimands coincide, as there is only one satisfying function, but under partial identification that may not be the case.

I also think the comment the authors provided to reviewer GhAo regarding the CRISPR use-case is interesting. Coming from more of a social science perspective, this is not a setting which would come up much, but I suspect this may be different in the case you're considering. Making this more clear in the text why this setting matters would help the paper's impact.
4 **Insufficient experiments:** Our runtime evaluation in appendix C, Fig. 4 (see also Fig. 3 in the rebuttal pdf), shows that the computational cost of our approach is not a severe limitation, i.e., runtime is not a real concern. While 25 replications already provide a good assessment of the finite sample variance, we happily increase this number to 100 for the revised version. From the first finished settings, the results are visually unchanged.
We agree that we would have loved to include a real-world application of the method, but point to our discussion of the fundamental difficulty of assessing our method in a real-world setting in our “limitations” setting (l.392 and following).
[1] Cinelli, Carlos, and Chad Hazlett. "An omitted variable bias framework for sensitivity analysis of instrumental variables." Available at SSRN 4217915 (2022).

[2] Vancak, Valentin, and Arvid Sjölander. "Sensitivity analysis of G‐estimators to invalid instrumental variables." Statistics in Medicine 42.23 (2023): 4257-4281.

[3] Kilbertus, Niki, Matt J. Kusner, and Ricardo Silva. "A class of algorithms for general instrumental variable models." Advances in Neural Information Processing Systems 33 (2020)

[4] Drineas, P., Mahoney, M.W. (2005). On the Nystrom Method for Approximating a Gram Matrix for Improved Kernel-Based Learning, 2005, Journal of Machine Learning Research, http://jmlr.org/papers/v6/drineas05a.html

[5] Li, Mu et al. “Large-Scale Nyström Kernel Matrix Approximation Using Randomized SVD.” IEEE Transactions on Neural Networks and Learning Systems 26 (2015): 152-164.

[6] Elisabeth Ailer, Jason Hartford, and Niki Kilbertus. 2023. Sequential underspecified instrument selection for cause-effect estimation. In Proceedings of the 40th International Conference on Machine Learning (ICML'23), Vol. 202. JMLR.org, Article 19, 408–420.
[1] Drineas, P., Mahoney, M.W. (2005). On the Nystrom Method for Approximating a Gram Matrix for Improved Kernel-Based Learning, 2005, Journal of Machine Learning Research, http://jmlr.org/papers/v6/drineas05a.html

[2] Li, Mu et al. “Large-Scale Nyström Kernel Matrix Approximation Using Randomized SVD.” IEEE Transactions on Neural Networks and Learning Systems 26 (2015): 152-164.

[3] Bennett, Andrew, et al. "Minimax Instrumental Variable Regression and $ L_2 $ Convergence Guarantees without Identification or Closedness." The Thirty Sixth Annual Conference on Learning Theory. PMLR, 2023.

[4] Cinelli, Carlos, and Chad Hazlett. "An omitted variable bias framework for sensitivity analysis of instrumental variables." Available at SSRN 4217915 (2022).

[5] Vancak, Valentin, and Arvid Sjölander. "Sensitivity analysis of G‐estimators to invalid instrumental variables." Statistics in Medicine 42.23 (2023): 4257-4281.

[6] Kilbertus, Niki, Matt J. Kusner, and Ricardo Silva. "A class of algorithms for general instrumental variable models." Advances in Neural Information Processing Systems 33 (2020)

[7] Sriperumbudur, Bharath K., Kenji Fukumizu, and Gert RG Lanckriet. "Universality, Characteristic Kernels and RKHS Embedding of Measures." Journal of Machine Learning Research 12.7 (2011).

[8] Schölkopf, Bernhard, and Alexander J. Smola. Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT press, 2002.