Probabilistic Factorial Experimental Design for Combinatorial Interventions
Authors: Divya Shyamal
Jiaqi Zhang
Caroline Uhler
Venue: ICML 2025 spotlightposter (2025)
Keywords: Experimental Design
Factorial Experiment
Combinatorial Interventions
Primary area: theory->active_learning_and_interactive_learning

Abstract: A _combinatorial intervention_, consisting of multiple treatments applied to a single unit with potential interactive effects, has substantial applications in fields such as biomedicine, engineering, and beyond. Given $p$ possible treatments, conducting all possible $2^p$ combinatorial interventions can be laborious and quickly becomes infeasible as $p$ increases. Here we introduce the _probabilistic factorial experimental design_, formalized from how scientists perform lab experiments. In this framework, the experimenter selects a dosage for each possible treatment and applies it to a group of units. Each unit independently receives a random combination of treatments, sampled from a product Bernoulli distribution determined by the dosages. Additionally, the experimenter can carry out such experiments over multiple rounds, adapting the design in an active manner. We address the optimal experimental design problem within a novel intervention model that imposes bounded-degree interactions between treatments. In the passive setting, we provide a closed-form solution for the near-optimal design. Our results prove that a dosage of $\frac{1}{2}$ for each treatment is optimal up to a factor of $1+O(\frac{\ln(n)}{n})$ for estimating any $k$-way interaction model, regardless of $k$, and imply that $O\big(kp^{3k}\ln(p)\big)$ observations are required to accurately estimate this model. For the multi-round setting, we provide a near-optimal acquisition function that can be numerically optimized. We also explore several extensions of the design problem and finally validate our findings through simulations.

Probabilistic Factorial Experimental Design for Combinatorial Interventions
Divya Shyamal 12
Jiaqi Zhang
13
Caroline Uhler" 13
Abstract
1. Introduction
In many domains, is often interest to consider the si- multameous application of multiple treatments/actions. For example, in cell biology, perturbing several genes is often necessary to induce transition in cell state (Takahashi & Yamanaka, 2006). While single treatment is constrained to limited range of possible effects, a combinatorial interven- tion comprising multiple treatments applied to the same unit - can result a much wider array of outcomes. Much of this potential stems from the interactive effects between treatments, rather than merely the additive contributions of each. For example, perturbing paralogs (a pair of genes) can have surprisingly larger effect than the sum of perturbing each gene individually, as one gene may compensate for the other, and only perturbing both simultaneously will effec- tively disrupt the pathway (Koonin, 2005). However, these interactions make the study of combinatorial interventions considerably more challenging than understanding single interventions alone, as each set of treatments may exhibit distinct interactions.
A combinatorial intervention, consisting of multi- ple treatments applied to a single unit with poten- tially interactive effects, has substantial applica- tions in fields such as biomedicine, engineering, and beyond. Given p possible treatments, conduct- ing all possible 2 combinatorial interventions be laborious and quickly becomes infeasible p increases. Here we introduce the probubilistic factorial experimental design, formalized from how scientists perform lab experiments. In this framework, the experimenter selects a dosage for each possible treatment and applies it to group of units. Each unit independently receives ran- dom combination of treatments, sampled from a product Bernoulli distribution determined by the dosages. Additionally, the experimenter can carry out such experiments over multiple rounds, adapt- ing the design in an active manner. We address the optimal experimental design problem within an intervention model that imposes -ounded-degree interactions between treatments. In the passive setting, we provide a closed-form solution for the near-optimal design. Our results prove that a dosage of 1/2 for each treatment is optimal up to a factor of 1 + (In())/n) for estimating any k-way interaction model, regardless of k, and im- ply that Okpp* (p)) observations are required to accurately estimate this model. For the multi- round setting, we provide near-optimal acquisi- tion function that can be numerically optimized. We also explore several extensions of the design problem and finally validate our findings through simulations.
From the design perspective, the problem of testing com- binatorial interventions to analyze the combined effects of various treatments is known as a factorial design (Fisher et al., 1966). Given p possible treatments with large p, is often infeasible to conduct all possible 2 combinatorial interventions, which corresponds to a full factorial design. To address the scalability challenge, fractional factorial de- signs are introduced, which test only subset of possible combinations. However, selecting this subset is difficult when prior knowledge is limited. Choosing a suboptimal subset may lead to a biased understanding of the experimen- tal landscape. In addition, performing a large and specified subset of combinatorial interventions can be laborious and impractical, each combination must be precisely assem- bled. In the perturbation example, this involves synthesizing unique guide sequence for each combination that targets the specific genes involved (Rood et al., 2024). However, when the combination size is large, this becomes infeasible as a longer guide sequence may lack sufficient penetrance to effectively enter the targeted cells. To tackle these issues, here formalize and study scalable and unbiased approach to design factorial experiments.
'Equal contribution 'Department of Electrical Engineering and Computer Science, MIT DDepartment of Mathematics, MIT EEric and Wendy Schmidt Center, Broad Institute. Correspondence to: Jiaqi Zhang sriezhang@mit.ddu Caroline Uhler <euhh l mit.
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
Inspired by how scientists perform library designs in the

---

Probabilistic Factorial Experimental Design for Combinatorial Interventions
lab (Yao al., 2024), we introduce probabilistic factorial factoroi experimental design. In this framework, the experimenter selects a dosage for each possible treatment and applies it to group of units. Each unit independently receives a random combination of treatments, sampled from a product Bernoulli distribution determined by the specified dosages. In the perturbation example mentioned above, this setup for- malizes a mulhilllllllicttyy of infection (MOI) perturbation experiment (Yao al., 2024), where multiple perturbations are applied at various MOI to a plate of cells, and each cell receives combination of perturbations randomly. The in- troduction of a probabilistic design via dosages allow us to interpolate between an unbiased but expensive full factorial design and a relatively scalable but restricted fractional fac- torial design. By adjusting the dosages, we can effectively scale up full factorial design by controlling the proportion of units receiving each combination in a realistic manner. Crucially, this approach remains unbiased as it does not re- quire restricting the experiment to predetermined subset of treatments. The question is then how to optimally design the dosages, e.g., in order to efficiently learn the interactions.
2. Related Works
Factorial design. Factorial experimental design has been extensively studied for its efficacy in evaluating multiple treatments simultaneously. Classical methods include full and fractional factorial designs (Fisher et al., 1966), and have been applied to various applications in biology, agri- culture, and others (c.f., (Hamrahan & Lu, 2006)). Full fac- torial design considers all possible treatment combinations, where each treatment may have multiple levels (Deming & Morgan, 1993; Lundstedt al., 1998; Dean & Voss, 1999). These experiments are sometimes conducted in multiple blocks, where each block is expected to have controlled condition of external factors and contains replicate of either all or partial combinations. When the number of total treatments increases, conducting such experiments quickly becomes infeasible. In these cases, fractional factorial de- sign are preferred where subset of carefully selected treat- ment combinations are tested (Gunst & Mason, 2009). A 2-m fractional design is one where 28-m samples are used, each with different combination (Box et al., 1978). These combinations are carefully selected to minimize aliasing, Aliasing occurs when, for the combinations selected, the interactions are linearly dependent (Gunst & Mason, 2009; Mukerjee & Wu, 2007). In a full factorial design, there is linear independence, so there is no confounding when the model is fit. In fractional design, some aliasing will always occur in full-degres model; however, methods proposed in the literature select combinations such that the aliasing of important effects (i.e. degrec-1 terms) does not occur (Gunst & Mason, 2009). With little prior knowledge, it is common to assume that low-order effects are more important than higher-order interactions and select designs to focus on low- order effects (Cheng, 2016). With low-degree assumption, aliasing can be avoided entirely. Fractional designs can be classified by their resolution (denoted by It), which deter- mines which interactions can be potentially confounded. For example, a Resolution V fractional design eliminates any confounding between lower than degree-3 interactions, appropriate for degree-2 functions (Montgomery, 2017). Of particular interest in literature are minimum aberration de- signs, which minimize the number of degree-l terms aliased with degree-R terms (Fries & Hunter, 1980; Cheng, 2016). However, scalability to high-dimensional problems remains a challenge, and efficient sampling methods such as Bayesian optimization are proposed (Mitchell et al., 1995; Kerr, 2001; Chang & Cheng. 2018).
Contributions. Our contributions are summarized below.
We propose and introduce the probabilistic factorial design, motivated by library design experiments in the lab (section 3.1). This setup assumes that treatments are randomly assigned to group of units according to a prescribed dosage vector. It provides scalable and flexible approach to implement factorial experiments, which we show to encapsulate both full and fractional factorial designs as special cases.
Within this framework, we address the problem of op- timal experimental design, which involves optimizing the dosage vectors based on given objective. Our main focus is on learning the underlying combinatorial intervention model using Boolean function represen- tation assuming bounded-order interactions.
In the passive setting (section 4.2), we prove that assigning a dosage of 1/2 to each treatment is near-optimal for estimating any k-way interac- tion model, leading to a sample complexity of Okk'** In(p)).
- In the active setting (section 4.3), we introduce an acquisition function that can be numerically optimized and demonstrate that it is also near- optimal in theory.
The probabilistic setting proposed in this paper serves as a flexible realization of factorial design that automatically generates design resembling either a full factorial or frac- tional factorial design, depending on the selected dosages. We formally discuss this in section 3.
We explore several extensions to the design problem, including constraints on limited supply, heteroskedas- tic multi-round noise, and emulation of a target com- binatorial distribution (section 5). Finally, we validate our thcoretical findings through simulated experiments (section 6).
2

---

Probabilistic Factorial Experimental Design for Combinatorial Interventions
Learning combinatorial interventions. Modeling com- binatorial interventions is crucial for understanding their interactions and designing experiments. There are multi- ple ways to model such interventions, often by imposing structures that relate different combinations. For example, the Bliss independence (Bliss, 1939) and Loewe additivity (Loewe, 1926) models are commonly used to describe addi- tive systems where no interactions between treatments are assumed.
plicity, we consider no interference between units, where each unit independently receives a combinatorial interven- tion at random. Denote the intervention associated with unit m E [n]] (1....,7 n) by Xm E {-1,1},, where .mm.s = 1 if and only if it receives a combinatorial intervention that contains treatment i. Here Xm is randomly sampled ac- cording to a product Bernoulli distribution according to d. where
Lm,ii -
with probability di, with probability - di.
An alternative approach is to use structural causal model (SCM) and the principal of independent causal mechanisms (Eberhardt, 2007; Eberhardt & Scheines, 2007). In particu- lar, this assumes that (1) each virgle-variable intervention alters the dependency of that variable on its parent vari- ables according to the SCM, and (2) a combinatorial in- tervention modifies each involved variable according to its respective single-variable intervention and then combines these changes in a factorized joint distribution. Within this framework, various types of interventions, including do-, hard-, and noft-interventionss can be defined (e.g.. (Correa & Bareinboim, 2020; Zhang et al., 2023)). However, simi- lar to the Bliss independence and Loewe additivity models, SCM-based approaches cannot capture interactions between treatments.
(I)
The experimenter can carry out such experiments for T times, with different dosages d1.... dy, potentially in sequential and adaptive manner. In combinatorial perturba- tion example in section 1, the dosage vector formalizes the multiplicity of infection of each considered perturbation.
Note that this setup reduces to traditional two-level factorial design (Fisher et al., 1966) by choosing d {0. 1). In particular, for any combinatorial intervention consisting of treatments in S S [p], setting d, 1 if E S or else d, O gives rise to all units receiving S. Allowing for continuous d E [o, 1]" generalizes this setup by enabling the allocation of units to different combinatorial interventions in a realistic and effective manner controlled by d.
To model such interactions, one can use a generalized sur- face model, which can be instantiated via polynomial func- tions (Lee, 2010) or Gaussian processes (Shapovalova et al., 2022). Alternatively, Boolean functions provide another modeling framework (Agarwal et al., 2023a), where theo- retical tools such the Fourier transform can be leveraged (O' Do'Dnnelll, 2008). Agarwal et al. has employed this ap- proach, where sparsity and rank constraints are used to en- force structural assumptions on combinatorial interactions. In this paper, we also utilize Boolean functions, where we demonstrate their close relationship with generalized sur- face models. We show that interactions can be read-off from Fourier coefficients, allowing us formalize assumptions about the degree of interactions.
3.2. Outcome Models for Combinatorial Interventions
Under this setup, we are interested in estimating the average treatment effect of combinatorial interventions. For unit m, we observe its treatment assignment x,, and outcome Yn E R. We adopt the outcome model proposed by Agarwal et al. (2023b), where you corresponds to noisy observation of a real-valued Boolean function f 1-1, + R. i.e.,
Ym = =x(xm) + C.-
Here we assume cm is independent among different units and is normally distributed with mean and variance o². This model choice has the flexibility of allowing for interac- tions between arbitrary sets of treatments, as we illustrate below.
3. Setup and Model
In this section, we propose and define the setup of proba- bilistic factorial experimental design. We then introduce the outcome model we use to model combinatorial interventions and discuss its applicability to model interactions.
The class of sentation via the Fourier basis
Boolean functions admits a repre-
{xs(x)
II iEs
2 1 S S [p]
3.1. Probabilistic Factorial Design Setup
by
Consider p possible treatments with 2 total combinato- rial interventions. In a probabilistic factorial experimen- tal design, the experimenter chooses vector of dosages, denoted by d = (di...., dp) E [0, and applies the treatments at this level to n homogenous units. For sim-
f(x) = Σ 3sos(x).
SCIp
Here 3s = 2" Ey fly)os(y) (see Appendix A for details). The Fourier coefficients are interpretable in the sense that the polynomial instantiation of the generalized
3

---

Probabilistic Factorial Experimental Design for Combinatorial Interventions
response surface model (Lee, 2010) can be expressed in this form, where all the k- k-way interactions captured by
The columns of X' corresponds all possible combinations (including size S with interactions, i.e., S S [p] with |S] S k. The m-th row of X' corresponds to the Fourier characteristics of the observed combination Xm with
[3s S _ [p], |S| - k}.
In particular, the generalized response surface model can be written follows.
Xm.s = (xm) = II Tm.i.
iES
Polynomial Instantiation. To capture the nonlinear inter- actions between treatments, we can model the outcome of combinatorial intervention X via
Given that X is randomly drawn according to the dosages d and its columns are correlated, it is possible that it is ill-conditioned for a standard linear regression estimator. Therefore, in order to control the estimation error, we use truncated ordinary least squares (OLS) to estimate B:
[(x) - Σ 0. 1 +
Σ 1,==1
3 (x if K E x)- s
B a²
where ag represents the contribution in the final outcome by the interaction among treatments in S. This model can be represented via the Fourier representation (see Appendix A for details), where
otherwise.
Here À denotes the eigenvalues of X X and Y is the vector by stacking 1/m with E [m]. Note that this results in a null estimator when the cigenvalues are small. We use this to demonstrate the key ideas of our analysis in a simpler form In practice, when X' is ill-conditioned, alternative estimators such as ridge regression can used, where similar theoreti- cal results can be derived (see Appendix B for details). The truncated OLS estimator satisfies the following property, which we utilize in our analysis.
RT 2 T
Bs
SCT
In a bounded-order interaction model, ag 0 for large |S]. In particular, if a.s 0 for |S] > k. then Bs 0 for |S] > k according to Eq. (2). This motivates us to make the following assumptions on the Fourier coefficients.
Assumption 3.1 Gooudded-odde interactions). The out- come model exhibits counded-orded interactions, i.c., there exists o(p) such that
Lemma 4.1. Given a fixed design matrix X, the truncated OLS estimator satisfies
o²
131122 S
ss 0 if
S
> k.
Ax x)
(3)
We also assume that 3 is bounded in L2 norm.
K
o²
Ey [1|@ - -312
< min
B²}.
Assumption 3.2. (Boundedness of (3) There exists a con- stant B such that || < B.
A x)
4.2. Passive Setting
4. Optimal Experimental Design
In this scenario, the experimenter decides the choice of the dosages d in prospective fashion without considering any data collected in the past. This is in contrast to an active design, where collected data are utilized to decide the current design. Note that the first round of any active setting reduces the passive scenario, as there is no collected data.
In this section, we focus on optimal experimental design for learning the outcome model f. We consider extensions of these results in section 5. For the objective of learning f, we provide near-optimal design strategies for the choice of dosages in both passive and adaptive scenarios. We start by introducing the estimators of f. All formal proofs this section are deferred to Appendix B.
Suppose we have budget of n units. To select d such that we can obtain the most accurate estimate of B after observing these units, it is natural to optimize the following objective:
4.1. Estimators
||@ 3113
Estimating the Fourier coefficients accurately in turn gives an accurate estimate of as [x) fxx)]]2 ||ا B1|2. where f(x) Escpl (x) Therefore, it suffices to
Ep
(4)
We show that this objective has closed-form near-optimal solution of d = (1/2, 1/2), regardless of the order of the interactions.
focus on 3.
Denote the collected dataset D {Xmm: Vm) m [n]]. Let the design matrix be X' e Ra*K with K = - (P).
Theorem 4.2. For the truncated OLS estimator;
(422.-. ... 1/2) is optimal up to factor of 1 1 O In(n))

---

Probabilistic Factorial Experimental Design for Combinatorial Interventions
with respect to Eq. (4), In addition, the minimizer of Eq. (4) lies in an lx norm ball centered on the half dosage with radius O
d = (1/,,..., 1/2), we use the upper bound show that
Ko²
Infn)
ED [13 |
n11 6)
nd²²
Note that with the half dosage, the probability of observ- ing any particular combinatorial intervention S S [p] is P. Therefore in the passive setting, it is always optimal to evenly administer every treatment so that the observed combinatorial interventions follow a uniform distribution.
B² exp
K In 9
8K2
for any 0 < 8 < 1. For d such that max; 12d; - 1| > 0, we use the lower bound to show that,
(1 2exp Kln 8K2)
8²
ED [1 3|1]]
Proof sketch. In Lemma 4.1, we show how to bound the ex-
σ²
+ o²(K 1)
minf n2(1 - max, |2d, - 1 + 8)
pectation of the error ||, - 3112 with respect to randomness in the outcome Y. To obtain the optimal dosage for Eq. (4), we need to additionally take expectation with respect to the randomness of X' which is where the dosages enter as the combinatorial interventions x are sampled from Eq. (1). Note that B[ X] = n (d) E RK*, where
n2 + 6)
for any that d
> O. By choosing 8 (2 (2 we obtain (l/2,..., 1/2) is optimal up to a factor of 1 +
O [n(m))
As a corollary of the proof for Theorem 4.2 we can show the error of estimating B decays with a rate of n n11
E(d)s, = II (2d;- - 1),
(5)
VESAS'
Corollary 4.3. With d = (1/2.... ./2), there
for any S.S' C [p] such that |S| S k.!
Es 11.3
312] 1 2Ko +1
n
Intuition of the optimality of half dosages. For the standard OLS estimator, the expected squared error is
for n. > nag where no = 0 In In
Therefore in order to estimate a k-way interaction model
correctly, O(K3 In(K) - 0 Okp In(p)) samples suffice.
xx x)
4.3. Active Setting
If we directly swap T X' with its expected value, then we need to minimize
In this setting, the experimenter decides the choice of the dosages d sequentially in multiple rounds, where the ob- servations from previous rounds can be used to inform the choice of dosage. Note that as discussed in Section 4.2, the first round of the active setting degenerates to the passive setting, where the optimal choice is d = (2/2,...,1/2)
K
M(E(d))
Note that tr (E())) K for all d. Therefore, by the Cauchy- }x(E()))- is minimized if and
Consider round > 1. Denote D; as the collected data and let X₁ be the design matrix obtained by D, at round t < T. The goal is to minimize the following objective
Schwarz inequality, _ > only if (xd)) = 1 for all E [K]. which is satisfied when E(d) = I and (1/2..... 1/2.....1//4 1/2).
Ep [1 - 3 S.D.U...DD. U......]..
To formally show that the expected error is optimized with half dosages, we can use a concentration result for X x which can be obtained using an e-net argument (Vershynin, 2018) and Hoeffding's 's inequality. However, the eigenvalues of X' enters the error computation through the denomi- nators, which makes the computation difficult. particular, cannot be bounded due to explod-
(6)
In this scenario, we can not obtain a closed form solution as the optimal choice of d depends on pre-collected iDy U ... Dy-1, which can be arbitrary. However, we show it is possible to derive near-optimal objective that can be easily computed and numerically optimized.
E K ing terms when Amin (X' X') approaches zero. We resolve this difficulty by utilizing the bounds in Lemma 4.1. For
< {x x1) -()-
Theorem 4.4. The following choice of dosage:
1
'err A denotes the disjunctive union, i.c., SAS' (S U S') \ nS)).
d = argmin
(7)
de
A ((() + + +
5

---

Probabilistic Factorial Experimental Design for Combinatorial Interventions
Algorithm 1 Active probabilistic factorial experimental de-
setting where we have supply constraints on treatments, or where do not want assign unit too many treatments at once. Note that the constraint implies that the expected number of treatments assigned to a unit is at most L.
sign.
1: Initialize X - = 0MMM.
2: for t- = to I" do
3: if = then
In the passive setting, we derive a closed-fore form near-optimal dosage for the pure-additive model, i.e. k = in Assump- tion 3.1. This result requires understanding of the spectrum of (d) In the no-interaction case, we are able to derive the characteristic polynomial for E(d), which becomes dif- ficult when k > 1. However, empirical results show that the result, which we now state, to hold for ls > as well (see section 6).
4: set = (1/2, 1/2);
5: else
6: set d, argmin Σ K defo.11p
(x(d)) Exsl x:)
7: end if
8: Gather observations according to Eq. (1) and form
design matrix X4
9: Update X x x
X x 2 X
10: end for
Theorem 5.1. For the additive model with k = 1, among the dosages that satisfy the constraint in Eq. (8), the uniform dosage with d; = - for all i E [p] is optimal up to a factor
11: Return estimated B using all observations.
of 1 O [n((a)) with respect Eq. (9).
is optimal up to a factor of O [I(nn]
with respect to
Eq. (6).
For non-additive models and the active setting, we note that Theorem 4.4, where the feasible region of d is modified according to Eq. (8), to still hold. Therefore, although no closed-form solution can be derived, we can still obtain a near-optimal solution via numerical optimization.
In practice, we solve for dy by numerically optimizing the objective in Eq. (7) using the SLSQP solver Scipy (Virtanen al., 2020). The number of iterations for the op- timizer converge is roughly Op²²), and the complexity of each iteration is Onn + K3) (where the first term comes from the matrix multiplication of XT and the second term comes from computing the eigenvalues of (d)). Recall the definition of K to be the number of interactions under consideration, i.e. K Σx (t) = Opp²) for small k. Therefore, the overall complexity (npak+3 + p5k-3)
5.2. Heteroskedastic Multi-round Case
Our results can easily extend to the scenario where the noise in the outcomes varies by round. This case might be relevant when different rounds of experiments have systematic batch effects, e.g., if they are collected within different labs.
for small k. In practice, may recommend using a proxy, which only involves the inverse of the minimum eigenvalue: dr = ar0. Amin 3 X x We found that numerically optimizing this was significantly faster and that the solver was consistently accurate. While the complexity computed above should be the same for this approach, in practice it takes many less iterations to con- verge. We summarize the procedure for the active setting in Algorithm
Assume that in round t, the variance of the observed outcome noise is of. Note that in this setting, d (42,..., 1/2) is still near-optimal for the first round. How- ever, the optimal choice of dosage at round T becomes
K dy = argnni
de(o,1]p
1=1 X
E(d) + EL 03
1
2 x.)
where we now scale the observations at round f by o, use the truncated OLS estimator on this modified dataset (Eq. (6)), following weighted least squares approach.
1 and
5. Extensions
In this section, we consider several extensions and discuss how the design policy changes in different scenarios.
5.3. Limited Intervention Cardinality
Consider the scenario where the set of possible treatments that can be applied has limited cardinality:
5.1. Limited Supply Constraint
Here, we consider the case where we have additional con- straint on the possible dosages d:
||d|oo S T., for some 0 < T. < p.
Σ d L, for some 0 < L
p
P 2
(8)
Suppose that d4 * 0 for i E D, where the cardinality |D] is bounded by L. Then holds that X = (-1))5VPI X
i=1
Therefore the design matrix can be written
We assume L < g. as otherwise = -1/2,... 1/2) is feasible and therefore optimal. This case is inspired by a
X XDD
6

---

Probabilistic Factorial Experimental Design for Combinatorial Interventions
where XD denotes the submatrix of X' corresponding to columns Y.s with S _ D and Tn consists of one-hot vectors columns. In this case, we may estimate 3 only up to TDB, e.g., using the following truncated OLS estimator
6.1. Comparison to Fractional Factorial Design
Here we compare the half dosage versus a partial facto- rial design in the passive setting. We generate degree-1 1 Boolean function with p 8. We use 28-2 Resolution V design with 64 samples for each approach.
(x if (x S o otherwise.
n²
The fractional design returns a mean squared error of 0.14 0.062, where the half dosage gives 0.16 + 0.078 (averaged over 300 trials and with +1 std). With fewer samples, the careful selection of combinations will make a difference, the fractional design can outperform the half dosage. But in many cases, especially in biological applications, careful selection of combinations is not possible which is why the much more flexible dosage design is preferable, as it enables the administration of an exponential number of combinations by choosing a linear number of dosages.
Note that this has form similar to B, where using similar arguments in Section 4, we can show that d, 1/2 for i E D is near optimal. Thus, in the passive setting, the near- optimal strategy becomes selecting a subset of treatments D with [D] S and setting d, = 1/2 for i E D and d, = 0 for i & D. As the estimator for T 3 directly estimates entries Gg of 3 with S S D. one can select D based on prior preference of which coefficients of B3 are of interest.
5.4. Emulating a Target Combinatorial Distribution
However, in the active setting, the optimal dosage can out- perform fractional design. This is discussed further in Section 6.3.
We consider different problem that explores the possibility of emulating a target distribution of combinatorial interven- tions with one round of probabilistic factorial design.
6.2. Passive Setting
Formally, let be an arbitrary distribution over all possible combinatorial interventions, we are interested in approxi- mating q with choices of d. Denote Pa as the distribution over combinatorial interventions induced by dosage d. We use KL divergence D(g lI pa) measure the approxima- tion error. To optimize over d, note that pa is product distribution and we have
In Theorem 4.2, we show that d = (1/ 1/2) is optimal up to a factor of 1 + O [n(m)), ). Empirically, our validations build on the comparison of estimation error between half dosages and randomly sampled dosage vectors. We consider two different ways to generate dosages in this comparison, as described below.
Simulation 1. Here, we investigate the approximation of achieved by different dosages d based on their loe diistances from the 2 = (1/2,.... 1/2), i.e., ||a - dll We consider distances ranging from .4, where we sample 100 differ- ent dosage vectors each distance. For each dosage, we generate 20 sets of observations and regress each.
D(q 11 pa)= =4))) Σ t log(d)) (11 (1--0) log(1 d:),
where q: = Ex ((x;) is the marginal distribution of receiving treatment under the target distribution, and (-) denotes the entropy. Minimizing this equality quickly ob- tains dj = qs. which indicates choosing d based on the marginals of the target distribution. The minimal approxi- mation error is then
075
H())= ((1 @ .....9p). 9p);
118 111-
which means we can emulate target distribution well if it is closed to product distribution.
10
20
P 10
035
6. Experiments
We conduct experiments to validate our theoretical results, as well as show a comparison to fractional factorial design, using simulated data. We generate the outcome model f by sampling the Fourier coefficients from the uniform distri- bution, i.e., B ~ 1)), We noise the outcomes with standard Gaussian noise. In each of the following simu- lations, we keep 33 constant through all iterations of each run. Further details and the code repository can be found in Appendix D.
||d -11-
Figure I. Simulation 1. Average ||@3 - /3122 over 2000 different observation sets generated from 100 different dosages at cach given distance. The bars correspond to t.5 std over the 2000 observations. The curves are generated with values 10, A
2, n = 200; p = 20, = 2, n 1000; and p = 30, 2, n 1000.
7

---

Probabilistic Factorial Experimental Design for Combinatorial Interventions
10
118 P112
0.40
p-
cottnll
mindom
0.4
0.5
Figure 2. Simulation 2. Approximation error of uniform dosages. Bars correspond =.5 std over 500 observations per dosage. The curves are generated with values 10, & = n, = 200; p= 20, k 2. n- 1000; and p = 30, 2, n 1000.
Found
Figure 3, Active setting with relatively large n. Results are av- eraged over 20 trials, where p 15, k 2, 75. We limit the y-axis axis to 1, focusing on later rounds when the approximation error is small. Bars correspond 1.2 std.
We show these results for three different sets of p, k and n in Figure 6.2. These values are chosen such that the ratio K/n kept approximately constant under different number of total treatments, following Corollary 4.3: p 10,
2, n 200; p 20, k 2, n 1000; and p 30, 2, 1000.
2.1
1112
half
optimal
Simulation 2. Here, we only consider dosages where each treatment is administered at the same dosage, which we refer to as a uniform dosage. We consider dosage values ranging from .4 to .6, and generate 500 different observation sets for each dosage. We show the approximation error of f against the dosage value in Figure for the three different sets of p, k; and used in simulation 1.
0.3
partall random
Round
Results. In simulation 1, we see that the approximation error is generally increasing in |d 1 small n (on the scale of O(K), rather than poly(K) in Corollary 4.3), we see that the half dosage seems be optimal. In simulation 2, again see that the half dosage exhibits optimality, with U shaped curves dipping at .5.
Figure 4. Active setting with relative small high noise. Results are averaged over 50 trials, where p = 5, k = 1, 16, a = 5. Bars correspond to +.1 std.
2 xo. Even with relatively
Results. We see that random performs consistently worse that optimal and half. For high n (compared to K), the difference between opt imal and half is marginal (as seen in Figure 3). However, when is small, there is noticeable gap between optimal and half. In the case where there are not many samples (compared to features) per round, we find that the optimal acquisition strategy more clearly outperforms the half strategy. This is because when we have a smaller number of samples, we will need to "coorreet" as the distribution of combinations will be more lopsided and further away from the uniform distribution. Similarly, this is why opt imal can outperform partia all in a multiple-round setting, though it may be subpar in single round. Therefore, in scenarios where each round has few samples, we think it is worth computing the optimal acquisition dosage. When we have large n relative to the half strategy and optimal strategy perform very similarly.
6.3. Active Setting
Here, we carry 10 sequential experimental rounds. We compare our proposed choice of dosage in Theorem 4.4, which we refer to as opt amtimal to two baselines. The first baseline, referred to as random, randomly chooses a dosage from (0, 1)8 at each round. The second baseline, referred to as half, chooses the dosage of ! at each round. 2 On synthetic data, we find that opt aptimal and half per- form similarly when is relatively large, while clearly out- performing random (Figure 3). In settings where is rela- tively small, opt imal 1 outperforms half f (Figure 4). We also add partial design baseline, referred to as partiar ial (a Resolution V 25-1 design), in the small p setting. In carlier rounds, we see optimal performs the best, and pa partia rrtiall catches up after sufficiently many rounds.
8

---

Probabilistic Factorial Experimental Design for Combinatorial Interventions
113 B11
118 1912
10
0025
0000
0075
||d Bll-
[|d -111--
Figure 5. Limited Supply Constraint. Here d needs satisfy Em di S 2. The r-axis shows the -distanne from the T.Pp uniform dosage
Figure 6. Misspecification. The a-axis shows the 1 distance from the half dosage. We vary & from 2 to 4, where the true k = 5. We use 300, 100, 200 samples, respectively. 50 different dosages are sampled at each distance, with 20 iterations of each. Bars are 1.2 std.
2 and = 1000 constant. 50 different dosages are sampled at cach distance, with iterations each. Bars are over the 2000 squared errors at each distance.
vary p over 8, 9, 10, keeping
with hounded-degres interactions. We establish theoretical guarantees and near-optimal desgin strategies in both pas- sive and active learning settings. In the passive setting, we prove that a uniform dosage of 1/2 per treatment is near- optimal for estimating any k-way interaction model. In the active setting, we propose numerically optimizable acquisi- tion function and demonstrate its theoretical near-optimality. Additionally, extend approach to account for prac- tical constraints. including limited supply, heteroskedastic multi-round noise, and emulating target combinatorial dis- tributions. Finally, these thcoretical results are validated through simulated experiments.
6.4. Extensions
In Theorem 5.1, proved that the uniform dosage of L is optimal in the constrained case for the simple additive models. Empirically, we see that this holds for interactive models as well, both in simulations and in numerically op- timizing Eq. (4). For example, for the pairwise interaction case, Figure shows the approximation error versus the deviation from the suspected optimal dosage. We that with L 2, 1000, and varying p 8, 8,9, 10, the approximation error increases as we deviate from 4.
6.5. Misspecccii model
Limitations and Future Work. This work has several limitations and assumptions that may be interesting ad- dress in future work. First, assume product infection mechanism in the probabilistic design. However, this as- sumption may not hold in certain scenarios, such as when interference or censoring effects are present. For exam- ple, in cell biology, experiments conducted on tissue sam- ples may exhibit spatial interactions among neighboring cells. Additionally, certain treatment combinations may in- duce cell death, leading to a lack of observable units for those combinations. Second, our combinatorial interven- tion model could be extended to incorporate unit-specific covariates. The current model assumes that outcomes are determined solely by the received treatment, which suffices for homogenous units and average effects. However, incor- porating covariate-based models would enable finer-grained personalized creatennt-outconee predictions. Third, while we explore several extensions to the design problem, further investigations into alternative constraints, such as sparse interventions, and alternative objectives, such as optimizing specific outcome variables, could be valuable directions for future work.
In the case where do not know the true degree of the highest-order interaction, our model may be misspecified case. While our theoretical results do not support this case, we conduct experiments that show that the half dosage still appears to be optimal in a single-round setting. Here, we use Boolean function of full degree (with = 5), and vary & between and 4. So while the true function features interaction terms of all degrees, terms of interaction up A appear in We fit the model under these assumed values of k, and observe that a half dosage appears to still lead to the lowest estimation errors in Figure 6.
assumption is that only
7. Discussion
In this work, we propose and study probabilistic factorial de- sign, a scalable and flexible approach to implementing facto- rial experiments, which generalizes both full and fractional factorial designs. Within this framework, we tackle the op- timal design problem, focusing on learning combinatorial intervention models using Boolean function representations

---

Probabilistic Factorial Experimental Design for Combinatorial Interventions
Impact Statement
Eberhardt, F. Causation and intervention. Unpublished doctoral dissertation, Carnegie Mellon University, 93, 2007.
This paper presents theoretical work whose goal is to ad- vance the field of Machine Learning. There are many poten- tial societal consequences of our work, none which we feel must be specifically highlighted here.
Eberhardt, F. and Scheines, R. Interventions and causal inference. Philosophy of science, 74(5):981-99) 2007.
Fisher, R. A., Fisher, A., Genetiker, S., Fisher, R. A.,
Acknowledgements
Genetician, S., Britain, G., Fisher, R. A., and Généticien, S. The design of experiments, volume 21. Springer, 1966.
We thank the anonymous reviewers for helpful comments. D.S. was supported by the Advanced Undergraduate Re- search Opportunities Program at MIT. J.Z. was partially supported by an Apple AI/ML PhD Fellowship. C.U. partially supported by NCCIH/NTH (IDP2AT012345), ONR (N00014-22-1-21116 16 and N00014-24-1-2687) the United States Department of Energy (DE-SC0023187), the Eric and Wendy Schmidt Center at the Broad Institute, and a Simons Investigator Award.
Fries, A. and Hunter, W. G. Minimum aberration 2 designs. Technometrics, 22(4):601-608, 1980.
Gunst, R. F. and Mason, R. L. Fractional factorial design.
Wiley Interdisciplinary Reviews: Computational Statis- tics, 1()):23-224, 2009.
Hanrahan, G. and Lu, K. Application factorial and re- sponse surface methodology modern experimental de- sign and optimization. Critical reviews in analytical chemistry, 36(3-4): 141-151, 14111)) 2006.
References
Agarwal, A., Agarwal, A., and Vijaykumar, S. Synthetic combinations: A causal inference framework for combi- natorial interventions. Advances in Neural Information 2023a.
Kerr, M. K. Bayesian optimal fractional factorials. Statistica
Sinica, pp. 605-630, 2001.
Processing Systems,
Koonin, E. V. Orthologs, paralogs, and evolutionary ge-
nomics. Annu. Rev. Genet., 39(1):309-333 2005.
Agarwal, A., Agarwal, A., and Vijaykuma, S. Synthetic combinations: A causal inference framework for combi- natorial interventions. Advances in Neural Information Processing Systems, 1111915--92116 2023b.
Lee, S.-i. Drug interaction: focusing on response surface models. Korean journal of anesthesiology, 58(5):421- 434, 2010.
Loewe, S. Effect of combinations: mathematical basis of problem. Arch. Exp. Pathol. Pharmakol., 14:313-326. 1926.
Bliss, I. The toxicity of poisons applied jointly 1. Annals of applied biology, 26(3):585 -15,, 1939.
Box. G. E., Hunter, W. G., and Hunter, S. Statistics for Experimenters, volume 664. John Wiley & Sons, New York, 1978.
Lundstedt, T., Seifert, E., Abramo, L., Thelin, B., Nyström, Á., Pettersen, J., and Bergman, R. Experimental design and optimization. Chemometries and intelligent labora- tory systems, 42(1-2)33-40, 1998.
Chang, M.-C. and Cheng, C.-S. A bayesian approach to the selection of two-level multi-stratum factorial designs. The Annals of Statistics, 46(4):1779-1800) 2018.
Mitchell, J., Morris, M. D., and Ylvisaker, D. Two-level fractional factorials and bayesian prediction. Statistica Sinica, pp. 559-573, 1995.
Cheng, C.-S. Theory of factorial design. Chapman and Hall/CRC Boca Raton, FL, USA, 2016.
Montgomery, D. C. Design and Analysis of Experiments. John Wiley Sons, 2017.
Correa, J. and Bareinboim, E. A calculus for stochastic interventions: Causal effect identification and surrogate experiments. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 10093-10100, 2020.
Mukerjee, R. and Wu, F.J. A Modern Theory of Factorial Design. Springer Science & Business Media, 2007.
O' O'Donnell, R. Some topics in analysis of boolean functions. In Proceedings of the fortieth annual ACM symposium on Theory of computing, pp. 569-578, 2008
Dean, A. and Voss, D. Design and analysis of experiments. Springer, 1999.
Rood, J. E., Hupalowska, A., and Regev, A. Toward a foundation model causal cell and tissue biology with a perturbation cell and tissue atlas. Cell, 187(17) 4545, 2024.
Deming, S. N. and Morgan, S. Experimental design: a chemometric approach. Elsevier, 1993.
10

---

Probabilistic Factorial Experimental Design for Combinatorial Interventions
Shapovalova, Y., Heskes, T., and Dijkstra, T. Non- parametrie synergy modeling of chemical compounds with gaussian processes. BMC bioinformatics, 23:1-30, 2022.
Takahashi, K. and Yamanaka, S. Induction of pluripotent stem cells from mouse embryonic and adult fibroblast cultures by defined factors. cell, 126(4):663-676, 2006.
Vershynin, R. High-dimensional probability: An introduc- tion with applications in data science, volume 47. Cam- bridge university press, 2018.
Virtanen, P., Gommers, R., Oliphant, T. E.,, Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, et al. Scipy 1.0: fundamental algorithms for scientific computing in python. Nature methods, 17(3):261-272, 2020.
Yao, D., Binan, L., Bezney, J., Simonton, B., Freedman, J.. Frangich. C. Dey, K., Geiger-Schuller, K., Eraslan, Gusev, A., al. Scalable genetic screening for reg- ulatory circuits using compressed perturb-seq. Nature biotechnology, 42(8): 122--12922 2024.
Zhang, J., Cammarata, L., Squires, C., Sapsis, T. and Uhler, C. Active learning for optimal intervention design in causal models. Nature Machine Intelligence, 5(10): 1066-1075, 2023.

---

Probabilistic Factorial Experimental Design for Combinatorial Interventions
A. Proof for Section 3
A.I. Fourier Representation
Lemma A.1. f admits the representation f(x) = Escly) (sos(x), where Bs 1 20
Eye f(y)os(y).
Proof. Plugging in the value of As, we have
f(x) =
Σ
1
Σ
f(y)6s())
0s(x)
2
SCIpl
yE(-1,1)
Σ
f(y)
2F
SCIP yE
1,11"
Σ f(y)
Σ
II
2
yE(-1,17
SC
EE
20 {x)
f(x),
as Esgin Hes 2iy; = 2 x = and otherwise.
Lemma A.2. Consider a model on {-1,1}?, where vs((x) =1 1ffffx for all E S, and
g(x) Σ ag$ss(x).
SEiP
This model is specific case of our model, where a low-interaction constraint this model implies low-interaction constraint on our model.
Proof. We have
o(x) = Σ
asts
SC(P)
(x; + 1)
Σ
0g
2
(c
iE8
Σ
Σ II
2
as TES
SCIP)
Σ
as
(x))).
21s
T TCIpi
92T
Therefore,
g(x)
Bsús(x)
SC
where
As
Σ
2T
T2S
Note that ag = 0 for all |S] >> implies that B3 = 0 for all
|S| >
12

---

Probabilistic Factorial Experimental Design for Combinatorial Interventions
B. Proofs for Section 4
B.1. Properties of E(d
Lemma B.1. Let [d = [x(x)* Φ(x), where 6(x) the row vector composed of (x(x) for all S with [S] S k and x is
distributed according to the dosage d. Then the minimum eigenvalue of E(d) is at most 1, with equality iff 11 2 pp
Proof. First note that E(d is given by
L(d)s,s' = II (2d, 1)
IESAS'
Therefore, E is symmetric with diagonal elements equal 1. In addition. E(d) is positive semidefinite and hence has real, non-negative eigenvalues. Combined with the fact that the trace of E(d) is M, the mean of the eigenvalues must be 1. Therefore, the minimum eigenvalue is equal to if and only if all the eigenvalues are equal to 1. A real symmetric matrix has spectrum of only I's if and only if it is the identity. Noting that (( 2d, 1, E(d) I if and only if d; = 4
concluding the proof.
Lemma B.2. With E(d) defined above, we have Auin (E(d)) - min;(1 - 12d, 1|).
Proof. We proceed with proof by contradiction. Let c* min, (1 [2d 1I) and 1 argmin(1 - |2d, 1)).
(X(d)) > ct, then (7 c* IK is positive definite because 2dd) is positive semidefinite. Therefore, all leading principal minors of E(d) - c² must have positive determinants. Consider the 2 x 2 submatrix defined by the rows/columns
[2d; 1|
2d
corresponding to 0 and (i*) In Σ c'I this is
which has determinant 0. Note that this submatrix
2d 1
[2d 1|
a principal minor in a permuted version of Σ c² I, which is also positive definite. Therefore, we have a contradiction as
Σ E-c²l c-'Ixx k is not positive definite, and hence Amin (E(d) & min,(1 -2d, 1))-
B.2. Proof of Lemma 4.1
Lemma B.3 (Truncated OLS). Given a fixed design matrix X, the truncated OLS estimator satisfies the following property:
By [13 - -312] =
ΣE
o² M x)
if E
K x
x) S
B²a
|1.1113.
otherwise.
In particular, there is min x. o² xi
|| S EE [IIB 3
Li 7x X3 B2}.
Proof. We utilize the eigen cigen-decomposittonn UDUT of XT X. We have
E
3
OLS
3
E x1 c12]
E x1=x(222 x2 -xx=
E
-r
x)
s ar xx x) xx-x))
K
2 Xx(
Therefore, if DK
<
Ba S.. we use the OLS estimator which has MSE of ΣK x²
Xイ( Otherwise, if
DK x1(xxx
B² our estimator is 0R which has squared error of ||01] < Ba, This gives the desired result.
13

---

Probabilistic Factorial Experimental Design for Combinatorial Interventions
Lemma B.4 (OdS+Ridge estimator). Given fixed n. K design matrix X, the OLS OLS+Ridge estimator is defined by
B
OTS
B'n (X
Amin otherwise.
1
ridge
B
and satisfies
Ko² <
B Bnn
Ey
||3
B11
min
(9)
Amin (X"
X))
Ba Amimin (X" x)2 + Kno
Proof. The bound on OLS follows easily from the proof of Proposition B.3, where we have that
Ko²
Amin (XTx
Now, we analyze the ridge estimator. Recall the definition:
xIx y
where À is a chosen regularization parameter. The bias-variance decomposition gives us
E
3
rigge
3
=
E
3 ridgs -
=||
+
E
3 ridgo
ridge
We analyze each term separately. For the bias term, we have
E
3
ridge
3 = ((XTX' + xIx -1X Ix)
= =-X(X x + AB
that
E rigge
3
X2 32TT + AIx)
AIK)
X2 11111
1X x AIK)~
max |=|=1
x² 11B1 Amax (1x X + AI xx'x
X² 1
(Amin x² A)2
Now for the variance, we have (where once again, the eigen-decompossition x -UDUT)
rigec
3 idge
2
E ||(2T -
E
E
E [xxTxXXT X x
E |
-tr #
5 2X
-our (0-+))
Per [k) AIx)
xxxxx
(X d)2
(mmin (x) X)2'
14

---

Probabilistic Factorial Experimental Design for Combinatorial Interventions
With the knowledge that || B2, we choose
À r X)
B Amiu (X²
which gives us an overall bound of
(²
B Xmin XTT + o
Lemma B.5 (Concentration of (T) x). Let X be the (rundom) design matrix generated by dosage d. Then
P (1=22 x 1
st 21)
nt?2
K
8K2
where the first norm is the spectral norm, and E(d) is defined as in Lemmna B.
Proof. This proof loosely follows the proof of Theorem 4.5.1 in (Vershynin, 2018). Let A' be the (random) design matrix generated by dosage d. Recall that X X is a K x K matrix. Now, let M be 1 net on the unit sphere 5 with [W| & 9 9 (Vershynin, 2018). We have
- x 2(a> n
< x -D(1) 2,2
2 max sEN
max zEN
|xx0| x² ('E())
(10)
n
where the first norm is the spectral norm. This chain of inequalities follows from the definition of an e-net and the triangle inequality (Vershynin, 2018). Let X; denote the ith row of X, and define Z; == (X,, Then we have || || =
D (Ki, x)² Σi= 2 where ZZ S by Cauchy-Sehward [d)).. Therefore, by Hoeffding's inequality, have that
follows that i xf E(d)xl S 2K, as E[ =
P
1
Σ
2 -
E(d)ar 21 < 2exp
nt²
2K
Now, applying union bound over N and substituting into (9), we have
nt²
x x x
St
2 - 9k 2exp
8K2
nt2
= 1 1
Kln9
8K2
B.3. Proof of Theorem 4.2
Proof. We have that, under the half dosage.
=11
K
nE
B
< min
BB2
(x x)
K
d²
[|] xTxxx xTxx x I(a) > nB²
<P P
x x Tx
E(d)
+P
n
Am
Ka²
ni²
<
+
nB²
exp
exp
Kln
(11)
8 8
8K²
where we use Lemma 1. Lemma B.5, and Weyl's inequality (Amin(() Amia(B)) S ||A -B|| for real, symmetric A, B) the last step.
15

---

Probabilistic Factorial Experimental Design for Combinatorial Interventions
Next, we lower bound for any other dosage d. Let min; [2d, 1[). We have
K o²
11
nE
3
2 min
[1/11
(xx
X)
K
P
x' x E(d)
<
min
X (d) 0" 12 [|311²]
nd²
exp
Kln
min
{((E(d) 11311
8K2
nd²
a²
o²(K -1)
exx
K
min
+
n 1131
(12)
8K2
c
1+8
where the last step is by Lemma B.2 and Cauchy-Schii Schwarz:
K
K (E(()) +8)
Σ
Σ (.(E(()) +6
2 2k² K2
(13)
with equality if and only if (E)()) are equal for all i. In particular, because Amiu (E(d)) S c by Lemma B.2, we have that
K
Σ
1
1 K-1
(E(())
+8 c (E(dd) +8
c+
1+
applying Cauchy-Schward we did (13).
Setting 5 = 8, in 11 and ó2 in 12, the 1 dosage is optimal within factor of
Ka² 1-8 + nB2
exp
K
8K
no?
1 2 exp
K In
n63 (
min
Ko² 1+5
[||B1]3
S
which is the result of dividing expression 11 by expression 12, and plugging in c 1 in 12. We further have that for large enough, if
-1
Ko² 1-6 + nB
exp K In
OK(K- -1
o²
82
K In
ns3 8K?
82
1 1 2exp
then d results in a lower mean squared error than the 1 dosage. This expression is the result of solving for e such that expression 12 is greater than expression 11.
(2ln(1)) n
1/2 have optimality a factor of
Choosing 8, 52
1 1 O In(n)
(110 ((n(m) In(m))
and that the optimal solution d' must satisfy 2 100
i.c.
In n
d' -
<
2
16

---

Probabilistic Factorial Experimental Design for Combinatorial Interventions
This result be extended allow for arbitrary distributions over combinations, rather than product distributions over treatments as induced by dosages:
Theorem B.6. Allowing for any distribution over combinations, the uniform distribution over combinations is optimal to factor of at most 1 + O [n(n)
In particular, as - oc, the uniform distribution minimizes the mean squared error of the
truncated OLS estimator.
Proof. The same argument used to show Lemma B.1 can be extended to arbitrary distributions, Let E(g) = Ex(x)" T "(x) where (xx) is distributed according the distribution y over combinations. Then, once again, E(g) has trace K for all g. and Ex A. (x())) minimized when E(g) is the identity matrix (refer to the Cauchy-Schwarz argument above). This achieved by g = U(1-1,1)") i.c. the uniform distribution over combinations. Therefore, may repeat the argument above to get the same result on the optimality factor of the uniform distribution in this more general case.
B.4. Proof of Theorem 4.4
Proof. Let P p= Eisl XT X,, where X; is the design matrix from round i. Now, define
d argmin d
( + P)
We begin by showing an upper bound on when the design matrix at round t is generated by d'. Let X' denote the cumulative design matrix after rounds, so that X = X2 X4 + nP, We have
nE || 8
o²
S min
B²}
XXT
( X + +P) - >8) nB²
Amin (XT x)
0²
nd²
+ nB2 exp
Kln9
(14)
( + P) -
8K2
Where we use Lemma B.5 and Weyl's ' inequality, as in the proof of Theorem 4.2. Next, we lower bound for any other dosage d. We have
1
K
0²
n.
β - 13
2 n min
|1
X (X' X)
o²
> P
xTx x (E(d) P)
<0 min
(x(E(d) + P) n [|
n
nd² 8K2
K
x²
2
1 1- eexp
Ing
min
(E + P) + n [|
(15)
Setting 5 = 51 in 14 and 5 = dg in 15, the d" is optimal to within a factor of
ΣK ((S(d nB² exp K In
x²
nd
SK
1 exx K In
nd SR²
min EK
g2 ((S n ||
which is the result of dividing expression 14 by expression 15.
17

---

Probabilistic Factorial Experimental Design for Combinatorial Interventions
Choosing 81 = 62 = (2ln(n) 1/2
we have optimality of d" to a factor of at most
1+0 0 In(n)
C. Proof for Section 5
C.I. Proof of Theorem 5.1
Proof. Following the proof of theorems 4.2, and noting that both terms in Eq. (9) are decreasing in Amin
Amin(X2 (X* p leads to E(d) with the highest minimum eigenvalue. When k = 1, [(d can be written as below. Let G = 1 (2d, - 1)?. Define the following two matrices: y is the p-length column vector with y, = 2d. 1, and C the diagonal matrix with CN+ = c. Then
X), it suffices
to show that among dosages satisfying Σ d: L, the uniform dosage with values
1 y
yd
E(d)
vy
+
We compute (ed() >Ix) using the formula for the determinant of block matrix and the matrix determinant lemma. Let G 1 (2d,- 1)2. Then for # 1, c2 for any i, we have
(c((E) AIx) = (1 A) dey +
AIx
1 x yz")
x
Σ Ci
(1 - A) (c A)
s=1 X
Therefore, the eigenvalues can be 1, c2 for any i, or the solutions to 1 -
d
1-q
= 0. Define
I-X
Li=1 q-x
1 4
(a(A) =
C -
where q's are defined according to d.
WLOG, assume 61 S Cg ... S Cp. We first note that the minimum eigenvalue must lie in (0, as (A() root in this interval. This is because g(0) 1 (unless the d; = 0 or for some i, in which case E(d) is singular) and limx-+e ((A) = -c..
(A) must have
Now, let d be the uniform dosage with elements f.. so that c =1 ( 21 -1) 1)² The minimum eigenvalue here is given by
1 ( + +1 p(1 p(1++ Vle + +1 -1))))) 40 x- 2
so it suffices show that for any dosage (satisfying the constraint) that
X (a()))
implying that there is root to (A) that less than or equal to X Note that ga- (X*)= 0, it suffices to show that
ci > x ga(A)) S ga- (A*).
Now, treating g as a function of c = (C1, (21,,2, Ca,,..,,..00 cp) parametrized by X, it suffices to show that g concave in c. This would imply that a maximizer exists at a uniform dosage (since g is symmetric in c), and that dosage must be as h(c) = 1-c
-2 decreasing in c. We have concavity of g the Hessian is diagonal matrix with the äth element being [c:-x-}3 S 0 as
C1 > X*. Therefore, the minimum eigenvalue E(d) is indeed maximized at d and we may proceed with the proof as in Theorem 4.2.
18

---

Probabilistic Factorial Experimental Design for Combinatorial Interventions
D. Experiment Details
Code can be found at the linked repository. Below we give a few additional details of our experiments.
Hardware and libraries. Experiments were run on device with a 16 core Intel Core Ultra 7 165H processor with 32 GB RAM, and NVIDIA RTX 4000 Mobile Ada Generation 12 GB GPU. The code is implemented in Python, utilizing the cupy and numba libraries, among others. The active design optimization was done using scipy SLSQP solver.
19


Reviews:
[{"review_id": "QQIVrBE3XG", "reviewer": "Reviewer_bJhy", "summary": "The paper introduces a probabilistic factorial experimental design to address the optimal experimental design problem for combinatorial interventions. \nThe contribution of the paper:\n1. The paper introduces a probabilistic factorial experimental design for a given choice of dosage vector.\n2. The paper provides a closed-form solution for the near-optimal design for passive and active settings.\n3. The authors explore extending the design framework to incorporate constraints and noisy scenarios.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for appreciating our theoretical framework and scalability challenges it addressed. Below, we address the concerns and questions brought up by the reviewer.\n\n> The use of Boolean functions and Fourier transforms is not new, as similar approaches have been explored in prior work, such as Agarwal, A., Agarwal, A., and Vijaykumar, S.Synthetic Combinations: A Causal Inference Framework for Combinatorial Interventions.\n    \nWe thank the reviewer for this comment. As noted in section 2 (last paragraph), we used Boolean functions to model combinatorial interventions, as it can easily model the scenario where the outcome is mainly driven by low-order effects in the absence of higher-order interactions -- a common assumption in fractional factorial design. In addition, it captures generalized surface models (see section 3.1). However, the main contribution of the paper lies not in the use of Boolean functions, but in the proposal of the novel probabilistic experimental framework and the accompanying theoretical analysis of the dosage choice (described in detail in section 1), which, to our knowledge, has not been explored in prior work.\n\n> The proposed design strategies appear to depend on the choice of dosage (may require prior knowledge from experimenters), which is a subset of the full factorial design. As a result, the outcomes of the proposed approach may be suboptimal. Could the authors elaborate more on this and discuss how to address it?\n\nA full factorial design can be formulated within our framework. In particular, there would be $2^p$ rounds, each with $1$ sample. In order to fix the sample, the dosage would be chosen to be deterministic, i.e. $\\mathbf{d}\\in \\\\{0,1\\\\}^p$. In addition, when the number of samples is large enough for a full factorial design to be implemented, the half dosage is closely related to this design as the half dosage induces a uniform distribution over combinations. Therefore in such cases, the two approaches perform similarly. Per our theoretical results, we suggest the experimenter uses the half dosage, which requires no prior knowledge.", "claims_and_evidence": "Yes, the paper provides the theoretical proofs and empirical evidence to support the claims.", "methods_and_evaluation_criteria": "The authors validated the proposed approach using a simulated dataset for both passive and active settings.", "theoretical_claims": "Yes", "experimental_designs_or_analyses": "Yes", "relation_to_broader_scientific_literature": "Building on previous work, this paper utilizes Boolean functions and Fourier transforms to establish the theoretical foundation of its approach.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "**Strengths:**\n1. The paper addresses a significant gap in scalability challenges in factorial design with combinatorial interventions.\n2. The theoretical framework is robust, with clear assumptions and derivations.\n\n**Weaknesses:**\n1. The use of Boolean functions and Fourier transforms is not new, as similar approaches have been explored in prior work, such as *Agarwal, A., Agarwal, A., and Vijaykumar, S.Synthetic Combinations: A Causal Inference Framework for Combinatorial Interventions*.", "other_comments_or_suggestions": "No", "questions_for_authors": "1. The proposed design strategies appear to depend on the choice of dosage (may require prior knowledge from experimenters), which is a subset of the full factorial design. As a result, the outcomes of the proposed approach may be suboptimal. Could the authors elaborate more on this and discuss how to address it?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Loeina1C0L", "reviewer": "Reviewer_D2ja", "summary": "This paper is concerned with the problem of experimental design in the high dimensional factorial setting where users may be administered combinations of treatments, and the aim is to administer a subset of treatments such that all combinations are recovered. The authors frame this problem in terms of the Fourier transform of boolean functions and assuming that the treatment status can be relaxed to probabilities of treatment. After this transformation the authors use tools from optimal experimental design for the selection mechanism. Extensions are provided to subsets and heteroskedastic settings. Empirical results show strong performance.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for appreciating our method and the thoughtful suggestions. We would like to address the concerns and questions of the reviewer as below.\n\n> The authors should have a broader literature review of the partial factorial design literature.\n\nWe thank the reviewer for this suggestion. We will add the following paragraph to Section 2 to expand our discussion of the partial factorial design literature. In addition, we are happy to include any specific references the reviewer believes would further strengthen our coverage of related work.\n\n\"A $2^{-m}$ fractional design is one where $2^{p-m}$ samples are used, each with a different combination [1]. These combinations are carefully selected to minimize aliasing. Aliasing occurs when, for the combinations selected, the interactions are linearly dependent [2][3]. In a full factorial design, there is linear independence so there is no confounding when the model is fit. In a fractional design, some aliasing will always occur in a full-degree model; however, methods proposed in literature select combinations such that the aliasing of important effects (i.e. degree-1 terms) does not occur [2]. With a low-degree assumption, aliasing can be avoided entirely. Fractional designs can be classified by their *resolution* (denoted by $R$), which determines which interactions can be potentially confounded. For example, a Resolution V fractional design eliminates any confounding between lower than degree-3 interactions, appropriate for degree-2 functions [4]. Of particular interest in literature are *minimum aberration designs*, which minimize the number of degree-$l$ terms aliased with degree-$R-l$ terms [5][6].\"\n\n> I would have like to seen a more complete comparison to partial factorial experiments.\n\nWe thank the reviewer for this suggestion. We have conducted an additional experiment, where we compare the half dosage versus a partial factorial design in the passive setting. Here, we generate a degree-$1$ Boolean function with $p=8$. We use a $2^{8-2}$ Resolution $V$ design with $64$ samples for each approach. Results are shown below, averaged over $300$ trials and with $\\pm 1$ std.\n\n| Fractional design |Half dosage|\n| -------- | -------- | \n|$.14\\pm .062$| $.16\\pm.078$|\n\nWith fewer samples, the careful selection of combinations will make a difference, so the fractional design can outperform the half dosage. But in many cases, especially in biological applications, careful selection of combinations is not possible which is why the much more flexible dosage design is preferable, as it enables the administration of an exponential number of combinations by choosing a linear number of dosages.\n\nHowever, in the active setting, the optimal dosage can outperform a fractional design. Please see the experiment in response to Reviewer CWrH, under \"... the authors could include more existing active learning methods as baselines\".\n\n> I am curious how this approach (specifically the active learning setting) interacts with adjustment using user covariates.\n\nWe thank the reviewer for this question. We could assume the following setup in the passive setting: there are $m$ users with known covariates $\\mathbf{c}\\_i\\in \\mathbb{R}^l$, each of which receives the $n$ combinations determined by the dosage (so that we have a total of $mn$ samples). Assuming the covariates have a linear relationship with the outcome, i.e. $y\\_i = \\beta\\_u\\mathbf{c}\\_i+f(\\mathbf{x})$, then the optimal dosage in the passive setting is $\\mathbf{d}\\_u^* = \\text{argmin}\\_{\\mathbf{d}\\in [0,1]^p} \\sum\\_{i = 1}^{l+K}\\frac{1}{\\lambda\\_i(A)}$ where $A=\\begin{bmatrix}\n    \\sum\\_{i=1}^m\\mathbf{c}\\_i\\mathbf{c}\\_i^T&\\sum\\_{i=1}^m \\mathbf{c}\\_i\\mathbf{y}(\\mathbf{d})^T\\\\\\\\\n    \\sum\\_{i=1}^m \\mathbf{y}(\\mathbf{d})\\mathbf{c}\\_i^T&\\Sigma(\\mathbf{d})\n\\end{bmatrix}$, with $\\mathbf{y}(\\mathbf{d})\\_S = \\prod\\_{i\\in S} (2d\\_i-1)\\in \\mathbb{R}^{K}$ and $\\Sigma(\\mathbf{d})$ is as defined in the paper. We conjecture that $\\mathbf{d}_u^*$ is still the half dosage. To extend to the active setting, the same objective is used as in the paper except $\\Sigma(\\mathbf{d})$ is replaced with $A$. We are happy to consider alternative models of user covariates if the reviewer has any specific suggestions.\n\n---\nReferences:\n\n$[1]$ Box, George EP, William H. Hunter, and Stuart Hunter. Statistics for experimenters. Vol. 664. New York: John Wiley and sons, 1978.\n\n$[2]$ Gunst, Richard F., and Robert L. Mason. \"Fractional factorial design.\" Wiley Interdisciplinary Reviews: Computational Statistics 1.2 (2009): 234-244.\n\n$[3]$ Mukerjee, Rahul, and CF Jeff Wu. A modern theory of factorial design. Springer Science & Business Media, 2007.\n\n$[4]$ Montgomery, Douglas C. Design and analysis of experiments. John Wiley & Sons, 2017.\n\n$[5]$ Fries, Arthur, and William G. Hunter. \"Minimum aberration $2^{k–p}$ designs.\" Technometrics 22.4 (1980): 601-608.\n\n$[6]$ Cheng, Ching-Shui. Theory of factorial design. Boca Raton, FL, USA: Chapman and Hall/CRC, 2016.", "claims_and_evidence": "All theoretical claims made are well supported by theory provided in the paper.", "methods_and_evaluation_criteria": "Yes, the method is quite sensible (and interesting), evaluation criterion is appropriate.", "theoretical_claims": "Yes, I reviewed all proofs and they are sound to my reading.", "experimental_designs_or_analyses": "Yes. The experiments are sound, though I would have like to seen a more complete comparison to partial factorial experiments.", "relation_to_broader_scientific_literature": "This paper addresses an interesting and highly relevant problem of factorial experimental design. While the problem itself dates back to Fisher, the authors provide a nice contribution to the literature.", "essential_references_not_discussed": "The authors should have a broader literature review of the partial factorial design literature.", "other_strengths_and_weaknesses": "Overall, I think this paper is a creative approach to the problem of design of factorial experiments. \nMy main complaint, as I mention above, is that the experimental evaluation here is severely limited.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "I am curious how this approach (specifically the active learning setting) interacts with adjustment using user covariates. Does this change the design considerations?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "N/A"}}, {"review_id": "LQPjJYtjos", "reviewer": "Reviewer_CWrH", "summary": "This paper introduces probabilistic factorial experimental design for combinatorial interventions, where each treatment is assigned a dosage between 0 and 1, and units randomly receive treatments based on these probabilities. \nThis framework generalizes both full and fractional factorial designs by allowing random assignment of treatment combinations rather than deterministic selection. \nThe authors model outcomes using Boolean functions with Fourier expansions to capture bounded-order interactions. \nThey prove that uniform half-dosage allocation ($d_i = 0.5$) is near-optimal in single-round experiments, with optimality up to a factor of $1 + O(\\frac{\\operatorname{ln}(n)}{n})$. For multi-round experiments (i.e., active learning setting), they develop an acquisition strategy that adapts dosages based on previous observations. The work also addresses practical constraints like limited treatment supply. Experiments results on simulated datasets demonstrate that the proposed strategies outperform random dosage selection.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for appreciating our theoretical analysis, as well as for their many valuable suggestions. Below, we address the reviewer's concerns and lay out modifications we will make according to the reviewer's suggestions.\n\n> My only concern is that all the evaluations are conducted on synthetic data. It would be great if the authors could conduct some experiments on semi-synthetic or real-world datasets.\n\nDue to the character limit, we refer the reviewer to our response to Reviewer jf24's first point under \"Simulation results on real-world dataset are missing\".\n\n> The simulations use synthetic data generated from the same model class assumed in the theory, but this might not reflect the robustness of the proposed framework to model misspecification.\n\nWe thank the reviewer for pointing this out. While Boolean functions are universal approximators, our low-degree assumption can cause misspecification, as recognized by the reviewer. In many applications the low-degree assumptions holds, especially in biology, but degree misspecification may still exist. To address the reviewer's concern, we have conducted an additional experiment where the model is misspecified. Please see the response to Reviewer jf24, under the comment about \"sensitivity analysis.\" Here, we use a real-world full-degree Boolean function ($k=p$), and fit assuming lesser values of $k$. \n\n> In the active setting, the authors could include more existing active learning methods as baselines beyond random and half-dosage strategies for a more comprehensive evaluation.\n\nWe appreciate the reviewer's suggestion. For the comparison with passive baselines, we have added an additional baseline based on partial factorial design. Results are shown in the table below. For the comparison with active strategies, since multiple combinatorial interventions are drawn in each round (administered by a selection of dosage), we are not aware of existing methods that can be easily adapted to this setting. However, we would be happy to include additional baselines if the reviewer has specific suggestions.\n\nHere we compare a Resolution $V$ $2^{5-1}$ fractional design versus our optimal strategy and half dosages. Each round has $16$ samples, with $p=5$ and $k=1$.\n\n|  | Round 1 | Round 2 |Round 3| Round 4| Round 5|\n| -------- | -------- | -------- | -------- | -------- | -------- |\n| Optimal dosage     | $.54$| $\\bf{.16}$|$\\bf{.10}$|$\\bf{.08}$|$\\bf{.07}$|\n| Half dosage    |$.52$|$.23$|$.14$|$.09$|$.08$||\n|Fractional factorial design|$\\bf{.28}$|$.21$|$.15$|$.12$|$.09$|\n\nBolded entries show the lowest loss among each round, where we see that our optimal dosage strategy outperforms the other strategies after the first round.\n\n> The computational complexity of the active learning approach is not thoroughly discussed.\n\nThe number of iterations for the optimizer to converge is roughly $O(p^3)$, and the complexity of each iteration is $O(nK^2 + K^3)$ (where the first term comes from the matrix multiplication of $\\mathcal{X}^T\\mathcal{X}$ and the second term comes from computing the eigenvalues of $\\Sigma(\\mathbf{d})$). Recall the definition of $K$ to be the number of interactions under consideration, i.e. $K = \\sum_{i=0}^k {p\\choose i} = O(p^k)$ for small $k$. Therefore, the overall complexity is $O(np^{3k+3}+p^{6k+3})$ for small $k$. In practice, we may recommend using a proxy, which only involves the inverse of the minimum eigenvalue: $\\bf{d}_{T} = \\text{argmin}\\_{\\bf{d}\\in[0,1]^p}\\frac{1}{\\lambda\\_{\\min}\\left(\\Sigma(\\bf{d})+\\frac{1}{n}\\sum\\_{t=1}^{T-1}{\\mathcal{X}_t^\\top\\mathcal{X}_t}\\right)}$. We found that numerically optimizing this was significantly faster and that the solver was consistently accurate. While the complexity computed above should be the same for this approach, in practice it takes many less iterations to converge.\n\n> The empirical results suggest ... A more thorough discussion of this trade-off between computational complexity and performance gain would strengthen the paper.\n\nWe thank the reviewer for this suggestion and will include a discussion of this trade-off in our paper. In the case where there are not many samples (compared to features) per round, we find that the optimal acquisition strategy more clearly outperforms the half strategy. This is because when we have a smaller number of samples, we will need to \"correct\" as the distribution of combinations will be more lopsided and further away from the uniform distribution. Therefore, in scenarios where each round has few samples, we think it is worth computing the optimal acquisition dosage. When we have a large $n$ relative to $p$, the half strategy and optimal strategy perform very similarly. While the computational complexity of finding the optimal strategy can quickly scale, in practice it only takes a matter of seconds to compute.\n\n---\nReference:\n$[1]$ Montgomery, Douglas C. Design and analysis of experiments. John Wiley & Sons, 2017.", "claims_and_evidence": "The paper's claims are generally well-supported by theoretical analysis and empirical evidence.", "methods_and_evaluation_criteria": "The methods and evaluation criteria in this paper are appropriate for the problem of optimal experimental design for combinatorial interventions. My only concern is that all the evaluations are conducted on synthetic data. It would be great if the authors could conduct some experiments on semi-synthetic or real-world datasets.", "theoretical_claims": "I checked the proof of Theorem 4.2, which establishes the near-optimality of half-dosage allocation. The proof appears sound, using concentration inequalities and eigenvalue properties to bound the estimation error.", "experimental_designs_or_analyses": "I checked the experimental designs in Section 6, both the passive setting and active setting simulations. The authors appropriately test the theoretical claims by comparing estimation errors across different dosage strategies. I have some minor concerns:\n\n1. The simulations use synthetic data generated from the same model class assumed in the theory, but this might not reflect the robustness of the proposed framework to model misspecification.\n\n2. In the active setting, the authors could include more existing active learning methods as baselines beyond random and half-dosage strategies for a more comprehensive evaluation.", "relation_to_broader_scientific_literature": "This paper extends classical factorial design literature by introducing a probabilistic framework that addresses scalability issues in traditional full and fractional factorial designs. The active learning component relates to Bayesian experimental design and sequential experimental design, though with acquisition functions specific to the probabilistic factorial framework. The work also complements recent advances in causal inference for combinatorial interventions and provides theoretical foundations for experimental practices used in biological perturbation experiments.", "essential_references_not_discussed": "All the essential related works are discussed.", "other_strengths_and_weaknesses": "Strengths: The theoretical analysis in this paper is rigorous and well-organized, making it easy to understand the theoretical results and their practical implications.\n\nWeaknesses:\n\n1. Please see my comments in the previous parts.\n\n2. The computational complexity of the active learning approach is not thoroughly discussed.\n\n3. The empirical results suggest that the optimal acquisition strategy only outperforms the half strategy slightly in the active setting. This raises questions about whether the half strategy might be preferable in practice since it requires no computation or learning procedure. A more thorough discussion of this trade-off between computational complexity and performance gain would strengthen the paper.", "other_comments_or_suggestions": "Please see Other Strengths And Weaknesses.", "questions_for_authors": "please see Other Strengths And Weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FYtoIUiAXV", "reviewer": "Reviewer_jf24", "summary": "This paper studies the combinatorial intervention problem. The authors propose a probabilistic factorial experimental design, where each unit independently receives a random combination of treatments according to specified dosages. They derive a closed-form solution for the near-optimal design in the passive setting and a numerically optimizable solution for the near-optimal design in the active setting. Simulation results are provided to validate their findings.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for finding our extensions enlightening. We find the reviewer's suggestions insightful and accordingly lay out additional experiments and their results. \n\n> Simulation results on real-world dataset are missing.\n\nWe thank the reviewer for sharing this concern. Our paper is concerned with how an experimenter could optimally construct a dataset through choice of dosage, and therefore, experiments with real-world data necessitate close collaboration with individuals engaged in experimentation. However, we could have simulated the performance of various dosages using a real-world dataset, if this dataset included samples with all combinations. For example, for each combination generated by a given dosage, we could draw a corresponding point from the dataset to construct a new dataset consistent with the dosage. Unfortunately, we are not aware of such complete datasets. We plan to collaborate with biological experimenters in the future to create datasets based on our results.\n\nWhile existing real-world datasets are not feasible for experimentation, we added a semi-synthetic simulation in the following way: we use a real-world Boolean function with $p=5$, which is a reliability function originally presented in Quality and Reliability Engineering International of full-degree [1]. We create datasets based on this function, where each sample corresponds to a combination (according to the dosage) and the corresponding value of the function, with noise added. We conduct experiments using this function, results of which can be found in tabular format below. We note that $p=5$ is relatively low, but we were unable to find a completely-defined Boolean function of higher dimension.\n\n> However, the paper could be strengthened by including a sensitivity analysis that varies k.\n\nWe thank the reviewer for this suggestion. We have accordingly conducted the following experiment: we use a real-world Boolean function with $p=5$ and of full degree (described above). We replicate the first experiment in our paper, where we investigate the effect of $||\\mathbf{d}-\\frac{1}{2}||_\\infty$. We investigate values of $k$ ranging from $2$ through $4$. We display the average loss ($\\mathbb{E}_x\\left[||f(x) - \\hat{f}(x)||_2^2\\right]$) over $200$ dosages at each distance (where we perform $20$ trials with each dosage), to be displayed in graphical format in our paper. Here we use $n=200$ samples.\n\n\n|| $0$ | $.02$ | $.04$ | $.06$ | $.08$ | $.1$ | $.12$ | $.14$ | $.16$ | $.18$ |\n| -- | -- | -- |--| -- | -- | -- |--| -- | -- | -- |\n|$k=2$     |$3.49$|$3.50$|$3.53$|$3.58$|$3.64$|$3.71$|$3.81$|$3.98$|$4.12$|$4.27$|\n| $k=3$     |$.59$|$.59$|$.61$|$.62$|$.64$|$.67$|$.74$|$.78$|$.87$|$1.11$|\n| $k=4$     |$.58$|$.58$|$.72$|$.65$|$.95$|$1.17$|$1.39$|$1.73$|$1.90$|$2.30$|\n\n\nEven when the model is misspecified, we see that the half dosage appears optimal and observe the loss increase as we move further away from the half dosage.\n\n> The x-axis in Figure 2 seems to be the dosage value rather than $||d-\\frac{1}{2}||_\\infty$\n\nWe thank the reviewer for catching this, which we will fix accordingly in the paper.\n\n> According to Figures 1 and 2, $\\bf{d}=(\\frac{1}{2}, \\ldots , \\frac{1}{2})$ appears to be exactly optimal, rather than merely near-optimal. Do the authors conjecture that this is indeed optimal for general k? Similarly, in the general constrained case, is the uniform dosage of $L/p$ conjectured to be optimal?\n\nBased on our experiments and the general heuristic that in linear regression, one would like features which are \"spread out,\" we conjecture that the half dosage is optimal and that the uniform dosage in the constrained case is also optimal for general $k$. It is difficult to prove exact optimality, as we must compare the quantity  $\\mathbb{E}_{\\mathcal{X}}\\left[\\sum\\_{i=1}^K \\frac{1}{\\lambda_i(\\mathcal{X}^T\\mathcal{X})}\\right]$ across different dosages. While we were able to show that the inner quantity concentrates as the number of samples grows, it is not clear to us how to compute the mean for a fixed number of samples. \n\n---\nReference:\n$[1]$ Montgomery, Douglas C. Design and analysis of experiments. John Wiley & Sons, 2017.", "claims_and_evidence": "They are generally well-supported to me. The simulation results clearly align with the theories.", "methods_and_evaluation_criteria": "The simulations are carefully designed to validate the theoretical results. For example, the stated near-optimality of setting the dosage to 1/2 for each treatment in the passive setting is clearly demonstrated in Figures 1 and 2. Additionally, the simulations for the active setting illustrate the effectiveness of adaptively choosing the dosages in accordance with the proposed theory.", "theoretical_claims": "The proofs appear sound to me.", "experimental_designs_or_analyses": "The experimental designs are generally sound. However, the paper could be strengthened by including a sensitivity analysis that varies $k$.", "relation_to_broader_scientific_literature": "The proposed probabilistic factorial design includes both full and fractional factorial designs in the literature as special cases. It serves as a flexible realization of a factorial design.", "essential_references_not_discussed": "I don't see any obvious missing references.", "other_strengths_and_weaknesses": "Strengths: The paper is generally well written and smooth to follow. Then extensions are enlightening.\nWeaknesses: Simulation results on real-world dataset are missing.", "other_comments_or_suggestions": "The x-axis in Figure 2 seems to be the dosage value rather than $\\\\|d-\\frac{1}{2}\\\\|_{\\infty}$.", "questions_for_authors": "According to Figures 1 and 2, $\\mathbf{d} = (1 / 2, \\ldots, 1 / 2)$ appears to be exactly optimal, rather than merely near-optimal. Do the authors conjecture that this is indeed optimal for general $k$? Similarly, in the general constrained case, is the uniform dosage of $L / p$ conjectured to be optimal?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: Short summary: This paper proposes a clever probabilistic factorial experimental design to address the optimal experimental design problem for combinatorial interventions.
All reviews unanimously vote to accept this paper (either as “weak accept” or “accept”). 
The reviews acknowledge that the paper addresses a key technical gap and provides a creative solution. The reviews also find the theoretical contributions solid and robust. The paper is also well-written making it easy to understand. 
One key limitation identified in the reviews is the severely lacking experimental section. The future improvements of the paper and follow up work could center around more exhaustive experiments and real-world datasets and deeper discussion of computational complexity.
Decision: Accept (spotlight poster)