SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\hat {\boldsymbol {\alpha}} _ {u} = \underset {\boldsymbol {\alpha}} {\operatorname {a r g m i n}} \frac {1}{| \Pi_ {u} |} \| \mathbf {Y} _ {\Pi_ {u}} - \boldsymbol {\chi} (\Pi_ {u}) \boldsymbol {\alpha} \| _ {2} ^ {2} + \lambda_ {u} \| \boldsymbol {\alpha} \| _ {1} \tag {3}
$$
where recall that  $\chi(\Pi_u) = [\chi^\pi : \pi \in \Pi_u] \in \mathbb{R}^{|\Pi_u| \times 2^p}$ . Then, for any donor unit-combination pair  $(u, \pi)$ , let  $\hat{\mathbb{E}}[Y_u^{(\pi)}] = \langle \hat{\alpha}_u, \chi^\pi \rangle$  denote the estimate of the potential outcome  $\mathbb{E}[Y_u^{(\pi)}]$ .
Step 2: Vertical Regression. Next, we estimate potential outcomes for all units  $n \in [N] \backslash \mathcal{I}$ . To do so, we define some required notation. For  $\Pi_S \subseteq \Pi$ , define the vector of estimated potential outcomes  $\hat{\mathbb{E}}[\mathbf{Y}_u^{(\Pi_S)}] = [\hat{\mathbb{E}}[Y_u^{(\pi)}]: \pi \in \Pi^S] \in \mathbb{R}^{|\Pi_S|}$ . Additionally, let  $\hat{\mathbb{E}}[\mathbf{Y}_{\mathcal{I}}^{(\Pi_S)}] = [\hat{\mathbb{E}}[\mathbf{Y}_u^{(\Pi_S)}]: u \in \mathcal{I}] \in R^{|\Pi_S| \times |\mathcal{I}|}$ .
Step 2(a): Principal Component Regression. Perform a singular value decomposition (SVD) of  $\hat{\mathbb{E}}[\mathbf{Y}_{\mathcal{I}}^{(\Pi_n)}]$  to get  $\hat{\mathbb{E}}[\mathbf{Y}_{\mathcal{I}}^{(\Pi_n)}] = \sum_{l=1}^{\min(|\Pi_n|,|\mathcal{I}|)} \hat{s}_l \hat{\mu}_l \hat{\nu}_l^T$ . Using a hyper-parameter  $\kappa \leq \min(|\Pi_n|,|\mathcal{I}|)^3$ , compute  $\hat{\mathbf{w}}^n \in \mathbb{R}^{|\mathcal{I}|}$  as follows:
$$
\hat {\mathbf {w}} ^ {n} = \left(\sum_ {l = 1} ^ {\kappa} \hat {s} _ {l} ^ {- 1} \hat {\nu} _ {l} \hat {\boldsymbol {\mu}} _ {l} ^ {T}\right) \mathbf {Y} _ {\Pi_ {n}} \tag {4}
$$
Step 2(b): Estimation. Using  $\hat{\mathbf{w}}^n = [\hat{w}_u^n : u \in \mathcal{I}]$ , we have the following estimate for any  $\pi \in \Pi$
$$
\hat {\mathbb {E}} \left[ Y _ {n} ^ {(\pi)} \right] = \sum_ {u \in \mathcal {I}} \hat {w} _ {u} ^ {n} \hat {\mathbb {E}} \left[ Y _ {u} ^ {(\pi)} \right] \tag {5}
$$
(a) Observation pattern
(b) Horizontal regression
Figure 1: (a) depicts an example of a particular observation pattern with outcome for unit-combination pair  $(n,\pi)$  missing. (b) demonstrates horizontal regression for donor unit  $u$  where we estimate potential outcome  $\mathbb{E}[Y_u^{(\pi)}]$ . (c) visualizes vertical regression where we transfer estimated outcomes from the donor set  $\mathcal{I}$  to  $(n,\pi)$ .
(c) Vertical regression
Suitability of Lasso and PCR. Lasso is appropriate for the horizontal regression because  $\alpha_{u}$  is sparse. However, Synthetic Combinations allows for any ML algorithm (e.g., neural networks, random forests) to be used in the first step. This flexibility allows an analyst to tailor the horizontal learning procedure, and include prior information. However, we leave this model-agnostic analysis as future work, and focus on the Lasso for simplicity. PCR is appropriate for the vertical regression because  $\mathcal{A}$  is low rank. As [3, 4] show, PCR implicitly regularizes the regression by adapting to the rank of the covariates  $(\mathbf{Y}_{\Pi_n})$ , i.e., the out-of-sample error of PCR scales with  $r$  rather than the ambient covariate dimension.
Determining Donor Set  $\mathcal{I}$ . Synthetic Combinations requires the existence of a subset of units  $\mathcal{I} \subset [N]$  such that we are able to (i) accurately estimate their potential outcomes under all possible combinations, and (ii) transfer these estimated outcomes to a unit  $n \in [N] \setminus \mathcal{I}$ . Theoretically, we detail sufficient conditions on the observation pattern such that we are able to perform (i) and (ii) accurately via the Lasso and PCR respectively. In practice, we recommend the following to determine  $\mathcal{I}$ . For every unit  $n \in [N]$ , learn a separate Lasso model  $\alpha_{n}$  and assess its performance through cross-validation (CV). Assign units with low CV error (with a pre-determined threshold) as the donor set  $\mathcal{I}$ ,
3Both  $\lambda$  and  $\kappa$  can be chosen in a data-driven manner (e.g., via CV) as discussed in [17] and [4] respectively.
and estimate outcomes  $\hat{\mathbb{E}}[Y_u^{(\pi)}]$  for every unit  $u \in \mathcal{I}$  and  $\pi \in \Pi$ . For non-donor units, PCR performance can also be assessed via k-fold CV. For units with low PCR error, linear span inclusion (Assumption 4.1(b)) and the assumptions required for the generalization for PCR likely hold, and hence we estimate their potential outcomes as in (5). For units with large PCR error, it is either unlikely that these set of assumptions holds or that  $|\Pi_n|$  is not large enough (i.e., additional experiments need to be run for this unit), and hence we do not recommend estimating their counterfactuals.
6 Synthetic Combinations Theoretical Analysis
In this section, we establish finite-sample consistency of Synthetic Combinations, starting with a discussion of the additional assumptions required for our results.
6.1 Additional Assumptions
Assumption 6.1 (Bounded Potential Outcomes). We assume that  $\mathbb{E}[Y_n^{(\pi)}] \in [-1, 1]$  for any unit-combination pair  $(n, \pi)$ .
Assumption 6.2 (Sub-Gaussian Noise). Conditioned on  $\mathcal{A}$  for any unit-combination pair  $(n,\pi)$ ,  $\epsilon_n^\pi$  are independent zero-mean sub-Gaussian random variables with  $Var[\epsilon_n^\pi \mid \mathcal{A}] \leq \sigma^2$  and  $\| \epsilon_n^\pi \mid \mathcal{A} \|_{\psi_2} \leq C\sigma$  for some constant  $C > 0$ .
Assumption 6.3 (Incoherence of Donor Fourier characteristics). For every unit  $u \in \mathcal{I}$ , assume  $\chi(\Pi_u)$  satisfies incoherence:  $\left\| \frac{\chi(\Pi_u)^T \chi(\Pi_u)}{|\Pi_u|} - \mathbf{I}_{2^p} \right\|_{\infty} \leq \frac{C'}{s}$ , for a universal constant  $C' > 0$ .
To define our next set of assumptions, we introduce necessary notation. For any subset of combinations  $\Pi_S \subset \Pi$ , let  $\mathbb{E}[\mathbf{Y}_{\mathcal{I}}^{(\Pi_S)}] = [\mathbb{E}[\mathbf{Y}_u^{(\Pi_S)}]: u \in \mathcal{I}] \in \mathbb{R}^{|\Pi_S| \times |\mathcal{I}|}$ .
Assumption 6.4 (Donor Unit Balanced Spectrum). For a given unit  $n \in [N] \setminus \mathcal{I}$ , let  $r_n$  and  $s_1 \ldots s_{r_n}$  denote the rank and non-zero singular values of  $\mathbb{E}[\mathbf{Y}_{\mathcal{I}}^{(\Pi_n)} \mid \mathcal{A}]$ . We assume that the singular values are well-balanced, i.e., for universal constants  $c, c' > 0$ , we have that  $s_{r_n} / s_1 \geq c$ , and  $\| \mathbb{E}[\mathbf{Y}_{\mathcal{I}}^{(\Pi_n)} \mid \mathcal{A}] \|_F^2 \geq c' |\Pi_n| |\mathcal{I}|$ .
Assumption 6.5 (Subspace Inclusion). For a given unit  $n \in [N] \setminus \mathcal{I}$  and intervention  $\pi \in \Pi \setminus \Pi_n$ , assume that  $\mathbb{E}[\mathbf{Y}_{\mathcal{I}}^{(\pi)}]$  lies within the row-span of  $\mathbb{E}[\mathbf{Y}_{\mathcal{I}}^{(\Pi_n)}]$ .
Assumption 6.3 is necessary for finite-sample consistency when estimating  $\alpha_{n}$  via the Lasso estimator (3), and is commonly made when studying the Lasso [36]. Incoherence can also seen as a inclusion criteria for a unit  $n$  to be included in the donor set  $\mathcal{I}$ . Assumption 6.3 for example holds (with high probability) if  $\Pi_{u}$  is chosen uniformly at random and grows as  $\omega(s^{2}p)$  as shown in Lemma 2 of [32]. Assumption 6.4 requires that the non-zero singular values of  $\mathbb{E}[\mathbf{Y}_{\mathcal{I}}^{(\Pi_n)} \mid \mathcal{A}]$  are well-balanced. This assumption is standard when studying PCR [3, 5], and within the econometrics literature [10, 22]. It can also be empirically validated by plotting the spectrum of  $\hat{\mathbb{E}}[\mathbf{Y}_{\mathcal{I}}^{(\Pi_n)}]$ ; if the singular spectrum of  $\hat{\mathbb{E}}[\mathbf{Y}_{\mathcal{I}}^{(\Pi_n)}]$  displays a natural elbow point, then Assumption 6.4 is likely. Assumption 6.5 is also commonly made when analyzing PCR [4, 5, 2]. It can be thought of as a "causal transportability" condition from the model learnt using  $\Pi_{n}$  to the interventions  $\pi \in \Pi \setminus \Pi_{n}$ . That is, subspace inclusion allows us to generalize well, and accurately estimate  $\mathbb{E}[Y_{n}^{(\pi)} \mid \mathcal{A}]$  using  $\langle \hat{\mathbb{E}}[\mathbf{Y}_{\mathcal{I}}^{(\Pi_n)}], \hat{\mathbf{w}}^n \rangle$ .
6.2 Finite Sample Consistency
The following result establishes finite-sample consistency of Synthetic Combinations. Without loss of generality, we will focus on estimating the pair of quantities  $(\mathbb{E}[Y_u^{(\pi)}],\mathbb{E}[Y_n^{(\pi)}])$  for a given donor unit  $u\in \mathcal{I}$ , and non-donor unit  $n\in [N]\backslash \mathcal{I}$  under treatment assignment  $\pi \in \Pi$ . To simplify notation, we will use  $O_{p}$  notation: for any sequence of random vectors  $X_{n}$ ,  $X_{n} = O_{p}(\gamma_{n})$  if, for any  $\epsilon >0$ , there exists constants  $c_{\epsilon}$  and  $n_{\epsilon}$  such that  $\mathbb{P}(\| X_n\| _2\geq c_\epsilon \gamma_n)\leq \epsilon$  for every  $n\geq n_{\epsilon}$ . Similarly, we define  $\tilde{O}_p(\gamma_n)$  which suppresses logarithmic terms. We will further absorb dependencies on  $\sigma$  into  $\tilde{O}_p(\cdot)$ .
Theorem 6.6 (Finite Sample Consistency of Synthetic Combinations). Conditioned on  $\mathcal{A}$ , let Assumptions 3.1-4.1, and 6.1-6.5 hold. Then, the following statements hold.
(a) For the given donor unit-combination pair  $(u,\pi)$ , let the Lasso regularization parameter satisfy  $\lambda_u = \Omega (\sqrt{\frac{p}{|\Pi_u|}})$ . Then, we have that:  $|\hat{\mathbb{E}} [Y_u^{(\pi)}] - \mathbb{E}[Y_u^{(\pi)}]| = \tilde{O}_p\Big(\sqrt{\frac{s^2p}{|\Pi_u|}}\Big)$ .
(b) For the given unit-combination pair  $(n,\pi)$  where  $n\in [N]\backslash \mathcal{I}$ , let  $\kappa = \mathrm{rank}(\mathbb{E}[\mathbf{Y}_{\mathcal{I}}^{(\Pi_n)}])\coloneqq r_n$ . Then, provided that  $\min_{u\in \mathcal{I}}|\Pi_u| := M = \omega (r_n^2 s^2 p)$ , we have that:
$$
\left| \hat {\mathbb {E}} [ Y _ {n} ^ {(\pi)} ] - \mathbb {E} [ Y _ {n} ^ {(\pi)} ] \right| = \tilde {O} _ {p} \left(\frac {r _ {n} ^ {2} \sqrt {s ^ {2} p}}{\sqrt {M \times \min \{| \Pi_ {n} | , | \mathcal {I} | \}}} + \frac {r _ {n} ^ {2} s ^ {2} p \sqrt {| \mathcal {I} |}}{M} + \frac {r _ {n}}{| \Pi_ {n} | ^ {1 / 4}}\right)
$$
Establishing Theorem 6.6 requires a novel analysis of error-in-variables (EIV) linear regression. Specifically, the general EIV linear model is as follows:  $Y = \mathbf{X}\beta +\epsilon$ ,  $\mathbf{Z} = \mathbf{X} + \mathbf{H}$ , where  $Y$  and  $\mathbf{Z}$  are observed. In our case,  $Y = \mathbf{Y}_{\Pi_n}$ ,  $\mathbf{X} = \mathbb{E}[\mathbf{Y}_{\mathcal{I}}^{(\Pi_n)}]$ ,  $\beta = \mathbf{w}^n$ ,  $\mathbf{Z} = \hat{\mathbb{E}} [\mathbf{Y}_{\mathcal{I}}^{(\Pi_n)}]$ , and  $\mathbf{H}$  is the error arising in estimating  $\mathbb{E}[\mathbf{Y}_{\mathcal{I}}^{(\Pi_n)}]$  via the Lasso. Typically one assumes that  $\mathbf{H}$  is a matrix of independent sub-gaussian noise. Our analysis requires a novel worst-case analysis of  $\mathbf{H}$  (due to the 2-step regression of Lasso and then PCR), in which each entry of is  $\mathbf{H}$  simply bounded.
Next, we describe the conditions placed on  $\Pi_n$ ,  $M$ ,  $\mathcal{I}$  to achieve consistent estimation of  $(\mathbb{E}[Y_u^{(\pi)}], \mathbb{E}[Y_n^{(\pi)}])$ . To simplify the discussion, we assume that  $\min \{| \Pi_n |, |\mathcal{I} | \} = |\mathcal{I}|$ . This condition can be enforced in practice by simply picking a subset of donor units such that  $|\mathcal{I}| \leq |\Pi_n|$  when performing PCR (i.e., step 2 of Synthetic Combinations). Given this assumption, it can be verified that  $|\Pi_n|$  and  $M$  need to scale as  $\omega(r_n^4)$  and  $\omega(r_n^4 s^2 p)$  respectively to achieve  $\max \left( |\hat{\mathbb{E}}[Y_u^{(\pi)}] - \mathbb{E}[Y_u^{(\pi)}] |, |\hat{\mathbb{E}}[Y_n^{(\pi)}] - \mathbb{E}[Y_n^{(\pi)}] | \right) = \tilde{o}_p(1)$ . Next, we present a corollary that discusses how quickly the parameters  $r_n$ ,  $s$ , and  $p$  can grow with the number of observations  $|\Pi_n|$  and  $M$ .
Corollary 6.7. With the set-up of Theorem 6.6, if (a)  $r_n = o(|\Pi_n|^{1/4})$  and (b)  $s = o\left(\sqrt{M / (p \max\{|\Pi_n|, |\mathcal{I}|\})}\right)$  then  $\max\left(|\hat{\mathbb{E}}[Y_u^{(\pi)}] - \mathbb{E}[Y_u^{(\pi)}]\right|, |\hat{\mathbb{E}}[Y_n^{(\pi)}] - \mathbb{E}[Y_n^{(\pi)}]|$  =  $\tilde{o}_p(1)$  as  $M, |\Pi_n|, |\mathcal{I}| \to \infty$ .
Corollary 6.7 quantifies how  $s, r_n$  can scale with the number of observations to achieve consistency. That is, Corollary 6.7 reflects the maximum "complexity" allowed for a given sample size.
6.3 Sample Complexity
We discuss the sample complexity of Synthetic Combinations to estimate all  $N \times 2^p$  causal parameters, and compare it to that of other methods. To ease our discussion, we will ignore dependence on logarithmic factors and  $\sigma$ . Even if potential outcomes  $Y_{n}^{(\pi)}$  were observed for all unit-combination pairs, consistently estimating  $\mathbb{E}[\mathbf{Y}_N^{(\Pi)}]$  is not trivial. This is because, we only get to observe a single and noisy version  $Y_{n\pi} = \langle \alpha_n, \chi^\pi \rangle + \epsilon_n^\pi$ . Hypothetically, if we observe  $K$  independent samples of  $Y_{n\pi}$  for a given  $(n,\pi)$ , denoted by  $Y_{n\pi}^{1}, \ldots, Y_{n\pi}^{K}$ , the maximum likelihood estimator would be the empirical average  $\frac{1}{K} \sum_{i=1}^{K} Y_{n\pi}^{i}$ . The empirical average would concentrate around  $\mathbb{E}[Y_{n}^{(\pi)}]$  at a rate  $O(1/\sqrt{K})$  and hence would require  $K = \Omega(\delta^{-2})$  samples to estimate  $\mathbb{E}[Y_{n}^{(\pi)}]$  within error  $O(\delta)$ . Therefore, this naive (unimplementable) solution would require  $N \times 2^p \times \delta^{-2}$  observations to estimate  $\mathbb{E}[\mathbf{Y}_N^{(\Pi)}]$ .
On the other hand, Synthetic Combinations produces consistent estimates of the potential outcome despite being given at most only a single noisy sample of each potential outcome. As the discussion after Theorem 6.6 shows, if  $|\mathcal{I}| \leq |\Pi_n|$ , Synthetic Combinations require  $|\mathcal{I}| \times r^4 s^2 p / \delta^2$  observations for the donor set, and  $(N - |\mathcal{I}|) \times r^4 / \delta^4$  observations for the non-donor units to achieve an estimation error of  $O(\delta)$  for all  $N \times 2^p$  causal parameters. Hence, we have that the number of observations required to achieve an estimation error of  $O(\delta)$  for all pairs  $(n,\pi)$  scales as  $O\big(\mathrm{poly}(r / \delta) \times (N + s^2 p)\big)$ .
Sample Complexity Comparison to Other Methods. Horizontal regression: An alternative algorithm would be to learn an individual model for each unit  $n \in [N]$ . That is, run a separate horizontal regression via the Lasso for every unit. This alternative algorithm has sample complexity that scales at least as  $O(N \times s^2 p / \delta^2)$  rather than  $O\big(\mathrm{poly}(r) / \delta^4 \times (N + s^2 p)\big)$  required by Synthetic Combinations. It suffers because it does not utilize any structure across units (i.e., the low-rank property of  $\mathcal{A}$ ), whereas Synthetic Combinations captures the similarity between units via PCR.
Matrix completion: Synthetic Combinations can be thought of as a matrix completion method; estimating  $\mathbb{E}[Y_n^{(\pi)}]$  is equivalent to imputing  $(n,\pi)$ -th entry of the observation matrix  $\mathbf{Y} \in \{\mathbb{R} \cup \star\}^{N \times 2^p}$ , where recall  $\star$  denotes an unobserved unit-combination outcome. Under the low-rank property (Assumption 3.1(b)) and various models of missingness (i.e., observation patterns), recent works on matrix completion [14, 29, 6] (see Section 2 for an overview) have established that estimating  $\mathbb{E}[Y_n^{(\pi)}]$  to an accuracy  $O(\delta)$  requires at least  $O(\mathrm{poly}(r / \delta) \times (N + 2^p))$  samples. This is because matrix completion techniques do not leverage the sparsity of  $\alpha_{n}$ . Moreover, matrix completion results typically report error in the Frobenius norm, whereas we give entry-wise guarantees. This leads to an extra factor of  $s$  in our analysis as it requires proving convergence for  $\| \hat{\alpha}_n - \alpha_n \|_1$  rather than  $\| \hat{\alpha}_n - \alpha_n \|_2$ . Hence, Synthetic Combinations combines the best of both approaches by leveraging both the structure of the potential outcomes and the similarity across units.
Natural Lower Bound on Sample Complexity. We provide an informal discussion on the lower bound sample-complexity to estimate all  $N \times 2^p$  potential outcomes. As established in Lemma E.1,  $\mathcal{A}$  has at most  $rs$  non-zero columns. Counting the parameters in the singular value decomposition of  $\mathcal{A}$ , only  $r \times (N + rs)$  free parameters are required to be estimated. Hence, a natural lower bound on the sample complexity scales as  $O(Nr + r^2 s)$ . Hence, Synthetic Combinations is only sub-optimal by a factor (ignoring logarithmic factors) of  $sp$  and  $\mathrm{poly}(r)$ . As discussed earlier, an additional factor of  $s$  can be removed if we focus on deriving Frobenius norm error bounds. It remains as interesting future work to derive estimation procedures that are able to achieve this lower bound.
In this work, we formulate a causal inference framework for combinatorial interventions, a setting that is ubiquitous in practice. We propose a model that imposes both unit-specific structure, and latent similarity across units. Under this model, we propose an estimation procedure, Synthetic Combinations, that exploits the sparsity and low-rankness of the Fourier coefficients to efficiently estimate all  $N \times 2^p$  causal parameters. We formally establish finite-sample consistency of Synthetic Combinations in an observational setting. Our work also naturally suggests future directions for research such as extending Synthetic Combinations to permutations over items (i.e., rankings), or providing an analysis of Synthetic Combinations that is agnostic to the horizontal regression algorithm used. A related line of work to the question above is also deriving estimation algorithms that can achieve the sample complexity lower bound discussed in Section 6.3.
We thank Alberto Abadie, Peng Ding, Giles Hooker, Devavrat Shah, Vasilis Syrgkanis, and Bin Yu for useful discussions and feedback.
[1] A. Abadie, A. Diamond, and J. Hainmueller. Synthetic control methods for comparative case studies: Estimating the effect of california's tobacco control program. Journal of the American statistical Association, 105(490):493-505, 2010.
[2] A. Agarwal and R. Singh. Causal inference with corrupted data: Measurement error, missing values, discretization, and differential privacy. arXiv preprint arXiv:2107.02780, 2021.
[3] A. Agarwal, D. Shah, D. Shen, and D. Song. On robustness of principal component regression. Advances in Neural Information Processing Systems, 32, 2019.
[4] A. Agarwal, D. Shah, and D. Shen. On principal component regression in a high-dimensional error-in-variables setting. arXiv preprint arXiv:2010.14449, 2020.
[5] A. Agarwal, D. Shah, and D. Shen. Synthetic interventions. arXiv preprint arXiv:2006.07691, 2020.
[6] A. Agarwal, M. Dahleh, D. Shah, and D. Shen. Causal matrix completion. arXiv preprint arXiv:2109.15154, 2021.
[28] R. H. Keshavan, A. Montanari, and S. Oh. Matrix completion from a few entries. IEEE transactions on information theory, 56(6):2980-2998, 2010.
[29] W. Ma and G. H. Chen. Missing not at random in matrix completion: The effectiveness of estimating missingness probabilities under a low nuclear norm assumption. Advances in neural information processing systems, 32, 2019.
[30] R. Mazumder, T. Hastie, and R. Tibshirani. Spectral regularization algorithms for learning large incomplete matrices. The Journal of Machine Learning Research, 11:2287-2322, 2010.
[31] E. Mossel, R. O'Donnell, and R. P. Servedio. Learning juntas. In Proceedings of the thirty-fifth annual ACM symposium on Theory of computing, pages 206-212, 2003.
[32] S. Negahban and D. Shah. Learning sparse boolean polynomials. In 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pages 2032-2036. IEEE, 2012.
[33] L. T. Nguyen, J. Kim, and B. Shim. Low-rank matrix completion: A contemporary survey. IEEE Access, 7:94215-94237, 2019.
[34] R. O'Donnell. Some topics in analysis of boolean functions. In Proceedings of the fortieth annual ACM symposium on Theory of computing, pages 569-578, 2008.
[35] B. Recht. A simpler approach to matrix completion. Journal of Machine Learning Research, 12(12), 2011.
[36] P. Rigollet and J.-C. Hütter. High dimensional statistics. Lecture notes for course 18S997, 813 (814):46, 2015.
[37] P.-Å. Wedin. Perturbation bounds in connection with singular value decomposition. BIT Numerical Mathematics, 12(1):99-111, 1972.
[38] C. J. Wu and M. S. Hamada. Experiments: planning, analysis, and optimization. John Wiley & Sons, 2011.
[39] A. Zhao and P. Ding. Regression-based causal inference with factorial experiments: estimands, model specifications and design-based properties. Biometrika, 109(3):799-815, 2022.
A Combinatorial Inference Applications
In this section, we discuss how classical models and applications could relate to our proposed potential outcome model. In particular, we discuss two well-studied models for functions over combinations: low-degree Boolean polynomials and  $k$ -Juntas. We also discuss applications such as factorial design experiments and recommendation systems.
Low-degree Boolean Polynomials. A special instance of sparse Boolean functions is a low-degree polynomial which we define as follows. For a positive integer  $d \leq p$ ,  $f: \{-1, 1\}^p \to \mathbb{R}$  is a  $d$ -degree polynomial if its Fourier transform satisfies the following for any input  $\mathbf{x} \in \{-1, 1\}^p$
$$
f (\mathbf {x}) = \sum_ {S \subset | p |, | S | \leq d} \alpha_ {S} \chi_ {S} (\mathbf {x}). \tag {6}
$$
In this setting, we see that  $s \leq \sum_{i=0}^{d} \binom{p}{i} \approx p^d$ . That is, degree  $d$ -polynomials impose sparsity on the potential outcome by limiting the degree of interaction between interventions.
Applications of low-degree Boolean polynomials. We discuss two applications when the potential outcomes can be modeled as a low-degree Boolean polynomial.
Factorial Design Experiments. Factorial design experiments consist of  $p$  treatments where each treatment arm can take on a discrete set of values, and units are assigned different combinations of these treatments. In the special case that each treatment arm only takes on two possible values, this experimental design mechanism is referred to as a  $2^{p}$  factorial experiment. Factorial design experiments are widely employed in the social sciences, agricultural and industrial applications [20, 18, 38]. A common strategy to determine treatment effects in this setting is to assume that the potential outcome only depends on main effects and pairwise interactions with higher-order interactions being negligible [11, 21, 24]. That is, analysts implicitly enforce sparsity of  $\alpha_{n}$  by setting  $d = 2$ . We refer the reader to [39] for a detailed discussion of various estimation strategies and their validity in different settings. The model we propose (see Assumption 3.1) captures these various modeling choices commonly used in factorial design experiments by imposing sparsity on  $\alpha_{n}$ . Further, as we show later, Synthetic Combinations can adapt to low-degree polynomials without pre-specifying the degree,  $d$ , i.e., it automatically adapts to the inherent level of interactions in the data. The additional assumption we make however is that there is structure across units, i.e., the matrix  $\mathcal{A} = [\alpha_{n}]_{n\in [N]}$  is low-rank.