SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Optimal Team Selection. Another use for combinatorial inference is to find the optimal configuration for a team. For example, a soccer team can be interested in finding the configuration of positions (i.e., defensive, midfield, offensive players) that leads to the highest winning percentage. Here, units  $n$  are different teams, and combinations  $\pi$  represent possible configurations of the team. The potential outcomes  $\mathbb{E}[Y_n^{(\pi)}]$  represent the winning percentage for a given team  $n$  under a particular configuration  $\pi$ , with the coefficients of  $\alpha_{n}$  representing the effect of different configurations on the win rate. In this setting, enforcing sparsity of  $\alpha_{n}$  via low-degree polynomials may be appropriate because the number of wins may depend on all the players, however only interactions between players of the same type matter while higher-order interactions between players of different types can be negligible. Returning to the example of a soccer team, the win rate can depend strongly on the synergy of the offensive players, whereas interactions between offensive and defensive players have a smaller effect. Further, since different teams might play in similar ways and can have the same optimal configuration, one potential way to capture this structure across teams is by placing a low-rank structure on  $\mathcal{A}$ .
$k$ -Juntas. Another special case of sparse Boolean functions are  $k$ -Juntas which only depend on  $k < p$  input variables. More formally, a function  $f: \{-1, 1\}^p \to \mathbb{R}$  is a  $k$ -junta if there exists a set  $K \subset [p]$  with  $|K| = k$  such that the fourier expansion of  $f$  can be represented as follows
$$
f (\mathbf {x}) = \sum_ {S \subset K} \alpha_ {S} \chi_ {S} (\mathbf {x}) \tag {7}
$$
Therefore, in the setting of the  $k$ -Junta, the sparsity index  $s \leq 2^k$ . In contrast to low-degree polynomials where the function depends on all  $p$  variables but limits the degree of interaction between variables,  $k$ -Juntas only depend on  $k$  variables but allow for arbitrary interactions amongst them.
Applications of  $k$ -Juntas. We discuss two applications when the potential outcomes can be modeled as a  $k$ -Junta.
Recommendation Systems. Recommendation platforms such as Netflix are often interested in recommending a combination of movies that maximizes a user's engagement with a platform. Here, units  $n$  can be individual users, and  $\pi$  represents combinations of different movies. The potential outcomes  $\mathbb{E}[Y_n^{(\pi)}]$  are the engagement levels for a unit  $n$  when presented with a combination of movies  $\pi$ , with  $\alpha_{n}$  representing the preferences for that user. In this setting, a user's engagement with the platform may only depend on a small subset of movies. For example, a user who is only interested in fantasy will only remain on the platform if they are recommended movies such as Harry Potter. Under this behavioral model, the potential outcomes (i.e., engagement levels) can be modeled as a  $k$ -Junta with the non-zero coefficients of  $\alpha_{n}$  representing the combinations of the  $k$  movies that affect engagement level for a user  $n$ . Our potential outcome observational model (Assumption 3.1) captures this form of sparsity while also reflecting the low-rank structure commonly assumed when studying recommendation systems. Once again, our results show Synthetic Combinations can adapt to  $k$ -Juntas without prespecifying the subset of features  $K$ , i.e., it automatically learns the important features for a given user.
Knock-down Experiments in Genomics. A key task in genomics is to identify which set of genes are responsible for a phenotype (i.e., physical trait of interest such as blood pressure) in a given individual. To do so, geneticists use knock-down experiments which measure the difference in the phenotype after eliminating the effect of a set of genes in an individual. To encode this process in the language of combinatorial causal inference, we can think of units  $n$  as different individuals, an action as knocking a particular gene out, and  $\pi$  as a combination of genes that are knocked out. The potential outcomes  $\mathbb{E}[Y_n^{(\pi)}]$  is the expression of the phenotype for a unit  $n$  when the combination of genes  $\pi$  are eliminated via knock-down experiments, and the coefficients of  $\alpha_{n}$  represent the effect of different combination of genes on the phenotype. A typical assumption in genomics is that phenotypes only depend on a small set of genes and their interactions. In this setting, we can model this form of sparsity by thinking of the potential outcome function as a  $k$ -junta, as well as capturing the similarity between the effect of genes on different individuals via our low-rank assumption.
B Proof of Theorem 4.2
Proof. Below, the symbol  $\stackrel{AX}{=}$  and  $\stackrel{DX}{=}$  imply that the equality follows from Assumption  $X$  and Definition  $X$ , respectively. We begin with the proof of Theorem 4.2 (a). Proof of Theorem 4.2 (a): For a donor unit  $u \in \mathcal{I}$  and  $\pi \in \Pi \backslash \Pi_u$ , we have
$$
\begin{array}{l} \mathbb {E} \left[ Y _ {u} ^ {(\pi)} \mid \mathcal {A} \right] \stackrel {{A 3. 1}} {{=}} \mathbb {E} \left[ \left\langle \boldsymbol {\alpha} _ {u}, \boldsymbol {\chi} ^ {\pi} \right\rangle + \epsilon_ {u} ^ {\pi} \mid \mathcal {A} \right] \\ \stackrel {A 3. 1 (c)} {=} \langle \alpha_ {u}, \chi^ {\pi} \rangle \mid \mathcal {A} \\ = \left\langle \alpha_ {u}, \chi^ {\pi} \right\rangle \mid \mathcal {A}, \mathcal {D} \\ \end{array}
$$
$$
\begin{array}{l} \stackrel {A 3. 1 (b)} {=} \langle \boldsymbol {\alpha} _ {u}, \tilde {\boldsymbol {\chi}} _ {u} ^ {\pi} \rangle \mid \mathcal {A}, \mathcal {D} \\ \stackrel {A 4. 1 (a)} {=} \left\langle \boldsymbol {\alpha} _ {u}, \sum_ {\pi_ {u} \in \Pi_ {u}} \beta_ {\pi_ {u}} ^ {\pi} \tilde {\chi} _ {u} ^ {\pi_ {u}} \right\rangle \mid \mathcal {A}, \mathcal {D} \\ = \sum_ {\pi_ {u} \in \Pi_ {u}} \beta_ {\pi_ {u}} ^ {\pi} \left\langle \boldsymbol {\alpha} _ {u}, \tilde {\boldsymbol {\chi}} _ {u} ^ {\pi_ {u}} \right\rangle \mid \mathcal {A}, \mathcal {D} \\ \stackrel {A 3. 1 (c), A 3. 2} {=} \sum_ {\pi_ {u} \in \Pi_ {u}} \beta_ {\pi_ {u}} ^ {\pi} \mathbb {E} [ \langle \boldsymbol {\alpha} _ {u}, \tilde {\boldsymbol {\chi}} _ {u} ^ {\pi_ {u}} \rangle + \epsilon_ {u} ^ {\pi_ {u}} \mid \mathcal {A}, \mathcal {D} ] \\ \stackrel {A 3. 1} {=} \sum_ {\pi_ {u} \in \Pi_ {u}} \beta_ {\pi_ {u}} ^ {\pi} \mathbb {E} \left[ Y _ {u, \pi_ {u}} \mid \mathcal {A}, \mathcal {D} \right] \\ \end{array}
$$
Proof of Theorem 4.2 (b): For a donor unit  $n \in [N] \backslash I^D$  and  $\pi \in \Pi \backslash \Pi_n$ , we have
$$
\begin{array}{l} \mathbb {E} [ Y _ {n} ^ {(\pi)} \mid \mathcal {A} ] \overset {A 3. 1} {=} \mathbb {E} [ \langle \pmb {\alpha} _ {n}, \pmb {\chi} ^ {\pi} \rangle + \epsilon_ {n} ^ {\pi} \mid \mathcal {A} ] \\ \stackrel {A 3. 1 (c)} {=} \langle \alpha_ {n}, \chi^ {\pi} \rangle \mid \mathcal {A} \\ = \left\langle \boldsymbol {\alpha} _ {n}, \chi^ {\pi} \right\rangle \mid \mathcal {A}, \mathcal {D} \\ \stackrel {A 4. 1 (b)} {=} \left\langle \sum_ {u \in \mathcal {I}} w _ {u} ^ {n} \boldsymbol {\alpha} _ {u}, \boldsymbol {\chi} ^ {\pi} \right\rangle \mid \mathcal {A}, \mathcal {D} \\ = \sum_ {u \in \mathcal {I}} w _ {u} ^ {n} \left\langle \boldsymbol {\alpha} _ {u}, \boldsymbol {\chi} ^ {\pi} \right\rangle \mid \mathcal {A}, \mathcal {D} \\ \stackrel {A 3. 1 (\underline {{c}}), A 3. 2} {=} \sum_ {u \in \mathcal {I}} w _ {u} ^ {n} \mathbb {E} [ \langle \boldsymbol {\alpha} _ {u}, \boldsymbol {\chi} ^ {\pi} \rangle + \epsilon_ {u} ^ {\pi} \mid \mathcal {A}, \mathcal {D} ] \\ \stackrel {A 3. 1} {=} \sum_ {u \in \mathcal {I}} w _ {u} ^ {n} \mathbb {E} \left[ Y _ {u} ^ {(\pi)} \mid \mathcal {A}, \mathcal {D} \right] \\ = \sum_ {u \in \mathcal {I}} \sum_ {\pi_ {u} \in \Pi_ {u}} w _ {u} ^ {n} \beta_ {\pi_ {u}} ^ {\pi} \mathbb {E} \left[ Y _ {u, \pi_ {u}} \mid \mathcal {A}, \mathcal {D} \right] \\ \end{array}
$$
where the last equality follows from Theorem 4.2 (a).
C Proof of Theorem 6.6
C.1 Proof of Theorem 6.6 (a)
$$
\begin{array}{l} \hat {\mathbb {E}} \left[ Y _ {u} ^ {(\pi)} \right] - \mathbb {E} \left[ Y _ {u} ^ {(\pi)} \right] = \left\langle \hat {\boldsymbol {\alpha}} ^ {u}, \chi^ {\pi} \right\rangle - \left\langle \boldsymbol {\alpha} _ {u}, \chi^ {\pi} \right\rangle \\ = \left\langle \hat {\alpha} _ {u} - \alpha_ {u}, \chi^ {\pi} \right\rangle \\ \leq \left\| \hat {\alpha} _ {u} - \alpha_ {u} \right\| _ {1} \left\| \chi^ {\pi} \right\| _ {\infty} \\ = \left\| \hat {\boldsymbol {\alpha}} _ {u} - \boldsymbol {\alpha} _ {u} \right\| _ {1} \tag {8} \\ \end{array}
$$
To finish the proof, we quote the following Theorem which we adapt to our notation.
Theorem C.1 (Theorem 2.18 in [36]). Fix the number of samples  $n \geq 2$ . Assume that the linear model  $Y = \mathbf{X}\theta^{*} + \epsilon$ , where  $\mathbf{X} \in \mathbb{R}^{n \times d}$  and  $\epsilon$  is a sub-gaussian random variable with noise variance  $\sigma^2$ . Moreover, assume that  $\| \theta^{*} \|_{0} \leq k$ , and that  $\mathbf{X}$  satisfies the incoherence condition (Assumption 6.3) with parameter  $k$ . Then, the lasso estimator  $\hat{\theta}^{L}$  with regularization parameter defined by
$$
2 \tau = 8 \sigma \sqrt {\log (2 d) / n} + 8 \sigma \sqrt {\log (1 / \delta) / n}
$$
$$
\left\| \theta^ {*} - \hat {\theta} ^ {L} \right\| _ {2} ^ {2} \leq k \sigma^ {2} \frac {\log (2 d / \delta)}{n} \tag {9}
$$
Further, as in established in the proof of Theorem 2.18 in [36],  $\| \theta^{*} - \hat{\theta}^{L}\|_{1}\leq \sqrt{k}\| \theta^{*} - \hat{\theta}^{L}\|_{2}$ . Note that the set-up of Theorem C.1 holds in our setting with the following notational changes:  $Y = \mathbf{Y}_{\Pi_u}$ ,  $\mathbf{X} = \chi (\Pi_u)$ ,  $\theta^{*} = \alpha_{u}$ ,  $\hat{\theta}^{L} = \hat{\alpha}_{u}$ ,  $k = s$  as well as our assumptions on the regularization parameter  $\lambda_{u}$  and that  $\epsilon_{u}^{\pi}$  is sub-gaussian (Assumption 6.2). Applying Theorem C.1 gives us
$$
\left\| \hat {\boldsymbol {\alpha}} _ {u} - \boldsymbol {\alpha} _ {u} \right\| _ {1} = O _ {p} \left(\sqrt {\frac {s ^ {2} p}{\left| \Pi_ {u} \right|}}\right)
$$
Substituting this bound into (8) yields the claimed result.
C.2 Proof of Theorem 6.6 (b)
For any matrix  $\mathbf{A}$  with orthonormal columns, let  $\mathcal{P}_A = \mathbf{A}\mathbf{A}^T$  denote the projection matrix on the subspace spanned by the columns of  $\mathbf{A}$ . Define  $\tilde{\mathbf{w}}^n = \mathcal{P}_{V_\mathcal{I}^{(\Pi_n)}}\mathbf{w}^n$ , where  $\mathbf{V}_\mathcal{I}^{(\Pi_n)}$  are the right singular vectors of  $\mathbb{E}[\mathbf{Y}_\mathcal{I}^{(\Pi_n)}]$ . Let  $\Delta_w^n = \hat{\mathbf{w}}^n - \tilde{\mathbf{w}}^n \in \mathbb{R}^{|\mathcal{I}|}$ , and  $\Delta_\mathcal{I}^\pi = \hat{\mathbb{E}}[\mathbf{Y}_\mathcal{I}^{(\pi)}] - \mathbb{E}[\mathbf{Y}_\mathcal{I}^{(\pi)}] \in \mathbb{R}^{|\mathcal{I}|}$ . Denote  $\Delta_E = \max_{u \in \mathcal{I}, \pi \in \Pi} \left| \hat{\mathbb{E}}[Y_u^{(\pi)}] - \mathbb{E}[Y_u^{(\pi)}] \right|$ . In order to proceed, we first state the following result,
Lemma C.2. Let the set-up of Theorem 6.6 hold. Then, we have
$$
\mathbb {E} \left[ Y _ {n} ^ {(\pi)} \right] = \langle \mathbb {E} \left[ \mathbf {Y} _ {\mathcal {I}} ^ {(\pi)} \right], \tilde {\mathbf {w}} ^ {n} \rangle
$$
Using Lemma C.2, and the notation established above, we have
$$
\begin{array}{l} \left| \hat {\mathbb {E}} [ Y _ {n} ^ {(\pi)} ] - \mathbb {E} [ Y _ {n} ^ {(\pi)} ] \right| = \left| \langle \hat {\mathbb {E}} [ \mathbf {Y} _ {\mathcal {I}} ^ {(\pi)} ], \hat {\mathbf {w}} ^ {n} \rangle - \langle \mathbb {E} [ \mathbf {Y} _ {\mathcal {I}} ^ {(\pi)} ], \tilde {\mathbf {w}} ^ {n} \rangle \right| \\ \leq \left| \left\langle \Delta_ {\mathcal {I}} ^ {\pi}, \tilde {\mathbf {w}} ^ {n} \right\rangle \right| + \left| \left\langle \mathbb {E} \left[ \mathbf {Y} _ {\mathcal {I}} ^ {(\pi)} \right], \Delta_ {w} ^ {n} \right\rangle \right| + \left| \left\langle \Delta_ {w} ^ {n}, \Delta_ {\mathcal {I}} ^ {\pi} \right\rangle \right| \tag {10} \\ \end{array}
$$
From Assumption 6.5, it follows that  $\mathbb{E}[\mathbf{Y}_{\mathcal{I}}^{(\pi)}] = \mathcal{P}_{V_{\mathcal{I}}^{(\Pi_n)}}\mathbb{E}[\mathbf{Y}_{\mathcal{I}}^{(\pi)}]$ , where  $\mathbf{V}_{\mathcal{I}}^{(\Pi_n)}$  are the right singular vectors of  $\mathbb{E}[\mathbf{Y}_{\mathcal{I}}^{(\Pi_n)}]$ . Using this in (10) gives us
$$
\left| \hat {\mathbb {E}} \left[ Y _ {n} ^ {(\pi)} \right] - \mathbb {E} \left[ Y _ {n} ^ {(\pi)} \right] \right| \leq | \langle \Delta_ {\mathcal {I}} ^ {\pi}, \tilde {\mathbf {w}} ^ {n} \rangle | + \left| \langle \mathbb {E} \left[ \mathbf {Y} _ {\mathcal {I}} ^ {(\pi)} \right], \mathcal {P} _ {V _ {\mathcal {I}} ^ {(\Pi_ {n})}} \Delta_ {w} ^ {n} \rangle \right| + | \langle \Delta_ {w} ^ {n}, \Delta_ {\mathcal {I}} ^ {\pi} \rangle | \tag {11}
$$
Below, we bound the three terms on the right-hand-side of (11) separately. Before we bound each term, we state a Lemma that will be useful to help us establish the results.
Lemma C.3. Let Assumptions 3.1, 4.1, 6.1, and 6.4 hold. Then,  $\| \tilde{\mathbf{w}}^n\|_2 \lesssim \sqrt{\frac{r_n}{|\mathcal{T}|}}$
Bounding Term 1. By Holder's inequality and Lemma C.3, we have that
$$
\left| \left\langle \Delta_ {\mathcal {I}} ^ {\pi}, \tilde {\mathbf {w}} ^ {n} \right\rangle \right| \leq \left\| \tilde {\mathbf {w}} ^ {n} \right\| _ {1} \left\| \Delta_ {\mathcal {I}} ^ {\pi} \right\| _ {\infty} \leq \left\| \tilde {\mathbf {w}} ^ {n} \right\| _ {1} \Delta_ {E} \leq \sqrt {\left| \mathcal {I} \right|} \| \tilde {\mathbf {w}} ^ {n} \| _ {2} \Delta_ {E} \lesssim \sqrt {r _ {n}} \Delta_ {E} \tag {12}
$$
This concludes the analysis for the first term.
Bounding Term 2. By Cauchy-Schwarz and Assumption 6.1 we have,
$$
\begin{array}{l} \left| \langle \mathbb {E} [ \mathbf {Y} _ {\mathcal {I}} ^ {(\pi)} ], \mathcal {P} _ {V _ {\mathcal {I}} ^ {(\Pi_ {n})}} \Delta_ {w} ^ {n} \rangle \right| \leq \| \mathbb {E} [ \mathbf {Y} _ {\mathcal {I}} ^ {(\pi)} ] \| _ {2} \| \mathcal {P} _ {V _ {\mathcal {I}} ^ {(\Pi_ {n})}} \Delta_ {w} ^ {n} \| _ {2} \\ \leq \sqrt {| \mathcal {I} |} \| \mathcal {P} _ {V _ {\mathcal {I}} ^ {(\Pi_ {n})}} \Delta_ {w} ^ {n} \| _ {2} \tag {13} \\ \end{array}
$$
We now state a lemma that will help us conclude our bound of Term 2. The proof is given in Appendix D.3.
Lemma C.4. Let the set-up of Theorem 6.6 hold. Then,
$$
\begin{array}{l} \left\| \mathcal {P} _ {V _ {\mathbb {I}} ^ {(\Pi_ {n})}} \Delta_ {w} ^ {n} \right\| _ {2} \\ = \tilde {O} _ {p} \left(r _ {n} ^ {2} \left[ \frac {\Delta_ {E}}{\sqrt {| \mathcal {I} |} \min  \left\{\sqrt {| \Pi_ {n} |} , \sqrt {| \mathcal {I} |} \right\}} + \Delta_ {E} ^ {2} \right] + \frac {r _ {n} ^ {3 / 2} \Delta_ {E}}{\sqrt {| \mathcal {I} |}} + \frac {r _ {n} \left(1 + \sqrt {\Delta_ {E}} r _ {n} ^ {1 / 4}\right)}{\sqrt {| \mathcal {I} |} | \Pi_ {n} | ^ {1 / 4}}\right) \tag {14} \\ \end{array}
$$
Incorporating the result of the lemma above into (13) gives us
$$
\begin{array}{l} \left| \langle \mathbb {E} [ \mathbf {Y} _ {\mathcal {I}} ^ {(\pi)} ], \mathcal {P} _ {V _ {\mathcal {I}} ^ {(\Pi_ {n})}} \Delta_ {w} ^ {n} \rangle \right| \\ = \tilde {O} _ {p} \left(r _ {n} ^ {2} \left[ \frac {\Delta_ {E}}{\min  \left\{\sqrt {| \Pi_ {n} |} , \sqrt {| \mathcal {I} |} \right\}} + \sqrt {| \mathcal {I} |} \Delta_ {E} ^ {2} \right] + r _ {n} ^ {3 / 2} \Delta_ {E} + \frac {r _ {n} \left(1 + \sqrt {\Delta_ {E}} r _ {n} ^ {1 / 4}\right)}{| \Pi_ {n} | ^ {1 / 4}}\right) \tag {15} \\ \end{array}
$$
Bounding Term 3. By Holder's inequality, we have that
$$
\begin{array}{l} | \langle \Delta_ {w} ^ {n}, \Delta_ {\mathcal {L}} ^ {\pi} \rangle | \leq \| \Delta_ {w} ^ {n} \| _ {2} \| \Delta_ {\mathcal {L}} ^ {\pi} \| _ {2} \\ \leq \sqrt {| \mathcal {I} |} \| \Delta_ {w} ^ {n} \| _ {2} \| \Delta_ {\mathcal {I}} ^ {\pi} \| _ {\infty} \\ \leq \sqrt {| \mathcal {T} |} \Delta_ {E} \| \Delta_ {w} ^ {n} \| _ {2} \tag {16} \\ \end{array}
$$
We now state a proposition that will help us conclude our proof of Term 3. The proof is given in Appendix D.4.
Proposition C.5. Let the set-up of Theorem 6.6 hold. Then, conditioned on  $\mathcal{A}$ , we have
$$
\tilde {\mathbf {w}} ^ {n} - \tilde {\mathbf {w}} ^ {n} = \tilde {O} _ {p} \left(r _ {n} \left[ \frac {\| \tilde {\mathbf {w}} ^ {n} \| _ {2}}{\min  \left\{\sqrt {| \Pi_ {n} |} , \sqrt {| \mathcal {I} |} \right\}} + \Delta_ {E} \right]\right) \tag {18}
$$
As a result of Proposition C.5 and Lemma C.3, we have the following bound for Term 3,
$$
\left| \left\langle \Delta_ {w} ^ {n}, \Delta_ {\mathcal {I}} ^ {\pi} \right\rangle \right| = \tilde {O} _ {p} \left(r _ {n} ^ {3 / 2} \left[ \frac {\Delta_ {E}}{\min  \left\{\sqrt {| \Pi_ {n} |} , \sqrt {| \mathcal {I} |} \right\}} + \sqrt {| \mathcal {I} |} \Delta_ {E} ^ {2} \right]\right) \tag {19}
$$
Collecting Terms. Combining equations (12), (15), (19) gives us
$$
\begin{array}{l} \left| \hat {\mathbb {E}} \left[ Y _ {n} ^ {(\pi)} \right] - \mathbb {E} \left[ Y _ {n} ^ {(\pi)} \right] \right| \\ = \tilde {O} _ {p} \left(r _ {n} ^ {2} \left[ \frac {\Delta_ {E}}{\min  \left\{\sqrt {| \Pi_ {n} |} , \sqrt {| \mathcal {I} |} \right\}} + \sqrt {| \mathcal {I} |} \Delta_ {E} ^ {2} \right] + r _ {n} ^ {3 / 2} \Delta_ {E} + \frac {r _ {n} \left(1 + \sqrt {\Delta_ {E}} r _ {n} ^ {1 / 4}\right)}{| \Pi_ {n} | ^ {1 / 4}}\right) \tag {20} \\ \end{array}
$$
By Theorem 6.6 (a), we have that
$$
\Delta_ {E} = \max  _ {u \in \mathcal {I}} O _ {p} \left(\sqrt {\frac {s ^ {2} p}{| \Pi_ {u} |}}\right) = O _ {p} \left(\sqrt {\frac {s ^ {2} p}{M}}\right) \tag {21}
$$
where we remind the reader that  $M = \min_{u\in \mathcal{I}}|\Pi_u|$ . Substituting (21), and our assumption that  $M = \omega (r_n^2 s^2 p)$  into (20), we get
$$
\left| \hat {\mathbb {E}} \left[ Y _ {n} ^ {(\pi)} \right] - \mathbb {E} \left[ Y _ {n} ^ {(\pi)} \right] \right| = \tilde {O} _ {p} \left(\frac {r _ {n} ^ {2} \sqrt {s ^ {2} p}}{\sqrt {M \times \min  \{| \Pi_ {n} | , | \mathcal {I} | \}}} + \frac {r _ {n} ^ {2} s ^ {2} p \sqrt {| \mathcal {I} |}}{M} + \frac {r _ {n}}{| \Pi_ {n} | ^ {1 / 4}}\right) \tag {22}
$$
D Proofs of Helper Lemmas for Theorem 6.6.
In this section we provide proofs of Lemmas C.2, C.3, C.4, and Proposition C.5 which were required for the Proof of Theorem 6.6.
D.1 Proof of Lemma C.2
By Assumption 3.1 and 4.1 (b), we have
$$
\begin{array}{l} \mathbb {E} \left[ Y _ {n} ^ {(\pi)} \right] = \mathbb {E} \left[ \left\langle \boldsymbol {\alpha} _ {n}, \boldsymbol {\chi} ^ {\pi} \right\rangle + \epsilon_ {n} ^ {\pi} \right] \\ = \mathbb {E} [ \langle \boldsymbol {\alpha} _ {n}, \boldsymbol {\chi} ^ {\pi} \rangle ] \\ = \sum_ {u \in \mathcal {I}} w _ {u} ^ {n} \mathbb {E} [ \langle \boldsymbol {\alpha} _ {u}, \boldsymbol {\chi} ^ {\pi} \rangle ] \\ = \sum_ {u \in \mathcal {I}} w _ {u} ^ {n} \mathbb {E} \left[ Y _ {u} ^ {(\pi)} \right] \\ = \mathbb {E} \left[ \mathbf {Y} _ {\mathcal {I}} ^ {(\pi)} \right] ^ {T} \mathbf {w} ^ {n} \\ \end{array}
$$
From Assumption 6.5, we have that  $\mathbb{E}[\mathbf{Y}_{\mathcal{I}}^{(\pi)}] = \mathcal{P}_{V_{\mathcal{I}}^{(\Pi_n)}}\mathbb{E}[\mathbf{Y}_{\mathcal{I}}^{(\pi)}]$ . Substituting this into the equation above completes the proof.
D.2 Proof of Lemma C.3
For simplicity, denote  $\mathbb{E}[\mathbf{Y}_n^{(\Pi_n)}] = \mathbb{E}[\mathbf{Y}^{(\Pi_n)}]$ . By definition,  $\tilde{\mathbf{w}}^n$  is the solution to the following optimization program
$$
\min  _ {\mathbf {w} \in \mathbb {R} ^ {| \mathcal {I} |}} \| \mathbf {w} \| _ {2}
$$
$$
\mathrm {s . t .} \quad \mathbb {E} \left[ \mathbf {Y} ^ {\left(\Pi_ {n}\right)} \right] = \mathbb {E} \left[ \mathbf {Y} _ {\mathcal {I}} ^ {\left(\Pi_ {n}\right)} \right] \mathbf {w} \tag {23}
$$
Let  $\mathbf{U}_{\mathcal{I}}^{(\Pi_n)}, \mathbf{\Sigma}_{\mathcal{I}}^{(\Pi_n)}, \mathbf{V}_{\mathcal{I}}^{(\Pi_n)}$  denote the SVD of  $\mathbb{E}[\mathbf{Y}_{\mathcal{I}}^{(\Pi_n)}]$ . Further, let  $\mathbf{U}_{\mathcal{I}}^{(\Pi_n),r_n}, \mathbf{\Sigma}_{\mathcal{I}}^{(\Pi_n),r_n}, \mathbf{V}_{\mathcal{I}}^{(\Pi_n),r_n}$  denote the rank  $r_n$  truncation of the SVD. Then, define  $\mathbf{w}_{r_n} = \mathbf{V}_{\mathcal{I}}^{(\Pi_n),r_n}(\mathbf{\Sigma}_{\mathcal{I}}^{(\Pi_n),r_n})^\dagger (\mathbf{U}_{\mathcal{I}}^{(\Pi_n),r_n})^T\mathbb{E}[\mathbf{Y}^{(\Pi_n)}]$ , where  $\dagger$  is pseudo-inverse. We first show that  $\mathbf{w}_{r_n}$  is a solution to (23).
$$
\begin{array}{l} \mathbb {E} \left[ \mathbf {Y} _ {\mathcal {I}} ^ {\left(\Pi_ {n}\right)} \right] \mathbf {w} _ {r _ {n}} = \left(\mathbf {U} _ {\mathcal {I}} ^ {\left(\Pi_ {n}\right)} \boldsymbol {\Sigma} _ {\mathcal {I}} ^ {\left(\Pi_ {n}\right)} \left(\mathbf {V} _ {\mathcal {I}} ^ {\left(\Pi_ {n}\right)}\right) ^ {T}\right) \mathbf {V} _ {\mathcal {I}} ^ {\left(\Pi_ {n}\right), r _ {n}} \left(\boldsymbol {\Sigma} _ {\mathcal {I}} ^ {\left(\Pi_ {n}\right), r _ {n}}\right) ^ {\dagger} \left(\mathbf {U} _ {\mathcal {I}} ^ {\left(\Pi_ {n}\right), r _ {n}}\right) ^ {T} \mathbb {E} \left[ \mathbf {Y} ^ {\left(\Pi_ {n}\right)} \right] \\ = \left(\sum_ {i = 1} ^ {r _ {n}} s _ {i} u _ {i} v _ {i} ^ {T}\right) \left(\sum_ {j = 1} ^ {r _ {n}} \frac {1}{s _ {j}} v _ {j} u _ {j} ^ {T} \mathbb {E} \left[ \mathbf {Y} ^ {\left(\Pi_ {n}\right)} \right]\right) \\ = \sum_ {i, j = 1} ^ {r _ {n}} \frac {s _ {i}}{s _ {j}} u _ {i} v _ {i} ^ {T} v _ {j} u _ {j} ^ {T} \mathbb {E} \left[ \mathbf {Y} ^ {\left(\Pi_ {n}\right)} \right] \\ = \sum_ {i = 1} ^ {r _ {n}} u _ {i} u _ {i} ^ {T} \mathbb {E} \left[ \mathbf {Y} ^ {\left(\Pi_ {n}\right)} \right] \\ = \mathbb {E} \left[ \mathbf {Y} ^ {\left(\Pi_ {n}\right)} \right] \\ \end{array}
$$
Next, we bound  $\| \mathbf{w}_{r_n}\| _2$  using Assumptions 6.1 and 6.4 as follows
$$
\begin{array}{l} \| \mathbf {w} _ {r _ {n}} \| _ {2} \leq \| (\boldsymbol {\Sigma} _ {\mathcal {I}} ^ {(\Pi_ {n}), r _ {n}}) ^ {\dagger} \| _ {2} \| \mathbb {E} [ \mathbf {Y} ^ {(\Pi_ {n})} ] \| _ {2} \\ \leq \frac {\sqrt {| \Pi_ {n} |}}{s _ {r _ {n}} (\mathbb {E} [ \mathbf {Y} _ {\mathcal {I}} ^ {(\Pi_ {n})} ])} \\ \leq \sqrt {\frac {c r _ {n}}{| \mathcal {I} |}} \\ \end{array}
$$
$$
\| \tilde {\mathbf {w}} ^ {n} \| _ {1} \leq \sqrt {| \mathcal {I} |} \| \tilde {\mathbf {w}} _ {r _ {n}} \| _ {2} \leq \sqrt {c r _ {n}}
$$
D.3 Proof of Lemma C.4
First, we introduce some necessary notation required for the proof. Let  $\hat{\mathbb{E}}\left[\mathbf{Y}_{\mathcal{I}}^{(\Pi_n)}\right] = \hat{\mathbf{U}}_{\mathcal{I}}^{(\Pi_n)}\hat{\mathbf{S}}_{\mathcal{I}}^{(\Pi_n)}\hat{\mathbf{V}}_{\mathcal{I}}^{(\Pi_n)}$  denote the rank  $r_n$  SVD of  $\hat{\mathbb{E}}\left[\mathbf{Y}_{\mathcal{I}}^{(\Pi_n)}\right]$ . Then, to establish Lemma C.4, consider the following decomposition:
$$
\mathcal {P} _ {V _ {\mathfrak {I}} ^ {(\Pi_ {n})}} \Delta_ {w} ^ {n} = \left(\mathcal {P} _ {V _ {\mathfrak {I}} ^ {(\Pi_ {n})}} - \mathcal {P} _ {\hat {V} _ {\mathfrak {I}} ^ {(\Pi_ {n})}}\right) \Delta_ {w} ^ {n} + \mathcal {P} _ {\hat {V} _ {\mathfrak {I}} ^ {(\Pi_ {n})}} \Delta_ {w} ^ {n}
$$
We bound each of these terms separately again.
Bounding Term 1. We have
$$
\left\| \left(\mathcal {P} _ {V _ {\mathcal {I}} ^ {(\Pi_ {n})}} - \mathcal {P} _ {\hat {V} _ {\mathcal {I}} ^ {(\Pi_ {n})}}\right) \Delta_ {w} ^ {n} \right\| _ {2} \leq \left\| \mathcal {P} _ {V _ {\mathcal {I}} ^ {(\Pi_ {n})}} - \mathcal {P} _ {\hat {V} _ {\mathcal {I}} ^ {(\Pi_ {n})}} \right\| _ {\mathrm {o p}} \| \Delta_ {w} ^ {n} \| _ {2} \tag {24}
$$
Theorem D.1 (Wedin's Theorem [37]). Given  $\mathbf{A},\mathbf{B}\in \mathbb{R}^{m\times n}$ , let  $(\mathbf{U},\mathbf{V}),(\hat{\mathbf{U}},\hat{\mathbf{V}})$  denote their respective left and right singular vectors. Further, let  $(\mathbf{U}_k,\mathbf{V}_k)\in (\text{respectively},(\hat{\mathbf{U}}_k,\hat{\mathbf{V}}_k))$  correspond to the truncation of  $(\mathbf{U},\mathbf{V})$  (respectively,  $(\hat{\mathbf{U}},\hat{\mathbf{V}})$ ), respectively, that retains the columns corresponding to the top  $k$  singular values of  $\mathbf{A}$  (respectively,  $\mathbf{B}$ ). Let  $s_i$  represent the  $i$ -th singular values of  $A$ . Then,  $\max (\| \mathcal{P}_{U_k} - \mathcal{P}_{\hat{U}_k}\|_{op},\| \mathcal{P}_{V_k} - \mathcal{P}_{\hat{V}_k}\|_{op})\leq \frac{2\|\mathbf{A} - \mathbf{B}\|_{op}}{s_k - s_{k + 1}}$
Applying Theorem D.1 gives us