SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \mathbb {E} \left[ (\frac {| \overline {{S}} |}{n} \cdot \left(\widehat {\tau} _ {\overline {{S}}} - \frac {1}{m} \sum_ {i \in T} (\mathbf {x} _ {i} - \overline {{\mathbf {x}}} ^ {\overline {{S}}}) ^ {\top} \widetilde {\mathbf {v}} ^ {S, \mathbf {y}}\right) - \frac {| \overline {{S}} |}{n} \cdot \tau_ {\overline {{S}}}) (\frac {| S |}{n} \cdot \widehat {\tau} _ {S} - \frac {| S |}{n} \cdot \tau_ {S}) | S, \mathbf {y} \right] \\ = \mathbb {E} \left[ \left(\frac {| \overline {{S}} |}{n} \cdot \left(\widehat {\tau} _ {\overline {{S}}} - \frac {1}{m} \sum_ {i \in T} (\mathbf {x} _ {i} - \overline {{\mathbf {x}}} ^ {\overline {{S}}}) ^ {\top} \widetilde {\mathbf {v}} ^ {S, \mathbf {y}}\right) - \frac {| \overline {{S}} |}{n} \cdot \tau_ {\overline {{S}}}\right) | S, \mathbf {y} \right] \cdot \mathbb {E} \left[ \left(\frac {| S |}{n} \cdot \widehat {\tau} _ {S} - \frac {| S |}{n} \cdot \tau_ {S}\right) | S, \mathbf {y} \right]. \\ \end{array}
$$
Since  $\mathbb{E}\left[\widehat{\tau}_{\overline{S}}\right] = \tau_{\overline{S}}$  and  $\frac{1}{|\overline{S}|}\sum_{i\in \overline{S}}\mathbf{x}_i = \overline{\mathbf{x}}^{\overline{S}}$ , the above quantity is equal to zero. Now note that since we use the Horvitz-Thompson estimator to estimate  $\widehat{\tau}$ , we have
$$
\mathbb {E} \left[ \left(\frac {| S |}{n} \cdot \widehat {\tau} _ {S} - \frac {| S |}{n} \cdot \tau_ {S}\right) ^ {2} | S, y \right] = \left(\frac {| S |}{n}\right) ^ {2} \frac {1}{| S | ^ {2}} \| \boldsymbol {\mu} _ {S} \| _ {2} ^ {2} \leq \frac {1 0 0 d \cdot \log (d / \delta)}{n ^ {2} \cdot \epsilon^ {2}} \| \boldsymbol {\mu} \| _ {\infty} ^ {2}. \tag {19}
$$
Now note that  $\hat{\mathbf{b}}$  is fixed given  $S, \mathbf{y}$ . Let
$$
\widetilde {\mathbf {b}} ^ {\overline {{S}}} = \operatorname * {a r g   m i n} _ {\mathbf {b} \in \mathbb {R} ^ {d}} \left[ \frac {1}{\phi} \| \mathbf {X} _ {\overline {{S}}} \mathbf {b} - \boldsymbol {\mu} _ {\overline {{S}}} \| _ {2} ^ {2} + \frac {\zeta^ {2}}{(1 - \phi)} \| \mathbf {b} \| _ {2} ^ {2} \right].
$$
Then by Theorem 19,
$$
\begin{array}{l} \mathbb {E} \left[ \left(\frac {| \overline {{S}} |}{n} \cdot \left(\widehat {\tau_ {\overline {{S}}}} - \frac {1}{m} \sum_ {i \in T} (\mathbf {x} _ {i} - \overline {{\mathbf {x}}} ^ {\overline {{S}}}) ^ {\top} \widetilde {\mathbf {v}} ^ {S, \mathbf {y}}\right) - \frac {| \overline {{S}} |}{n} \cdot \tau_ {\overline {{S}}}\right) ^ {2} | S, \mathbf {y} \right] \\ \leq \left(\frac {| \overline {{S}} |}{n}\right) ^ {2} \left(\frac {1}{m \cdot | \overline {{S}} |} \frac {1}{\phi} \left\| \mathbf {X} _ {\overline {{S}}} \widetilde {\mathbf {b}} ^ {\overline {{S}}} - \boldsymbol {\mu} _ {\overline {{S}}} \right\| _ {2} ^ {2} + \frac {1}{m ^ {2}} \frac {\zeta^ {2}}{(1 - \phi)} \left\| \widetilde {\mathbf {b}} ^ {\overline {{S}}} \right\| _ {2} ^ {2} + \frac {1}{m \cdot | \overline {{S}} |} \left\| (\overline {{\mathbf {X}}} ^ {\overline {{S}}} - \mathbf {X} _ {\overline {{S}}}) \widetilde {\mathbf {v}} ^ {S, \mathbf {y}} - (\overline {{\mathbf {t}}} ^ {\overline {{S}}} - \mathbf {t} _ {\overline {{S}}}) \right\| _ {2} ^ {2}\right) \\ \leq \frac {1}{m n} \frac {1}{\phi} \left\| \mathbf {X} _ {\overline {{S}}} \widetilde {\mathbf {b}} ^ {\overline {{S}}} - \boldsymbol {\mu} _ {\overline {{S}}} \right\| _ {2} ^ {2} + \frac {1}{m ^ {2}} \frac {\zeta^ {2}}{(1 - \phi)} \left\| \widetilde {\mathbf {b}} ^ {\overline {{S}}} \right\| _ {2} ^ {2} + \frac {1}{m n} \left\| (\overline {{\mathbf {X}}} ^ {\overline {{S}}} - \mathbf {X} _ {\overline {{S}}}) \widetilde {\mathbf {v}} ^ {S, \mathbf {y}} - (\overline {{\mathbf {t}}} ^ {\overline {{S}}} - \mathbf {t} _ {\overline {{S}}}) \right\| _ {2} ^ {2} \\ \leq \frac {1}{m n} \frac {1}{\phi} \left\| \mathbf {X} _ {\overline {{S}}} \mathbf {b} ^ {*} - \boldsymbol {\mu} _ {\overline {{S}}} \right\| _ {2} ^ {2} + \frac {1}{m ^ {2}} \frac {\zeta^ {2}}{(1 - \phi)} \left\| \mathbf {b} ^ {*} \right\| _ {2} ^ {2} + \frac {1}{m n} \left\| (\overline {{\mathbf {X}}} ^ {\overline {{S}}} - \mathbf {X} _ {\overline {{S}}}) \widetilde {\mathbf {v}} ^ {S, \mathbf {y}} - (\overline {{\mathbf {t}}} ^ {\overline {{S}}} - \mathbf {t} _ {\overline {{S}}}) \right\| _ {2} ^ {2} \\ \leq \frac {1}{m n} \frac {1}{\phi} \| \mathbf {X b} ^ {*} - \boldsymbol {\mu} \| _ {2} ^ {2} + \frac {1}{m ^ {2}} \frac {\zeta^ {2}}{(1 - \phi)} \| \mathbf {b} ^ {*} \| _ {2} ^ {2} + \frac {1}{m n} \left\| (\overline {{\mathbf {X}}} ^ {\overline {{S}}} - \mathbf {X} _ {\overline {{S}}}) \widetilde {\mathbf {v}} ^ {S, \mathbf {y}} - (\overline {{\mathbf {t}}} ^ {\overline {{S}}} - \mathbf {t} _ {\overline {{S}}}) \right\| _ {2} ^ {2} \tag {20} \\ \end{array}
$$
We now use the guarantees of leverage score sampling to bound the following
$$
\left\| (\overline {{\mathbf {X}}} ^ {\overline {{S}}} - \mathbf {X} _ {\overline {{S}}}) \widetilde {\mathbf {v}} ^ {S, \mathbf {y}} - (\overline {{\mathbf {t}}} ^ {\overline {{S}}} - \mathbf {t} _ {\overline {{S}}}) \right\| _ {2} ^ {2}.
$$
The matrix  $\overline{\mathbf{X}}^{\overline{S}}$  denotes a matrix with a size equal to  $\overline{\mathbf{X}}_{\overline{S}}$  such that all of its rows are equal to the average row of  $\overline{\mathbf{X}}_{\overline{S}}$ . Similarly,  $\overline{\mathbf{t}}^{\overline{S}}$  is a vector with a size equal to  $\overline{\mathbf{t}}_{\overline{S}}$  such that all of its entries are equal to the average entry of  $\overline{\mathbf{t}}_{\overline{S}}$ . Similar to the proof of Theorem 12, let  $\widetilde{\mathbf{X}}^{\overline{S}} := \left[(\mathbf{X}_{\overline{S}} - \overline{\mathbf{X}}^{\overline{S}}) \mathbf{1}\right]$ , and
$$
\mathbf {Z} ^ {\overline {{S}}} = \left[ \begin{array}{c} (\mathbf {X} _ {\overline {{S}}} - \overline {{\mathbf {X}}} ^ {\overline {{S}}}) \\ \sqrt {\lambda} \mathbf {I} _ {d} \end{array} \right], \widetilde {\mathbf {Z}} ^ {\overline {{S}}} = \left[ \begin{array}{c c} (\mathbf {X} _ {\overline {{S}}} - \overline {{\mathbf {X}}} ^ {\overline {{S}}}) & \mathbf {1} \\ \sqrt {\lambda} \mathbf {I} _ {d} & \mathbf {0} \end{array} \right].
$$
Moreover, let  $\widetilde{\mathbf{y}}^{(1,\overline{S})},\widetilde{\mathbf{y}}^{(0,\overline{S})}\in \mathbb{R}^{|S| + d}$  be vectors that are equal to  $\mathbf{y}_{\overline{S}}^{(1)},\mathbf{y}_{\overline{S}}^{(0)}$  , respectively, on their first  $|\overline{S} |$  entries, and the rest of their entries are zero. Also let  $\widetilde{\mathbf{t}}^{\overline{S}} = \widetilde{\mathbf{y}}^{(1,\overline{S})} - \widetilde{\mathbf{y}}^{(0,\overline{S})}$  . Then by Theorem 1, the vector  $\tilde{\mathbf{v}}$  in Algorithm 4 satisfies
$$
\begin{array}{l} \mathbb {E} \left[ \left\| 2 \widetilde {\mathbf {Z}} ^ {\overline {{S}}} \widetilde {\mathbf {v}} - \widetilde {\mathbf {t}} ^ {\overline {{S}}} \right\| _ {2} ^ {2} | S \right] \leq \frac {d}{\alpha} \cdot (1 + \epsilon) \| \boldsymbol {\mu} _ {\overline {{S}}} \| _ {\infty} ^ {2} + (1 + \epsilon) \min  _ {\mathbf {b} \in \mathbb {R} ^ {d + 1}} \left\| \widetilde {\mathbf {Z}} ^ {\overline {{S}}} \mathbf {b} - \widetilde {\mathbf {t}} ^ {\overline {{S}}} \right\| _ {2} ^ {2} \\ \leq \frac {d}{\alpha} \cdot (1 + \epsilon) \| \boldsymbol {\mu} \| _ {\infty} ^ {2} + (1 + \epsilon) \min  _ {\mathbf {b} \in \mathbb {R} ^ {d + 1}} \left\| \widetilde {\mathbf {Z}} ^ {\overline {{S}}} \mathbf {b} - \widetilde {\mathbf {t}} ^ {\overline {{S}}} \right\| _ {2} ^ {2} \\ \end{array}
$$
$$
\left\| \widetilde {\mathbf {Z}} ^ {\overline {{S}}} \mathbf {b} - \widetilde {\mathbf {t}} ^ {\overline {{S}}} \right\| _ {2} ^ {2} = \left\| \widetilde {\mathbf {X}} ^ {\overline {{S}}} \mathbf {b} - \mathbf {t} _ {\overline {{S}}} \right\| _ {2} ^ {2} + \lambda \| \mathbf {b} _ {1: d} \| _ {2} ^ {2},
$$
and similar to the proof of Theorem 12, one can see that
$$
\min _ {\mathbf {b} \in \mathbb {R} ^ {d}} \left[ \left\| (\mathbf {X} _ {\overline {{S}}} - \overline {{\mathbf {X}}} ^ {\overline {{S}}}) \mathbf {b} - (\mathbf {t} _ {\overline {{S}}} - \overline {{\mathbf {t}}} ^ {\overline {{S}}}) \right\| _ {2} ^ {2} + \lambda \| \mathbf {b} \| _ {2} ^ {2} \right] = \min _ {\mathbf {b} \in \mathbb {R} ^ {d + 1}} \left[ \left\| \widetilde {\mathbf {X}} ^ {\overline {{S}}} \mathbf {b} - \mathbf {t} _ {\overline {{S}}} \right\| _ {2} ^ {2} + \lambda \| \mathbf {b} _ {1: d} \| _ {2} ^ {2} \right]
$$
$$
\left\| 2 \widetilde {\mathbf {Z}} ^ {\overline {{S}}} \widetilde {\mathbf {v}} - \widetilde {\mathbf {t}} ^ {\overline {{S}}} \right\| _ {2} ^ {2} = \left\| (\mathbf {X} _ {\overline {{S}}} - \overline {{\mathbf {X}}} ^ {\overline {{S}}}) \widehat {\mathbf {v}} + \widetilde {\mathbf {v}} _ {d + 1} \cdot \mathbf {1} - \mathbf {t} _ {\overline {{S}}} \right\| _ {2} ^ {2} + \lambda \| \widehat {\mathbf {v}} \| _ {2} ^ {2}.
$$
Again similar to the proof of Theorem 12, one can show that
$$
\left\| \left(\mathbf {X} _ {\overline {{S}}} - \overline {{\mathbf {X}}} ^ {\overline {{S}}}\right) \widehat {\mathbf {v}} - \left(\mathbf {t} _ {\overline {{S}}} - \overline {{\mathbf {t}}} ^ {\overline {{S}}}\right) \right\| _ {2} ^ {2} \leq \left\| \left(\mathbf {X} _ {\overline {{S}}} - \overline {{\mathbf {X}}} ^ {\overline {{S}}}\right) \widehat {\mathbf {v}} + \widetilde {\mathbf {v}} _ {d + 1} \cdot \mathbf {1} - \mathbf {t} _ {\overline {{S}}} \right\| _ {2} ^ {2}
$$
Combining the above and noting that  $\lambda \cdot \| \widehat{\mathbf{v}}\| _2^2\geq 0$  , we have
$$
\begin{array}{l} \mathbb {E} \left[ \left\| \left(\mathbf {X} _ {\overline {{S}}} - \overline {{\mathbf {X}}} ^ {\overline {{S}}}\right) \widehat {\mathbf {v}} - \left(\mathbf {t} _ {\overline {{S}}} - \overline {{\mathbf {t}}} ^ {\overline {{S}}}\right) \right\| _ {2} ^ {2} \right] \\ \leq \frac {d}{\alpha} \cdot (1 + \epsilon) \| \boldsymbol {\mu} \| _ {\infty} ^ {2} + (1 + \epsilon) \min  _ {\mathbf {b} \in \mathbb {R} ^ {d}} \left[ \left\| (\mathbf {X} _ {\overline {{S}}} - \overline {{\mathbf {X}}} ^ {\overline {{S}}}) \mathbf {b} - (\mathbf {t} _ {\overline {{S}}} - \overline {{\mathbf {t}}} ^ {\overline {{S}}}) \right\| _ {2} ^ {2} + \lambda \| \mathbf {b} \| _ {2} ^ {2} \right] \\ \leq \frac {d}{\alpha} \cdot (1 + \epsilon) \| \boldsymbol {\mu} \| _ {\infty} ^ {2} + (1 + \epsilon) \left[ \left\| (\mathbf {X} _ {\overline {{S}}} - \overline {{\mathbf {X}}}) \widehat {\mathbf {b}} - (\mathbf {t} _ {\overline {{S}}} - \overline {{\mathbf {t}}}) \right\| _ {2} ^ {2} + \lambda \left\| \widehat {\mathbf {b}} \right\| _ {2} ^ {2} \right] \\ \end{array}
$$
Now since  $(\mathbf{X}_{\overline{S}} - \overline{\mathbf{X}}^{\overline{S}})\widehat{\mathbf{b}} -(\mathbf{t}_{\overline{S}} - \overline{\mathbf{t}}^{\overline{S}})$  is orthogonal to the vector of all ones, we have
$$
\begin{array}{l} \left\| (\mathbf {X} _ {\overline {{S}}} - \overline {{\mathbf {X}}} ^ {\overline {{S}}}) \widehat {\mathbf {b}} - (\mathbf {t} _ {\overline {{S}}} - \overline {{\mathbf {t}}} ^ {\overline {{S}}}) \right\| _ {2} ^ {2} \leq \left\| (\mathbf {X} _ {\overline {{S}}} - \overline {{\mathbf {X}}} ^ {\overline {{S}}}) \widehat {\mathbf {b}} - (\mathbf {t} _ {\overline {{S}}} - \overline {{\mathbf {t}}} ^ {\overline {{S}}}) \right\| _ {2} ^ {2} + \left\| (\overline {{\mathbf {X}}} ^ {\overline {{S}}} - \overline {{\mathbf {X}}} _ {\overline {{S}}}) \widehat {\mathbf {b}} - (\overline {{\mathbf {t}}} ^ {\overline {{S}}} - \overline {{\mathbf {t}}} _ {\overline {{S}}}) \right\| _ {2} ^ {2} \\ = \left\| \left(\mathbf {X} _ {\overline {{S}}} - \overline {{\mathbf {X}}} _ {\overline {{S}}}\right) \widehat {\mathbf {b}} - \left(\mathbf {t} _ {\overline {{S}}} - \overline {{\mathbf {t}}} _ {\overline {{S}}}\right) \right\| _ {2} ^ {2} \\ \leq \left\| (\mathbf {X} - \overline {{\mathbf {X}}}) \widehat {\mathbf {b}} - (\mathbf {t} - \overline {{\mathbf {t}}}) \right\| _ {2} ^ {2}. \\ \end{array}
$$
Table 4: Variance of different methods for estimating ATE with few samples.  $\pmb{\mu} = \mathbf{y}^{(1)} + \mathbf{y}^{(0)}$ ,  $\mathbf{t} = \mathbf{y}^{(1)} - \mathbf{y}^{(0)}$ .  $0 < \phi < 1$  is an input parameter.  $0 < \epsilon < 1$  is an error parameter.  $\zeta$  is the maximum  $\ell_2$  norm over rows of  $\mathbf{X}$ .  $\mathbf{b}^* = \arg \min_{b \in \mathbb{R}^d} \left[ \frac{1}{\phi} \| \mathbf{X}\mathbf{b} - \boldsymbol{\mu} \|_2^2 + \frac{1}{1 - \phi} \zeta^2 \| \mathbf{b} \|_2^2 \right]$ , and  $\widehat{\mathbf{b}} = \arg \min_{\mathbf{b} \in \mathbb{R}^d} \left\| (\mathbf{X} - \overline{\mathbf{X}}) \widehat{\mathbf{b}} - (\mathbf{t} - \overline{\mathbf{t}}) \right\|_2^2 + \lambda \| \widehat{\mathbf{b}} \|_2^2$ , where  $\lambda = \frac{6 \cdot \log(d / \delta)}{\epsilon^2} \cdot \zeta^2$ .
$$
\begin{array}{l} \mathbb {E} \left[ \left\| \left(\mathbf {X} _ {\overline {{S}}} - \overline {{\mathbf {X}}} ^ {\overline {{S}}}\right) \widehat {\mathbf {v}} - \left(\mathbf {t} _ {\overline {{S}}} - \overline {{\mathbf {t}}} ^ {\overline {{S}}}\right) \right\| _ {2} ^ {2} \right] \tag {21} \\ \leq \frac {d}{\alpha} \cdot (1 + \epsilon) \| \boldsymbol {\mu} \| _ {\infty} ^ {2} + (1 + \epsilon) \left[ \left\| (\mathbf {X} - \overline {{\mathbf {X}}}) \widehat {\mathbf {b}} - (\mathbf {t} - \bar {\mathbf {t}}) \right\| _ {2} ^ {2} + \lambda \left\| \widehat {\mathbf {b}} \right\| _ {2} ^ {2} \right]. \\ \end{array}
$$
the results follow by picking  $\alpha = d / n$  and combining (18), (19), (20), and (21).
We compared the variance bounds obtained by our approach for ATE estimation with partial observation with other methods in Table 4.
E Detailed Description of Datasets and Extra Experiments
The costliest operation in our algorithms is to compute (ridge) leverage scores. This can be performed in input-sparsity time (i.e.,  $\tilde{O} (\mathrm{nnz} + d^3)$ , where nnz denotes the number of nonzero entries in the covariate matrix  $\mathbf{X}$  [10]). Therefore, our algorithms are much more efficient than to the GSW design, which has a running time of  $O(n^{2}d)$  [18]. This significant difference in the running time is also illustrated in our experiments, as shown in Table 6.
Some details of the datasets we used in our experiments are presented in Table 5. Moreover, the comparison of our ATE approach with two other methods (Lin's [24] and difference in means) is presented in Table 7 and Figure 2.
F Gram-Schmidt Walk Design
Harshaw et al [18] introduced Algorithm 5 for balanced experimental design.