SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof. Fix  $\varepsilon > 0$ . From Proposition 3.4, if  $\mathcal{Q}$  is public-privately learnable, then it is list learnable to list size  $\ell(\alpha, \beta)$  with  $m_L(\alpha, \beta)$  samples, where
$$
\ell (\alpha , \beta) = \frac {1 0}{9} \exp \left(\varepsilon \cdot n \left(\frac {\alpha}{2}, \frac {\beta}{1 0}, \varepsilon\right)\right) \quad \text {a n d} \quad m _ {L} (\alpha , \beta) = m _ {P} \left(\frac {\alpha}{2}, \frac {\beta}{1 0}, \varepsilon\right).
$$
Proposition 3.5 implies  $\mathcal{Q}$  admits  $(\tau (\alpha ,\beta),t(\alpha ,\beta),m_C(\alpha ,\beta))$  sample compression, where
$$
\tau (\alpha , \beta) = m _ {L} (\alpha , \beta) = m _ {P} \left(\frac {\alpha}{2}, \frac {\beta}{1 0}, \varepsilon\right),
$$
$$
t (\alpha , \beta) = \log_ {2} (\ell (\alpha , \beta)) = \frac {\log (\frac {1 0}{9}) + \varepsilon \cdot n \left(\frac {\alpha}{2} , \frac {\beta}{1 0} , \varepsilon\right)}{\log (2)},
$$
$$
m _ {C} (\alpha , \beta) = m _ {L} (\alpha , \beta) = m _ {P} \left(\frac {\alpha}{2}, \frac {\beta}{1 0}, \varepsilon\right).
$$
As a consequence of Corollary E.4, Fact E.3, and Proposition 3.1, we have the following result about the public-private learnability of mixture distributions.
Theorem E.5 (Public-private learning for mixture distributions). Suppose  $\mathcal{Q} \subseteq \Delta(\mathcal{X})$  is public-privately learnable with  $m(\alpha, \beta, \varepsilon)$  public samples and  $n(\alpha, \beta, \varepsilon)$  private samples. Then for any  $k \geq 1$ ,  $\mathcal{Q}^{\oplus k}$ , the class of  $k$ -mixtures of  $\mathcal{Q}$ , is public-privately learnable with  $m_k(\alpha, \beta, \varepsilon)$  public samples and  $n_k(\alpha, \beta, \varepsilon)$  private samples, where
$$
\begin{array}{l} m _ {k} (\alpha , \beta , \varepsilon) = O \left(\frac {k \log \left(\frac {k}{\beta}\right)}{\alpha} \cdot m \left(\frac {\alpha}{3 6}, \frac {\beta}{2 0}, \varepsilon_ {0}\right)\right), \\ n _ {k} (\alpha , \beta , \varepsilon) = O \left(\left(\frac {1}{\alpha^ {2}} + \frac {1}{\varepsilon \alpha}\right) \cdot \left(\varepsilon_ {0} k \cdot n \left(\frac {\alpha}{3 6}, \frac {\beta}{2 0}, \varepsilon_ {0}\right) + \right. \right. \\ k \log \left(\frac {k \log \left(\frac {k}{\beta}\right)}{\alpha} \cdot m \left(\frac {\alpha}{3 6}, \frac {\beta}{2 0}, \varepsilon_ {0}\right)\right) \cdot m \left(\frac {\alpha}{3 6}, \frac {\beta}{2 0}, \varepsilon_ {0}\right) + \log \left(\frac {1}{\beta}\right)\left. \right) \\ \end{array}
$$
for any choice of  $\varepsilon_0 > 0$
We give an example of an application of this result. Consider the class of Gaussians over  $\mathbb{R}^d$ , for which there exists a public-private learner that uses  $m = O(d)$  public samples and  $n = O\left(\frac{d^2}{\alpha^2} + \frac{d^2}{\varepsilon \alpha}\right) \cdot \text{polylog}\left(d, \frac{1}{\alpha}, \frac{1}{\beta}\right)$  private samples [BKS22]. Then Theorem E.5 implies that there exists a public-private learner for the class of  $k$ -mixtures of Gaussians that uses  $m_k = O\left(\frac{kd \log(k / \beta)}{\alpha}\right)$  public samples and
$$
n _ {k} = O \left(\left(\frac {1}{\alpha^ {2}} + \frac {1}{\varepsilon \alpha}\right) \cdot \left(\varepsilon_ {0} k \left(\frac {d ^ {2}}{\alpha^ {2}} + \frac {d ^ {2}}{\varepsilon_ {0} \alpha}\right) + k d\right)\right) \cdot \operatorname {p o l y l o g} \left(d, k, \frac {1}{\alpha}, \frac {1}{\beta}\right)
$$
private samples for any  $\varepsilon_0 > 0$
With the choice of  $\varepsilon_0 = \alpha$ , we get a private sample complexity of  $n_k = O\left(\frac{kd^2}{\alpha^3} + \frac{kd^2}{\alpha^2\varepsilon}\right)$ . polylog  $\left(d, k, \frac{1}{\alpha}, \frac{1}{\beta}\right)$ . Notably, this private sample complexity, obtained by specializing the general result of Theorem E.5, suffers some loss compared to our learner for mixtures of Gaussians from Corollary E.2.
Product distributions. We start by mentioning a fact from  $\mathrm{[ABDH^{+}20]}$ , which says that if a compression scheme exists for a class of distributions  $\mathcal{Q}$ , then there exists a compression scheme for the class of  $k$ -products of  $\mathcal{Q}$ .
Fact E.6 (Compression for product distributions [ABDH+20, Lemma 4.6]). If a class of distributions  $\mathcal{Q}$  admits  $(\tau (\alpha ,\beta),t(\alpha ,\beta),m(\alpha ,\beta))$  r-robust sample compression, then for any  $k\geq 1$ , the class of  $k$ -products of  $\mathcal{Q}$  admits  $(\tau_{k}(\alpha ,\beta),t_{k}(\alpha ,\beta),m_{k}(\alpha ,\beta))$  r-robust sample compression, where  $\tau_k,t_k,m_k: (0,1]\to \mathbb{N}$  are as follows:
$$
\tau_ {k} (\alpha , \beta) = k \cdot \tau \left(\frac {\alpha}{k}, \beta\right), \quad t _ {k} (\alpha , \beta) = k \cdot t \left(\frac {\alpha}{k}, \beta\right), \quad m _ {k} (\alpha , \beta) = \log_ {3} \left(\frac {3 k}{\beta}\right) \cdot m \left(\frac {\alpha}{k}, \beta\right).
$$
As a consequence of Corollary E.4, Fact E.6, and Proposition 3.1, we have the following result about the public-private learnability of product distributions.
Theorem E.7 (Public-private learning for mixture distributions). Suppose  $\mathcal{Q} \subseteq \Delta(\mathcal{X})$  is public-privately learnable with  $m(\alpha, \beta, \varepsilon)$  public samples and  $n(\alpha, \beta, \varepsilon)$  private samples. Then for any  $k \geq 1$ ,  $\mathcal{Q}^{\otimes k}$ , the class of  $k$ -products of  $\mathcal{Q}$  over  $\mathcal{X}^k$ , is public-privately learnable with  $m_k(\alpha, \beta, \varepsilon)$  public samples and  $n_k(\alpha, \beta, \varepsilon)$  private samples, where
$$
\begin{array}{l} m _ {k} (\alpha , \beta , \varepsilon) = O \left(\log \left(\frac {k}{\beta}\right) \cdot m \left(\frac {\alpha}{1 2 k}, \frac {\beta}{2 0}, \varepsilon_ {0}\right)\right), \\ n _ {k} (\alpha , \beta , \varepsilon) = O \left(\left(\frac {1}{\alpha^ {2}} + \frac {1}{\varepsilon \alpha}\right) \cdot \left(\varepsilon_ {0} k \cdot n \left(\frac {\alpha}{1 2 k}, \frac {\beta}{2 0}, \varepsilon_ {0}\right) + \right. \right. \\ k \log \left(\log \left(\frac {k}{\beta}\right) \cdot m \left(\frac {\alpha}{1 2 k}, \frac {\beta}{2 0}, \varepsilon_ {0}\right)\right) \cdot m \left(\frac {\alpha}{1 2 k}, \frac {\beta}{2 0}, \varepsilon_ {0}\right) + \log \left(\frac {1}{\beta}\right))\left. \right) \\ \end{array}
$$
As an example, for the class of Gaussians over  $\mathbb{R}$ , there exists a public-private learner that requires  $m = O(1)$  public samples and  $n = O\left(\frac{1}{\alpha^2} + \frac{1}{\varepsilon\alpha}\right) \cdot \text{polylog}\left(\frac{1}{\alpha}, \frac{1}{\beta}\right)$  private samples [BKS22]. Then Theorem E.7 implies that there exists a public-private learner for the class of  $k$ -products of Gaussians that requires  $m_k = O\left(\log (k / \beta)\right)$  public samples and  $n_k = \left(\left(\frac{1}{\alpha^2} + \frac{1}{\varepsilon\alpha}\right) \cdot \left(\varepsilon_0 k\left(\frac{1}{\alpha^2} + \frac{1}{\varepsilon_0\alpha}\right) + k\right)\right) \cdot \text{polylog}\left(k, \frac{1}{\alpha}, \frac{1}{\beta}\right)$  private samples, for any choice of  $\varepsilon_0 > 0$ . Note that if we were to apply Fact E.6 to Fact 4.1 after setting  $d = 1$  in the latter, and then apply Proposition 3.1, we would obtain a better sample complexity in terms of the private data than what we would after combining Corollary E.1 (setting  $d = 1$ ) and Theorem E.7 here. However, Theorem E.7 is a more versatile framework, so some loss is to be expected again.
F Statements and proofs for Section 5 – Agnostic and distribution-shifted public-private learning
Theorem 5.2 gives us an agnostic and a distribution-shifted learner for Gaussians over  $\mathbb{R}^d$ , as stated in the following corollary.
Corollary F.1 (Agnostic and distribution-shifted public-private learner for Gaussians). Let  $d \geq 1$ . For any  $\alpha, \beta \in (0,1]$  and  $\varepsilon > 0$ , there exists  $\frac{1}{3}$ -shifted 3-agnostic public-private learner for the class of Gaussians in  $\mathbb{R}^d$  that uses  $m$  public samples and  $n$  private samples, where
$$
\begin{array}{l} m = O \left(d \log \left(\frac {1}{\beta}\right)\right), \\ n = O \left(\frac {d ^ {2} \log \left(\frac {d}{\alpha}\right) + \log \left(\frac {1}{\beta}\right)}{\alpha^ {2}} + \frac {d ^ {2} \log \left(\frac {d}{\alpha}\right) + \log \left(\frac {1}{\beta}\right)}{\alpha \varepsilon}\right). \\ \end{array}
$$
G Statements and proofs for Section 6 - Lower bounds
Lemma G.1. Let  $\mathcal{Q} \subseteq \Delta(\mathcal{X})$  and  $m \in \mathbb{N}$ . For a subclass  $\mathcal{C} \subseteq \mathcal{Q}$ , denote by  $\mathcal{U}(\mathcal{C})$  the uniform distribution over  $\mathcal{C}$ . Suppose there exists a sequence of distribution classes  $(\mathcal{Q}_k)_{k=1}^{\infty}$ , with each  $\mathcal{Q}_k \subseteq \mathcal{Q}$ , and a set  $B \subseteq \mathcal{X}^m$  such that following holds:
1. There exists  $\eta \in (0,1]$  and  $k_{\eta} \in \mathbb{N}$  with
$$
\mathop{\mathbb{P}}_{\substack{Q\sim \mathcal{U}(\mathcal{Q}_{k})\\ \boldsymbol {X}\sim Q^{m}}}\left\{\boldsymbol {X}\in B\right\} \geq \eta
$$
for all  $k \geq k_{\eta}$ .
2. There exist  $c > 0$  and  $\alpha \in (0,1]$  such that, defining  $(u_k)_{k=1}^{\infty}$ ,  $(r_k)_{k=1}^{\infty}$ , and  $(s_k)_{k=1}^{\infty}$  as
$$
u_{k}:= \sup_{\substack{\boldsymbol {x}\in B\\ q\in \mathcal{Q}_{k}}}q^{m}(\boldsymbol {x}),
$$
$$
r_{k}:= \sup_{p\in \mathcal{Q}_{k}}\underset {Q\sim \mathcal{U}(\mathcal{Q}_{k})}{\mathbb{P}}\{\mathrm{TV}(p,Q)\leq 2\alpha \} ,
$$
$$
s_{k}:= \inf_{\boldsymbol {x}\in B}\mathbb{P}_{Q\sim \mathcal{U}(\mathcal{Q}_{k})}\left\{Q^{m}(\boldsymbol {x})\geq c\cdot u_{k}\right\} ,
$$
$$
\lim  _ {k \rightarrow \infty} \frac {r _ {k}}{s _ {k}} = 0.
$$
Then for any  $\ell \in \mathbb{N}$ , there does not exist any  $(\frac{\alpha\eta}{4},\frac{\alpha\eta}{4},\ell)$ -list learner for  $\mathcal{Q}$  that uses  $m$  samples.
Proof. We provide a proof by contradiction. Suppose for some  $\ell \in \mathbb{N}$ , we have an  $(\frac{\alpha\eta}{4},\frac{\alpha\eta}{4},\ell)$ -list learner for  $\mathcal{Q}$  using  $m$  samples, denoted by  $\mathcal{L}:\mathcal{X}^m\to \{L\subseteq \Delta (\mathcal{X}):|L|\leq \ell \}$ . Then for all  $k\in \mathbb{N}$  we have that
$$
\underset { \begin{array}{c} Q \sim \mathcal {U} \left(\mathcal {Q} _ {k}\right) \\ \boldsymbol {X} \sim Q ^ {m} \end{array} } {\mathbb {E}} \left[ \operatorname {d i s t} (Q, \mathcal {L} (\boldsymbol {X})) \right] \leq \left(1 - \frac {\alpha \eta}{4}\right) \cdot \frac {\alpha \eta}{4} + \frac {\alpha \eta}{4} \cdot 1 \leq \frac {\alpha \eta}{2}. \tag {1}
$$
Now, since  $\lim_{k\to \infty}\frac{r_k}{s_k} = 0$ , there exists  $k_0\geq k_\eta \in \mathbb{N}$ , such that
$$
\frac {r _ {k _ {0}} \cdot u _ {k _ {0}} \cdot \ell}{s _ {k _ {0}} \cdot c u _ {k _ {0}}} \leq \frac {1}{1 1}, \tag {2}
$$
$$
\underset { \begin{array}{c} Q \sim \mathcal {U} \left(\mathcal {Q} _ {k _ {0}}\right) \\ X \sim Q ^ {m} \end{array} } {\mathbb {P}} \left\{\boldsymbol {X} \in B \right\} \geq \eta . \tag {3}
$$
Fix any  $\pmb{x} \in B$ , and let  $R = \{q \in \mathcal{Q}_{k_0} : \mathrm{dist}(q, \mathcal{L}(\pmb{x})) \leq \alpha\}$  and  $S = \{q \in \mathcal{Q}_{k_0} : q^m(\pmb{x}) \geq cu_{k_0}\}$  (note that both  $R$  and  $S$  depend on  $\pmb{x}$ ).
For  $i\in [\ell ]$  , further let  $R_{i} = \{q\in \mathcal{Q}_{k_{0}}:\mathrm{TV}(q,\mathcal{L}(\pmb {x})_{i})\leq \alpha \}$  , so that  $R = \cup_{i = 1}^{\ell}R_{i}$
Now, fix  $i \in [\ell]$ . Assuming that  $R_i \neq \emptyset$ , consider any  $p \in R_i$ . For any  $q \in R_i$ , we have  $\mathrm{TV}(p,q) \leq 2\alpha$ . Hence,  $R_i \subseteq \{q \in \mathcal{Q}_{k_0} : \mathrm{TV}(p,q) \leq 2\alpha\}$ . Regardless of whether  $R_i$  is empty,
$$
\underset {Q \sim \mathcal {U} (\mathcal {Q} _ {k _ {0}})} {\mathbb {P}} \{Q \in R _ {i} \} \leq \sup  _ {p \in \mathcal {Q} _ {k _ {0}}} \underset {Q \sim \mathcal {U} (\mathcal {Q} _ {k _ {0}})} {\mathbb {P}} \{\mathrm {T V} (p, Q) \leq 2 \alpha \} = r _ {k _ {0}}.
$$
Moreover, we can conclude that
$$
\mathbb {P} _ {Q \sim \mathcal {U} \left(\mathcal {Q} _ {k _ {0}}\right)} \{Q \in R \} \leq \sum_ {i = 1} ^ {\ell} \mathbb {P} _ {Q \sim \mathcal {U} \left(\mathcal {Q} _ {k _ {0}}\right)} \{Q \in R _ {i} \} \leq r _ {k _ {0}} \cdot \ell . \tag {4}
$$
Observe that this implies, since  $u_{k_0} \geq q^m(\pmb{x})$ ,
$$
\int_ {R} q ^ {m} (\boldsymbol {x}) f _ {Q} (q) d q \leq u _ {k _ {0}} \int_ {R} f _ {Q} (q) d q = u _ {k _ {0}} \underset {Q \sim \mathcal {U} \left(\mathcal {Q} _ {k _ {0}}\right)} {\mathbb {P}} \{Q \in R \} \leq u _ {k _ {0}} \cdot r _ {k _ {0}} \cdot \ell \tag {5}
$$
an inequality we will use momentarily. We can now write
(By definition of  $S$  and (5))
$$
\begin{array}{l} \underset { \begin{array}{c} Q \sim \mathcal {U} (\mathcal {Q} _ {k _ {0}}) \\ \boldsymbol {X} \sim Q ^ {m} \end{array} } {\mathbb {E}} \left[ \operatorname {d i s t} (Q, \mathcal {L} (\boldsymbol {X})) \mid \boldsymbol {X} = \boldsymbol {x} \right] = \int_ {\mathcal {Q} _ {k _ {0}}} f _ {Q | \boldsymbol {X}} (q \mid \boldsymbol {x}) \cdot \operatorname {d i s t} (q, \mathcal {L} (\boldsymbol {x})) d q \\ \geq \int_ {S \backslash R} f _ {Q | \mathbf {X}} (q \mid \mathbf {x}) \cdot \operatorname {d i s t} (q, \mathcal {L} (\mathbf {x})) d q \\ \geq \alpha \int_ {S \backslash R} f _ {Q | \boldsymbol {X}} (q \mid \boldsymbol {x}) d q \\ \geq \alpha \left(\int_ {S} f _ {Q | \mathbf {X}} (q \mid \mathbf {x}) d q - \int_ {R} f _ {Q | \mathbf {X}} (q \mid \mathbf {x}) d q\right) \\ = \alpha \left(\int_ {S} \frac {q ^ {m} (\boldsymbol {x}) f _ {Q} (q)}{f _ {X} (\boldsymbol {x})} d q - \int_ {R} \frac {q ^ {m} (\boldsymbol {x}) f _ {Q} (q)}{f _ {X} (\boldsymbol {x})} d q\right) \\ = \alpha \frac {1}{f _ {X} (\boldsymbol {x})} \left(\int_ {S} q ^ {m} (\boldsymbol {x}) f _ {Q} (q) d q - \int_ {R} q ^ {m} (\boldsymbol {x}) f _ {Q} (q) d q\right) \\ \geq \alpha \frac {1}{f _ {X} (\boldsymbol {x})} \left(c u _ {k _ {0}} \int_ {S} f _ {Q} (q) d q - u _ {k _ {0}} \cdot \ell \cdot r _ {k _ {0}}\right) \\ = \alpha \frac{1}{f_{X}(\boldsymbol{x})}\left(cu_{k_{0}}\underset {Q\sim \mathcal{U}(\mathcal{Q}_{k_{0}})}{\mathbb{P}}\left\{Q\in S\right\} -u_{k_{0}}\cdot \ell \cdot r_{k_{0}}\right). \\ \end{array}
$$
Plugging (4) in, along with the definition of  $s_{k_0}$ , we have,
$$
\begin{array}{l} \underset { \begin{array}{c} Q \sim \mathcal {U} (Q _ {k _ {0}}) \\ \boldsymbol {X} \sim Q ^ {m} \end{array} } {\mathbb {E}} \left[ \operatorname {d i s t} (Q, \mathcal {L} (\boldsymbol {X})) \mid \boldsymbol {X} = \boldsymbol {x} \right] \geq \alpha \frac {1}{f _ {X} (\boldsymbol {x})} \left(c u _ {k _ {0}} \cdot s _ {k _ {0}} - u _ {k _ {0}} \cdot \ell \cdot r _ {k _ {0}}\right) \\ \geq \alpha \frac {1}{f _ {X} (\boldsymbol {x})} \left(1 0 \cdot u _ {k _ {0}} \cdot \ell \cdot r _ {k _ {0}}\right) \quad (k _ {0} \text {f r o m E q u a t i o n 2}) \\ \geq 1 0 \alpha \int_ {R} \frac {q ^ {m} (\boldsymbol {x}) f _ {Q} (q)}{f _ {X} (\boldsymbol {x})} d q \tag {by(5)} \\ = 10\alpha \cdot \underset { \begin{array}{c} Q \sim \mathcal {U} (\mathcal {Q} _ {k _ {0}}) \\ X \sim Q ^ {m} \end{array} } {\mathbb {P}} \left\{Q \in R   \big |   \boldsymbol {X} = \boldsymbol {x} \right\}. \\ \end{array}
$$
Integrating over all  $\pmb{x} \in B$  and using Inequality 3,
$$
\begin{array}{l} \underset { \begin{array}{c} Q \sim \mathcal {U} (\mathcal {Q} _ {k _ {0}}) \\ \boldsymbol {X} \sim Q ^ {m} \end{array} } {\mathbb {E}} [ \mathrm {d i s t} (Q, \mathcal {L} (\boldsymbol {X})) ] \geq \underset { \begin{array}{c} Q \sim \mathcal {U} (\mathcal {Q} _ {k _ {0}}) \\ \boldsymbol {X} \sim Q ^ {m} \end{array} } {\mathbb {P}} \left\{\boldsymbol {X} \in B \right\} \cdot \underset { \begin{array}{c} Q \sim \mathcal {U} (\mathcal {Q} _ {k _ {0}}) \\ \boldsymbol {X} \sim Q ^ {m} \end{array} } {\mathbb {E}} [ \mathrm {d i s t} (Q, \mathcal {L} (\boldsymbol {X})) | \boldsymbol {X} \in B ] \\ \geq \eta \cdot \underset { \begin{array}{c} Q \sim \mathcal {U} (\mathcal {Q} _ {k _ {0}}) \\ X \sim Q ^ {m} \end{array} } {\mathbb {E}} [ \operatorname {d i s t} (Q, \mathcal {L} (X)) | X \in B ] \\ \geq \eta \cdot 10\alpha \cdot \underset { \begin{array}{c} Q \sim \mathcal {U} (\mathcal {Q} _ {k _ {0}}) \\ X \sim Q ^ {m} \end{array} } {\mathbb {P}} \{\operatorname {d i s t} (Q, \mathcal {L} (X)) \leq \alpha   |   X \in B \}. \\ \end{array}
$$
If  $\mathbb{P}\{\mathrm{dist}(Q,\mathcal{L}(\pmb {X}))\leq \alpha \mid \pmb {X}\in B\} \geq \frac{1}{10}$ , then  $\mathbb{E}\left[\mathrm{dist}(Q,\mathcal{L}(\pmb {X}))\right]\geq \alpha \eta$ , contradicting (1). Otherwise,
$$
\begin{array}{l} \mathbb {E} \left[ \operatorname {d i s t} (Q, \mathcal {L} (\boldsymbol {X})) \right] \geq \eta \cdot \mathbb {E} \left[ \operatorname {d i s t} (Q, \mathcal {L} (\boldsymbol {X})) \mid \boldsymbol {X} \in B \right] \\ \geq \eta \cdot \alpha \cdot \mathbb {P} \left\{\operatorname {d i s t} (Q, \mathcal {L} (\boldsymbol {X})) > \alpha \mid \boldsymbol {X} \in B \right\} \\ \geq \eta \cdot (\alpha \cdot (1 - \frac {1}{1 0})), \\ \end{array}
$$
also contradicting Equation 1.
Proof of Theorem 6.1. To prove Theorem 6.1, it suffices to find, for every  $d \geq 2$ , a sequence of subclasses  $(\mathcal{Q}_k)_{k=1}^{\infty}$  and a set  $B \in (\mathbb{R}^d)^{d-1}$  that indeed satisfy the conditions of Lemma G.1. In what follows, we fix an arbitrary  $d \geq 2$ .
The construction of the sequence of hard subclasses. Let  $e_d = [0, 0, \dots, 1]^\top \in \mathbb{R}^d$ . We define the following sets:
$$
T = \left\{\left[ \begin{array}{c} t \\ 0 \end{array} \right] \in \mathbb {R} ^ {d}: t \in \mathbb {R} ^ {d - 1} \text {w i t h} \| t \| _ {2} \leq \frac {1}{2} \right\},
$$
$$
C = \left\{\left[ \begin{array}{c} t \\ \lambda \end{array} \right] \in \mathbb {R} ^ {d}: t \in \mathbb {R} ^ {d - 1} \text {w i t h} \| t \| _ {2} \leq \frac {1}{2} \text {a n d} \lambda \in [ 1, 2 ] \subseteq \mathbb {R} \right\}.
$$
That is,  $T$  is a  $\frac{1}{2}$ -disk (a disk with radius  $\frac{1}{2}$ ) in  $\mathbb{R}^{d-1}$  embedded onto the  $(d-1)$ -dimensional hyperplane in  $\mathbb{R}^d$  spanning the first  $(d-1)$  dimensions (axes), centered at the origin.  $C$  is a cylinder of unit length and radius  $\frac{1}{2}$  placed unit distance away from  $T$  in the positive  $e_d$ -direction.
Let  $S^{d - 1} = \{x\in \mathbb{R}^d:\| x\| _2 = 1\}$  be the unit-sphere, centered at the origin, in  $\mathbb{R}^d$ , and let
$$
N = \left\{u \in S ^ {d - 1}: | u \cdot e _ {d} | \leq \frac {\sqrt {3}}{2} \right\}.
$$
That is,  $N$  is the set of vectors  $u$  on the unit hypersphere with angle  $\geq \frac{\pi}{6}$  from  $e_d$ . For  $u \in N$ , define the "rotatiou" matrix
$$
R _ {u} = \left[ \begin{array}{c c c c} | & | & & | \\ u & v _ {2} & \ldots & v _ {d} \\ | & | & & | \end{array} \right] \in \mathbb {R} ^ {d \times d}
$$
where  $\{v_{2},\ldots ,v_{d}\}$  is any orthonormal basis for  $\{u\}^{\perp}$  (where  $\{u\}^{\perp}$  denotes the subspace orthogonal to the subspace spanned by the set of vectors  $\{u\}$ ).<sup>4</sup>
Now, for  $\sigma > 0$ ,  $t \in T$ , and  $u \in N$ , define the Gaussian
$$
G (\sigma , t, u) = \mathcal {N} \left(t, R _ {u} \left[ \begin{array}{c c c} \sigma^ {2} & & \\ & 1 & O \\ & O & \ddots \\ & & 1 \end{array} \right] R _ {u} ^ {\top}\right) \in \Delta (\mathbb {R} ^ {d}).
$$
For all  $k\geq 1$  , let
$$
\mathcal {Q} _ {k} = \left\{G \left(\frac {1}{k}, t, u\right): t \in T, u \in N \right\}.
$$
That is, each  $\mathcal{Q}_k$  is a class of "flat" (i.e., near  $(d - 1)$ -dimensional) Gaussians in  $\mathbb{R}^d$ , with  $\sigma^2 = \frac{1}{k^2}$  variance on a single thin direction  $u$  and unit variance in all other directions. Their mean vectors come from a point on the hyperplanar disk  $T$  (which we recall is a  $(d - 1)$ -dimensional disk orthogonal to  $e_d$ ), and the thin direction  $u$  comes from  $N$  (which is  $S^{d - 1}$  excluding points that form angle  $< \frac{\pi}{6}$  with  $e_d$ ). As  $k \to \infty$ , the Gaussians get flatter.
Lower bounding the weight of  $B$ . We start with the following claim, which shows the probability that  $d - 1$  samples drawn the uniform mixture of  $\mathcal{Q}_k^{d - 1}$  all fall into the cylinder  $C$  can be uniformly lower bounded by an absolute constant, independent of  $k$ .
Claim G.2. Let  $B$  be the set of all possible vectors of  $d - 1$  points in the cylinder  $C$ , i.e.,  $B = C^{d - 1} \in (\mathbb{R}^d)^{d - 1}$ . There exists  $\eta > 0$  such that for  $k \geq 10$ ,
$$
\mathop{\mathbb{P}}_{\substack{Q\sim \mathcal{U}(\mathcal{Q}_{k})\\ \boldsymbol {X}\sim Q^{d - 1}}}\left\{\boldsymbol {X}\in B\right\} \geq \eta .
$$
Proof of Claim G.2. Consider the inscribed cylinder  $C' \subseteq C$
$$
C ^ {\prime} = \left\{\left[ \begin{array}{c} t \\ \lambda \end{array} \right] \in \mathbb {R} ^ {d}: t \in \mathbb {R} ^ {d - 1} \text {w i t h} \| t \| _ {2} \leq \frac {1}{3} \text {a n d} \lambda \in \left[ \frac {4}{3}, \frac {5}{3} \right] \subseteq \mathbb {R} \right\}.
$$
Technically,  $R_{u}$  is an equivalence class of matrices since we do not specify which orthonormal basis of  $\{u\}^{\perp}$ . However, as it turns out, the choice of the orthonormal basis of  $\{u\}^{\perp}$  does not matter since they all result in the same Gaussian densities in the preceding definition of  $G(\sigma, t, u)$ .
Also, consider  $T' \subseteq T$  and  $N' \subseteq N$ :
$$
T ^ {\prime} = \left\{\left[ \begin{array}{c} t \\ 0 \end{array} \right] \in \mathbb {R} ^ {d}: t \in \mathbb {R} ^ {d - 1} \text {w i t h} \| t \| _ {2} \leq \frac {1}{4} \right\},
$$
$$
N ^ {\prime} = \left\{u \in S ^ {d - 1}: | u \cdot e _ {d} | \leq \frac {1}{3 6} \right\}.
$$
Now, fix  $u \in N'$  and  $t \in T'$ . Define the plane going through  $t$  with normal vector  $u$  as,
$$
P (u, t) = \left\{t + x: x \in \mathbb {R} ^ {d} \text {w i t h} x \cdot u = 0 \right\}.
$$
First, we show  $P(u,t) \cap C'$  contains a  $(d - 1)$ -dimensional region. Consider,
$$
y = \left[ \begin{array}{c} t \\ \frac {3}{2} \end{array} \right].
$$
The projection onto  $P(u,t)$  of  $y$  is given by,
$$
y ^ {\prime} = (y - t) - ((y - t) \cdot u) u + t = \left[ \begin{array}{c} t \\ \frac {3}{2} \end{array} \right] - c u,
$$
where  $|c| = |(y - t) \cdot u| \leq \frac{3}{2} \cdot \frac{1}{36} = \frac{1}{24}$ . Since  $\|t\|_2 \leq \frac{1}{4}$ , the norm of the first  $(d-1)$  dimensions of  $y'$  is  $\leq \frac{1}{4} + \frac{1}{24} \leq \frac{1}{3}$  and  $y_d' \in [\frac{3}{2} - \frac{1}{24}, \frac{3}{2} + \frac{1}{24}]$ , and so  $y' \in C'$ . Moreover, adding any  $z$  with  $z \cdot u = 0$  and  $\|z\|_2 \leq \frac{1}{24}$  results in  $y' + z$  with the norm of the first  $d-1$  dimensions being at most  $\frac{1}{4} + \frac{1}{24} + \frac{1}{24} \leq \frac{1}{3}$  and  $(y' + z)_d \in [\frac{3}{2} - \frac{1}{12}, \frac{3}{2} + \frac{1}{12}]$ . Hence,  $y' + z \in C'$ . This shows that  $P(u, t) \cap C'$  contains a  $(d-1)$ -dimensional subspace, since it contains a  $(d-1)$ -dimensional disk of radius  $\frac{1}{24}$ .
Next, let
$$
M = \left\{p + s u: p \in C ^ {\prime} \cap P (u, t), s \in \left[ - \frac {1}{6}, \frac {1}{6} \right] \subseteq \mathbb {R} \right\}.
$$
That is,  $M$  is a rectangular "extrusion" of  $C' \cap P(u,t)$  along both its normal vectors. Indeed, we have  $M \subseteq C$ , since adding a vector of length  $\leq \frac{1}{6}$  cannot take a point in  $C'$  outside of  $C$ . We also have that  $M$  is a  $d$ -dimensional region, so
$$
\underset {X \sim G (1 / 1 0, t, u)} {\mathbb {P}} \left\{X \in C \right\} \geq \underset {X \sim G (1 / 1 0, t, u)} {\mathbb {P}} \left\{X \in M \right\} > 0.
$$
Note that for  $\sigma \leq \frac{1}{10}$ , we have
$$
\underset {X \sim G (\sigma , t, u)} {\mathbb {P}} \{X \in M \} \geq \underset {X \sim G (1 / 1 0, t, u)} {\mathbb {P}} \{X \in M \}.
$$
This is because any  $x \in M$  can be written as  $t + x + cu$ , where  $x$  is such that  $x \cdot u = 0$ , and  $|c| \leq \frac{1}{6}$ . Plugging in this decomposition of  $x$  into the densities of  $G(1/10, u, t)$  and  $G(\sigma, u, t)$ , and simplifying yields the above.
To conclude, for  $k > 10$ , we have