SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \epsilon_ {\mathrm {K L}} = \mathbb {E} _ {x \sim \mathcal {D} _ {p r e}} [ \mathrm {K L} (p ^ {*} (x _ {i} | x _ {- i}) | | p (x _ {i} | x _ {- i})) ] (25) \\ \geq \Pr_ {x _ {- i} \sim \mathcal {D} _ {p r e}} [ f (x, i) \cdot f ^ {*} (x, i) <   0 ] \cdot \min  _ {x _ {- i}: f (x, i) \cdot f ^ {*} (x, i) <   0} \mathrm {K L} \left(p ^ {*} \left(x _ {i} \mid x _ {- i}\right) \| p \left(x _ {i} \mid x _ {- i}\right)\right) (26) \\ \geq \mu \cdot \Pr_ {x _ {- i} \sim \mathcal {D} _ {D S}} [ f (x, i) \cdot f ^ {*} (x, i) <   0 ] \cdot \min  _ {x _ {- i}: f (x, i) \cdot f ^ {*} (x, i) <   0} \mathrm {K L} \left(p ^ {*} \left(x _ {i} \mid x _ {- i}\right) | | p \left(x _ {i} \mid x _ {- i}\right)\right) (27) \\ \geq \epsilon_ {C L S} \cdot 2 \mu \cdot p _ {b} ^ {2} \min  _ {\varepsilon_ {p} \geq 0} \left\{(1 - e ^ {- \varepsilon_ {p}}) + k \cdot \min  _ {j \in A (i)} e ^ {b _ {j} ^ {*} - \varepsilon_ {b} - \varepsilon_ {p}} \cdot \left(e ^ {\sigma \left(\frac {\Gamma}{2} - 2 \varepsilon_ {b} - \varepsilon_ {p}\right)} - 1\right) \right\} ^ {2}. (28) \\ \end{array}
$$
Here  $\mathcal{D}_{pre}$  and  $\mathcal{D}_{DS}$  are the data distributions of pre-training and downstream task, and the second inequality comes from Assumption 2.
Now we have established the relationship between  $\epsilon_{\mathrm{KL}}$  and  $\epsilon_{CLS}$ , and the only step left is to bound the minimum over  $\varepsilon_p$ . For notation simplicity, we define  $\Lambda \coloneqq \frac{\Gamma}{2} - 2\epsilon_b$ .
If  $\varepsilon_{p} \leq \frac{\Lambda}{3}$ , we have  $(1 - e^{-\varepsilon_{p}}) \geq (1 - e^{-\frac{\Lambda}{3}})$ .
If  $\varepsilon_{p} > \frac{\Lambda}{3}$ , we know that
$$
e ^ {b _ {j} ^ {*} - \varepsilon_ {b} - \varepsilon_ {p}} \cdot \left(e ^ {\frac {\gamma}{2 k \max  _ {j \in A (i)} | a _ {j} ^ {*} |} - 2 \varepsilon_ {b} - \varepsilon_ {p}} - 1\right) > e ^ {b _ {j} ^ {*} - \varepsilon_ {b} - \frac {\Lambda}{3}} \cdot \left(e ^ {\frac {2 \Lambda}{3}} - 1\right) = e ^ {b _ {j} ^ {*} - \varepsilon_ {b}} \cdot \sinh \left(\frac {\Lambda}{3}\right).
$$
$$
\begin{array}{l} \left. \min  _ {\varepsilon_ {p} \geq 0} \left\{\left(1 - e ^ {- \varepsilon_ {p}}\right) + k \cdot \min  _ {j \in A (i)} e ^ {b _ {j} ^ {*} - \varepsilon_ {b} - \varepsilon_ {p}} \cdot \left(e ^ {\sigma \left(\frac {\Gamma}{2} - 2 \varepsilon_ {b} - \varepsilon_ {p}\right)} - 1\right) \right\} \right. (29) \\ \geq \min  \left\{1 - e ^ {- \frac {\Lambda}{3}}, 2 k \cdot \min  _ {j \in A (i)} e ^ {b _ {j} ^ {*} - \varepsilon_ {b}} \cdot \sinh \left(\frac {\Lambda}{3}\right) \right\} (30) \\ \end{array}
$$
Plugging this into (28) gives us
$$
\epsilon_ {\mathrm {K L}} \geq \epsilon_ {C L S} \cdot 2 \mu \cdot p _ {b} ^ {2} \min  \left\{\left(1 - e ^ {- \frac {\Lambda}{3}}\right) ^ {2}, 4 k ^ {2} \cdot \min  _ {j \in [ k ]} e ^ {2 b _ {j} ^ {*} - 2 \varepsilon_ {b}} \cdot \sinh^ {2} \left(\frac {\Lambda}{3}\right) \right\}. \tag {31}
$$
Since  $\Gamma > 8\epsilon_{b}$ , we know that  $\frac{\Gamma}{4} < \Lambda < 3$ , so  $(1 - e^{-\frac{\Lambda}{3}})^{2} > \left(\frac{\Lambda}{6}\right)^{2} > \frac{\Gamma^{2}}{576}$ .
Besides,  $\min_{j\in [k]}b_j^*\geq \epsilon_b - \log (4k)$  implies  $4k^{2}\cdot \min_{j\in [k]}e^{2b_{j}^{*} - 2\varepsilon_{b}}\geq \frac{1}{4}$ . We also have  $\sinh^2 (\frac{\Lambda}{3}) > \left(\frac{\Lambda}{3}\right)^2 >\frac{\Gamma^2}{144}$ .
Thus,  $\min \left\{(1 - e^{-\frac{\Lambda}{3}})^2, 4k^2 \cdot \min_{j \in [k]} e^{2b_j^* - 2\varepsilon_b} \cdot \sinh^2\left(\frac{\Lambda}{3}\right)\right\} > \frac{\Gamma^2}{576}$ . Plugging this into (31) finishes the proof of Theorem 3.
B.1 Further discussions of Theorem 3
Potentially improving the upper bound by better thresholding. We are taking the minimum over two terms in the proof of Theorem 3. These two terms come from two parts of the difference between the ground-truth probability and our learned probability: the bulk part and the top part. Both parts are functions of  $\epsilon_{p}$ , which measures the approximation error for the bulk probability. A large  $\epsilon_{p}$  will incur a large error in the bulk part while a smaller  $\epsilon_{p}$  will make the top part larger. We are setting  $\frac{\Lambda}{3}$  as the threshold for the current bound in Theorem 3. The bound could be improved by choosing other thresholds if we know more about the values of  $\{b_j^*\}_{j=1}^k$  and the normalized margin  $\Gamma$ .
More general cases for thresholds  $\{b_j^*\}_{j=1}^k$  and normalized margin  $\Gamma$ . The thresholds  $\{b_j^*\}_{j=1}^k$  can be considered as the model's sensitivity to the logits. A smaller threshold indicates a higher sensitivity. In the proof of Theorem 3, we are assuming that the thresholds are not very small. In general cases where the thresholds can be small, an important observation is that: For the same set of samples, decreasing the thresholds by some number  $s$  will increase the normalized margin by  $s$  as well. Since both the thresholds and the normalized margin are in the exponent in our bound (sinh function is close to exponential when  $\Gamma$  is large), these two effects can cancel out to a large degree and don't influence the bound by much.
Prompt engineering can help with downstream performance. Prompt engineering can help with the downstream task performance in multiple ways. One direct way is to make the data distribution closer to that of the pre-training stage to increase  $\mu$ . Furthermore, it can also indirectly help improve the bound by decreasing the number of activated neurons, increasing the margin, etc.
C Proof of Lemma 1
Here we provide a proof of Lemma 1, with the proofs of the necessary technical lemmas postponed to Appendix D.
Proof. For all  $j \in [n]$ , we set  $a_{j} = a_{j}^{*}$  and  $b_{j} = b_{j}^{*}$ . Since  $i$  is fixed in the proof, we will omit  $i$  when the selection of  $i$  is clear from context.
WLOG, assume  $x \in \mathrm{POS}$  because otherwise we can flip the sign of all  $a_j^*$ 's. In this case, from Assumption 3 we know that  $f^*(x, i) \geq \gamma$ . Therefore,  $f(x, i) \cdot f^*(x, i) < 0$  implies  $f(x, i) < 0$ , so  $f^*(x, i) - f(x, i) > \gamma$ .
There are two possible ways to make  $f(x, i)$  smaller than  $f^{*}(x, i)$ : Making the neurons with positive coefficients smaller or making those with negative coefficients larger. The total difference must be at least  $\gamma$ , so there must exist one sign whose change in value is at least  $\frac{\gamma}{2}$ . Formally, we decompose the
difference  $f^{*}(x,i) - f(x,i)$  into two terms:
$$
\begin{array}{l} f ^ {*} (x, i) - f (x, i) = \sum_ {j: a _ {j} ^ {*} > 0} a _ {j} ^ {*} \left(\sigma \left(\left\langle v _ {- i} ^ {*} (x), v _ {j} ^ {*} - v _ {0} ^ {*} \right\rangle - b _ {j} ^ {*}\right) - \sigma \left(\left\langle v _ {- i} (x), v _ {j} - v _ {0} \right\rangle - b _ {j} ^ {*}\right)\right) (32) \\ + \sum_ {j: a _ {j} ^ {*} <   0} \left| a _ {j} ^ {*} \right| \left(\sigma \left(\left\langle v _ {- i} (x), v _ {j} - v _ {0} \right\rangle - b _ {j} ^ {*}\right) - \sigma \left(\left\langle v _ {- i} ^ {*} (x), v _ {j} ^ {*} - v _ {0} ^ {*} \right\rangle - b _ {j} ^ {*}\right)\right). (33) \\ \end{array}
$$
Note that at least one of the two terms on the right-hand side must be at least  $\frac{\gamma}{2}$  because their sum is at least  $\gamma$ . We first consider the first case, i.e.,  $\sum_{j:a_j^* >0}a_j^*\left(\sigma (\langle v_{-i}^* (x),v_j^* -v_0^*\rangle -b_j^*) - \sigma (\langle v_{-i}(x),v_j - v_0\rangle -b_j^*)\right)\geq \frac{\gamma}{2}$ . The analysis of the second term is almost the same as the first one.
$$
\Delta_ {j} := \left(\left\langle v _ {- i} ^ {*} (x), v _ {j} ^ {*} - v _ {0} ^ {*} \right\rangle - b _ {j} ^ {*}\right) - \sigma \left(\left\langle v _ {- i} (x), v _ {j} - v _ {0} \right\rangle - b _ {j} ^ {*}\right), \tag {34}
$$
(n.b.  $\sigma$  only appears in the second term) which implicitly depends on  $x$ , and define the set
$$
S (x) := \{j: a _ {j} ^ {*} > 0, \Delta_ {j} > 0 \}. \tag {35}
$$
If  $\Delta_j > 0$ , we must have  $\langle v_{-i}^*(x), v_j^* - v_0^*\rangle - b_j^* > 0$  because  $\mathrm{ReLU}$  is non-negative. This means that the neurons corresponding to the words in  $S(x)$  must be activated.
Intuitively,  $S(x)$  is the set of words whose corresponding neuron in our student model has a smaller value than the ground-truth model, and  $\Delta_j$  for a word  $j \in S(x)$  is the difference between these two neurons. In other words, if we want to make a mistake in classifying a sample  $x$ ,  $\Delta_j$  is the obstacle for the neuron to overcome, and the sum of all these obstacles must be at least the margin  $\gamma$ . Formally, we have the following lemma for  $S(x)$  and lower bound for  $\Delta_j$ :
Lemma 2.  $S(x)\neq \emptyset$
Lemma 3.  $\sum_{j\in S(x)}\Delta_j\geq \frac{\gamma}{2\max_{j\in S(x)}a_j^*}.$
Considering the TV distance, we get
$$
\begin{array}{l} \sum_ {j = 1} ^ {n} \left| p ^ {*} \left(x _ {i} = j \mid x _ {- i}\right) - p \left(x _ {i} = j \mid x _ {- i}\right) \right| (36) \\ \geq \sum_ {j \in B (x, i)} | p ^ {*} (x _ {i} = j | x _ {- i}) - p (x _ {i} = j | x _ {- i}) | + \sum_ {j \in S (x)} | p ^ {*} (x _ {i} = j | x _ {- i}) - p (x _ {i} = j | x _ {- i}) | (37) \\ \geq \left| \sum_ {j \in B (x, i)} p ^ {*} \left(x _ {i} = j \mid x _ {- i}\right) - \sum_ {j \in B (x, i)} p \left(x _ {i} = j \mid x _ {- i}\right) \right| + \sum_ {j \in S (x)} \left| p ^ {*} \left(x _ {i} = j \mid x _ {- i}\right) - p \left(x _ {i} = j \mid x _ {- i}\right) \right| (38) \\ = \left| \frac {Z _ {\text {b u l k}} ^ {*} (x , i)}{Z ^ {*} (x , i)} - \frac {Z _ {\text {b u l k}} (x , i)}{Z (x , i)} \right| + \sum_ {j \in S (x)} | p ^ {*} (x _ {i} = j | x _ {- i}) - p (x _ {i} = j | x _ {- i}) |, (39) \\ \end{array}
$$
where the first inequality comes from Assumption 5, i.e.,  $S(x) \subseteq [k] \subseteq [n] \setminus B(x,i)$ .
We define  $\varepsilon_{p} := \left|\log \frac{Z_{\mathrm{bulk}}^{*}(x,i)}{Z^{*}(x,i)} - \log \frac{Z_{\mathrm{bulk}}(x,i)}{Z(x,i)}\right|$ , which is the approximation error of the log bulk probability and an important quantity to trade off the two terms in the TV distance. There are two terms in (39). The first term corresponds to the bulk probability and the second term corresponds to the activated neurons. When  $\varepsilon_{p}$  is large, the bulk probability has a large approximation error, resulting in a large TV distance. We lower bound this term in the following lemma:
$$
\left| \frac {Z _ {\operatorname {b u l k}} ^ {*} (x , i)}{Z ^ {*} (x , i)} - \frac {Z _ {\operatorname {b u l k}} (x , i)}{Z (x , i)} \right| \geq p _ {b} \left(1 - e ^ {- \varepsilon_ {p}}\right). \tag {40}
$$
The right-hand side of the above lemma is an increasing function of  $\varepsilon_{p}$ , and this lower bound becomes large when the approximation error of the bulk probability is large.
In the other regime when  $\varepsilon_{p}$  is small, the bulk partition function must be approximated accurately, i.e.,  $\frac{Z_{\mathrm{bulk}}^{*}(x,i)}{Z^{*}(x,i)}\approx \frac{Z_{\mathrm{bulk}}(x,i)}{Z(x,i)}$  . Thus,
$$
\begin{array}{l} \left| p ^ {*} \left(x _ {i} = j \mid x _ {- i}\right) - p \left(x _ {i} = j \mid x _ {- i}\right) \right| = \left| \frac {\exp \left(\left\langle v _ {- i} ^ {*} (x) , v _ {j} ^ {*} \right\rangle\right)}{Z ^ {*} (x , i)} - \frac {\exp \left(\left\langle v _ {- i} (x) , v _ {j} \right\rangle\right)}{Z (x , i)} \right| (41) \\ \approx \frac {Z _ {\mathrm {b u l k}} ^ {*} (x , i)}{Z ^ {*} (x , i)} \left| \frac {\exp \left(\left\langle v _ {- i} ^ {*} (x) , v _ {j} ^ {*} \right\rangle\right)}{Z _ {\mathrm {b u l k}} ^ {*} (x , i)} - \frac {\exp \left(\left\langle v _ {- i} (x) , v _ {j} \right\rangle\right)}{Z _ {\mathrm {b u l k}} (x , i)} \right|. (42) \\ \end{array}
$$
From Assumption 7 we know that the bulk partition functions can be accurately approximated using the vectors  $v_{0}^{*}$  and  $v_{0}$ . Thus,  $\frac{\exp(\langle v_{-i}^{*}(x), v_{j}^{*}\rangle)}{Z_{\mathrm{bulk}}^{*}(x,i)} \approx \exp (\langle v_{-i}^{*}(x), v_{j}^{*} - v_{0}^{*}\rangle)$ . Therefore, this difference is approximately  $|\exp (\langle v_{-i}^{*}(x), v_{j}^{*} - v_{0}^{*}\rangle) - \exp (\langle v_{-i}(x), v_{j} - v_{0}\rangle)| \approx \exp (\langle v_{-i}(x), v_{j} - v_{0}\rangle)(e^{\Delta_{j}} - 1)$ . As  $\varepsilon_{p}$  becomes smaller, the aforementioned approximations become more accurate, making the TV distance suffer from a term that is roughly proportional to  $(e^{\Delta_{j}} - 1)$ . This is formalized in the following lemma:
Lemma 5. For all  $j\in S(x)$
$$
\left| p ^ {*} \left(x _ {i} = j \mid x _ {- i}\right) - p \left(x _ {i} = j \mid x _ {- i}\right) \right| \geq e ^ {b _ {j} ^ {*} - \varepsilon_ {b} - \varepsilon_ {p}} \cdot p _ {b} \cdot \left(e ^ {\sigma \left(\Delta_ {j} - 2 \varepsilon_ {b} - \varepsilon_ {p}\right)} - 1\right). \tag {43}
$$
Plugging Lemma 4 and 5 into (39) gives us
$$
\begin{array}{l} \operatorname {T V} \left(p ^ {*} \left(x _ {i} \mid x _ {- i}\right), p \left(x _ {i} \mid x _ {- i}\right)\right) (44) \\ \geq \left| \frac {Z _ {\operatorname {b u l k}} ^ {*} (x , i)}{Z ^ {*} (x , i)} - \frac {Z _ {\operatorname {b u l k}} (x , i)}{Z (x , i)} \right| + \sum_ {j \in S (x)} \left| \frac {\exp \left(\langle v _ {- i} ^ {*} (x) , v _ {j} ^ {*} \rangle\right)}{Z ^ {*} (x , i)} - \frac {\exp \left(\langle v _ {- i} (x) , v _ {j} \rangle\right)}{Z (x , i)} \right| (45) \\ \geq p _ {b} \left(1 - e ^ {- \varepsilon_ {p}}\right) + \sum_ {j \in S (x)} e ^ {b _ {j} ^ {*} - \varepsilon_ {b} - \varepsilon_ {p}} \cdot p _ {b} \cdot \left(e ^ {\sigma \left(\Delta_ {j} - 2 \varepsilon_ {b} - \varepsilon_ {p}\right)} - 1\right) (46) \\ \geq p _ {b} \left(1 - e ^ {- \varepsilon_ {p}}\right) + p _ {b} \min  _ {j \in S (x)} e ^ {b _ {j} ^ {*} - \varepsilon_ {b} - \varepsilon_ {p}} \cdot \sum_ {j \in S (x)} \left(e ^ {\sigma \left(\Delta_ {j} - 2 \varepsilon_ {b} - \varepsilon_ {p}\right)} - 1\right) (47) \\ \geq p _ {b} \left(1 - e ^ {- \varepsilon_ {p}}\right) + p _ {b} \min  _ {j \in S (x)} e ^ {b _ {j} ^ {*} - \varepsilon_ {b} - \varepsilon_ {p}} \cdot | S (x) | \cdot \left(e ^ {\sum_ {j \in S (x)} \sigma \left(\Delta_ {j} - 2 \varepsilon_ {b} - \varepsilon_ {p}\right) / | S (x) |} - 1\right), (48) \\ \end{array}
$$
where the last inequality comes from Jensen's inequality.
Let  $g(u,v) \coloneqq u \cdot \left( e^{\frac{v}{u}} - 1 \right)$  for  $u, v > 0$ , then  $\frac{\partial g}{\partial u} = e^{\frac{v}{u}} \left( 1 - \frac{v}{u} - e^{-\frac{v}{u}} \right) \leq 0$ , so from  $|S(x)| \leq k$  we know that
$$
| S (x) | \cdot \left(e ^ {\sum_ {j \in S (x)} \sigma \left(\Delta_ {j} - 2 \varepsilon_ {b} - \varepsilon_ {p}\right) / | S (x) |} - 1\right) \geq k \cdot \left(e ^ {\sum_ {j \in S (x)} \sigma \left(\Delta_ {j} - 2 \varepsilon_ {b} - \varepsilon_ {p}\right) / k} - 1\right).
$$
Using the fact that  $\sum_{i}\sigma (y_{i}) = \sum_{i}\max \{y_{i},0\} \geq \max \{\sum_{i}y_{i},0\} = \sigma (\sum_{i}y_{i}),$
$$
\sum_ {j \in S (x)} \sigma \left(\Delta_ {j} - 2 \varepsilon_ {b} - \varepsilon_ {p}\right) \geq \sigma \left(\sum_ {j \in S (x)} \left(\Delta_ {j} - 2 \varepsilon_ {b} - \varepsilon_ {p}\right)\right) \geq \sigma \left(\frac {\gamma}{2 \max  _ {t \in S (x)} a _ {t} ^ {*}} - 2 k \cdot \varepsilon_ {b} - k \varepsilon_ {p}\right), \tag {49}
$$
where the last inequality follows from Lemma 3. Therefore,
$$
\operatorname {T V} \left(p ^ {*} \left(x _ {i} \mid x _ {- i}\right), p \left(x _ {i} \mid x _ {- i}\right)\right) \tag {50}
$$
$$
\geq p _ {b} \left(1 - e ^ {- \varepsilon_ {p}}\right) + p _ {b} \cdot k \min  _ {j \in S (x)} e ^ {b _ {j} ^ {*} - \varepsilon_ {b} - \varepsilon_ {p}} \cdot \left(e ^ {\sigma \left(\frac {\gamma}{2 k \max  _ {t \in S (x)} a _ {t} ^ {*}} - 2 \cdot \varepsilon_ {b} - \varepsilon_ {p}\right)} - 1\right). \tag {51}
$$
Since we do not have any assumption for  $\varepsilon_{p}$ , we take the minimum over all possible values of  $\varepsilon_{p}$  and get the following bound:
$$
\begin{array}{l} \operatorname {T V} \left(p ^ {*} \left(x _ {i} \mid x _ {- i}\right), p \left(x _ {i} \mid x _ {- i}\right)\right) (52) \\ \geq p _ {b} \min  _ {\varepsilon_ {p} \geq 0} \left\{\left(1 - e ^ {- \varepsilon_ {p}}\right) + k \cdot \min  _ {j \in S (x)} e ^ {b _ {j} ^ {*} - \varepsilon_ {b} - \varepsilon_ {p}} \cdot \left(e ^ {\sigma \left(\frac {\gamma}{2 k \max  _ {t \in S (x)} a _ {t} ^ {*}} - 2 \cdot \varepsilon_ {b} - \varepsilon_ {p}\right)} - 1\right) \right\}. (53) \\ \end{array}
$$
Above we have derived a lower bound for the TV distance between  $p^*$  and  $p$  when the first term of (33) is at least  $\frac{\gamma}{2}$ . When the second term of (33) is at least  $\frac{\gamma}{2}$ , the proof is symmetric to that of the first term and we only need to exchange the role of parameters from the teacher model and student model and the related definitions. Therefore, with the second term being at least  $\frac{\gamma}{2}$ , we have
$$
\operatorname {T V} \left(p ^ {*} \left(x _ {i} \mid x _ {- i}\right), p \left(x _ {i} \mid x _ {- i}\right)\right) \tag {54}
$$
$$
\geq p _ {b} \min  _ {\varepsilon_ {p} \geq 0} \left\{\left(1 - e ^ {- \varepsilon_ {p}}\right) + k \cdot \min  _ {j \in S ^ {\prime} (x)} e ^ {b _ {j} ^ {*} - \varepsilon_ {b} - \varepsilon_ {p}} \cdot \left(e ^ {\sigma \left(\frac {\gamma}{2 k \max  _ {t \in S ^ {\prime} (x)} | a _ {t} ^ {*} |} - 2 \cdot \varepsilon_ {b} - \varepsilon_ {p}\right)} - 1\right) \right\}, \tag {55}
$$
where  $S'(x) \coloneqq \{j : a_j^* < 0, \Delta_j' > 0\}$  and  $\Delta_j' \coloneqq (\langle v_{-i}(x), v_j - v_0 \rangle - b_j^*) - \sigma(\langle v_{-i}^*(x), v_j^* - v_0^* \rangle - b_j^*)$ . Since  $S(x) \cup S'(x) \subseteq [k]$ , merging (53) and (55) finishes the proof of Lemma 1.
D Detailed proofs of auxiliary lemmas
Proof. Assume by way of contradiction that  $S(x) = \emptyset$ . Then by the definition of  $S(x)$ , we know that for all  $j$  such that  $a_j^* > 0$ ,
$$
\begin{array}{l} \sigma \left(\left\langle v _ {- i} ^ {*} (x), v _ {j} ^ {*} - v _ {0} ^ {*} \right\rangle - b _ {j} ^ {*}\right) - \sigma \left(\left\langle v _ {- i} (x), v _ {j} - v _ {0} \right\rangle - b _ {j}\right) (56) \\ \leq \left(\left\langle v _ {- i} ^ {*} (x), v _ {j} ^ {*} - v _ {0} ^ {*} \right\rangle - b _ {j} ^ {*}\right) - \sigma \left(\left\langle v _ {- i} (x), v _ {j} - v _ {0} \right\rangle - b _ {j}\right) = \Delta_ {j} \leq 0. (57) \\ \end{array}
$$
$$
\sum_ {j: a _ {j} ^ {*} > 0} a _ {j} ^ {*} \left(\sigma (\langle v _ {- i} ^ {*} (x), v _ {j} ^ {*} - v _ {0} ^ {*} \rangle - b _ {j} ^ {*}) - \sigma (\langle v _ {- i} (x), v _ {j} - v _ {0} \rangle - b _ {j})\right) \leq 0 <   \frac {\gamma}{2},
$$
which is a contradiction. Thus,  $S(x) \neq \emptyset$ .
Proof. Note that for all  $j \in S(x)$ ,  $\langle v_{-i}^{*}(x), v_{j}^{*} - v_{0}^{*} \rangle - b_{j}^{*} > \sigma(\langle v_{-i}(x), v_{j} - v_{0} \rangle - b_{j}^{*}) \geq 0$ , so  $\sigma(\langle v_{-i}^{*}(x), v_{j}^{*} - v_{0}^{*} \rangle - b_{j}^{*}) = \langle v_{-i}^{*}(x), v_{j}^{*} - v_{0}^{*} \rangle - b_{j}^{*}$ . Therefore,
$$
\begin{array}{l} \sum_ {j \in S (x)} a _ {j} ^ {*} \Delta_ {j} = \sum_ {j \in S (x)} a _ {j} ^ {*} \left(\left(\left\langle v _ {- i} ^ {*} (x), v _ {j} ^ {*} - v _ {0} ^ {*} \right\rangle - b _ {j} ^ {*}\right) - \sigma \left(\left(\left\langle v _ {- i} (x), v _ {j} - v _ {0} \right\rangle - b _ {j} ^ {*}\right)\right)\right) (58) \\ = \sum_ {j \in S (x)} a _ {j} ^ {*} \left(\sigma \left(\langle v _ {- i} ^ {*} (x), v _ {j} ^ {*} - v _ {0} ^ {*} \rangle - b _ {j} ^ {*}\right) - \sigma \left(\langle v _ {- i} (x), v _ {j} - v _ {0} \rangle - b _ {j} ^ {*}\right)\right) (59) \\ \geq \frac {\gamma}{2}, (60) \\ \end{array}
$$
$$
\sum_ {j \in S (x)} \Delta_ {j} = \frac {\sum_ {j \in S (x)} \max  _ {t \in S (x)} a _ {t} ^ {*} \Delta_ {j}}{\max  _ {t \in S (x)} a _ {t} ^ {*}} \geq \frac {\sum_ {j \in S (x)} a _ {j} ^ {*} \Delta_ {j}}{\max  _ {t \in S (x)} a _ {t} ^ {*}} \geq \frac {\gamma}{2 \max  _ {t \in S (x)} a _ {t} ^ {*}}. \tag {61}
$$
Proof. If  $\varepsilon_{p} = 0$ , both sides of the inequality equal 0, and the inequality holds. When  $\varepsilon_{p} > 0$
$$
\begin{array}{l} \left| \frac {Z _ {\text {b u l k}} ^ {*} (x , i)}{Z ^ {*} (x , i)} - \frac {Z _ {\text {b u l k}} (x , i)}{Z (x , i)} \right| (62) \\ = \frac {Z _ {\text {b u l k}} ^ {*} (x , i)}{Z ^ {*} (x , i)} \cdot \left| 1 - \frac {Z _ {\text {b u l k}} (x , i)}{Z (x , i)} \cdot \frac {Z ^ {*} (x , i)}{Z _ {\text {b u l k}} ^ {*} (x , i)} \right| (63) \\ = \frac {Z _ {\text {b u l k}} ^ {*} (x , i)}{Z ^ {*} (x , i)} \cdot \left| 1 - \exp \left(\log \frac {Z _ {\text {b u l k}} (x , i)}{Z (x , i)} - \log \frac {Z _ {\text {b u l k}} ^ {*} (x , i)}{Z ^ {*} (x , i)}\right) \right| (64) \\ \left. \geq \frac {Z _ {\text {b u l k}} ^ {*} (x , i)}{Z ^ {*} (x , i)} \cdot \left| 1 - \exp \left(- \left| \log \frac {Z _ {\text {b u l k}} (x , i)}{Z (x , i)} - \log \frac {Z _ {\text {b u l k}} ^ {*} (x , i)}{Z ^ {*} (x , i)} \right|\right) \right| \right. (65) \\ \geq p _ {b} \left(1 - e ^ {- \varepsilon_ {p}}\right). (66) \\ \end{array}
$$
Proof. We know that the word probabilities come from a log-linear model, so
$$
| p ^ {*} (x _ {i} = j | x _ {- i}) - p (x _ {i} = j | x _ {- i}) | = \left| \frac {\exp (\langle v _ {- i} ^ {*} (x) , v _ {j} ^ {*} \rangle)}{Z ^ {*} (x , i)} - \frac {\exp (\langle v _ {- i} (x) , v _ {j} \rangle)}{Z (x , i)} \right|.
$$
If the probability  $p(x_{i} = j|x_{-i})$  is reasonably large, i.e., its corresponding neuron is activated, then we can use that with  $\Delta_j$  to bound this difference. In other words, when  $\langle v_{-i}(x),v_j - v_0\rangle \geq b_j^*$ , we know that
$$
\begin{array}{l} \left(\left\langle v _ {- i} ^ {*} (x), v _ {j} ^ {*} \right\rangle - \log Z ^ {*}\right) - \left(\left\langle v _ {- i} (x), v _ {j} \right\rangle - \log Z\right) (67) \\ = \left(\left\langle v _ {- i} ^ {*} (x), v _ {j} ^ {*} \right\rangle - \log Z _ {\mathrm {b u l k}} ^ {*}\right) - \left(\left\langle v _ {- i} (x), v _ {j} \right\rangle - \log Z _ {\mathrm {b u l k}}\right) + \left(\log \frac {Z _ {\mathrm {b u l k}} ^ {*}}{Z ^ {*}} - \log \frac {Z _ {\mathrm {b u l k}}}{Z}\right) (68) \\ \geq \left(\left\langle v _ {- i} ^ {*} (x), v _ {j} ^ {*} \right\rangle - \log Z _ {\mathrm {b u l k}} ^ {*}\right) - \left(\left\langle v _ {- i} (x), v _ {j} \right\rangle - \log Z _ {\mathrm {b u l k}}\right) - \varepsilon_ {p} (69) \\ = \left\langle v _ {- i} ^ {*} (x), v _ {j} ^ {*} - v _ {0} ^ {*} \right\rangle - \left\langle v _ {- i} (x), v _ {j} - v _ {0} \right\rangle + \left(\left\langle v _ {- i} ^ {*} (x), v _ {0} ^ {*} \right\rangle - \log Z _ {\mathrm {b u l k}} ^ {*}\right) (70) \\ - \left(\left\langle v _ {- i} (x), v _ {0} \right\rangle - \log Z _ {\mathrm {b u l k}}\right) - \varepsilon_ {p} (71) \\ \geq \Delta_ {j} - 2 \varepsilon_ {b} - \varepsilon_ {p}. (72) \\ \end{array}
$$
Note that the last inequality (72) comes from Assumption 7 and the following property of  $\Delta_j$ :
$$
\begin{array}{l} \langle v _ {- i} ^ {*} (x), v _ {j} ^ {*} - v _ {0} ^ {*} \rangle - \langle v _ {- i} (x), v _ {j} - v _ {0} \rangle = \langle v _ {- i} ^ {*} (x), v _ {j} ^ {*} - v _ {0} ^ {*} \rangle - b _ {j} ^ {*} - \left(\langle v _ {- i} (x), v _ {j} - v _ {0} \rangle - b _ {j} ^ {*}\right) (73) \\ \geq \left\langle v _ {- i} ^ {*} (x), v _ {j} ^ {*} - v _ {0} ^ {*} \right\rangle - b _ {j} ^ {*} - \sigma \left(\left\langle v _ {- i} (x), v _ {j} - v _ {0} \right\rangle - b _ {j} ^ {*}\right) (74) \\ = \Delta_ {j}. (75) \\ \end{array}
$$
Then we bound the difference in probabilities: for all  $j \in S(x)$ ,