SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\left(\alpha_ {t, \ell} - \epsilon\right) x _ {\ell} A x _ {\ell} ^ {\top} \leq a _ {t} W _ {Q} W _ {K} ^ {\top} x _ {\ell} ^ {\top} \leq \left(\alpha_ {t, \ell} + \epsilon\right) x _ {\ell} A x _ {\ell} ^ {\top}
$$
Now we proceed to the main body of the proof. Assume that  $\| x_{\ell}\|_{2} = 1$  for all  $\ell$ . Using Lemma (B.1), if  $a_{t}x_{t + 1}^{\top}\geq (1 - \delta)\| a_{t}\|_{2}$ , then we have
$$
\left| \left\| a _ {t} \right\| _ {2} ^ {- 1} a _ {t} W _ {Q} W _ {K} ^ {\top} x _ {\ell} ^ {\top} - x _ {t + 1} W _ {Q} W _ {K} ^ {\top} x _ {\ell} ^ {\top} \right| \leq \sqrt {2 \delta} \left\| W _ {Q} W _ {K} ^ {\top} x _ {\ell} ^ {\top} \right\| _ {2}
$$
Recall that  $\lambda_{Q},\lambda_{K}$  are the maximum singular values of  $W_{Q}$  and  $W_{K}$ , respectively. Then it holds that  $\left\| W_{Q}W_{K}^{\top}x_{\ell}^{\top}\right\|_{2}\leq \lambda_{Q}\lambda_{K}\left\| x_{\ell}\right\|_{2}$ . Using  $\| x_{\ell}\|_{2} = 1$ , we have
$$
\left| \left\| a _ {t} \right\| _ {2} ^ {- 1} a _ {t} W _ {Q} W _ {K} ^ {\top} x _ {\ell} ^ {\top} - x _ {t + 1} W _ {Q} W _ {K} ^ {\top} x _ {\ell} ^ {\top} \right| \leq \sqrt {2 \delta} \lambda_ {Q} \lambda_ {K}
$$
$$
\begin{array}{l} \left\| a _ {t} \right\| _ {2} = \left\| \left(\sum_ {j = 1} ^ {t - 1} \alpha_ {t, j} x _ {j}\right) W _ {V} W _ {O} \right\| \\ \leq \lambda_ {O} \lambda_ {V} \left\| \sum_ {j = 1} ^ {t - 1} \alpha_ {t, j} x _ {j} \right\| _ {2} \\ \leq \lambda_ {O} \lambda_ {V} \sum_ {j = 1} ^ {t - 1} \alpha_ {t, j} \| x _ {j} \| _ {2} \\ = \lambda_ {O} \lambda_ {V} \\ \end{array}
$$
Then since  $\delta \leq \left(\frac{c\epsilon}{\lambda_Q\lambda_K\lambda_V\lambda_O}\right)^2$  we have
$$
\left| \left\| a _ {t} \right\| _ {2} ^ {- 1} a _ {t} W _ {Q} W _ {K} ^ {\top} x _ {\ell} ^ {\top} - x _ {t + 1} W _ {Q} W _ {K} ^ {\top} x _ {\ell} ^ {\top} \right| \leq \frac {2 c \epsilon}{\lambda_ {V} \lambda_ {O}} \leq \frac {2 c \epsilon}{\| a _ {t} \| _ {2}}
$$
Since by Lemma (B.2), we have
$$
\left| a _ {t} W _ {Q} W _ {K} ^ {\top} x _ {\ell} ^ {\top} - \alpha_ {t, \ell} x _ {\ell} A x _ {\ell} ^ {\top} \right| \leq \epsilon x _ {\ell} ^ {\top} a x _ {\ell}
$$
It must hold that
$$
\left| x _ {t + 1} W _ {Q} W _ {K} ^ {\top} x _ {\ell} ^ {\top} - \left\| a _ {t + 1} \right\| _ {2} ^ {- 1} \alpha_ {t, \ell} x _ {\ell} A x _ {\ell} ^ {\top} \right| \leq \frac {\epsilon}{\left\| a _ {t} \right\| _ {2}} x _ {\ell} ^ {\top} a x _ {\ell} + \frac {2 c \epsilon}{\left\| a _ {t} \right\| _ {2}}
$$
Since  $x_{\ell}^{\top}ax_{\ell}\geq c$ , it holds that
$$
\frac {2 c \epsilon}{\| a _ {t} \| _ {2}} \leq \frac {2 \epsilon}{\| a _ {t} \| _ {2}} x _ {\ell} ^ {\top} a x _ {\ell}
$$
$$
\left| x _ {t + 1} W _ {Q} W _ {K} ^ {\top} x _ {\ell} ^ {\top} - \left\| a _ {t} \right\| _ {2} ^ {- 1} \alpha_ {t, \ell} x _ {\ell} A x _ {\ell} ^ {\top} \right| \leq \frac {3 \epsilon}{\left\| a _ {t} \right\| _ {2}} x _ {\ell} ^ {\top} a x _ {\ell}
$$
$$
\frac {x _ {\ell} A x _ {\ell} ^ {\top}}{\| a _ {t} \| _ {2}} \left(\alpha_ {t, \ell} - 3 \epsilon\right) \leq x _ {t + 1} W _ {Q} W _ {K} ^ {\top} x _ {\ell} ^ {\top} \leq \frac {x _ {\ell} A x _ {\ell} ^ {\top}}{\| a _ {t} \| _ {2}} \left(\alpha_ {t, \ell} + 3 \epsilon\right)
$$
Let  $\{\tilde{x}_t\}_{t=0}^T$  denote the tokens generated by the transformer with budget KV cache as in Algorithm 2 with  $m = 1$ :
$$
\tilde {x} _ {t + 1} = \mathcal {F} (\tilde {a} _ {t}), \text {w h e r e} \tilde {a} _ {t} = \operatorname {s o f t m a x} \left(^ {1} / _ {t} \cdot \tilde {x} _ {t} W _ {Q} \tilde {\mathcal {K}} _ {t} ^ {\top}\right) \tilde {\mathcal {V}} _ {t} ^ {\top} W _ {O}
$$
For reference purposes, let  $\{x_{t}\}_{t = 0}^{T}$  denote the tokens generated by a vanilla transformer defined in (5). We re-state Theorem 4.1 below, which bounds the difference  $\| x_{t} - \tilde{x}_{t}\|_{2}$
Theorem B.2. Let  $\lambda_1, \lambda_2$  denote the largest singular values of  $W_1$  and  $W_2$  in (6). Let
$$
\beta_ {t, j} = \frac {\exp \left(^ {1} / _ {t} \cdot \tilde {x} _ {t} W _ {Q} W _ {K} ^ {\top} \tilde {x} _ {j} ^ {\top}\right)}{\sum_ {i = 1} ^ {t - 1} \exp \left(^ {1} / _ {t} \cdot \tilde {x} _ {t} W _ {Q} W _ {K} ^ {\top} \tilde {x} _ {i} ^ {\top}\right)}
$$
$$
\mathbb {E} \left[ \| x _ {t} - \tilde {x} _ {t} \| _ {2} \right] \leq \frac {2 . 1 (1 - B / T _ {\max})}{(1 - \epsilon) ^ {2}} \left(k - (k - 1) \left(\frac {1 - \epsilon}{B / T _ {\max} - \epsilon}\right) ^ {1 / (k - 1)}\right)
$$
Define  $m_{k,j} = \mathbb{I}\left\{j\in S_t\right\}$ . With the definition of  $m_{k,j},\tilde{a}_t$  can be written as
$$
\tilde {a} _ {t} = \left(\sum_ {j = 1} ^ {t - 1} \tilde {\alpha} _ {t, j} \tilde {x} _ {j}\right) W _ {V} W _ {O}; \quad \tilde {\alpha} _ {t, j} = \frac {m _ {k , j} \exp \left(1 / t \cdot \tilde {x} _ {t} W _ {Q} W _ {K} ^ {\top} \tilde {x} _ {j} ^ {\top}\right)}{\sum_ {i = 1} ^ {t - 1} m _ {k , j} \exp \left(1 / t \cdot \tilde {x} _ {t} W _ {Q} W _ {K} ^ {\top} \tilde {x} _ {i} ^ {\top}\right)} \tag {8}
$$
Our first lemma shows the Lipschitzness of the attention module.
Lemma B.3. Consider two sequences of tokens  $\{x_i\}_{i=1}^t$  and  $\{y_i\}_{i=1}^t$  where  $\| x_i \|_2 = \| y_i \|_2 = 1$  for all  $i \in [t]$ . Define  $X_{t-1}, Y_{t-1} \in \mathbb{R}^{(t-1) \times d}$  as the matrices whose  $i$ th row are  $x_i$  and  $y_i$ , respectively. Let  $\Delta_t = \| x_t - y_t \|_2$ . Then we have
$$
\left\| s o f t m a x \left(\frac {1}{t} x _ {t} W _ {Q} W _ {K} ^ {\top} X _ {t - 1} ^ {\top}\right) _ {2} - s o f t m a x \left(\frac {1}{t} y _ {t} W _ {Q} W _ {K} ^ {\top} Y _ {t - 1} ^ {\top}\right) \right\| _ {2} \leq 2 \frac {\sqrt {t - 1}}{t} \lambda_ {Q} \lambda_ {K} \Delta_ {t}
$$
Proof. We can decompose the difference as
$$
\begin{array}{l} \left\| \operatorname {s o f t m a x} \left(\frac {1}{t} x _ {t} W _ {Q} W _ {K} ^ {\top} X _ {t - 1} ^ {\top}\right) - \operatorname {s o f t m a x} \left(\frac {1}{t} y _ {t} W _ {Q} W _ {K} ^ {\top} Y _ {t - 1} ^ {\top}\right) \right\| _ {2} \\ \leq \left\| \operatorname {s o f t m a x} \left(\frac {1}{t} x _ {t} W _ {Q} W _ {K} ^ {\top} X _ {t - 1} ^ {\top}\right) - \operatorname {s o f t m a x} \left(\frac {1}{t} x _ {t} W _ {Q} W _ {K} ^ {\top} Y _ {t - 1} ^ {\top}\right) \right\| _ {2} \\ + \left\| \operatorname {s o f t m a x} \left(\frac {1}{t} x _ {t} W _ {Q} W _ {K} ^ {\top} Y _ {t - 1} ^ {\top}\right) - \operatorname {s o f t m a x} \left(\frac {1}{t} y _ {t} W _ {Q} W _ {K} ^ {\top} Y _ {t - 1} ^ {\top}\right) \right\| _ {2} \\ \end{array}
$$
By the Lipschitzness of softmax, we have
$$
\begin{array}{l} \left\| \operatorname {s o f t m a x} \left(\frac {1}{t} x _ {t} W _ {Q} W _ {K} ^ {\top} X _ {t - 1} ^ {\top}\right) - \operatorname {s o f t m a x} \left(\frac {1}{t} x _ {t} W _ {Q} W _ {K} ^ {\top} Y _ {t - 1} ^ {\top}\right) \right\| _ {2} \\ \leq \frac {1}{t} \left\| x _ {t} W _ {Q} W _ {K} ^ {\top} \left(X _ {t - 1} - Y _ {t - 1}\right) ^ {\top} \right\| _ {2} \\ \leq \frac {1}{t} \lambda_ {Q} \lambda_ {K} \| x _ {t} \| _ {2} \| X _ {t - 1} - Y _ {t - 1} \| _ {2} \\ \end{array}
$$
Since  $\| x_{t}\|_{2} = 1$  and  $\| X_{t - 1} - Y_{t - 1}\| _2 = \left(\sum_{j = 1}^{t - 1}\| x_j - y_j\| _2\right)^{\frac{1}{2}}\leq \sqrt{t - 1}\Delta_t$ , we have
$$
\left\| \mathsf {s o f t m a x} \left(x _ {t} W _ {Q} W _ {K} ^ {\top} X _ {t - 1} ^ {\top}\right) - \mathsf {s o f t m a x} \left(x _ {t} W _ {Q} W _ {K} ^ {\top} Y _ {t - 1} ^ {\top}\right) \right\| _ {2} \leq \frac {\sqrt {t - 1}}{t} \lambda_ {Q} \lambda_ {K} \Delta_ {t}
$$
$$
\begin{array}{l} \left\| \operatorname {s o f t m a x} \left(\frac {1}{t} x _ {t} W _ {Q} W _ {K} ^ {\top} Y _ {t - 1} ^ {\top}\right) - \operatorname {s o f t m a x} \left(\frac {1}{t} y _ {t} W _ {Q} W _ {K} ^ {\top} Y _ {t - 1} ^ {\top}\right) \right\| _ {2} \\ \leq \frac {1}{t} \left\| \left(x _ {t} - y _ {t}\right) W _ {Q} W _ {K} ^ {\top} Y _ {t - 1} ^ {\top} \right\| _ {2} \\ \leq \frac {1}{t} \lambda_ {Q} \lambda_ {K} \left\| Y _ {t - 1} \right\| _ {F} \left\| x _ {t} - y _ {t} \right\| _ {2} \\ \end{array}
$$
Since  $\| x_{t} - y_{t}\|_{2} = \Delta_{t}$  and  $\| Y_{t - 1}\| _2 = \sqrt{t - 1}$ , we have
$$
\left\| \operatorname {s o f t m a x} \left(\frac {1}{t} x _ {t} W _ {Q} W _ {K} ^ {\top} Y _ {t - 1} ^ {\top}\right) - \operatorname {s o f t m a x} \left(\frac {1}{t} y _ {t} W _ {Q} W _ {K} ^ {\top} Y _ {t - 1} ^ {\top}\right) \right\| _ {2} \leq \frac {\sqrt {t - 1}}{t} \lambda_ {Q} \lambda_ {K} \Delta_ {t}
$$
Combining the two bounds gives
$$
\left\| \operatorname {s o f t m a x} \left(\frac {1}{\sqrt {t}} x _ {t} W _ {Q} W _ {K} ^ {\top} X _ {t - 1} ^ {\top}\right) - \operatorname {s o f t m a x} \left(\frac {1}{\sqrt {t}} y _ {t} W _ {Q} W _ {K} ^ {\top} Y _ {t - 1} ^ {\top}\right) \right\| _ {2} \leq 2 \frac {\sqrt {t - 1}}{t} \lambda_ {Q} \lambda_ {K} \Delta_ {t}
$$
Our second lemma shows the difference between the output of the sampled and vanilla transformer when the input is the same.
Lemma B.4. Let  $\tilde{a}_t$  be defined as in (8). Define  $b_{t}$  as
$$
b _ {t} = \left(\sum_ {j = 1} ^ {t - 1} \beta_ {t, j} \tilde {x} _ {j}\right) W _ {V} W _ {O}; \quad \beta_ {t, j} = \frac {\exp \left(1 / t \cdot \tilde {x} _ {t} W _ {Q} W _ {K} ^ {\top} \tilde {x} _ {j} ^ {\top}\right)}{\sum_ {i = 1} ^ {t - 1} \exp \left(^ {1} / t \cdot \tilde {x} _ {t} W _ {Q} W _ {K} ^ {\top} \tilde {x} _ {i} ^ {\top}\right)} \tag {9}
$$
Assume that  $\| x_{j}\|_{2} = 1$  for all  $j\in [t]$ . Then we have
$$
\| \tilde {a} _ {t} - b _ {t} \| _ {2} \leq \lambda_ {V} \lambda_ {O} \sum_ {j \notin \hat {S} _ {t}} \beta_ {t, j}
$$
Proof. A direction computation yields
$$
\tilde {a} _ {t} - b _ {t} = \left(\sum_ {j = 1} ^ {t - 1} \left(\tilde {\alpha} _ {t, j} - \beta_ {t, j}\right) \tilde {x} _ {j}\right) W _ {V} W _ {O}
$$
Thus,  $\| \tilde{a}_t - b_t\| _2$  can be bounded as
$$
\| \tilde {a} _ {t} - b _ {t} \| _ {2} \leq \lambda_ {V} \lambda_ {O} \sum_ {j = 1} ^ {t - 1} (\tilde {\alpha} _ {t, j} - \beta_ {t, j}) \| \tilde {x} _ {j} \| _ {2} = \lambda_ {V} \lambda_ {O} \sum_ {j = 1} ^ {t - 1} (\tilde {\alpha} _ {t, j} - \beta_ {t, j})
$$
since  $\| \tilde{x}_j\| _2 = 1$  for all  $j\in [t]$ . Now we analyze  $\tilde{\alpha}_{t,j} - \beta_{t,j}$ . Let  $\hat{S}_t = S_t\setminus \{t\}$ . Then  $m_{k,j} = 1$  if and only if  $j\in \hat{S}_t$ . For convenience, let  $r_{t,j} = 1 / t\cdot \tilde{x}_tW_QW_K^\top \tilde{x}_j^\top$ . Thus,  $\beta$  can be written as
$$
\beta_ {t, j} = \frac {\exp (r _ {t , j})}{\sum_ {i \in \hat {S} _ {t}} \exp (r _ {t , i}) + \sum_ {i \notin \hat {S} _ {t}} \exp (r _ {t , i})}
$$
Furthermore, for all  $j \notin \hat{S}_t$ , we have  $\tilde{\alpha}_{t,j} = 0$ . For all  $j \in \hat{S}_t$ , we have
$$
\tilde {\alpha} _ {t, j} = \frac {\exp (r _ {t , j})}{\sum_ {i \in \hat {S} _ {t}} \exp (r _ {t , i})}
$$
Therefore, for all  $j \in \hat{S}_t$ , we have
$$
\begin{array}{l} \beta_ {t, j} - \tilde {\alpha} _ {t, j} = \exp \left(r _ {t, j}\right) \cdot \frac {\sum_ {i \notin \hat {S} _ {t}} \exp \left(r _ {t , i}\right)}{\left(\sum_ {i \in \hat {S} _ {t}} \exp \left(r _ {t , i}\right)\right) \left(\sum_ {i \in \hat {S} _ {t}} \exp \left(r _ {t , i}\right) + \sum_ {i \notin \hat {S} _ {t}} \exp \left(r _ {t , i}\right)\right)} \\ = \frac {\exp \left(r _ {t , j}\right)}{\sum_ {i \in \hat {S} _ {t}} \exp \left(r _ {t , i}\right)} \cdot \frac {\sum_ {i \notin \hat {S} _ {t}} \exp \left(r _ {t , i}\right)}{\sum_ {i \in \hat {S} _ {t}} \exp \left(r _ {t , i}\right) + \sum_ {i \notin \hat {S} _ {t}} \exp \left(r _ {t , i}\right)} \\ = \tilde {\alpha} _ {t, j} \sum_ {i \notin \hat {S} _ {t}} \beta_ {t, j} \\ \end{array}
$$
Therefore, the bound of  $\| \tilde{a}_t - b_t\| _2$  can be written as
$$
\left\| \tilde {a} _ {t} - b _ {t} \right\| _ {2} \leq \lambda_ {V} \lambda_ {O} \left(\sum_ {j \in \hat {S} _ {t}} ^ {t - 1} \tilde {\alpha} _ {t, j} \sum_ {i \notin \hat {S} _ {t}} \beta_ {t, j} - \sum_ {j \notin \hat {S} _ {t}} \beta_ {t, j}\right) = 2 \lambda_ {V} \lambda_ {O} \sum_ {j \notin \hat {S} _ {t}} \beta_ {t, j}
$$
where the last equality follows from  $\sum_{j\in \hat{S}_t}\tilde{\alpha}_{t,j} = 1$
Our last lemma shows the Lipschitzness of the MLP in (6).
Lemma B.5. Let  $\lambda_1, \lambda_2$  denote the largest singular values of  $W_1, W_2$  in (6). For all  $x_1, x_2 \in \mathbb{R}^d$ , we have
$$
\left\| \mathcal {F} \left(x _ {1}\right) - \mathcal {F} \left(x _ {2}\right) \right\| \leq \left(1 + \lambda_ {1} \lambda_ {2}\right) \| x _ {1} - x _ {2} \| _ {2}
$$
Proof. Direct computation yields
$$
\begin{array}{l} \| \mathcal {F} (x _ {1}) - \mathcal {F} (x _ {2}) \| = \| \left(x _ {1} + W _ {2} \operatorname {r e l u} \left(W _ {1} x _ {1}\right)\right) - \left(x _ {2} + W _ {2} \operatorname {r e l u} \left(W _ {1} x _ {2}\right)\right) \| \\ \leq \left\| x _ {1} - x _ {2} \right\| _ {2} + \left\| W _ {2} \operatorname {r e l u} \left(W _ {1} x _ {1}\right) - W _ {2} \operatorname {r e l u} \left(W _ {1} x _ {2}\right) \right\| \\ \leq \left\| x _ {1} - x _ {2} \right\| _ {2} + \lambda_ {2} \left\| \operatorname {r e l u} \left(W _ {1} x _ {1}\right) - \operatorname {r e l u} \left(W _ {1} x _ {2}\right) \right\| \\ \leq \left\| x _ {1} - x _ {2} \right\| _ {2} + \lambda_ {2} \left\| W _ {1} \left(x _ {1} - x _ {2}\right) \right\| _ {2} \\ \leq \left\| x _ {1} - x _ {2} \right\| _ {2} + \lambda_ {1} \lambda_ {1} \left\| x _ {1} - x _ {2} \right\| _ {2} \\ = \left(1 + \lambda_ {1} \lambda_ {2}\right) \| x _ {1} - x _ {2} \| _ {2} \\ \end{array}
$$
where in the third inequality we use the fact that  $\mathsf{relu}(\cdot)$  is 1-Lipschitz.
Now we turn to the proof of our main theorem. Combining all of the results, we have
$$
\begin{array}{l} a _ {t} - \tilde {a} _ {t} = \left(\sum_ {j = 1} ^ {t - 1} \alpha_ {t, j} x _ {j}\right) W _ {V} W _ {O} - \left(\sum_ {j = 1} ^ {t - 1} \tilde {\alpha} _ {t, j} \tilde {x} _ {j}\right) W _ {V} W _ {O} \\ = \underbrace {\left(\sum_ {j = 1} ^ {t - 1} \alpha_ {t , j} x _ {j}\right) W _ {V} W _ {O} - \left(\sum_ {j = 1} ^ {t - 1} \alpha_ {t , j} \tilde {x} _ {j}\right) W _ {V} W _ {O}} _ {\tau_ {1}} \\ + \underbrace {\left(\sum_ {j = 1} ^ {t - 1} \alpha_ {t , j} \tilde {x} _ {j}\right) W _ {V} W _ {O} - \left(\sum_ {j = 1} ^ {t - 1} \beta_ {t , j} \tilde {x} _ {j}\right) W _ {V} W _ {O}} _ {\tau_ {2}} \\ + \underbrace {\left(\sum_ {j = 1} ^ {t - 1} \beta_ {t , j} \tilde {x} _ {j}\right) W _ {V} W _ {O} - \left(\sum_ {j = 1} ^ {t - 1} \tilde {\alpha} _ {t , j} \tilde {x} _ {j}\right) W _ {V} W _ {O}} _ {\tau_ {3}} \\ \end{array}
$$
Therefore, by triangle inequality, we have
$$
\left\| a _ {t} - \tilde {a} _ {t} \right\| _ {2} \leq \left\| \mathcal {T} _ {1} \right\| _ {2} + \left\| \mathcal {T} _ {2} \right\| _ {2} + \left\| \mathcal {T} _ {3} \right\| _ {2} \tag {10}
$$
To start, the magnitude of  $\mathcal{T}_1$  can be bounded as
$$
\begin{array}{l} \left\| \mathcal {T} _ {1} \right\| _ {2} = \left\| \left(\sum_ {j = 1} ^ {t - 1} \alpha_ {t, j} \left(x _ {t, j} - \tilde {x} _ {t, j}\right)\right) W _ {V} W _ {O} \right\| _ {2} \\ \leq \lambda_ {V} \lambda_ {O} \left\| \sum_ {j = 1} ^ {t - 1} \alpha_ {t, j} \left(x _ {t, j} - \tilde {x} _ {t, j}\right) \right\| \\ \leq \lambda_ {V} \lambda_ {O} \sum_ {j = 1} ^ {t - 1} \alpha_ {t, j} \| x _ {t, j} - \tilde {x} _ {t, j} \| _ {2} \\ \leq \lambda_ {V} \lambda_ {O} \Delta_ {t} \sum_ {j = 1} ^ {t - 1} \alpha_ {t, j} \\ = \lambda_ {V} \lambda_ {O} \Delta_ {t} \\ \end{array}
$$
where in the third inequality we use  $\| x_{t,j} - \tilde{x}_{t,j}\|_2 = \Delta_t$  and in the last equality we use  $\sum_{j=1}^{t-1}\alpha_{t,j} = 1$ . To bound the magnitude of  $\mathcal{T}_2$ , we apply Lemma B.3, which shows that  $\|\alpha_t - \beta_t\| \leq 2\frac{\sqrt{t-1}}{t}\lambda_Q\lambda_K\Delta_t$  to get that
$$
\begin{array}{l} \left\| \mathcal {T} _ {2} \right\| _ {2} = \left\| \left(\sum_ {j = 0} ^ {t - 1} \left(\alpha_ {t, j} - \beta_ {t, j}\right) \tilde {x} _ {j}\right) W _ {V} W _ {O} \right\| _ {2} \\ \leq \lambda_ {V} \lambda_ {O} \left\| \left(\sum_ {j = 0} ^ {t - 1} \left(\alpha_ {t, j} - \beta_ {t, j}\right) \tilde {x} _ {j}\right) \right\| _ {2} \\ \leq \lambda_ {V} \lambda_ {O} \sum_ {j = 0} ^ {t - 1} \left| \alpha_ {t, j} - \beta_ {t, j} \right| \| \tilde {x} _ {j} \| _ {2} \\ \leq \lambda_ {V} \lambda_ {O} \| \alpha_ {t} - \beta_ {t} \| _ {1} \\ \leq \sqrt {t - 1} \lambda_ {V} \lambda_ {O} \| \alpha_ {t} - \beta_ {t} \| _ {2} \\ \leq 2 \left(1 - \frac {1}{t}\right) \lambda_ {Q} \lambda_ {K} \lambda_ {V} \lambda_ {O} \Delta_ {t} \\ \end{array}
$$
Lastly, to bound the magnitude of  $\mathcal{T}_3$ , we use Lemma B.4 to get that
$$
\| \mathcal {T} _ {3} \| _ {2} \leq 2 \lambda_ {V} \lambda_ {O} \sum_ {j \notin \hat {S} _ {t}} \beta_ {t, j}
$$
Putting things together for (10), we have
$$
\left\| a _ {t} - \tilde {a} _ {t} \right\| _ {2} \leq \lambda_ {V} \lambda_ {O} \left(2 \sum_ {j \notin \hat {S} _ {t}} \beta_ {t, j} + (2 \lambda_ {Q} \lambda_ {K} + 1) \Delta_ {t}\right)
$$
By Lemma B.5 we can further show that
$$
\left\| x _ {t + 1} - \tilde {x} _ {t + 1} \right\| _ {2} \leq \left(1 + \lambda_ {1} \lambda_ {2}\right) \lambda_ {V} \lambda_ {O} \left(2 \sum_ {j \notin \tilde {S} _ {t}} \beta_ {t, j} + \left(2 \lambda_ {Q} \lambda_ {K} + 1\right) \Delta_ {t}\right)
$$
By Theorem B.3, we have that with probability at least  $1 - T_{\mathrm{max}} \exp \left(-\frac{\epsilon^2 b^2 (T_{\mathrm{min}} - 1)}{(k - 2)^2 (u - b)^2}\right) - T_{\mathrm{max}} \exp \left(-\frac{2 (T_{\mathrm{min}} - 1) (1 - B / T_{\mathrm{max}})^2}{(1 - \epsilon)^2}\right)$ , it holds for all  $t \in [T_{\mathrm{min}}, T_{\mathrm{max}}]$  that
$$
\mathbb {E} \left[ \sum_ {j \notin \hat {S} _ {t}} \beta_ {t, j} \right] \leq \frac {(1 - B / T _ {\max})}{0 . 9 8 (1 - \epsilon) ^ {2}} \left(k - (k - 1) \left(\frac {1 - \epsilon}{B / T _ {\max} - \epsilon}\right) ^ {\frac {1}{k - 1}}\right) := \Delta_ {\max}
$$
Given that  $\mathbb{E}\left[\| x_t - \tilde{x}_t\|\right] \leq 2\Delta_{\max}$ , we have
$$
\begin{array}{l} \mathbb {E} \left[ \| x _ {t + 1} - \tilde {x} _ {t + 1} \| _ {2} \right] \leq (1 + \lambda_ {1} \lambda_ {2}) \lambda_ {V} \lambda_ {O} \left(2 \Delta_ {\max } + 2 \left(2 \lambda_ {Q} \lambda_ {K} + 1\right) \Delta_ {\max }\right) \\ \leq 4 \lambda_ {V} \lambda_ {O} (1 + \lambda_ {1} \lambda_ {2}) (1 + \lambda_ {Q} \lambda_ {K}) \Delta_ {\max } \\ \end{array}
$$
Thus, as long as  $\lambda_V\lambda_O(1 + \lambda_1\lambda_2)(1 + \lambda_Q\lambda_K)\leq \frac{1}{2}$ , we can guarantee that
$$
\mathbb {E} \left[ \left\| x _ {t + 1} - \tilde {x} _ {t + 1} \right\| _ {2} \right] \leq 2 \Delta_ {\max }
$$
Thus, for all  $t \in [T_{\min}, T_{\max}]$ , we have that
$$
\mathbb {E} \left[ \| x _ {t} - \tilde {x} _ {t} \| _ {2} \right] \leq \frac {2 . 1 (1 - ^ {B} / T _ {\max })}{(1 - \epsilon) ^ {2}} \left(k - (k - 1) \left(\frac {1 - \epsilon}{B / T _ {\max } - \epsilon}\right) ^ {\frac {1}{k - 1}}\right)
$$
B.3 Budgeted Cache
Theorem B.3. Let  $\beta_{t,j}$  be sampled from some power-law distribution  $f(x) = c(x + b)^{-\gamma}$  with support on  $[0, u - b)$  for some  $k > 2$  and  $u \geq 5b$ . Let  $S_t$  be defined in Theorem B.2, and define  $\hat{S}_t = S_t \setminus \{t\}$ . Then with probability at least  $1 - T_{\max} \exp \left(-\frac{\epsilon^2 b^2 (T_{\min} - 1)}{(k - 2)^2 (u - b)^2}\right) - T_{\max} \exp \left(-\frac{2 (T_{\min} - 1) (1 - B)^2}{(1 - \epsilon)^2}\right)$  it holds for all  $t \in T$  that
$$
\mathbb {E} \left[ \sum_ {j \notin \hat {S} _ {t}} \beta_ {t, j} \right] \leq \frac {\left(1 - B / T _ {\max }\right)}{0 . 9 8 (1 - \epsilon) ^ {2}} \left(k - (k - 1) \left(\frac {1 - \epsilon}{B / T _ {\max } - \epsilon}\right) ^ {\frac {1}{k - 1}}\right) \tag {11}
$$
We consider the case of maintaining a budget of  $B$  by dropping the smallest  $\beta_{t,j}$ 's. Assume that  $v_{j}$  has pdf  $f(x) = c(x + b)^{-k}$  with support on  $[0, u - b)$ . To make things precise, we first compute  $c$
$$
c = \left(\int_ {0} ^ {u - b} (x + b) ^ {- k} d x\right) ^ {- 1} = \frac {k - 1}{b ^ {1 - k} - u ^ {1 - k}}
$$
To start, we notice that
$$
\int x (x + b) ^ {- k} = - \frac {(x + b) ^ {1 - k} ((k - 1) x + b)}{(k - 1) (k - 2)} := g (x)
$$
Let  $C = \sum_{j=1}^{t-1} v_j$ , then the expectation of  $C$  is
$$
\begin{array}{l} \mathbb {E} [ C ] = (t - 1) \mathbb {E} [ v _ {1} ] = (t - 1) \frac {k - 1}{b ^ {1 - k} - u ^ {1 - k}} \int_ {0} ^ {\infty} x (x + b) ^ {- k} d x \\ = (t - 1) \frac {k - 1}{b ^ {1 - k} - u ^ {1 - k}} (g (u) - g (0)) \\ = (t - 1) \frac {k - 1}{b ^ {1 - k} - u ^ {1 - k}} \left(\frac {b ^ {2 - k}}{(k - 1) (k - 2)} - \frac {u ^ {1 - k} ((k - 1) u - (k - 2) b)}{(k - 1) (k - 2)}\right) \\ = \frac {t - 1}{k - 2} \cdot \frac {b ^ {2 - k} - (k - 1) u ^ {2 - k} + (k - 2) b u ^ {1 - k}}{b ^ {1 - k} - u ^ {1 - k}} \\ \end{array}
$$
Let  $\Delta = \frac{b^{2 - k} - (k - 1)u^{2 - k} + (k - 2)bu^{1 - k}}{b^{1 - k} - u^{1 - k}}$ . By Hoeffding's inequality, we have that
$$
\mathbb {P} \left(C \leq (1 - \epsilon) \mathbb {E} [ C ]\right) \leq \exp \left(- \frac {2 \epsilon^ {2} \mathbb {E} [ C ] ^ {2}}{(t - 1) (u - b) ^ {2}}\right)
$$
This implies that with probability at least  $1 - \exp \left(-\frac{2\epsilon^2\Delta^2(t - 1)}{(k - 2)^2(u - b)^2}\right)$  we have
$$
C \geq (1 - \epsilon) \Delta \frac {t - 1}{k - 2}
$$
Now, we proceed to bound  $\sum_{j\notin \hat{S}_t}\beta_{t,j}$  where  $\hat{S}_t = \{j\in [t - 1]:\beta_{t,j}\geq \frac{\gamma}{C}\}$ . Equivalently, we can bound  $C^{-1}\sum_{j = 1}^{t - 1}\mathbb{I}\left\{v_j\leq \gamma \right\} v_j$ . Its expectation is given by
$$
\begin{array}{l} \mathbb {E} \left[ C ^ {- 1} \sum_ {j = 1} ^ {t - 1} \mathbb {I} \left\{v _ {j} \leq \gamma \right\} v _ {j} \right] \leq \frac {k - 2}{(t - 1) \Delta (1 - \epsilon)} \mathbb {E} \left[ \sum_ {j = 1} ^ {t - 1} \mathbb {I} \left\{v _ {j} \leq \gamma \right\} v _ {j} \right] \\ = \frac {k - 2}{\Delta (1 - \epsilon)} \cdot \frac {k - 1}{b ^ {1 - k} - u ^ {1 - k}} \int_ {0} ^ {\gamma} x (x + b) ^ {- k} d x \\ = \frac {(k - 1) (k - 2)}{\Delta (1 - \epsilon) (b ^ {1 - k} - u ^ {1 - k})} (g (\gamma) - g (0)) \\ \end{array}
$$
We pause here and study how small can we choose  $\gamma$ . Notice that
$$
\mathbb {E} \left[ \sum_ {j = 1} ^ {t - 1} \mathbb {I} \left\{v _ {j} \leq \gamma \right\} \right] = (t - 1) \mathbb {P} (v _ {j} \leq \gamma) = (t - 1) \cdot \frac {b ^ {1 - k} - (\gamma + b) ^ {1 - k}}{b ^ {1 - k} - u ^ {1 - k}}
$$
By Hoeffding's inequality again, we have that
$$
\begin{array}{l} \mathbb {P} \left(\sum_ {j = 1} ^ {t - 1} \mathbb {I} \left\{v _ {j} \leq \gamma \right\} \geq (1 - \epsilon) (t - 1) \cdot \frac {b ^ {1 - k} - (\gamma + b) ^ {1 - k}}{b ^ {1 - k} - u ^ {1 - k}}\right) \\ \leq \exp \left(- \frac {2 (t - 1) \epsilon^ {2} \left(b ^ {1 - k} - (\gamma + b) ^ {1 - k}\right) ^ {2}}{\left(b ^ {1 - k} - u ^ {1 - k}\right) ^ {2}}\right) \\ \end{array}
$$
Enforcing  $\sum_{j=1}^{t-1} \mathbb{I}\{v_j \leq \gamma\} \geq T_{\max} - B$  gives  $(\gamma + b)^{1-k} \leq b^{1-k} - \frac{1 - B / T_{\max}}{1 - \epsilon} (b^{1-k} - u^{1-k})$ , which can be satisfied as long as  $\gamma \geq \left(\left(\frac{B / T_{\max} - \epsilon}{1 - \epsilon}\right)^{\frac{1}{1-k}} - 1\right)b$ . Therefore
$$
g (\gamma) = - \left(b ^ {1 - k} - \frac {1 - B / T _ {\max}}{1 - \epsilon} (b ^ {1 - k} - u ^ {1 - k})\right) \frac {b + (k - 1) \gamma}{(k - 1) (k - 2)}
$$
We further notice that
$$
b ^ {1 - k} - \frac {1 - B / T _ {\max}}{1 - \epsilon} (b ^ {1 - k} - u ^ {1 - k}) \geq \frac {B / T _ {\max} - \epsilon}{1 - \epsilon} (b ^ {1 - k} - u ^ {1 - k})
$$
$$
\begin{array}{l} \mathbb {E} \left[ C ^ {- 1} \sum_ {j = 1} ^ {t - 1} \mathbb {I} \left\{v _ {j} \leq \gamma \right\} v _ {j} \right] \leq \frac {b (1 - B / T _ {\max})}{\Delta (1 - \epsilon) ^ {2}} - \frac {(k - 1) (B / T _ {\max} - \epsilon) \gamma}{\Delta (1 - \epsilon) ^ {2}} \\ \leq \frac {b (1 - B / T _ {\max })}{\Delta (1 - \epsilon) ^ {2}} \left(k - (k - 1) \left(\frac {1 - \epsilon}{B / T _ {\max } - \epsilon}\right) ^ {\frac {1}{k - 1}}\right) \\ \end{array}
$$
Notice that if  $u\geq 5b$  , we have
$$
\Delta = b - (k - 1) \left(\frac {u}{b}\right) ^ {1 - k} \cdot \frac {b - u}{b ^ {1 - k} - u ^ {1 - k}} \leq 0. 9 8 b
$$