SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Finally, we aggregate over the last window of the 1000 sequences to recover a full cipher key, in case certain letters never appear in the last window of certain sequences. We again recover a full cipher key via majority vote. In Figure 4b, we show the highly accurate predicted cipher key recovered from ciphertext produced using the example ground truth cipher key in Figure 4a.
To perform a more detailed analysis showing the Bayesian deciphering process of the lexinvariant model, we use the logits of the probe to recover the predicted distribution of the cipher key  $\mathcal{P}(\pi | x_1, \dots, x_{n-1})$ . Instead of taking the majority vote of the predicted decipher tokens in the window, we take the mean of logits predicted for each ciphered token. This essentially gives a locally averaged predicted distribution of cipher key matrices. Specifically, the cipher key matrices are generated across windows of 50 characters, and the probabilities are averaged over 1000 input sequences encoded using the same ground truth cipher. As shown in Figure 5, the predicted distribution of cipher key matrix becomes sharper as the prefix becomes longer.
3.4 In-context Bayesian Deciphering Examples
Here, we show several qualitative examples of in-context Bayesian deciphering. We first show how the lex invariant LM maintains uncertainty over possible lexical permutations while iteratively updating them at each index, using examples from a character-level lexinvariant model. Then, we also show an example of semantic in-context deciphering with a 32K vocabulary lexinvariant model, where the meaning of a novel word is inferred relative to common words in-context.
3.4.1 Uncertainty over Lexical Permutations
In Figure 6a, we input the following ciphered sequence to the frozen character-level lexinvariant LM with the probe: "I saw lots of people in town today, walking and talking around me. I greeted my friend Alice and my classmate Alex. I saw a guy, Joe, walking outside carrying a zat. Joe's zat was taken off zy wind. Today's wind was strong, so Joe's zat flew zackward. Joe lost Joe's zat for good. Joe will miss Joe's zat." For each instance of z in the sequence, we display the predicted deciphering of that instance as a row of probabilities across non-cipher letters a-z.
The lexinvariant model starts off assuming uniform probability for all possible lexical permutations  $\pi$ . After seeing more and more text, the lexinvariant model quickly realizes that  $\mathbf{z}$  only has a few main plausible decipherings (b, h, c, m). Eventually, the lexinvariant model is able to narrow the possibilities down to  $\mathbf{z}$  maps to  $\mathbf{b}$  near the end of the sequence. The predicted probabilities shift with the seen context accordingly, demonstrating an example of how the predicted cipher key is iteratively updated at each index.
Figure 6b shows another example with a similar set up, but with text: "I saw a man in the pazk with a zat. The man was walking with the zat zight beside him. I've never seen anything like that before." While context initially suggests that  $\mathbf{z}$  may be deciphered as c, it becomes clear that  $\mathbf{z}$  must correspond to  $\mathbf{r}$  after the appearance of "right". The disambiguation is reflected in the depicted probabilities.
In Figure 6c and 6d, we show two deciphering examples over code. We consider two code examples in which it is initially ambiguous whether the character  $\mathsf{z}$  deciphers to : or  $\{\cdot\}$ . The ambiguity is eventually resolved by the use of Python-like or Java-like syntax.
3.4.2 Semantic Deciphering
In addition to character-level deciphering, we show examples of semantic deciphering with the larger vocabulary of 32k. Although the lexinvariant LM could not possibly figure out the true lexical permutation among 32k tokens using a small 512 context, it is possible to construct a simple context that repetitively uses simple words so that these words are easier to decipher. Then the lexinvariant LM can decipher the approximate semantics of the rare symbols relative to other easier-to-decipher words.
One example is the following: given the prompt 'crash!' 'aaa!' i looked up from my cup of coffee. 'crash!' - that was the cafe window. and 'aaa!' [... more text...] what one here is a drink  $\leftarrow$  - restaurants  $\leftarrow$  - music  $\leftarrow$  - coffee  $\leftarrow$  - father  $\leftarrow$  the one here that drink is, where the word coffee, music, and father all only appear once before the question and restaurants appeared 4 times, the model is able to correctly answer that coffee is drinkable. See the full example in the appendix.
3.5 Synthetic Reasoning Tasks
As discussed in the introduction, lexical flexibility is correlated with in-context reasoning performance, as demonstrated by existing large LMs. Thus, we study whether the lexinvariant model also learns in-context reasoning capabilities through the challenging lexinvariant training.
Specifically, we measure the performance of lexinvariant models over two pure in-context symbol manipulation tasks: LookUp, where the task is to predict the next token based on the given lookup table, e.g. A->2C->4G->5C-> (should predict 4 here); and Permutation, where the task is to permute an arbitrary subsequence of the given sequence the same way as in the given few demonstrations, e.g. A 2 C->C A4 1 D-> (should predict D 4 here). In each of the tasks, the symbols are randomly sampled from the vocabulary so that we measure the pure reasoning ability independent from any knowledge of specific words. We measure the model performance in terms of generated token accuracy over 1000 examples. The results are shown in Table 1. As shown in the table, the lexinvariant models achieve drastically higher accuracy, with an average of 4X improvement.
Table 1: Accuracy over synthetic reasoning tasks.
Dataset	Vocab	LookUp Acc	Permutation Acc
Standard	LI	Standard	LI
Pile	char	48.50	91.80	27.66	59.35
32k	21.45	92.10	22.84	55.63
Wiki-40B	char	38.25	59.70	20.77	60.51
32k	8.75	59.35	9.94	50.91
Github	char	42.40	86.65	21.03	71.59
32k	4.25	80.20	8.59	67.39
(a) True deciphering: “ $\mathbf{z}$ ” → “ $\mathfrak{b}$ ”,  $T = 1$ .
(b) True deciphering: “ $\mathbf{z}$ ”  $\rightarrow$  “ $\mathbf{r}$ ”,  $T = 1$ .
Figure 6: Probe predictions for deciphering “z” at each occurrence of “z” in context.
(c) True deciphering: “ $\mathbf{z}$ ”  $\rightarrow$  “ $\{\}$ ,  $T = 2$ .
(d) True deciphering: “ $z$ ”  $\rightarrow$  “ $\vdots$ ”,  $T = 3$ .
3.6 Regularizing Language Models with Lexinvariance
Although lexinvariant LM has various interesting properties, it is not suitable for practical tasks since it would require the context to be extremely long so that all required words and knowledge are defined in the context. Here, we explore how to construct more practical semi-lexinvariant LMs that maintain some properties of lexinvariant LMs via regularization. We emphasize that this exploration is intended to be illustrative rather than directly improving state-of-the-art.
Instead of using random Gaussian embedding matrices in place of a learned embedding matrix entirely, we can use random embeddings for only some of the tokens in each sequence, while others use the learned embedding. This means that the learned LM assumes that certain tokens have stable meanings but not others, which can be seen as a form of regularization towards lexinvariance. Specifically, we randomly select tokens to randomize based on a Bernoulli distribution, which can essentially be seen as a form of dropout on token embeddings. On the BIG-bench tasks, we found that a model with dropout rate  $p = 0.2$  for randomization was  $25\%$  more likely to improve performance than to harm performance when evaluated with three shots, relative to a comparably-sized LM, with improvements especially over retrieval type of tasks. See full details in the Appendix G.
More broadly, this regularization view could potentially bring the benefit of lexinvariant LMs to practical applications. For example, the regularization could improve 1) the robustness of LMs by making them less sensitive to adversarial attacks or noise in the input data, 2) generalization across different languages or domains by being less tied to specific lexical items and more prone to learn the shared language structure, and 3) reasoning over more realistic tasks as we have started to explore with BIG-Bench. These areas are promising directions for future work to explore.
4.1 Symbol Grounding
Beyond a modeling choice, the main question of our paper (that being whether an LM can learn language without a stable token representation) is also analogous to the symbol grounding problem: Can meaning be acquired when symbols are not even grounded stably, i.e. they can be mapped to completely random meanings in different sequences? It has long been argued by the symbol grounding literature that symbolic representations must be grounded bottom-up in nonsymbolic representations [11], with famous arguments like Searle's Chinese room: It describes a person in a room given a step-by-step set of instructions by which they can respond to Chinese text with reasonable-sounding Chinese text. To an outside observer, the person in the room appears to understand Chinese, but the individual does not know a word of Chinese. This is widely used to argue that understanding language requires grounding the symbols in the real world. It leads to an ongoing debate on whether LMs can learn meaning purely from large amounts of text, without grounding to any real-world objects [4]. Although intuitively, lexinvariant LMs appear one step further removed from physical grounding than standard LM, we find that given enough context they can still infer the meaning of symbols based on lexical structures within the context.
4.2 Group invariances and Data augmentation
Our implementation of lexinvariant LMs can be seen as performing a form of very aggressive data augmentation, where we randomize the identity of each token in each sequence. From this perspective, it is somewhat similar to the data recombination in [14, 2] and augmentation of named entities in [20], where certain parts of the sentence are swapped with other words while still maintaining the original grammatical structure. In contrast to these augmentations, the training for our lexinvariant LMs completely swaps out all parts of the input text.
4.3 Byte-level T5
There is existing work on absorbing tokenization completely into part of language modeling by using extremely small tokens, such as Byte-level T5 [25]. In the extreme, such a model would become closer and closer to lexinvariant LM, since bytes or bits have almost no stable meaning, so their embeddings are likely not used for prediction. In this paper, we study general lexinvariant LMs with the lexinvariant property baked in and without requiring specific tokenizers.
4.4 Deciphering Substitution Cipher using LMs
In general, solving substitution ciphers, where the cipher key is a permutation of the original alphabet, is a NP-hard problem when only having access to LMs that can assign probabilities to sequences [18]. There have been several works focusing on solving substitution ciphers using LMs, including approaches from searching over the permutation space guided by LMs' scores [12] to training a seq-to-seq model directly to perform deciphering as translation [3]. Although our work does not focus specifically on the task of deciphering substitution ciphers, we show that our lex invariant model can efficiently perform in-context deciphering as a byproduct of language modeling.
4.5 Reasoning
It has been shown that large language models acquire surprising in-context reasoning capabilities [6, 15, 23]. Many of them are related to lexical flexibility through training for purely next-token prediction, such as modified arithmetic, data reformatting, and redefining single word etc. However, LLMs also memorize an enormous amount of knowledge along the way, which is often unnecessary. This work can also be seen as an exploration of whether a (semi-)lexinvariant LM can discount knowledge and prioritize learning the diverse structural reasoning patterns in language, therefore achieving the strong reasoning capability of LLMs with a smaller model.
In this work, we define and study lexinvariant language models, which do not have stable embeddings and learn to infer the meaning of symbols in-context. We show several surprising properties of this model theoretically and empirically, including convergence to standard language modeling, in-context deciphering, and better reasoning capabilities. We also explore a less extreme lexinvariance regularized language model and demonstrate its potential for solving more practical tasks efficiently.
We thank Sang Michael Xie and Steven Cao for discussions and for providing feedback on our manuscript. This project is supported by Open Philanthropy Project Award. Qian Huang is supported by Open Philanthropy AI fellowship.
R. Teehan, R. Yang, S. J. Singh, S. M. Mohammad, S. Anand, S. Dillavou, S. Shleifer, S. Wiseman, S. Gruetter, S. Bowman, S. S. Schoenholz, S. Han, S. Kwatra, S. A. Rous, S. Ghazarian, S. Ghosh, S. Casey, S. Bischoff, S. Gehrmann, S. Schuster, S. Sadeghi, S. S. Hamdan, S. Zhou, S. Srivastava, S. Shi, S. Singh, S. Asaadi, S. S. Gu, S. Pachchigar, S. Toshniwal, S. Upadhyay, S. Debnath, S. Shakeri, S. Thormeyer, S. Melzi, S. Reddy, S. P. Makini, S. hwan Lee, S. B. Torene, S. Hatwar, S. Dehaene, S. Divic, S. Ermon, S. R. Biderman, S. C. Lin, S. Prasad, S. T. Piantadosi, S. M. Shieber, S. Misherghi, S. Kiritchenko, S. Mishra, T. Linzen, T. Schuster, T. Li, T. Yu, T. A. Ali, T. Hashimoto, T.-L. Wu, T. Desbordes, T. Rothschild, T. Phan, T. Wang, T. Nkinyili, T. Schick, T. N. Kornev, T. Telleen-Lawton, T. Tunduny, T. Gerstenberg, T. Chang, T. Neeraj, T. Khot, T. O. Shultz, U. Shaham, V. Misra, V. Demberg, V. Nyamai, V. Raunak, V. V. Ramasesh, V. U. Prabhu, V. Padmakumar, V. Srikumar, W. Fedus, W. Saunders, W. Zhang, W. Vossen, X. Ren, X.F.Tong X.WuX.ShenY.YaghoobzadehY.LakretzY.SongY.BahriY.J.ChoiY.YangY.Hao Y.ChenY.BelinkovY.HouY.HouY.BaiZ.SeidZ.XinranZ.ZhaoZ.F.WangZ.J.WangZ.WangZ.WuS.Singh and U.Shaham Beyond the imitation game: Quantifying and extrapolating the capabilities of language models.ArXiv, abs/2206.04615, 2022
[24] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
[25] L. Xue, A. Barua, N. Constant, R. Al-Rfou, S. Narang, M. Kale, A. Roberts, and C. Raffel. Byt5: Towards a token-free future with pre-trained byte-to-byte models. Transactions of the Association for Computational Linguistics, 10:291-306, 2021.
Theorem A.1. Let  $x_{1}, \ldots, x_{n}$  be any token sequence generated by an arbitrary language distribution  $p$  with an alphabet of size  $d$ . Let  $p'(x_{1}, \ldots, x_{n}) = \mathbb{E}_{\pi}[p(\pi^{-1}(x_{1}), \ldots, \pi^{-1}(x_{n}))]$ . Then, for any  $0 < \epsilon, \delta < 1/2$ ,
with probability greater than  $1 - \delta$  when  $T \geq \frac{d}{\epsilon^4} \log \left( d, \frac{1}{\epsilon}, \frac{1}{\delta} \right)$ .
Proof. For any desired error  $0 < \epsilon < 1/2$  and failure rate  $0 < \delta < 1/2$ , we will first prove the analogous statement for KL divergence instead of  $\mathcal{L}_1$  distance, and then relate a bound on KL divergence back to  $\mathcal{L}_1$  distance via Pinsker's inequality.
Throughout the rest of proof, we will work with a parameter  $\epsilon' < O\left(\frac{\epsilon}{(\log(1 / \delta))^{1 / 4}}\right) < \frac{1}{2}$ , and will bound our KL divergence by  $\epsilon'$ .
To prove the bound in terms of KL divergence, it will be useful to ensure to work with a "smoothed" version of  $p$ , which we denote by  $\tilde{p}$ , for which every token has some nonzero probability,  $\sigma / d$ , of appearing at each timestep, for a parameter  $\sigma = \delta \epsilon' / T$ :
$$
\tilde {p} \left(x _ {T} \mid x _ {1}, \dots , x _ {T - 1}\right) = p \left(x _ {T} \mid x _ {1}, \dots , x _ {T - 1}\right) (1 - \sigma) + \frac {\sigma}{d}.
$$
Similarly, let  $\tilde{p}^{\prime}(x_1,\dots,x_n) = \mathbb{E}_{\pi}[\tilde{p}^{-1}(\pi (x_1),\dots,\pi^{-1}(x_n))]$ . We use  $\tilde{\mathcal{P}}$  to denote the probabilities under this change. With probability at least  $1 - \sigma T\geq 1 - \epsilon '\delta \geq 1 - \frac{\delta}{2}$ , the realized sequence  $x_{1},\ldots ,x_{n}$  drawn under  $p$  can be regarded as being drawn from  $\tilde{p}$  (as these distributions can be coupled with this probability).
The key idea is then to show that  $\tilde{p}^{\prime}(y_{t + 1}|y_{1:t})$ , where  $y_{t} = \pi^{*}(x_{t})$  for some ground truth  $\pi^{*}$  unknown to  $p^\prime$ , is equivalent to using the multiplicative weights algorithm to predict  $y_{t + 1}$  with the Hedge strategy, with the experts being each possible permutation of the tokens and the cost incurred by each expert being the negative log likelihood of the prediction. We denote  $\tilde{\mathcal{P}}_{\pi '}(y_{1:n}) = \tilde{\mathcal{P}} (y_{1:n}|\pi = \pi ') = \tilde{p} (\pi^{-1}(y_1),\dots,\pi^{-1}(y_n))$  and show this in Lemma A.2.
With this equivalence, we can then bound the difference between the prediction of  $p$  and  $p'$  as the regret of the multiplicative weights algorithm. Concretely, we show in Lemma A.3 that the regret of  $p'$  to any expert  $\pi$  is bounded as
$$
\frac {1}{T} \sum_ {t} ^ {T} \log \frac {\tilde {\mathcal {P}} _ {\pi} (y _ {t + 1} | y _ {1 : t})}{\tilde {p} ^ {\prime} (y _ {t + 1} | y _ {1 : t})} \leq 2 \epsilon^ {\prime 2}
$$
for  $T\geq \left(4\log^{2}\left(\frac{d}{\sigma}\right)\log (d!\right) / \epsilon^{\prime 4}.$
We can see  $\tilde{p}$  as the particular expert/permutation  $\tilde{\mathcal{P}}_I$ . And we can further only consider the special case that  $\pi^{*}$  is also the identity permutation, then the same result holds over  $x_{t}$  and with  $\tilde{\mathcal{P}}_{\pi}$  replaced by  $\tilde{p}$ , i.e.
$$
\frac {1}{T} \sum_ {t} ^ {T} \log \frac {\tilde {p} (x _ {t + 1} | x _ {1 : t})}{\tilde {p} ^ {\prime} (x _ {t + 1} | x _ {1 : t})} \leq 2 \epsilon^ {\prime 2}
$$
Now we want to convert this bound on regret in terms of log likelihood to KL divergence, and eventually to  $\mathcal{L}_1$  distance. To convert it to KL divergence regret, we construct a martingale:
$$
Z _ {i} = \sum_ {t = 1} ^ {i} \left(D _ {K L} (\tilde {p} (x _ {t + 1} | x _ {1: t}) \| \tilde {p} ^ {\prime} (x _ {t + 1} | x _ {1: t})) - \log \frac {\tilde {p} (x _ {t + 1} | x _ {1 : t})}{\tilde {p} ^ {\prime} (x _ {t + 1} | x _ {1 : t})}\right).
$$
We verify that this is a martingale in Lemma A.4, with differences bounded by  $2\log \frac{1}{\sigma}$ , and bound the probability that  $Z_{T}$  exceeds  $b = \log \frac{d}{\sigma}\sqrt{8T\log\frac{2}{\delta}}$  via Azuma's inequality Lemma A.6: with probability  $1 - \delta / 2$ , we have that  $|Z_{T}| \leq b$ .
Therefore, we have that with probability at least  $1 - \delta /2$
$$
\begin{array}{l} Z _ {T} = \sum_ {t = 1} ^ {T} \left(D _ {K L} \left(\tilde {p} \left(x _ {t + 1} \mid x _ {1: t}\right) \| \tilde {p} ^ {\prime} \left(x _ {t + 1} \mid x _ {1: t}\right)\right) - \log \frac {\tilde {p} \left(x _ {t + 1} \mid x _ {1 : t}\right)}{\tilde {p} ^ {\prime} \left(x _ {t + 1} \mid x _ {1 : t}\right)}\right) \leq b \\ \sum_ {t = 1} ^ {T} D _ {K L} \left(\tilde {p} \left(x _ {t + 1} \mid x _ {1: t}\right) \| \tilde {p} ^ {\prime} \left(x _ {t + 1} \mid x _ {1: t}\right)\right) \leq \sum_ {t = 1} ^ {T} \left(\log \frac {\tilde {p} \left(x _ {t + 1} \mid x _ {1 : t}\right)}{\tilde {p} ^ {\prime} \left(x _ {t + 1} \mid x _ {1 : t}\right)}\right) + b \\ \end{array}
$$
Putting this all together, since  $\frac{1}{T}\sum_{t}^{T}\log \frac{\tilde{p}_i(x_{t + 1}|x_{1:t})}{\tilde{p}'(x_{t + 1}|x_{1:t})}\leq 2\epsilon^{\prime 2}$  for  $T\geq \left(4\log^{2}\left(\frac{d}{\sigma}\right)\log (d!)\right) / \epsilon^{\prime 4}$ , we have the following:
$$
\sum_ {1} ^ {T} D _ {K L} \left(\tilde {p} \left(x _ {t + 1} \mid x _ {1: t}\right) \| \tilde {p} ^ {\prime} \left(x _ {t + 1} \mid x _ {1: t}\right)\right) \leq 2 \epsilon^ {\prime 2} T + b.
$$
We now convert our bound on KL divergence to a bound on  $\mathcal{L}_1$  distance via Pinsker's inequality:
$$
\| \tilde {p} (x _ {t + 1} | x _ {1: t}) - \tilde {p} ^ {\prime} (x _ {t + 1} | x _ {1: t}) \| _ {1} \leq \sqrt {\frac {1}{2} D _ {K L} (\tilde {p} (x _ {t + 1} | x _ {1 : t}) \| \tilde {p} ^ {\prime} (x _ {t + 1} | x _ {1 : t}))}.
$$
Further, at any given  $x_{t}$ , the difference between the redistributed probability distribution  $\tilde{p}$  and a unmodified probability distribution  $p$  is at most  $\sigma$ , so
$$
\left\| p \left(x _ {t + 1} \mid x _ {1: t}\right) - p ^ {\prime} \left(x _ {t + 1} \mid x _ {1: t}\right) \right\| _ {1} \leq \left\| \tilde {p} \left(x _ {t + 1} \mid x _ {1: t}\right) - \tilde {p} ^ {\prime} \left(x _ {t + 1} \mid x _ {1: t}\right) \right\| _ {1} + 2 \sigma .
$$
We are interested in the average  $\mathcal{L}_1$  across time steps:
$$
\begin{array}{l} \frac {1}{T} \sum_ {t = 1} ^ {T} \| p (x _ {t + 1} | x _ {1: t}) - p ^ {\prime} (x _ {t + 1} | x _ {1: t}) \| _ {1} \leq \frac {1}{T} \sum_ {t = 1} ^ {T} (\| \tilde {p} (x _ {t + 1} | x _ {1: t}) - \tilde {p} ^ {\prime} (x _ {t + 1} | x _ {1: t}) \| _ {1} + 2 \sigma) \\ \leq \frac {1}{T} \sum_ {t = 1} ^ {T} \sqrt {\frac {1}{2} D _ {K L} (\tilde {p} (x _ {t + 1} | x _ {1 : t}) \| \tilde {p} ^ {\prime} (x _ {t + 1} | x _ {1 : t}))} + 2 \sigma \\ \leq \frac {1}{T} \sqrt {T \sum_ {t = 1} ^ {T} \frac {1}{2} D _ {K L} \left(\tilde {p} \left(x _ {t + 1} \mid x _ {1 : t}\right) \| \tilde {p} ^ {\prime} \left(x _ {t + 1} \mid x _ {1 : t}\right)\right)} + 2 \sigma , \\ \end{array}
$$
where in the last inequality we applied Cauchy-Schwarz. Hence for  $T \geq \left(4\log^2\frac{d}{\sigma}\log (d!)\right) / \epsilon'^4$
$$
\begin{array}{l} \frac {1}{T} \sum_ {t = 1} ^ {T} \| p (x _ {t + 1} | x _ {1}, \dots , x _ {t}) - p ^ {\prime} (x _ {t + 1} | x _ {1}, \dots , x _ {t}) \| _ {1} \leq \frac {1}{T} \sqrt {\frac {T}{2} \left(2 \epsilon^ {\prime 2} T + b\right)} + 2 \sigma \\ \leq \sqrt {\epsilon^ {\prime 2} + \frac {b}{2 T}} + 2 \sigma . \\ \end{array}
$$
Simplifying this for  $b = \log \frac{d}{\sigma} \sqrt{8T \log \frac{2}{\delta}}$ ,  $T \geq \left(4 \log^2 \left(\frac{d}{\sigma}\right) \log (d!)\right) / \epsilon'^4$  and  $\sigma = \epsilon' \delta / T$ , we have
$$
\begin{array}{l} \frac {1}{T} \sum_ {t = 1} ^ {T} \| p (x _ {t + 1} | x _ {1}, \dots , x _ {t}) - p ^ {\prime} (x _ {t + 1} | x _ {1}, \dots , x _ {t}) \| _ {1} \leq \sqrt {\epsilon^ {\prime 2} + \frac {\sqrt {2 \log \frac {2}{\delta}}}{\sqrt {\log (d !)}} \epsilon^ {\prime 2}} + \frac {2 \epsilon^ {\prime} \delta}{T} \\ \leq \epsilon^ {\prime} (\frac {2 \delta}{T} + \sqrt {1 + \sqrt {\frac {2 \log \frac {2}{\delta}}{\log (d !)}}}) \\ \leq \epsilon^ {\prime} (1 + \sqrt {1 + \sqrt {2 \log \frac {2}{\delta}}}) \leq \epsilon^ {\prime} 2 \sqrt {2} (2 \log \frac {2}{\delta}) ^ {1 / 4}. \\ \end{array}
$$
We can bound this average  $L_{1}$  error by  $\epsilon$  if we set  $\epsilon' = \frac{\epsilon}{2\sqrt{2}(2\log\frac{2}{\delta})^{1/4}} < \frac{1}{2}$ , in which case our condition that  $T \geq \left(4\log^2\left(\frac{dT}{\delta\epsilon'}\right)\log(d!)\right)/\epsilon'^4$  becomes  $T \geq \left(512\log\frac{2}{\delta}\log^2\frac{dT}{\delta\epsilon'}\log(d!)\right)/\epsilon^4$ . The theorem now follows by simplifying this expression. Since  $\log\frac{2}{\delta} \leq 2\log\frac{1}{\delta}$ , and  $\log(d!) \leq d\log(d)$ , we can relax the condition on  $T$  as
$$
T \geq \left(1 0 2 4 \mathrm {l o g} \frac {1}{\delta} \mathrm {l o g} ^ {2} (\frac {d}{\delta \epsilon^ {\prime}}) \mathrm {l o g} ^ {2} (T) d \mathrm {l o g} (d)\right) / \epsilon^ {4} = \mathrm {l o g} ^ {2} (T) \frac {d}{\epsilon^ {4}} p o l y l o g (d, \frac {1}{\epsilon}, \frac {1}{\delta})
$$
To remove the  $\log^2 T$  from the right side, note that for any  $W > 10$ , if  $\mathrm{T} > 10W\log^2 W$ , then  $T > W\log^2 T$ , yielding the further relaxed the condition on  $T$  as