SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The  $L_{1}$  distance of suffixes from  $q_{01}, q_{11}$  that are shorter than  $L$  have also a distance of  $2\xi$ . On the other hand, any shorter prefix has a distance of 0. Since,  $L_{1}^{\mathsf{p}}$  sums all these distances, the minimum across the  $q_{0i}, q_{1i}$  pairs, is attained for  $q_{0L}$  and  $q_{1L}$ , which determines  $\mu_0 \geq 2\xi$ . Also, the distance between any other pair of states in the same layer is strictly higher, since they differ deterministically in some reward or observation. Hence, the  $L_{1}^{\mathsf{p}}$ -distinguishability of the entire RDP is  $\mu_0 = 2\xi$ . Now, we choose  $L = H/2$ ,  $\eta = 32\varepsilon/H$  and we assume  $\varepsilon \leq H\mu_0/64$ ,  $H \geq 2$ . We can verify that these choices are consistent with the previous assumption  $\min\{\xi, \eta\} \geq \frac{16\varepsilon}{H-L}$ . Substituting, the final requirement  $\varphi$  for the complete algorithm  $\mathfrak{A}$  is an exponential number of episodes in  $H$  or:
$$
| \mathcal {D} | \in \Omega \left(\frac {H}{\mu_ {0}} + \frac {C _ {\mathbf {R}} ^ {*} H ^ {2}}{\varepsilon^ {2}}\right) \tag {96}
$$
Now, for any  $H,\mu_0,C_{\mathbf{R}}^*,\varepsilon$  satisfying the previous assumptions, any algorithm cannot be  $(\varepsilon ,1 / 4)$  optimal for the instances in  $\mathbb{R}(H / 2,H,\mu_0,32\varepsilon /H)$  if Eq. (96) is not satisfied.
Note that in our RDP instance, the number of states and the horizon length scale linearly. So, we might equivalently write  $HO$  instead of  $H^2$ .