SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We prove the first inequality of Eq. (4) in a similar way. Let us re-define  $d = d_{\rho}(4\epsilon)$  and let  $d'$  be equal to the leftmost expression of Eq. (4). As before,  $d' \leq d$ . Then there exists a sequence  $h_1, \ldots, h_d \in \mathcal{H}$  satisfying the conditions of Definition 1 for some  $c \leq 1 - 4\epsilon$ . Then for all  $i \neq j$ ,  $\langle h_i, h_j \rangle \leq c + \frac{4\epsilon}{\sqrt{d}} \leq 1 - \epsilon$ , since  $d \geq 2$ . Moreover, for all  $i \neq j$  and  $i' \neq j'$ ,
$$
\left| \left\langle h _ {i}, h _ {j} \right\rangle - \left\langle h _ {i ^ {\prime}}, h _ {j ^ {\prime}} \right\rangle \right| = \left| \left\langle h _ {i}, h _ {j} \right\rangle - c + c - \left\langle h _ {i ^ {\prime}}, h _ {j ^ {\prime}} \right\rangle \right| \leq \frac {8 \epsilon}{\sqrt {d}} \leq \frac {1}{d ^ {\prime}},
$$
with the last inequality following from our choice of  $d'$ . Thus,  $h_1, \ldots, h_{d'}$ , the first  $d'$  hypotheses in the original sequence, satisfy Definition 4.
The second inequality of Eq. (4), now follows from the first inequality of Eq. (3), since if the second inequality of Eq. (4) does not hold then the leftmost expression of Eq. (3) must be at least  $d_{\rho}(\epsilon)$ , a contradiction. Likewise, the second inequality of Eq. (3), now follows from the first inequality of Eq. (4).
B.2 Lower bound setting
Definition 8 (SQ oracle (adversarial)). Let  $D$  be the input distribution over the domain  $X$ . For a tolerance parameter  $\tau > 0$ ,  $\mathcal{O}^{adv}(\tau) \coloneqq \mathcal{O}_{D,h^*}^{adv}(\tau)$  oracle is the oracle that for any query function  $h \in \mathcal{H}$ , returns a value  $v \in [\mu - \tau, \mu + \tau]$ , where  $\mu = \mathbb{E}_{x \sim D}[h(x)h^*(x)]$ .
Definition 9 (Sample oracle (statistical)). Let  $D$  be the input distribution over the domain  $X$ . The Sample oracle  $\mathcal{O} \coloneqq \mathcal{O}_{D,h^*}$  oracle is the oracle that given any function  $h \in \mathcal{H}$ , takes an independent random sample  $x$  from  $D$  and returns the value  $v = h(x)h^*(x)$ .
We will need the following results for our proof. The first is a reduction from an adversarial noise oracle to a statistical one. Specifically, consider the learning setting defined in Section 2, for a sample oracle  $\mathcal{O}$ . Let Alg be a (possibly randomized) algorithm for that setting. The following theorem shows a simulation of  $\mathcal{O}$  via  $\mathcal{O}^{adv}$  the SQ oracle.
Theorem 25 ([13], Theorem 3.13). Assume that  $\mathsf{Alg}$  outputs a  $\epsilon$ -approximation to  $h^*$  with probability at least  $\delta$ , using  $m$  samples from  $\mathcal{O}$ . Then, for any  $\delta' \in (0,1/4]$ , there exists a SQ algorithm  $\mathsf{Alg}'$  that uses at most  $m$  queries to  $\mathcal{O}^{adv}(\delta'^2/m)$  and outputs an  $\epsilon$ -approximation to  $h^*$  with probability at least  $\delta - \delta'$ .
Their result is obtained by simulating Alg using  $\mathcal{O}^{adv}$  as follows: for any query of Alg to  $\mathcal{O}$ , the response of  $\mathcal{O}^{adv}$  to that query is used as bias for a coin flip, which is then given to the learner as the simulated outcome of  $\mathcal{O}$ . They then prove that the true  $m$  samples of  $\mathcal{O}$  and the simulated coin flips are statistically close by bounding their distributional distance. This implies that the success probability of  $\mathrm{Alg}'$ , the simulated algorithm, is not much worse than that of Alg, the original algorithm.
We note that the result originally stated in [13] differs from Theorem 25 above in two ways. First, it reduces to a variant of  $\mathcal{O}^{adv}(\tau)$  with a tolerance  $\tau' \in [\tau, \sqrt{\tau}]$ . Thus, it holds for  $\mathcal{O}^{adv}(\tau)$  as well. Second, it is phrased in a more general setting of search problems over distributions, which captures the SQ model, as detailed in [13], Section 6.
The second result that is needed for our proof is the following lower bound due to [25].
Theorem 26 ([25], Theorem 8). Let  $\epsilon >0$ , and let  $\mathcal{H}\subseteq \{\pm 1\}^{\mathcal{X}}$  be a hypothesis space with strong  $SQ$  dimension  $d_{\mathrm{SQ}}\coloneqq \dim_{\mathrm{SQ}}(\mathcal{H},2\epsilon)\geq 3$  (see Definition 4). Then for any  $SQ$  algorithm Alg using  $m$  queries to  $\mathcal{O}^{adv}(\tau)$  with tolerance  $\tau \geq 2 / \sqrt{d_{\mathrm{SQ}}}$ , there exist  $h^*\in \mathcal{H}$  such that if Alg outputs an  $\epsilon$ -approximation to  $h^*$ , then  $m > d_{\mathrm{SQ}}\tau^2 /3$ .
B.2.1 Proof of Theorem 9
Set  $\delta = 2/3$ . Let  $D$  be a distribution over  $\mathcal{X}$ . Assume towards contradiction that there exists a learning algorithm  $\mathsf{Alg}$  such that for any  $h^* \in \mathcal{H}$ , given oracle access to  $\mathcal{O} := \mathcal{O}_{D,h^*}$  and using  $m \leq \sqrt[3]{d_{\mathrm{SQ}}} / 12$  samples from  $\mathcal{O}$ , the algorithm  $\mathsf{Alg}$  outputs an  $\epsilon$ -approximation to  $h^*$  with probability at least  $\delta$ .
We then apply Theorem 25 for  $\delta' = \delta/2$  to simulate the algorithm using  $\mathcal{O}^{adv} \coloneqq \mathcal{O}_{D,h^*}^{adv}$ . The resulting algorithm uses  $m \leq \sqrt[3]{d_{\mathrm{SQ}}} / 12$  queries to  $\mathcal{O}^{adv}(\tau)$  for  $\tau = \delta'^2 / m > 4 / (3\sqrt[3]{d_{\mathrm{SQ}}}) \geq 2 / \sqrt{d_{\mathrm{SQ}}}$  and has success probability of at least  $\delta - \delta' = \delta / 2 > 1/3$ . By Theorem 26 we obtain a contradiction, as  $m > d_{\mathrm{SQ}}\tau^2 / 3 > \sqrt[3]{d_{\mathrm{SQ}}} / 2$ .
C Missing proofs of Section 5
C.1 Proof of Theorem 11
Let  $d = \overline{\mathrm{d}}_{\rho}(\mathcal{Z},\alpha ,3\epsilon /2)$ . Note that the eluder dimension is always at least 1, so the theorem trivially holds if  $d\leq 9$ . In the remainder of the proof assume that  $d\geq 10$
From the definition of the monotonic dissimilarity dimension, there exists  $\tau \geq 3\epsilon /2$  such that  $d = \mathsf{d}(\mathcal{Z},\alpha ,\tau)$ . Let  $(f_{1},a_{1}),\ldots ,(f_{d},a_{d})$  be a sequence satisfying the dimension conditions for  $\tau$ .
We will show that the first  $\lceil d / 9\rceil$  elements of this sequence also satisfy the conditions of the eluder dimension for some  $\epsilon^{\prime}\geq \epsilon$ . Specifically, we will show that there is some  $\epsilon^{\prime}\geq \epsilon$  such that every element  $a_{j}$  with  $j\leq \lceil d / 9\rceil$  in the sequence above is  $\epsilon^\prime$ -independent of its predecessors. That is, we will show that for every such element  $a_{j}$ , there exists a pair of functions  $f,f^{\prime}\in \mathcal{F}$  that satisfy
$$
\sqrt {\sum_ {i = 1} ^ {j - 1} \left(f (a _ {i}) - f ^ {\prime} (a _ {i})\right) ^ {2}} \leq \epsilon^ {\prime},
$$
yet it also holds that  $f(a_{j}) - f^{\prime}(a_{j}) > \epsilon^{\prime}$ .
By definition of the dissimilarity dimension, there exists  $c \leq \alpha - \tau$  such that for all  $i < j$ ,
$$
\left| f _ {j} \left(a _ {i}\right) - c \right| = \left| \rho \left(\left(f _ {i}, a _ {i}\right) \mid \left(f _ {j}, a _ {j}\right)\right) - c \right| \leq \frac {\tau}{\sqrt {d}}. \tag {28}
$$
Then, by the triangle inequality,
$$
\left| f _ {j} \left(a _ {i}\right) - f _ {j + 1} \left(a _ {i}\right) \right| = \left| f _ {j} \left(a _ {i}\right) - c + c - f _ {j + 1} \left(a _ {i}\right) \right| \leq \left| f _ {j} \left(a _ {i}\right) - c \right| + \left| f _ {j + 1} \left(a _ {i}\right) - c \right| \leq \frac {2 \tau}{\sqrt {d}}. \tag {29}
$$
$$
\left(f _ {j} \left(a _ {i}\right) - f _ {j + 1} \left(a _ {i}\right)\right) ^ {2} \leq \frac {4 \tau^ {2}}{d}, \tag {30}
$$
and so for all  $j \leq \lceil d / 9 \rceil$  it holds that,
$$
\sum_ {i = 1} ^ {j - 1} \left(f _ {j} \left(a _ {i}\right) - f _ {j + 1} \left(a _ {i}\right)\right) ^ {2} <   \frac {4 \tau^ {2}}{9}. \tag {31}
$$
Next, recall that for all  $j \leq d$  we have  $f_{j}(a_{j}) \geq \alpha \geq c + \tau$  and  $f_{j+1}(a_{j}) \leq c + \frac{\tau}{\sqrt{d}}$ . Thus,
$$
f _ {j} \left(a _ {j}\right) - f _ {j + 1} \left(a _ {j}\right) \geq c + \tau - c - \frac {\tau}{\sqrt {d}} > \frac {2 \tau}{3}, \tag {32}
$$
where the last inequality holds for  $d \geq 10$ . Overall, Eqs. (31) and (32) then demonstrate that for  $\epsilon' = 2\tau / 3 \geq \epsilon$ , the element  $a_j$  is  $\epsilon'$ -independent of its predecessors, finishing the proof.
C.2 Proof of Theorem 13
Our proof uses the following result on ranks of perturbed identity matrices (see [2, Lemma 2.2]):
Lemma 27. Let  $\mathbf{A} \in \mathbb{R}^{d \times d}$  be a symmetric matrix such that  $A_{ii} = 1$  for all  $i$  and  $|A_{ij}| \leq 1 / \sqrt{d}$  for all  $i \neq j$ . Then  $\mathrm{rank}(\mathbf{A}) > d / 2$ .
The proof begins by constructing a matrix  $\mathbf{M}$  whose entries are derived from the evaluation values of elements that satisfy the dimension condition. Then we bound the rank of  $\mathbf{M}$  from above as well as from below. The lower bound is expressed in terms of the dimension  $d = \mathrm{d}_{\rho}(\mathcal{Z},\alpha ,\epsilon)$  while the upper bound is expressed in terms of  $n$ . Combining the bounds then yields the result of the theorem.
**Construction of M.** Let  $(f_{\theta_1}, \mathbf{a}_1), \ldots, (f_{\theta_d}, \mathbf{a}_d)$  denote the alternatives that satisfy the dimension conditions, with respect to some value  $c$  such that  $c \leq \alpha - \epsilon$  (see Definition 1). Define  $\mathbf{M}$  to be the  $d \times d$  matrix with entries  $M_{ij} = \langle \pmb{\theta}_i, \mathbf{a}_j \rangle - c$  for  $i, j \leq d$ . Note that all diagonal entries of  $\mathbf{M}$  are at least  $\alpha - c \geq \epsilon$ , and all other entries are in  $[- \frac{\epsilon}{\sqrt{d}}, \frac{\epsilon}{\sqrt{d}}]$ .
Upper bound on  $\mathrm{rank}(\mathbf{M})$ . Let  $\mathbf{K} \in \mathbb{R}^{d \times d}$  be the matrix of inner products,  $K_{ij} = \langle \pmb{\theta}_i, \mathbf{a}_j \rangle$ , and let  $\mathbf{U}$  be the Gram matrix for the set of vectors  $\pmb{\theta}_1, \dots, \pmb{\theta}_d, \mathbf{a}_1, \dots, \mathbf{a}_d$ . Then  $\mathbf{U}$  is a  $2d \times 2d$  matrix of the rank at most  $n$ , because the vectors are of the dimension  $n$  (see, e.g., [17, Theorem 7.2.10]), and  $\mathbf{K}$  is a submatrix of  $\mathbf{U}$ , so  $\mathrm{rank}(\mathbf{K}) \leq \mathrm{rank}(\mathbf{U}) \leq n$ . Moreover,  $\mathbf{M} = \mathbf{K} - c\mathbf{1}\mathbf{1}^\top$ , where  $\mathbf{1}$  is the all-ones vector in  $\mathbb{R}^d$ . Therefore, by subadditivity of rank,
$$
\operatorname {r a n k} (\mathbf {M}) = \operatorname {r a n k} (\mathbf {K} - c \mathbf {1 1} ^ {\top}) \leq \operatorname {r a n k} (\mathbf {K}) + \operatorname {r a n k} (- c \mathbf {1 1} ^ {\top}) \leq n + 1. \tag {33}
$$
Lower bound on rank(M). Let  $\mathbf{D} \in \mathbb{R}^{d \times d}$  be the diagonal matrix with entries  $D_{ii} = 1 / \sqrt{M_{ii}}$ . Since  $M_{ii} \geq \epsilon > 0$ , we have  $0 < D_{ii} \leq 1 / \sqrt{\epsilon}$ . Consider the matrix  $\mathbf{M}' = \mathbf{DMD}$ . Then
$$
\operatorname {r a n k} \left(\mathbf {M} ^ {\prime}\right) = \operatorname {r a n k} (\mathbf {D M D}) = \operatorname {r a n k} (\mathbf {M}), \tag {34}
$$
because the matrix  $\mathbf{D}$  is non-singular (see [17, Section 0.4.6(b)]). Furthermore, matrix  $\mathbf{M}'$  satisfies  $M_{ii}^{\prime} = 1$  for all  $i$  and
$$
| M _ {i j} ^ {\prime} | = | D _ {i i} M _ {i j} D _ {j j} | \leq \frac {1}{\sqrt {\epsilon}} \cdot \frac {\epsilon}{\sqrt {d}} \cdot \frac {1}{\sqrt {\epsilon}} = \frac {1}{\sqrt {d}}
$$
for all  $i \neq j$ . Consider the symmetric matrix  $\mathbf{S} = (\mathbf{M}' + (\mathbf{M}')^\top) / 2$ . Then, we also have  $S_{ii} = 1$  for all  $i$ , and  $|S_{ij}| \leq 1 / \sqrt{d}$  for all  $i \neq j$ . Thus, by Lemma 27,  $\mathrm{rank}(\mathbf{S}) > d / 2$ . Moreover, by the subadditivity of the rank
$$
d / 2 <   \operatorname {r a n k} (\mathbf {S}) \leq \operatorname {r a n k} \left(\mathbf {M} ^ {\prime} / 2\right) + \operatorname {r a n k} \left(\left(\mathbf {M} ^ {\prime}\right) ^ {\top} / 2\right) = 2 \operatorname {r a n k} \left(\mathbf {M} ^ {\prime}\right). \tag {35}
$$
Combining Eqs. (35), (34) and (33), we therefore obtain
$$
d / 2 <   2 \operatorname {r a n k} (\mathbf {M} ^ {\prime}) = 2 \operatorname {r a n k} (\mathbf {M}) \leq 2 n + 2,
$$
and so  $d < 4n + 4$ . Since  $d$  is an integer, we must have  $d \leq 4n + 3$ .
In the special case that  $\alpha = 1$ , we have  $\langle \pmb{\theta}_i, \mathbf{a}_i \rangle \geq 1$  for all  $i \leq d$ , which is only possible when  $\pmb{\theta}_i = \mathbf{a}_i$  for all  $i \leq d$ . As a result, the matrices  $\mathbf{M}$  and  $\mathbf{M}'$  are both symmetric, and thus  $\mathbf{S} = \mathbf{M}'$  and
$$
d / 2 <   \operatorname {r a n k} (\mathbf {S}) = \operatorname {r a n k} (\mathbf {M} ^ {\prime}) = \operatorname {r a n k} (\mathbf {M}) \leq n + 1,
$$
implying that  $d \leq 2n + 1$ .
C.3 Proof of Theorem 14
Using an existing bound on the eluder dimension for GLM bandits ([24], Proposition 7), and the fact that our dimension is bounded by the eluder dimension (Theorem 11) the result follows.
C.4 Proof of Theorem 15
Denote  $d = \mathsf{d}_{\rho}(\mathcal{Z}_b^{relu},1 - b,\epsilon)$ . Notice that since  $b < 1$ , for any  $\pmb{\theta}$ ,  $\mathbf{a} \in \mathcal{B}_n$  such that  $\pmb{\theta} \neq \mathbf{a}$  it holds that  $f_{\pmb{\theta},b}(\mathbf{a}) < f_{\pmb{\theta},b}(\pmb{\theta}) = 1 - b$ . Let  $(f_{\pmb{\theta}_1,b},\pmb{\theta}_1),\dots,(f_{\pmb{\theta}_d,b},\pmb{\theta}_d)$  be a sequence of elements satisfying the dimension definition, with respect to a corresponding scalar  $c \leq 1 - b - \epsilon$ . Since the evaluation is symmetric for ReLU functions, we can view this sequence as a set, and denote  $U = \{\pmb{\theta}_1,\dots,\pmb{\theta}_d\}$ . In addition, note that by the dimension definition, for all  $\pmb{\theta} \in U$ ,  $\| \pmb{\theta}\| = 1$ .
We start by proving an upper bound on  $d$ . Assume  $d \geq 9$ . First, consider the case  $c \leq \epsilon / 3$ . Let  $U_0$  be any subset of the unit sphere such that for all  $\pmb{\theta} \neq \pmb{\theta}'$  in  $U_0$  it holds that  $\langle \pmb{\theta}, \pmb{\theta}' \rangle \leq b + 2\epsilon / 3$ . Observe that for all such  $\pmb{\theta} \neq \pmb{\theta}'$  we have  $f_{\pmb{\theta}, b}(\mathbf{u}') = f_{\pmb{\theta}', b}(\pmb{\theta}) \in [0, 2\epsilon / 3]$ . Thus, we get that  $d \leq |U_0|$ .
A standard sphere covering argument shows that the size of such a set is upper bounded as follows. The  $\delta$ -covering number of the unit sphere is at most  $(3 / \delta)^n$  ([27], Cor. 4.2.13). Thus, there are at most  $(3 / \delta)^n$  points such that each pair  $\pmb{\theta} \neq \pmb{\theta}'$  satisfies  $\|\pmb{\theta} - \pmb{\theta}'\| \geq \delta$ , or equivalently  $\langle \pmb{\theta}, \pmb{\theta}' \rangle \leq 1 - \delta^2 / 2$ . By setting  $\delta = \sqrt{2(1 - b - 2\epsilon / 3)}$  we get that  $|U_0| \leq \left(\frac{3}{\delta}\right)^n \leq \left(\frac{3}{2\sqrt{2(\epsilon - 2\epsilon / 3)}}\right)^n \leq (4 / \sqrt{\epsilon})^n$ , which yields the desired bound.
Now, consider the case  $c > \epsilon / 3$ . In this case, for all  $i \neq j$ , we have that  $f_{\pmb{\theta}_j, b}(\pmb{\theta}_i) \geq c - \frac{\epsilon}{\sqrt{d}} \geq c - \epsilon / 3 > 0$ , and so  $\langle \pmb{\theta}_i, \pmb{\theta}_j \rangle > b$ . Let  $c' = c + b$ . Thus, it must also hold that,  $|\langle \pmb{\theta}_i, \pmb{\theta}_j \rangle - c'| \leq \frac{\epsilon}{\sqrt{d}}$  for all  $i \neq j$ . Note that  $c' < 1 - \epsilon$ . Then, by applying Lemma 13 we get that  $d$  is upper bounded by  $2n + 4$ . Overall, the bound in the claim holds.
Next, we show a lower bound on  $d = \mathsf{d}_{\rho}(\mathcal{Z}_{1 - \epsilon}^{relu},\epsilon ,\epsilon)$ , by lower bounding the size of the set  $U$  defined above. We now apply a sphere packing argument, which shows that there exists such a set  $U$  with size  $|U|\geq (1 / 2\epsilon)^{n / 2}$ . We follow a similar argument as above. Specifically, the  $\delta$ -packing number of the unit sphere is at most  $(1 / \delta)^n$  ([27], Cor. 4.2.13). By plugging in  $\delta = \sqrt{2\epsilon}$ , yields the desired bound.
C.5 Proof of Proposition 12
We start with an auxiliary lemma:
Lemma 28 (dissimilarity subadditivity). Let  $\mathcal{Z}_1$  and  $\mathcal{Z}_2$  be two sets, and let  $\alpha \in \mathbb{R}$  and  $\epsilon > 0$ . Denote  $\mathcal{Z} = \mathcal{Z}_1 \cup \mathcal{Z}_2$  and let  $\rho: \mathcal{Z} \times \mathcal{Z} \to \mathbb{R}$  be an evaluation function. Then,
$$
\mathsf {d} _ {\rho} (\mathcal {Z}, \alpha , \epsilon) \leq \mathsf {d} _ {\rho} (\mathcal {Z} _ {1}, \alpha , \epsilon) + \mathsf {d} _ {\rho} (\mathcal {Z} _ {2}, \alpha , \epsilon)
$$
$$
\overline {{\mathsf {d}}} _ {\rho} (\mathcal {Z}, \alpha , \epsilon) \leq \overline {{\mathsf {d}}} _ {\rho} (\mathcal {Z} _ {1}, \alpha , \epsilon) + \overline {{\mathsf {d}}} _ {\rho} (\mathcal {Z} _ {2}, \alpha , \epsilon).
$$
Proof. Let  $z_{1}, \ldots, z_{d} \subseteq \mathcal{Z}$  such that there exists  $c \leq \alpha - \epsilon$  with  $|\rho(z_{i}|z_{j}) - c| \leq \frac{\epsilon}{\sqrt{d}}$  for all  $i < j$ , and  $\rho(z_{i}|z_{i}) \geq \alpha$ . Let  $I_{1}, I_{2} \subseteq [d]$  be disjoint sets of indices with  $\{z_{i}\}_{i \in I_{1}} \subseteq \mathcal{Z}_{1}$  and  $I_{2} = [d] \setminus I_{1}$ . Consider the sub-sequence  $z_{\ell_{1}}, \ldots, z_{\ell_{|I_{1}|}}$  ordered by appearance in  $z_{1}, \ldots, z_{d}$  of elements in  $I_{1}$ . By definition for all  $i < j$ ,
$$
\left| \rho (z _ {\ell_ {i}} | z _ {\ell_ {j}}) - c \right| \leq \frac {\epsilon}{\sqrt {d}}
$$
Therefore the sequence  $z_{\ell_1}, \ldots, z_{\ell_{|I_1|}}$  satisfies  $|\rho(z_{\ell_i}|z_{\ell_j}) - c| \leq \frac{\epsilon}{\sqrt{|I_1|}}$  for all  $i < j$  and therefore  $|I_1| \leq \mathsf{d}_{\rho}(\mathcal{Z}_1, \alpha, \epsilon)$ . The same logic implies  $|I_2| \leq \mathsf{d}_{\rho}(\mathcal{Z}_2, \alpha, \epsilon)$ .
To get the monotonic version, note that if  $\epsilon^{*} = \arg \max_{\epsilon^{\prime}\geq \epsilon}\mathrm{d}_{\rho}(\mathcal{Z},\alpha ,\epsilon^{\prime})$
$$
\begin{array}{l} \bar {\mathsf {d}} _ {\rho} (\mathcal {Z}, \alpha , \epsilon) = \mathsf {d} _ {\rho} (\mathcal {Z}, \alpha , \epsilon^ {*}) \\ \leq \mathrm {d} _ {\rho} \left(\mathcal {Z} _ {1}, \alpha , \epsilon^ {*}\right) + \mathrm {d} _ {\rho} \left(\mathcal {Z} _ {2}, \alpha , \epsilon^ {*}\right) \\ \leq \bar {\mathrm {d}} _ {\rho} (\mathcal {Z} _ {1}, \alpha , \epsilon) + \bar {\mathrm {d}} _ {\rho} (\mathcal {Z} _ {2}, \alpha , \epsilon) \\ \end{array}
$$
where the first inequality is by the first statement of the lemma which was proved above, the next inequality is by definition of the monotonic dimension, and the last inequality follows by  $\epsilon \leq \epsilon^{*}$ .
We can now construct the classes that demonstrate the separation of the eluder and dissimilarity dimension.
We consider two overlapping semicircles, indexed by  $j \in \{0, 1\}$ , and defined as
$$
U _ {0} = \left\{(\cos x, \sin x): x \in \left(- \frac {\pi}{2}, \frac {\pi}{2}\right) \right\}, \text {a n d} U _ {1} = \left\{(\cos x, \sin x): x \in (0, \pi) \right\}.
$$
For each  $j\in \{0,1\}$ , and any  $N\in \mathbb{N}$  and  $\epsilon >0$ , we define the function class
$$
\mathcal {F} _ {j, N, \epsilon} := \left\{f _ {\mathbf {v}, S, \sigma}: \mathbf {v} \in \mathcal {C} \backslash U _ {j}, S \subseteq U _ {j}, | S | = N, \sigma \in \{\pm \epsilon \} ^ {S} \right\},
$$
containing functions
$$
f _ {\mathbf {v}, S, \sigma} (\mathbf {a}) = \left\{ \begin{array}{l l} 0 & \text {i f} \mathbf {a} \in U _ {j} \backslash S, \\ \sigma (\mathbf {a}) & \text {i f} \mathbf {a} \in S, \\ \langle \mathbf {v}, \mathbf {a} \rangle & \text {i f} \mathbf {a} \in \mathcal {C} \backslash U _ {j}. \end{array} \right. \tag {36}
$$
In words, the functions in the class  $\mathcal{F}_{j,N,\epsilon}$  are linear outside of the semicircle  $U_{j}$ , and zero in the semicircle  $U_{j}$ , except for a set of size  $N$ , where they can take any combination of values  $+\epsilon$  and  $-\epsilon$ . For any  $N \in \mathbb{N}$  and  $\epsilon > 0$ , we define the class  $\mathcal{F}_{N,\epsilon} := \bigcup_{j \in \{0,1\}} \mathcal{F}_{j,N,\epsilon}$  and show that this class has a constant dissimilarity dimension but its eluder dimension is at least  $N$ .
Finally, consider the action set  $\mathcal{A} = \mathcal{C}$  and the function class  $\mathcal{F}_{N,\epsilon}$  as defined above. Let  $\mathcal{Z}_{N,\epsilon} = \mathcal{F}_{N,\epsilon} \times \mathcal{A}$ ,  $\rho = \rho_{\mathrm{bandits}}$  and  $\epsilon \in (0,1/2)$ . We show that  $\dim_{\mathrm{E}}(\mathcal{F}_{N,\epsilon},\epsilon) \geq N$ , but  $\mathrm{d}_{\rho}(\mathcal{Z}_{N,\epsilon},1,\epsilon) \leq 16$ . First we prove the following lower bound on the eluder dimension of  $\mathcal{F}_{j,N,\epsilon}$ .
Lemma 29. The eluder dimension of  $\mathcal{F}_{j,N,\epsilon}$  satisfies  $\dim_{\mathrm{E}}(\mathcal{F}_{j,N,\epsilon},\epsilon)\geq N$  for all  $j\in \{0,1\}$ .
Proof. Let  $\mathbf{a}_1, \ldots, \mathbf{a}_N$  be an arbitrary set of points in  $U_j$  and consider the functions  $\{f_i\}_{i=1}^{N+1} \subseteq \mathcal{F}_{j,N,\epsilon,S,\mathbf{v}}$  that for all  $i, i' \leq N$  satisfy:
$$
f _ {i} (\mathbf {a} _ {i ^ {\prime}}) = \left\{ \begin{array}{l l} \epsilon & \text {i f i ^ {\prime} \neq i} \\ - \epsilon & \text {i f i ^ {\prime} = i}, \end{array} \right.
$$
and  $f_{N + 1}(\mathbf{a}_i) = \epsilon$  for all  $i\leq N$ . We now show that for all  $i\leq N$ , the action  $\mathbf{a}_i$  is  $\epsilon$ -independent of  $\mathbf{a}_1,\ldots ,\mathbf{a}_{i - 1}$  with respect to  $\mathcal{F}_{j,N,\epsilon ,S,\mathbf{v}}$ . This holds since  $\sqrt{\sum_{j = 1}^{i - 1}(f_i(\mathbf{a}_j) - f_{N + 1}(\mathbf{a}_j))^2} = 0$  while  $|f_{i}(\mathbf{a}_{i}) - f_{N + 1}(\mathbf{a}_{i})| = 2\epsilon >\epsilon$ . This finalizes the proof.
Lemma 30. Denote  $\mathcal{Z}_{N,\epsilon} = \mathcal{F}_{N,\epsilon}\times \mathcal{A}$  and let  $\epsilon \in (0,1 / 2)$ . Then,  $\mathtt{d}_{\rho}(\mathcal{Z}_{N,\epsilon},1,\epsilon)\leq 16$
Proof. Denote by  $\mathcal{Z}_{j,N,\epsilon} = \mathcal{F}_{j,N,\epsilon} \times \mathcal{A}$  for all  $j \in \{0,1\}$ . We start by showing that for all  $j \in \{0,1\}$ ,  $\mathrm{d}_{\rho}(\mathcal{Z}_{j,N,\epsilon}, 1,\epsilon) \leq 8$ . Let  $z_1, \ldots, z_d$  be a maximal sequence certifying the dissimilarity dimension  $\mathrm{d}_{\rho}(\mathcal{Z}_{N,\epsilon}, 1,\epsilon) \geq d$ , i.e., it holds that,
$$
\left| \rho \left(z _ {i} \mid z _ {i ^ {\prime}}\right) - c \right| \leq \frac {\epsilon}{\sqrt {d}} \text {f o r a l l} i <   i ^ {\prime}, \quad \text {w h i l e} \rho \left(z _ {i} \mid z _ {i}\right) \geq 1.
$$
Since  $\rho(z_i|z_i) \geq 1$ , it must be the case that  $z_i = (f_{\mathbf{v}_i, S_i, \sigma_i}, \mathbf{v}_i)$  with  $\mathbf{v}_i \notin U_j$  (since otherwise the self evaluations would be strictly less than 1). This implies that  $\rho(z_i|z_j) = \langle \mathbf{v}_i, \mathbf{v}_j \rangle$  for all  $i < j$ . Consequently, the score evaluations of all  $z_1, \ldots, z_d$  are equivalent to the score evaluations of the linear problem defined by  $\mathbf{v}_1, \ldots, \mathbf{v}_d$ . Thus Theorem 13 implies the maximum length of such a sequence can be of size at most  $2 \times 2 + 4 = 8$ . Finally the sub-additivity of the dissimilarity dimension (see Lemma 28) implies,
$$
\mathrm {d} _ {\rho} \left(\mathcal {Z} _ {N, \epsilon}, 1, \epsilon\right) \leq \sum_ {j = 0} ^ {1} \mathrm {d} _ {\rho} \left(\mathcal {Z} _ {j, N, \epsilon}, 1, \epsilon\right) \leq 1 6.
$$
Combining the results of Lemmas 29 and 30 finalizes the proof of Proposition 12.
D Multi-Armed Bandits
In this section we explore the dissimilarity dimension of the  $K$ -armed bandit problem. In this setting the learner interacts with a set of  $K$  arms and at every step of a sequential interaction pulls an arm  $a_{t} \in [K]$  and receives a reward  $r_{t}$  such that  $\mathbb{E}[r_t] = \mu_{i_t}$  where  $\mu_{a_t}$  is the mean reward of arm  $a_{t}$ . For simplicity we will assume  $\mu_{a} \in [0,1]$  for all  $a \in [K]$  and that  $|r_t| \leq 1$ .