SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Theorem 6 (Regret). Let  $\delta, T > 0$ , and let  $d = \overline{\mathsf{d}}_{\rho}(\mathcal{Z}, \alpha, 1 / T)$  for some set  $\mathcal{Z}$ , evaluation function  $\rho$  and  $\alpha \leq \alpha^{*}$ . Let  $\mathsf{Alg}$  be an interactive estimation algorithm with  $\alpha$ -large self-evaluations and a decaying estimation error with some  $C_{T,\delta}$ . If  $C_{T,\delta} \geq \ln(2T)$  then with probability at least  $1 - \delta$ , the regret of  $\mathsf{Alg}$  satisfies
$$
\operatorname {R e g r e t} (T, \alpha) \leq 1 + 1 2 d ^ {1. 2 5} \sqrt {C _ {T , \delta} T}.
$$
In the deterministic setting,  $\mathrm{Regret}(T,\alpha)\leq 1 + 12d^{1.5}$
For an algorithm with a decaying estimation error, the term  $C_{T,\delta}$  is sublinear in  $T$ , implying a sublinear regret in Theorem 6. For example, Algorithm 1 has a decaying estimation error with  $C_{T,\delta}$  that scales logarithmically with  $T / \delta$  for many standard function classes, and so the overall regret scales as  $O(\sqrt{T\log T})$  (see Corollary 18 in Appendix A.1).
To derive PAC generalization guarantees, we apply a variant of online-to-batch reduction to any algorithm with large self-evaluations and a decaying estimation error. The resulting approach, shown in Algorithm 2, satisfies the following guarantee (proved in Appendix A.7):
Theorem 7 (PAC generalization). Let  $\epsilon, \delta > 0$ , and let  $d = \overline{\mathsf{d}}_{\rho}(\mathcal{Z}, \alpha, \epsilon)$  for some set  $\mathcal{Z}$ , evaluation function  $\rho$  and  $\alpha \leq \alpha^{*}$ . Let  $\mathsf{Alg}$  be an interactive estimation algorithm with  $\alpha$ -large self-evaluations and a decaying estimation error with  $C_{T,\delta} \geq \ln(2T)$ , and suppose that we run Algorithm 2 with  $\mathsf{Alg}$  as the base algorithm,  $T \geq 64d^{2.5}(C_{T,\delta/2})/\epsilon^2$ ,  $n_1 = \lceil \log_2(4/\delta) \rceil$ , and  $n_2 = \lceil 128 \ln(8n_1/\delta)/\epsilon^2 \rceil$ . Then, with probability at least  $1 - \delta$ , the output  $\hat{z} \in \mathcal{Z}$  satisfies
$$
\rho (\hat {z} \mid z ^ {*}) \geq \alpha - \epsilon ,
$$
and the overall number of issued queries is  $O\left(\frac{d^{2.5}(C_{T,\delta / 2}) + \ln^2(1 / \delta)}{\epsilon^2}\right)$ .
In the deterministic setting, it suffices to run  $\mathsf{Alg}$  with  $T > 2d^{1.5}\ln (4 / \epsilon)$  and return  $\hat{z} = z_{\hat{t}}$  where  $\hat{t} = \operatorname{argmax}_{t\in \{1,\dots,T\}}r_t$  is the index of the largest observed reward. Then, with probability 1, we obtain  $\rho (\hat{z}\mid z^{*})\geq \alpha -\epsilon$  and issue at most  $O(d^{1.5}\ln (4 / \epsilon))$  queries.
Unknown  $\alpha^{*}$  and optimism. Algorithms 1 and 2 achieve performance guarantees with respect to a provided optimality level  $\alpha \leq \alpha^{*}$ . When it is not easy to provide a non-trivial  $\alpha$  (for example, when  $\alpha^{*}$  is unknown and cannot be non-trivially bounded), Algorithm 3 uses the optimistic least squares algorithmic template (see, e.g., [24]) to ensure  $\alpha^{*}$ -large self-evaluations with high probability and to achieve a sublinear  $\mathrm{Regret}(T,\alpha^{*})$ . Algorithm 3 takes as input a confidence radius parameter  $R$  of the same order as the decaying estimation error parameter  $C_{T,\delta}$  for Algorithm 1. We can then show that  $z^{*} \in \mathcal{Z}_{t}$  with high probability for all  $t \in \{1,\dots,T\}$ . Therefore,  $z_{t}$  must satisfy
Algorithm 3 Optimistic Interactive Estimation via Least Squares
1: Input: set of alternatives  $\mathcal{Z}$ , evaluation function  $\rho$ , number of steps  $T$ , confidence-set radius  $R$ .
2: for  $t = 1, \dots, T$  do
3: Compute confidence set
$$
\hat {z} _ {t} = \underset {z \in \mathcal {Z}} {\operatorname {a r g m i n}} \sum_ {i = 1} ^ {t - 1} \left(\rho (z _ {i} \mid z) - r _ {i}\right) ^ {2},
$$
$$
\mathcal {Z} _ {t} = \left\{z \in \mathcal {Z}: \sum_ {i = 1} ^ {t - 1} \left(\rho \left(z _ {i} \mid z\right) - \rho \left(z _ {i} \mid \hat {z} _ {t}\right)\right) ^ {2} \leq R \right\}.
$$
4: Submit the query  $z_{t} = \operatorname{argmax}_{z\in \mathcal{Z},}\rho (z\mid z)$ .
5: Observe reward  $r_t$ .
6: end for
$\rho (z_t|z_t)\geq \rho (z^* |z^*) = \alpha^*$ . In Appendix A.1 we show this modified version of the algorithm satisfies the decaying estimation error property. This technique allows us to achieve a sublinear  $\operatorname{Regret}(T,\alpha^{*})$  without knowing  $\alpha^{*}$  beforehand. Similar to the case of fixed  $\alpha$ , it is possible to derive a version of Algorithm 3 that leverages an online regression oracle. (See Appendix A.3 for details.)
4 Statistical queries
In this section we consider the statistical query (SQ) model, as defined in Example 3. In particular, we study the connection between our generalized framework and SQ learning, showing specifically that the dissimilarity dimension can be used to recover generalization bounds based on a known combinatorial parameter that characterizes SQ learning, called the strong SQ dimension. There are several notions of such a dimension [12, 25]. Here we focus on the one due to Szörenyi [25]:
Definition 4 (Strong SQ dimension, [25]). For a fixed distribution  $D$  over  $\mathcal{X}$ , the strong SQ dimension of a hypothesis class  $\mathcal{H} \subseteq \{\pm 1\}^{\mathcal{X}}$  with respect to some  $\epsilon > 0$ , denoted  $\dim_{\mathrm{SQ}}(\mathcal{H}, \epsilon)$ , is the largest number  $d$  for which there exist  $h_1, \ldots, h_d \in \mathcal{H}$  such that:
(a)  $|\langle h_i,h_j\rangle |\leq 1 - \epsilon$  for all  $1\leq i <   j\leq d,$  and
(b)  $|\langle h_i,h_j\rangle -\langle h_{i'}',h_{j'}\rangle |\leq \frac{1}{d}$  forall  $1\leq i <   j\leq d$ $1\leq i^{\prime} <   j^{\prime}\leq d,$
where  $\langle h,h^{\prime}\rangle \coloneqq \mathbb{E}_{x\sim D}[h(x)h^{\prime}(x)]$
The dissimilarity and strong SQ dimensions are closely related to one another in the sense of each providing a kind of polynomial bound on the other, as stated in the next proposition (see Appendix B.1 for the proof).
Proposition 8. Let  $D$  be a fixed distribution over  $\mathcal{X}$ , and let  $\mathcal{H} \subseteq \{\pm 1\}^{\mathcal{X}}$  be a hypotheses class. For  $\epsilon > 0$ , let  $d_{\mathrm{SQ}}(\epsilon) = \dim_{\mathrm{SQ}}(\mathcal{H}, \epsilon)$ , and let  $d_{\rho}(\epsilon) = d_{\rho_{\mathrm{SQ}}}(\mathcal{H}, 1, \epsilon)$ .
If  $d_{\rho}(\epsilon)\geq 2$  then
$$
\left. \min  \left\{d _ {\mathrm {S Q}} (\epsilon), \left\lfloor 4 \epsilon^ {2} \left(d _ {\mathrm {S Q}} (\epsilon)\right) ^ {2} \right\rfloor \right\} \leq d _ {\rho} (\epsilon) \leq \max  \left\{d _ {\mathrm {S Q}} (\epsilon / 4), 4 \epsilon^ {2} \left(d _ {\mathrm {S Q}} (\epsilon / 4) + 1\right) ^ {2} \right\}. \right. \tag {3}
$$
Similarly, if  $d_{\rho}(4\epsilon) \geq 2$  then
$$
\min  \left\{d _ {\rho} (4 \epsilon), \left\lfloor \frac {\sqrt {d _ {\rho} (4 \epsilon)}}{8 \epsilon} \right\rfloor \right\} \leq d _ {\mathrm {S Q}} (\epsilon) \leq \max  \left\{d _ {\rho} (\epsilon), \frac {\sqrt {d _ {\rho} (\epsilon) + 1}}{2 \epsilon} \right\}. \tag {4}
$$
We next give a lower bound based on the strong SQ dimension, which together with Proposition 8 will allow us to lower bound sample complexity of any interactive estimation algorithm in the SQ setting in terms of the dissimilarity dimension.
Theorem 9 (SQ lower bound). Let  $\epsilon > 0$ , and let  $\mathcal{H} \subseteq \{\pm 1\}^{\mathcal{X}}$  be a hypothesis class with strong SQ dimension  $d_{\mathrm{SQ}} = \dim_{\mathrm{SQ}}(\mathcal{H}, 2\epsilon) \geq 11$ . Let  $\mathsf{Alg}$  be any interactive estimation algorithm with the property that for any target  $h^{*} \in \mathcal{H}$ ,  $\mathsf{Alg}$  outputs an  $\epsilon$ -approximation to  $h^{*}$  with probability at least  $2/3$  using at most  $m$  queries. Then  $m > \sqrt[3]{d_{\mathrm{SQ}}} / 12$ .
The proof relies on a reduction to a lower bound of Szörenyi [25]. However, the lower bound of Szörenyi [25] holds within an SQ model that differs from ours, in that it allows adversarial query responses. Therefore, we first need to show how to obtain a learning algorithm  $\mathsf{Alg}^{\prime}$  that can be used with an adversarial oracle from an interactive estimation algorithm  $\mathsf{Alg}$  that uses an unbiased stochastic query oracle (as we assume in this work). To do this, we apply the reduction technique developed by Feldman et al. [13]. (See Appendix B.2 for the full proof and additional details.)
Combining Theorem 9 and Proposition 8 yields a lower bound on the sample complexity of interactive estimation in the SQ setting, for a sufficiently small  $\epsilon$ , in terms of the dissimilarity dimension:
Corollary 10. Let  $\epsilon > 0$ , and let  $\mathcal{H} \subseteq \{\pm 1\}^{\mathcal{X}}$  be a hypothesis class with strong SQ dimension  $\dim_{\mathrm{SQ}}(\mathcal{H}, 2\epsilon) \geq 11$ . Let  $d_{\rho}(\epsilon) = d_{\rho_{\mathrm{SQ}}}(\mathcal{H}, 1, \epsilon)$ . Assume  $\epsilon \leq 1 / (2\sqrt{d_{\rho}(\epsilon)})$ . Let  $\mathsf{Alg}$  be any interactive estimation algorithm with the property that for any target  $h^{*} \in \mathcal{H}$ ,  $\mathsf{Alg}$  outputs an  $\epsilon$ -approximation to  $h^{*}$  with probability at least  $2/3$  using at most  $m$  queries. Then  $m > \sqrt[3]{d_{\rho}(\epsilon)} / 12$ .
5 Bandits
In this section we focus on the bandits setting described in Example 2. We study the relationship between the dissimilarity dimension and the eluder dimension [24], a common combinatorial dimension for bounding regret of bandit algorithms. We show that eluder dimension can be used to upper bound the dissimilarity dimension, and we also highlight the cases when dissimilarity dimension leads to a tighter analysis.
Throughout this section we follow the setup introduced in Example 2. We consider an action set  $\mathcal{A}$ , a class  $\mathcal{F}$  of reward functions  $f: \mathcal{A} \to [-1,1]$ , and a target reward function  $f^{*} \in \mathcal{F}$ . We map this to our setting by considering the set of alternatives  $\mathcal{Z} = \mathcal{F} \times \mathcal{A}$ , evaluation function  $\rho_{\mathrm{bandits}}((f,a) \mid (f',a')) = f'(a)$  and the target  $(f^{*},a^{*})$ , where  $a^{*} = \operatorname{argmax}_{a \in \mathcal{A}} f^{*}(a)$ .
5.1 Comparison with eluder dimension
We start by describing the relationship between our dimension and the eluder dimension. Following Russo and Van Roy [24], we define  $\epsilon$ -dependence and  $\epsilon$ -eluder dimension as follows:
Definition 5 ( $\epsilon$ -dependence). An action  $a \in \mathcal{A}$  is  $\epsilon$ -dependent on actions  $\{a_1, \ldots, a_n\} \subseteq \mathcal{A}$  with respect to  $\mathcal{F}$  if any pair of functions  $f, f' \in \mathcal{F}$  satisfying  $\sqrt{\sum_{i=1}^{n}(f(a_i) - f'(a_i))^2} \leq \epsilon$  also satisfies  $|f(a) - f'(a)| \leq \epsilon$ . Furthermore, an action  $a$  is  $\epsilon$ -independent of  $\{a_1, \ldots, a_n\}$  with respect to  $\mathcal{F}$  if it is not  $\epsilon$ -dependent on  $\{a_1, \ldots, a_n\}$ .
Definition 6 (ε-eluder dimension). The ε-eluder dimension  $\dim_{\mathrm{E}}(\mathcal{F},\epsilon)$  is the length  $d$  of the longest sequence of elements in  $\mathcal{A}$  such that every element is ε-independent of its predecessors. Moreover, the monotone eluder dimension is defined as  $\overline{\dim}_{\mathrm{E}}(\mathcal{F},\epsilon)\coloneqq \max_{\epsilon '\geq \epsilon}\dim_{\mathrm{E}}(\mathcal{F},\epsilon ')$ .
The next theorem shows that the dissimilarity dimension is upper bounded by the eluder dimension (see Appendix C.1 for a proof):
Theorem 11. Let  $\mathcal{Z} = \mathcal{F}\times \mathcal{A}$ ,  $\rho = \rho_{\mathrm{bandits}}$ ,  $\epsilon >0$ ,  $\alpha \leq \alpha^{*}$ . Then  $\bar{\mathrm{d}}_{\rho}(\mathcal{Z},\alpha ,3\epsilon /2)\leq 9\overline{\mathrm{d}\mathrm{i}\mathrm{m}}_{\mathrm{E}}(\mathcal{F},\epsilon)$ .
Nevertheless, as the next example shows, the eluder dimension can be arbitrarily large, while the dissimilarity dimension remains constant. In this example, the action set is a circle in  $\mathbb{R}^2$ , that is,  $\mathcal{A} = \mathcal{C} \coloneqq \{\mathbf{v} \in \mathbb{R}^2 : \| \mathbf{v} \| = 1\}$ . We fix two open semicircles  $U_0, U_1 \subseteq \mathcal{C}$  with positive  $x$  and  $y$  coordinates, respectively, and for any  $N \in \mathbb{N}$  and  $\epsilon > 0$ , construct a function class  $\mathcal{F}_{N,\epsilon}$  with all the functions  $f: \mathcal{A} \to [-1,1]$  obtained by the following process. First, pick one of the semicircles  $U_j$  and any  $N$  points from  $U_j$ . On each of these points,  $f$  can equal either  $+ \epsilon$  or  $-\epsilon$ . Everywhere else in  $U_j$ ,  $f$  equals zero, and everywhere outside  $U_j$ , it equals the linear function  $\langle \mathbf{v}, \mathbf{a} \rangle$  parameterized by some  $\mathbf{v} \in \mathcal{C} \setminus U_j$ . Thus, the functions are constructed to be "simple" (namely, linear) near the optimal action  $\mathbf{v}$ , but complex far from it. The eluder dimension is large to capture overall complexity, whereas the dissimilarity dimension is small to capture the simplicity near the optimum. (See Appendix C.5 for the formal construction of  $\mathcal{F}_{N,\epsilon}$  and the proof of Proposition 12.)
Proposition 12. Let  $\epsilon \in (0,1/2)$ ,  $N \in \mathbb{N}$  and consider the action set  $\mathcal{A} = \mathcal{C}$ . Then, there is a function class  $\mathcal{F}_{N,\epsilon} \subseteq [-1,1]^{\mathcal{A}}$ , such that for  $\mathcal{Z}_{N,\epsilon} := \mathcal{F}_{N,\epsilon} \times \mathcal{A}$ ,  $\rho = \rho_{\mathrm{bandits}}$ , it holds that  $\mathrm{d}_{\rho}(\mathcal{Z}_{N,\epsilon},1,\epsilon) \leq 16$ , but the eluder dimension is lower bounded as  $\dim_{\mathrm{E}}(\mathcal{F}_{N,\epsilon},\epsilon) \geq N$ .
Thus, our regret bound based on the dissimilarity dimension implies that (optimistic) least squares algorithms have a regret independent of  $N$ . The same analysis with the eluder dimension [24] yields
a regret bound scaling polynomially with  $N$ . This shows that in the cases when the function classes are simple near the optimum, but complex far from it, the dissimilarity dimension can better capture the statistical complexity of bandit optimization than the eluder dimension.
5.2 Dissimilarity dimension bounds
We next derive dissimilarity dimension bounds for several standard bandit classes. Existing bounds on eluder dimension can be used to immediately bound the dissimilarity dimension, but in several cases we are able to obtain tighter bounds.
We first consider linear bandits. Let  $\mathcal{B}_n = \{\mathbf{v} \in \mathbb{R}^n : \| \mathbf{v} \| \leq 1\}$  be the unit ball in  $\mathbb{R}^n$ . Actions are chosen from a set  $\mathcal{A} \subseteq \mathcal{B}_n$ ; the reward function class is  $\mathcal{F}^{lb} = \{f_\theta : \theta \in \Theta\}$ , where  $\Theta \subseteq \mathcal{B}_n$  and  $f_\theta(\mathbf{a}) = \langle \theta, \mathbf{a} \rangle$ . The corresponding set of alternatives is denoted  $\mathcal{Z}^{lb} = \mathcal{F}^{lb} \times \mathcal{A}$ . In this case we obtain the following bound (see Appendix C.2 for a proof):
Theorem 13 (Linear bandits). Let  $\mathcal{Z}^{lb}$  be as defined above, let  $\rho = \rho_{\mathrm{bandits}}$ , and let  $\epsilon > 0$ ,  $\alpha \leq \alpha^*$ . Then  $\mathsf{d}_{\rho}(\mathcal{Z}^{lb}, \alpha, \epsilon) \leq 4n + 3$ . Moreover, when  $\alpha = 1$ , then  $\mathsf{d}_{\rho}(\mathcal{Z}^{lb}, \alpha, \epsilon) \leq 2n + 1$ .
The proof proceeds by deriving an upper bound as well as a lower bound on the rank of the matrix  $\mathbf{M}$  with entries  $M_{ij} = \rho (z_i\mid z_j) - c$  obtained from elements  $z_{1},\ldots ,z_{d}$  that satisfy the dimension condition for  $d = \mathfrak{d}_{\rho}(\mathcal{Z},\alpha ,\epsilon)$  with a scalar  $c$ . The upper bound on the rank is  $n + 1$ , and the lower bound is  $d / 4$  (which can be tightened to  $d / 2$  when  $\mathbf{M}$  is symmetric). The upper bound is obtained by basic linear algebra and the lower bound from a standard result on ranks of perturbed identity matrices [2, Lemma 2.2]. Combining these bounds then yields the claim of Theorem 13. Similar to existing bounds on eluder dimension [24, Proposition 6], our bound in Theorem 13 is linear in  $n$ . However, the eluder dimension bound has an additional dependence on  $1 / \epsilon$ , while our bound does not.
Next, we consider generalized linear model (GLM) bandits. Similar to linear bandits, the action set is  $\mathcal{A} \subseteq \mathcal{B}_n$ , but the function class includes a nonlinearity. Specifically, we are provided with a function  $g: \mathbb{R} \to \mathbb{R}$  that is differentiable and strictly increasing, and consider the function class  $\mathcal{F}^{glm} = \{f_{\boldsymbol{\theta}}: \boldsymbol{\theta} \in \Theta\}$  where  $\Theta \subseteq \mathcal{B}_n$  and  $f_{\boldsymbol{\theta}}(\mathbf{a}) = g(\langle \boldsymbol{\theta}, \mathbf{a} \rangle)$ . Furthermore, we assume that there are  $\underline{h}, \overline{h} > 0$  such that for all  $\mathbf{a} \in \mathcal{A}$ ,  $\boldsymbol{\theta} \in \Theta$ , we have  $\underline{h} \leq g'(\langle \boldsymbol{\theta}, \mathbf{a} \rangle) \leq \overline{h}$ . Define  $r = \overline{h}/\underline{h}$ . We again denote  $\mathcal{Z}^{glm} = \mathcal{F}^{glm} \times \mathcal{A}$ . Using an existing bound on the eluder dimension for GLM bandits ([24], Proposition 7) and the fact that our dimension is bounded by the eluder dimension (Theorem 11), we obtain the following bound (see Appendix C.3 for a proof):
Theorem 14 (GLM bandits). Let  $\mathcal{Z}^{glm}$  be as defined above, let  $\rho = \rho_{\mathrm{bandits}}$ , and let  $\epsilon >0$ ,  $\alpha \leq \alpha^{*}$ . Then  $\mathsf{d}_{\rho}(\mathcal{Z}^{glm},\alpha ,\epsilon)\leq O(nr^2\log (\overline{h} /\epsilon))$
By considering a different proof technique, along the lines of Theorem 13, it might be possible to tighten this bound. We leave this extension for future work.
Next, we consider a bandit setting that is similar to GLMs, but in this case the non-linearity is provided by the non-differentiable rectified linear unit (ReLU) activation function  $relu(x) = \max \{x,0\}$ . We consider the action set  $\mathcal{A} = \mathcal{B}_n$ , and the set of reward functions  $\mathcal{F}^{relu}$  consisting of all functions of the form  $f_{\theta ,b}(\mathbf{a}) = relu(\langle \pmb {\theta},\mathbf{a}\rangle -b)$  for some  $\pmb {\theta}\in \mathcal{B}_n$  and  $b\in [0,1)$ . The subset of  $\mathcal{F}^{relu}$  with a fixed value of  $b$  is denoted  $\mathcal{F}_b^{relu}$ , and we consider the set of alternatives  $\mathcal{Z}_b^{relu} = \mathcal{F}_b^{relu}\times \mathcal{B}_n$ .
Unlike the classes considered above, this setting can be shown to be challenging to learn in the general case. Indeed, it turns out that eluder dimension (as well as a related measure called star dimension) is growing at least exponentially with  $n$  [21, 10]. The same lower bound can be shown for the dissimilarity dimension by a similar proof technique. The following theorem also provides an exponential upper bound, showing that in certain regimes the exponential dependence is tight (see Appendix C.4 for a proof):
Theorem 15 (ReLU bandits). Let  $\mathcal{Z}^{relu}$  be as defined above, let  $\rho = \rho_{\mathrm{bandits}}$ , and let  $\epsilon, b > 0$  such that  $b \leq 1 - \epsilon$ . Then  $\mathsf{d}_{\rho}(\mathcal{Z}_b^{relu}, 1 - b, \epsilon) = O\left(\epsilon^{-n/2}\right)$ , and  $\mathsf{d}_{\rho}(\mathcal{Z}_{1-\epsilon}^{relu}, \epsilon, \epsilon) = \Omega\left(\epsilon^{-n/2}\right)$ .
We note that previous work ([10], Theorem 5.1) has shown that for a function class of one-layer neural networks with ReLU activations, obtaining sublinear regret requires  $T = \Omega(\epsilon^{-(n-2)})$ .
In this paper, we have introduced a new model for interactive estimation and proposed a new combinatorial dimension, called dissimilarity dimension, to study the hardness of learning in this
model. In (stochastic, correlational) statistical query learning, our dimension is polynomially related to the strong SQ dimension. In bandits, our dimension is upper bounded by the eluder dimension, and there are examples where the dissimilarity dimension leads to much tighter regret bounds.
While this work provides an initial investigation of the dissimilarity dimension, many open questions remain. For example, our regret bound for the general setting scales as  $d^{1.25}$ . Is it possible to tighten this to linear dependence, as is the case, for example, for eluder dimension? On the algorithmic side, we currently require solving a least squares problem of size  $t$  in iteration  $t$ . Although we also introduce an algorithm that leverages an online regression oracle (see Appendix A.3), the oracle-based approach still requires solving a least squares problem (on the data smoothed by the oracle). Is it possible to derive dissimilarity-dimension-based regret bounds directly for the predictions produced by the oracle? Ultimately, we hope investigations of relationships between dissimilarity dimension and related notions may help us understand the hardness of learning in interactive settings.
Aldo Pacchiano would like to thank the support of the Eric and Wendy Schmidt Center at the Broad Institute of MIT and Harvard. This work was supported in part by funding from the Eric and Wendy Schmidt Center at the Broad Institute of MIT and Harvard.
[1] Alekh Agarwal, Daniel Hsu, Satyen Kale, John Langford, Lihong Li, and Robert Schapire. Taming the monster: A fast and simple algorithm for contextual bandits. In International Conference on Machine Learning, pages 1638-1646. PMLR, 2014.
[2] Noga Alon. Perturbed identity matrices have high rank: Proof and applications. Combinatorics, Probability and Computing, 18(1-2):3-15, 2009.
[3] Maria-Florina Balcan and Avrim Blum. On a theory of learning with similarity functions. In Proceedings of the 23rd international conference on Machine learning, pages 73–80, 2006.
[4] Shai Ben-David and Eli Dichterman. Learning with restricted focus of attention. Journal of Computer and System Sciences, 56(3):277-298, 1998.
[5] Shai Ben-David, Alon Itai, and Eyal Kushilevitz. Learning by distances. Information and Computation, 117(2):240-250, March 1995.
[6] Alina Beygelzimer, John Langford, Lihong Li, Lev Reyzin, and Robert Schapire. Contextual bandit algorithms with supervised learning guarantees. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pages 19-26. JMLR Workshop and Conference Proceedings, 2011.
[7] Nader H Bshouty and Vitaly Feldman. On using extended statistical queries to avoid membership queries. Journal of Machine Learning Research, 2(Feb):359-395, 2002.
[8] Yihua Chen, Eric K Garcia, Maya R Gupta, Ali Rahimi, and Luca Cazzanti. Similarity-based classification: Concepts and algorithms. Journal of Machine Learning Research, 10(3), 2009.
[9] Thomas M Cover. Universal portfolios. Mathematical finance, 1(1):1-29, 1991.
[10] Kefan Dong, Jiaqi Yang, and Tengyu Ma. Provable model-based nonlinear bandit and reinforcement learning: Shelve optimism, embrace virtual curvature. Advances in Neural Information Processing Systems, 34:26168-26182, 2021.
[11] Simon Du, Sham Kakade, Jason Lee, Shachar Lovett, Gaurav Mahajan, Wen Sun, and Ruosong Wang. Bilinear classes: A structural framework for provable generalization in rl. In International Conference on Machine Learning, pages 2826-2836. PMLR, 2021.
[12] Vitaly Feldman. A complete characterization of statistical query learning with applications to evolvability. Journal of Computer and System Sciences, 78(5):1444-1459, 2012.
[13] Vitaly Feldman, Elena Grigorescu, Lev Reyzin, Santosh S Vempala, and Ying Xiao. Statistical algorithms and a lower bound for detecting planted cliques. Journal of the ACM (JACM), 64(2): 1-37, 2017.