SlowGuess/NeurIPS_2023_OCR huggingface dataset:
For standard linear functions,  $\alpha = 0.5$  and our regret bound in such situations is tight w.r.t.  $\alpha$ . However, we intentionally made assumption 1 to be more general in order to accommodate other much more complex ways of estimating the  $\phi$  functions such as manifold regression (see, e.g., Yang and Dunson (2016)), non-parametric ways like k-nearest-neighbors to estimate phi, or to estimate complex non-smooth function (e.g., functions in Holder spaces). For example, when  $\phi$  is a function in Holder space  $H(\beta)$ , then the learning rate is  $T^{-2\beta/(2\beta+d)}$ , which is generally slower than  $T^{-0.5}$  and depends on  $\beta$  as well as the data dimension  $d$  (see, e.g., the note of Tibshirani (2017)). A visual illustration can be found in Figure 3.