SlowGuess/NeurIPS_2023_OCR huggingface dataset:
By specifying the maximum order  $K$ , we obtain  $2K$  features  $\{h_{t,2k-1}, h_{t,2k}\}_{k \in [1:K]}$  from this construction. An all-one feature is further added, making the dictionary size  $N = 2K + 1$ .
Again, we set the confidence hyperparameter  $\varepsilon = 1$  for our algorithm. The total loss as a function of the dictionary size  $N$  is plotted in Figure 6. Notably, the case of  $N = 0$  is equivalent to trivially following the advice of the given forecaster  $\mathcal{A}$ :  $x_{t} = a_{t} = z_{t - 1}$ . It can be seen that our fine-tuning framework ( $N > 0$ ) actually results in better performance, due to exploiting the structures in the error sequence  $z_{1:T} - a_{1:T}$ .
Figure 6: Testing our algorithm for temperature forecasting.
We also test the fine-tuning performance of the algorithm from [JC22]. Same as the above, we set  $\varepsilon = 1$ . The total loss achieved by this alternative algorithm is 8238, which is around the same as  $\mathcal{A}$  itself, and significantly higher than the total loss of our algorithm with moderate amount of features  $(N > 5)$ . This fits the intuition from this paper: the environment contains persistent dynamics, which the algorithm from [JC22] cannot handle.
F Additional discussion
MRA in online learning On a broader scope, wavelets embody the idea of Multi-Resolution Analysis (MRA), which is reminiscent of the classical geometric covering (GC) construction in adaptive online learning [DGSS15]. Such a construction starts from a class of GC time intervals, which are equivalent to the support of Haar wavelet features. On each GC interval, a static online learning algorithm is defined (corresponding to using an all-one feature, c.f., Section 2.2); and then, the outputs of these "local" algorithms are aggregated by a sleeping expert algorithm on top [LS15, JOWW17]. Algorithmically, our innovation is introducing sign changes in the features, accompanied by a different, additive way to aggregate base algorithms. For tackling nonstationarity, both approaches have their own strengths: the GC construction can produce strongly adaptive guarantees on subintervals of the time horizon, while our algorithm does not need a bounded domain. Their possible connections are intriguing.
Lipschitz vs strongly convex losses We also comment on the choice of loss functions in unconstrained dynamic OCO. Besides the Lipschitz assumption we impose, a fruitful line of works by Baby and Wang [BW19, BW20, BW21, BZW21, BW22] considered an alternative setting with strong convexity, motivated by the prevalence of the square loss in statistics. Their focus is primarily on bounded domains, as [BW19] showed that evaluated under the square loss, a lower bound for the unconstrained dynamic regret is  $\Omega(P^2)$ . A sublinear regret bound here requires  $P = o(\sqrt{T})$ , rather than  $P = o(T)$  with Lipschitz losses – that is, the environment is required to be “more static” than the typical requirement in the Lipschitz setting.
Essentially, such a behavior is due to the large penalty that the square loss imposes on outliers. An adversary in online learning can deliberately pick the loss functions such that some of the player's predictions are large outliers with "huge" (square) losses, while the offline optimal comparator sequence suffers zero losses. Using the Lipschitz losses instead may offer an advantage on unbounded domains, due to being more tolerant to these outliers. Furthermore, Lipschitz losses do not necessarily have minimizers – this is useful for decision problems (as opposed to estimation), where a ground truth may not exist.[17]
Future work For future works, several interesting questions could stem from this paper. For example,
- Our regret bound is stated against individual comparator sequences. One could investigate the implication of this result in stochastic environments, where the comparator statistics may take more concrete forms.
- Besides the sparsity and the energy studied in this paper, an interesting open problem is investigating alternative complexity measures of the comparator, possibly drawing connections to statistical learning theory.
- Our framework builds on pre-defined dictionary inputs. The quantitative benefit of using a data-dependent dictionary is unclear.
- Beyond wavelets, one may investigate the combination of the sparse coding framework with other function approximators, such as neural networks.
An example is financial investment without budget constraints: doubling the invested amount also doubles the return.