SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where  $C_5$  is an absolute constant concludes the parametric confidence bound. The upper confidence bound then simply follows: for any  $\pmb{x} \in \mathcal{X}$
$$
r (\pmb {x}) - \hat {\beta} _ {t, j ^ {\star}} ^ {\top} \phi_ {j ^ {\star}} (\pmb {x}) = \langle \pmb {\theta} _ {j ^ {\star}} - \hat {\beta} _ {t, j ^ {\star}}, \phi_ {j ^ {\star}} (\pmb {x}) \rangle \leq \left\| \pmb {\theta} _ {j ^ {\star}} - \hat {\beta} _ {t, j ^ {\star}} \right\| _ {2} \| \phi_ {j ^ {\star}} (\pmb {x}) \| _ {2} \leq \omega (t, \delta , d)
$$
where the last inequality holds with high probability simultaneously for all  $t \geq 1$ .
Proof of Lemma 15. Using Proposition 21 and the Cauchy-Schwarz inequality we obtain,
$$
\begin{array}{l} \tilde {R} _ {j ^ {\star}} (n) = \sum_ {t = 1} ^ {n} r \left(\boldsymbol {x} ^ {\star}\right) - r \left(\tilde {\boldsymbol {x}} _ {t, j}\right) \\ = \sum_ {t = 1} ^ {n} r (\boldsymbol {x} ^ {\star}) - \hat {r} _ {t} (\boldsymbol {x} ^ {\star}) + \hat {r} _ {t} (\boldsymbol {x} ^ {\star}) - \hat {r} _ {t} (\tilde {\boldsymbol {x}} _ {t, j}) + \hat {r} _ {t} (\tilde {\boldsymbol {x}} _ {t, j}) - r (\tilde {\boldsymbol {x}} _ {t, j}) \\ \leq \sum_ {t = 1} ^ {n} \left\| \boldsymbol {\theta} _ {j} - \hat {\boldsymbol {\theta}} _ {t, j} \right\| _ {2} \left(\| \phi_ {j} (\boldsymbol {x} ^ {\star}) \| _ {2} + \| \phi_ {j} (\tilde {\boldsymbol {x}} _ {t, j}) \| _ {2}\right) + \hat {r} _ {t} (\boldsymbol {x} ^ {\star}) - \hat {r} _ {t} (\tilde {\boldsymbol {x}} _ {t, j}) \\ \leq \sum_ {t = 1} ^ {n} \omega (t, \delta , d) \left(\| \phi_ {j} (\boldsymbol {x} ^ {\star}) \| _ {2} + \| \phi_ {j} (\tilde {\boldsymbol {x}} _ {t, j}) \| _ {2}\right) + \hat {r} _ {t} (\boldsymbol {x} ^ {\star}) - \hat {r} _ {t} (\tilde {\boldsymbol {x}} _ {t, j}) \\ \end{array}
$$
with probability  $1 - \delta$ . If the agent selects actions greedily, then  $\hat{r}_t(\pmb{x}^\star) \leq \hat{r}_t(\tilde{\pmb{x}}_{t,j})$ , and
$$
\tilde {R} _ {j ^ {\star}} (n) \leq \sum_ {t = 1} ^ {n} \omega (t, \delta , d) \left(\| \phi_ {j} (\boldsymbol {x} ^ {\star}) \| _ {2} + \| \phi_ {j} (\tilde {\boldsymbol {x}} _ {t, j}) \| _ {2}\right) \leq \sum_ {t = 1} ^ {n} 2 \omega (t, \delta , d)
$$
since the feature map is normalized to satisfy  $\| \phi_j(\cdot)\| \leq 1$ . If the agent selects actions optimistically according to the upper confidence bound of Proposition 21, then
$$
\hat {r} _ {t} (\tilde {\boldsymbol {x}} _ {t, j}) + \omega (t, \delta , d) \| \phi_ {j} (\tilde {\boldsymbol {x}} _ {t, j}) \| \geq \hat {r} _ {t} (\boldsymbol {x} ^ {\star}) + \omega (t, \delta , d) \| \phi_ {j} (\boldsymbol {x} ^ {\star}) \|
$$
$$
\hat {r} _ {t} (\boldsymbol {x} ^ {\star}) - \hat {r} _ {t} (\tilde {\boldsymbol {x}} _ {t, j}) \leq \omega (t, \delta , d) \| \phi_ {j} (\tilde {\boldsymbol {x}} _ {t, j}) \| - \omega (t, \delta , d) \| \phi_ {j} (\boldsymbol {x} ^ {\star}) \|
$$
$$
\tilde {R} _ {j ^ {\star}} (n) \leq \sum_ {t = 1} ^ {n} 2 \omega (t, \delta , d) \| \phi_ {j} (\tilde {\boldsymbol {x}} _ {t, j}) \| _ {2} \leq \sum_ {t = 1} ^ {n} 2 \omega (t, \delta , d).
$$
Then due to Proposition 21, with probability greater than  $1 - \delta$ , simultaneously for all  $n \geq 1$
$$
\begin{array}{l} \sum_ {t = 1} ^ {n} \omega (t, \delta , d) \leq \sum_ {t = 1} ^ {n} C _ {1} \sqrt {\frac {\sigma^ {2} d \log \left(\frac {t}{\tilde {\lambda} d} + 1\right) + 2 \sigma^ {2} \log (1 / \delta) + \tilde {\lambda} B ^ {2}}{\tilde {\lambda} + C _ {\min } t ^ {3 / 4}}} \left(1 + t ^ {- 3 / 8} \sqrt {\log (\frac {M d}{\delta}) + (\log \log t) _ {+}}\right) \\ \leq \tilde {C} _ {1} n ^ {5 / 8} \sqrt {\frac {\sigma^ {2} d \log \left(\frac {n}{\lambda d} + 1\right) + 2 \sigma^ {2} \log (1 / \delta) + \tilde {\lambda} B ^ {2}}{C _ {\min}}} \left(1 + C _ {\min } ^ {- 1} n ^ {- \frac {3}{8}} \sqrt {\log \left(\frac {M d}{\delta}\right) + (\log \log n) _ {+}}\right) \\ \end{array}
$$
E Time-Uniform Concentration Inequalities
We will make use of the elegant concentration results in Howard et al. [2021], which analyzes the boundary of sub-Gamma processes.
Definition 22 (Sub-Gamma process). Let  $(S_{t})_{t = 0}^{\infty}$  and  $(V_{t})_{t = 0}^{\infty}$  be real-valued processes adapted to  $(\mathcal{F}_t)_{t = 1}^\infty$  with  $S_0 = V_0 = 0$  and  $V_{t}$  non-negative. We say that  $S_{t}$  is sub-Gamma if for  $\lambda \in [0,1 / c)$ , there exists a supermartingale  $(M_t(\lambda))_{t = 0}^{\infty}$  w.r.t.  $\mathcal{F}_t$ , such that  $\mathbb{E}M_0 = 1$  and for all  $t\geq 1$ :
The following is a special case of Theorem 1 in Howard et al. [2021]. We have simplified it by making a few straightforward choices for the parameters used originally by Howard et al. [2021], which will yield an easier-to-use bound in our scenario.
Proposition 23 (Curved Boundary of Sub-Gamma Processes). Let  $(S_{t})_{t\geq 0}$  be sub-Gamma with scale parameter  $c$  and variance process  $(V_{t})_{t\geq 0}$ . Define the boundary
$$
\mathcal {B} _ {\alpha} (v) := \frac {5}{2} \sqrt {\max  \{v , 1 \} \left(\left(\log \log e v\right) _ {+} + \log \left(\frac {2}{\alpha}\right)\right)} + 3 c \left(\left(\log \log e v\right) _ {+} + \log \left(\frac {2}{\alpha}\right)\right),
$$
for  $v > 0$ , where  $(x)_{+} = \max (0,x)$ . Then,
$$
\mathbb {P} \left(\exists t: S _ {t} \geq \mathcal {B} _ {\alpha} (V _ {t})\right) \leq \alpha .
$$
Proof of Proposition 23. Suppose  $\xi(\cdot)$  denotes the Riemann zeta function. Theorem 1 in Howard et al. [2021] states that if  $(S_{t})_{t\geq 0}$  is a sub-Gamma process with variance process  $(V_{t})_{t\geq 0}$  then the boundary
$$
\mathcal {S} _ {\alpha} (v ^ {\prime}) = k _ {1} \sqrt {v ^ {\prime} \left(s \log \log (\eta v ^ {\prime}) + \log \left(\frac {\zeta (s)}{\alpha \log^ {s} \eta}\right)\right)} + c k _ {2} \left(s \log \log (\eta v ^ {\prime}) + \log \left(\frac {\zeta (s)}{\alpha \log^ {s} \eta}\right)\right).
$$
satisfies,
$$
\mathbb {P} (\exists t: S _ {t} \geq \mathcal {S} _ {\alpha} (\max  (V _ {t}, 1))) \leq \alpha
$$
$$
k _ {1} := \frac {\eta^ {1 / 4} + \eta^ {- 1 / 4}}{\sqrt {2}} \qquad \text {a n d} \qquad k _ {2} := (\sqrt {\eta} + 1) / 2
$$
and  $s, \eta \geq 1$ . Choosing  $s = 2$  and  $\eta = e$ , we obtain  $\zeta(2) = \pi^2 / 6 \leq 2$ . Furthermore, we have  $k_1 \leq \frac{3}{2}$  and  $k_2 \leq \frac{3}{2}$ . Then if  $v' \geq 1$  (which we will enforce by the construction  $v' = \max(1, v)$ ), we compute
$$
s \log \log (\eta v ^ {\prime}) + \log \left(\frac {\zeta (s)}{\alpha \log^ {s} \eta}\right) \leq 2 (\log \log e v ^ {\prime}) _ {+} + \log \left(\frac {2}{\alpha}\right).
$$
Therefore, we can upper bound (using our bounds on  $k_{1}, k_{2}$ )
$$
\mathcal {S} _ {\alpha} (v ^ {\prime}) \leq \frac {5}{2} \sqrt {v ^ {\prime} \left((\log \log e v ^ {\prime}) _ {+} + \log \left(\frac {2}{\alpha}\right)\right)} + 3 c \left((\log \log e v ^ {\prime}) _ {+} + \log \left(\frac {2}{\alpha}\right)\right).
$$
Now, since the boundary is given by  $S_{\alpha}(\max(v, 1))$  and  $v' = \max(v, 1) \geq 1$  we deduce that
$$
\mathcal {B} _ {\alpha} (v) := \frac {5}{2} \sqrt {\max  \{v , 1 \} \left(\left(\log \log e v\right) _ {+} + \log \left(\frac {2}{\alpha}\right)\right)} + 3 c \left(\left(\log \log e v\right) _ {+} + \log \left(\frac {2}{\alpha}\right)\right).
$$
is an any-time valid boundary.
Lemma 24 (Time-Uniform Two-sided Bernoulli). Let  $X_{1},\ldots ,X_{s},\ldots ,X_{t}$  be a martingale sequence of Bernoulli random variables with conditional mean  $\gamma_{s}$ . Then for all  $\delta >0$
$$
\mathbb {P} \left(\exists t: \left| \sum_ {s = 1} ^ {t} \left(X _ {s} - \gamma_ {s}\right) \right| \geq \frac {5}{2} \sqrt {t \left((\log \log t) _ {+} + \log (4 / \delta)\right)}\right) \leq \delta ,
$$
Proof of Lemma 24. By Proposition 23, we know that if  $S_{t}$  is sub-Gamma with variance process  $V_{t}$  and scale parameter  $c$ , then
$$
\mathbb {P} \left(\exists t: S _ {t} \geq \mathcal {B} _ {\delta} (V _ {t})\right) \leq \delta ,
$$
$$
\mathcal {B} _ {\delta} (v) := \frac {5}{2} \sqrt {\max  \{1 , v \} \left(\left(\log \log e v\right) _ {+} + \log (2 / \delta)\right)} + 3 c \left(\left(\log \log e v\right) _ {+} + \log (2 / \delta)\right).
$$
By Howard et al. [2020], we know that if  $(X_{t})_{t = 1}^{\infty}$  is a Bernoulli sequence, then  $S_{t} = \sum_{s = 1}^{t}(X_{s} - \gamma_{s})$  is sub-Gamma with variance process  $V_{t} = t$  and scale parameter  $c = 0$  (hence, sub-Gaussian). This implies,
$$
\mathbb {P} \left(\exists t: \sum_ {s = 1} ^ {t} \left(X _ {s} - \gamma_ {s}\right) \geq \frac {5}{2} \sqrt {t \left((\log \log t) _ {+} + \log (2 / \delta)\right)}\right) \leq \delta ,
$$
The above arguments also hold for the sequence  $Z_{s} = -X_{s}$ . Then taking a union bound and adjusting  $\delta \gets \delta / 2$  concludes the proof.
Lemma 25 (Time-Uniform Bernstein). Let  $(\xi_i)_{i=1}^{\infty}$  be a sequence of conditionally standard subgaussian variables, where each  $\xi_i$  is  $\mathcal{F}_{i-1} = \sigma(\xi_1, \ldots, \xi_i)$  measurable. Then, for  $v_i \in \mathbb{R}$  and  $\delta \in (0,1]$
$$
\mathbb {P} \left(\exists t: \sum_ {i = 1} ^ {t} (\xi_ {i} ^ {2} - 1) v _ {i} \geq \frac {5}{2} \sqrt {\max  \left\{1 , 4 \| \boldsymbol {v} _ {t} \| _ {2} ^ {2} \right\} \omega_ {\delta} (\| \boldsymbol {v} _ {t} \| _ {2})} + 1 2 \omega_ {\delta} (\| \boldsymbol {v} _ {t} \| _ {2}) \max  _ {i \geq 1} v _ {i}\right) \leq \delta
$$
where,  $\pmb{v}_t = (v_1,\dots ,v_t)\in \mathbb{R}^t$  and  $\omega_{\delta}(v)\coloneqq \left(\log \log (4ev^{2})\right)_{+} + \log (2 / \delta)$ .
Proof of Lemma 25. From Lemma 4,  $S_{t} = \sum_{i=1}^{t} (\xi_{i}^{2} - 1) v_{i}$  is sub-Gamma with variance process  $V_{t} = 4 \sum_{i=1}^{t} v_{i}^{2}$  and  $c = 4 \max_{i \geq 1} v_{i}$ . By Proposition 23, we know that if  $S_{t}$  is sub-Gamma with variance process  $V_{t}$  and scale parameter  $c$ , then
Lemma 26 (Time-Uniform Azuma-Hoeffding). Let  $X_{1}, \ldots, X_{n}$  be a martingale difference sequence such that  $|X_{t}| \leq B$  for all  $t > 1$  almost surely. Then for all  $\delta > 0$ ,
$$
\mathbb {P} \left(\exists t: \sum_ {s = 1} ^ {t} X _ {s} \geq \frac {5 B}{2} \sqrt {t \left((\log \log e t B ^ {2}) _ {+} + \log (2 / \delta)\right)}\right) \leq \delta ,
$$
Proof of Lemma 26. By Proposition 23, we know that if  $S_{t}$  is sub-Gamma with variance process  $V_{t}$  and scale parameter  $c$ , then
$$
\mathbb {P} \left(\exists t: S _ {t} \geq \mathcal {B} _ {\delta} \left(V _ {t}\right)\right) \leq \delta ,
$$
By [Howard et al., 2020], we know that if  $(X_{t})_{t = 1}^{\infty}$  is  $B$ -bounded martingale difference sequence, then  $S_{t} = \sum_{s = 1}^{t}X_{s}$  is sub-Gamma with variance process  $V_{t} = tB^{2}$  and scale parameter  $c = 0$ . This implies,
$$
\mathbb {P} \left(\exists t: S _ {t} \geq \frac {5 B}{2} \sqrt {t \left((\log \log e t B ^ {2}) _ {+} + \log (2 / \delta)\right)}\right) \leq \delta ,
$$
F Experiment Details
Figure 5: Examples of possible reward functions  $r(\cdot)$  in our experiments.
F.1 Hyper-Parameter Tuning Results
We implement 6 algorithms in our experiments, ETC [Algorithm 4, Hao et al., 2020], ETS (Algorithm 5), CORRAL [Algorithm 6, Agarwal et al., 2017], ALEXP (Algorithm 1), and Lastly UCB (Algorithm 3) with the oracle feature map  $\phi_{j\star}$  (Oracle), and UCB with the concatenated feature map  $\phi$  (Naive). The Python code is available on github.com/lasgroup/ALEXP. When algorithms require exploration, e.g., in the case of ETC or ALEXP, we simply set  $\pi = \mathrm{Unif}(\mathcal{X})$ . Figure 7 shows the results of our hyperparameter tuning experiment. To ensure that the curves are valid, we run each
Figure 6: Bench-marking ALEXP and other baselines. Complete version of Fig. 1 and Fig. 2.
Algorithm 2 GetPosterior
Inputs:  $H_{t},\phi ,\tilde{\lambda}$    
Let  $K_{t}\gets [\phi^{\top}(\pmb{x}_{i})\phi (\pmb{x}_{j})]_{i,j\leq t}$  , and  $V_{t}\gets (K_{t} + \tilde{\lambda}^{2}\pmb {I})$  , and  $\pmb {k}(\cdot)\gets [\pmb{\phi}^{\top}(\pmb{x}_i)\pmb {\phi}(\cdot)]_{i\leq t}$
Calculate  $\mu_t(\cdot) \gets \pmb{k}^T(\cdot)V_t^{-1}\pmb{y}_t$
Calculate  $\sigma_t(\cdot)\gets \sqrt{\phi^\top(\cdot)\phi(\cdot) - \pmb{k}^\top(\cdot)V_t^{-1}\pmb{k}(\cdot)}$
Return:  $\mu_t, \sigma_t$
Algorithm 3 UCB
Inputs:  $\tilde{\lambda},\beta_t,\phi$    
for  $t = 1,\ldots ,n$  do Choose  $\pmb {x}_t$  arg max  $u_{t - 1}(\pmb {x}) = \mu_{t - 1}(\pmb {x}) + \beta_t\sigma_{t - 1}(\pmb {x}).$ $\triangleright$  Choose actions optimistically Observe  $y_{t} = r(\pmb{x}_{t}) + \varepsilon_{t}$  .  $\triangleright$  Receive reward  $H_{t}\gets H_{t - 1}\cup \{(x_{t},y_{t})\}$ $\triangleright$  Append history Update  $\mu_t,\sigma_t\gets \mathrm{GetPosterior}(H_t,\phi ,\tilde{\lambda})$    
end for
configuration for 20 different random seeds, i.e. on different random environments. The shaded areas in Figure 7 show the standard error.
UCB. For all the experiments, we set the exploration coefficient of UCB to  $\beta_{t} = 2^{3}$  and choose the regression regularizer from  $\tilde{\lambda} \in \{0.01, 0.1, 0.5\}$ . We use PYTORCH [Paszke et al., 2017] for updating the upper confidence bounds, which requires more regularization for longer feature maps (e.g. when  $s = 8$ ,  $p = 2$ ), to be computationally stable.
Lasso. Every time we need to solve Eq. (1), we set  $\lambda_{t}$  according to the rate suggested by Theorem 3. To find a suitable constant scaling coefficient, we perform a hyper-parameter tuning experiment sampling 20 values in  $[10^{-5}, 10^{0}]$ . We choose  $\lambda_0 = 0.009$ , and scale  $\lambda_{t}$  with it across all experiments.
ALEXP. We set the rates for  $\gamma_{t}$  and  $\eta_t$  as prescribed by Theorem 1. For the scaling constants, we perform a hyper-parameter tuning experiment log-uniformly sampling 20 different configurations from  $\gamma_0\in [10^{-4},10^{-1}]$  and  $\eta_0\in [10^0,10^2 ]$  . For each problem instance (i.e. as  $s$  and  $p$  change) we repeat this process. However we observe that the optimal hyper-parameters work well across all problem instances.
ETC/ETS. For these algorithms, we separately tune  $n_0$  for each problem instance. We set  $\lambda_1 \propto \sqrt{\log M / n_0}$  according to Theorem 4.2 of [Hao et al., 2020] and scale it with  $\lambda_0 = 0.009$ , as stated before. We uniformly sample 10 different values where  $n_0 \in [2,80]$  since the horizon is  $n = 100$ . The optimal value often happens around  $n_0 = 20$ .
CORRAL. We set the rates of the parameters as  $\gamma = \mathcal{O}(1 / n)$  and  $\eta = \mathcal{O}(\sqrt{M / n})$  according to Agarwal et al. [2017, Theorem 5]. Then similar to ALEXP, we tune the scaling constants. The procedure for tuning the constants is identical to ALEXP, as in we use the same search interval, and try 10 different configurations for  $\gamma$  and  $\eta$ .
3To achieve the  $\sqrt{dT\log T}$  regret, one has to set  $\beta_{t} = \mathcal{O}(\sqrt{d\log T})$  as shown in Proposition 21.
Algorithm 4 ETC [Hao et al., 2020]
Inputs:  $n_0, n, \lambda_1, \pi$   
Let  $H_0 = \emptyset$   
for  $t = 1, \dots, n_0$  do  
Draw  $\boldsymbol{x}_t \sim \pi$ .  
Observe  $y_t = r(\boldsymbol{x}_t) + \varepsilon_t$ .  
 $H_t \gets H_{t-1} \cup \{(\boldsymbol{x}_t, y_t)\}$   
end for  
 $\hat{\theta}_{n_0} \gets \mathcal{L}(\boldsymbol{\theta}, H_{n_0}, \lambda_1)$   
for  $t = n_0 + 1, \dots, n$  do  
Choose  $\boldsymbol{x}_t = \arg \max \hat{\theta}_{n_0}^\top \phi(\boldsymbol{x})$   
end for
Algorithm 5 ETS
Inputs:  $n_0, n, \lambda_1, \tilde{\lambda}, \beta_t, \pi$   
Let  $H_0 = \emptyset$   
for  $t = 1, \dots, n_0$  do  
Draw  $\boldsymbol{x}_t \sim \pi$ .  
Observe  $y_t = r(\boldsymbol{x}_t) + \varepsilon_t$ .  
 $H_t \gets H_{t-1} \cup \{(\boldsymbol{x}_t, y_t)\}$   
end for  
 $\hat{\theta}_{n_0} \gets \mathcal{L}(\boldsymbol{\theta}, H_{n_0}, \lambda_1)$ $\hat{J} \gets \{j \mid \hat{\theta}_{n_0, j} \neq 0, j \in [M]\}$ $\phi_{\hat{j}}(\cdot) \gets [\phi_j(\cdot)]_{j \in \hat{J}}$   
for  $t = n_0 + 1, \dots, n$  do  
Choose  $\boldsymbol{x}_t = \arg \max u_{t-1}(\boldsymbol{x}) = \mu_{t-1}(\boldsymbol{x}) + \beta_t \sigma_{t-1}(\boldsymbol{x})$   
Observe  $y_t = r(\boldsymbol{x}_t) + \varepsilon_t$ $H_t \gets H_{t-1} \cup \{(\boldsymbol{x}_t, y_t)\}$   
Update  $\mu_t, \sigma_t \gets \text{GetPosterior}(H_t, \phi_{\hat{j}}, \tilde{\lambda})$   
end for
Algorithm 6 CORRAL [Agarwal et al., 2017]
Inputs:  $n,\gamma ,\eta$    
Initialize  $\beta = e^{1 / \ln n},\eta_{1,j} = \eta ,\rho_{1,j} = 2M$  for all  $j = 1,\ldots ,M$    
Set  $q_{1} = \bar{q}_{1} = \frac{1}{M}$  and initialize base agents  $(p_{1,1},\dots ,p_{1,M})$    
for  $t = 1,\dots ,n$  do Choose  $j_{t}\sim \bar{q}_{t}$  ▷Sample Agent Draw  $x_{t}\sim p_{t,j_{t}}$  Play action according to agent  $j_{t}$  Observe  $y_{t} = r(\pmb {x}_{t}) + \varepsilon_{t}$  Calculate IW estimates  $\hat{r}_{t,j} = \frac{y_t}{\bar{q}_{t,j}}\mathbb{I}\{j = j_t\}$  for all  $j = 1,\dots ,M$  Send  $\hat{r}_{t,j} = \frac{y_t}{\bar{q}_{t,j}}\mathbb{I}\{j = j_t\}$  to agents and get updated policies  $p_{t + 1,j}$ $q_{t + 1} = \mathrm{LOG - BARRIER - OMD}(q_t,\hat{r}_{t,j_t}e_{j_t},\eta_t)$  Update agent probabilities  $\bar{q}_{t + 1} = (1 - \gamma)q_{t + 1} + \gamma \frac{1}{M}$  Mix with exploratory distribution for  $j = 1,\dots ,M$  do Update parameters if  $\frac{1}{\bar{q}_{t + 1,j}} >\rho_{t,j}$  then  $\rho_{t + 1,j}\gets \frac{2}{\bar{q}_{t,j}}$  and  $\eta_{t + 1,j}\gets \beta \eta_{t,j}$  else  $\rho_{t + 1,j}\gets \rho_{t,j}$  and  $\eta_{t + 1,j}\gets \eta_{t,j}$  end if   
end for   
end for
Algorithm 7 LOG-BARRIER-OMD
Inputs:  $\pmb{q}_t, \ell_t, \pmb{\eta}_t$   
Find  $\xi \in [\min_j \ell_{t,j}, \max_j \ell_{t,j}]$  such that  $\sum_{j=1}^{M} (q_{t,j}^{-1} + \eta_{t,j}(\ell_{t,j} - \xi))^{-1} = 1$ .  
Return:  $\pmb{q}_{t+1}$  where  $q_{t+1,j}^{-1} = q_{t,j}^{-1} + \eta_{t,j}(\ell_{t,j} - \xi)$  for all  $j \in [M]$
Figure 7: Results for different hyper-parameters across different problem instances. ALEXP is robust to the choice of hyper-parameters.