SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\hat {\boldsymbol {\theta}} _ {t + 1} = \operatorname * {a r g m i n} _ {\| \boldsymbol {\theta} \| _ {2} \leq S} \frac {\| \boldsymbol {\theta} - \hat {\boldsymbol {\theta}} _ {t} \| _ {\mathbf {V} _ {t + 1}} ^ {2}}{2} + \langle \boldsymbol {\theta} - \hat {\boldsymbol {\theta}} _ {t}, \nabla \tilde {\ell} _ {t} (\hat {\boldsymbol {\theta}} _ {t}) \rangle
$$
10: Construct the confidence region
$$
\mathcal {C} _ {t + 1} = \left\{\boldsymbol {\theta} \in \mathbb {R} ^ {d} \mid \| \boldsymbol {\theta} - \hat {\boldsymbol {\theta}} _ {t + 1} \| _ {\mathbf {V} _ {t + 1}} ^ {2} \leq \gamma \right\}
$$
where  $\mathbf{V}_t = \mathbf{V}_{t-1} + \frac{\kappa}{2}\pmb{x}_{t-1}\pmb{x}_{t-1}^\top$  with  $\mathbf{V}_1 = \lambda\mathbf{I}_d$ . Here,  $\kappa$  is defined in Assumption 2 and  $\lambda = \max\{1, \kappa/2\}$ . With the processed action-reward pair  $(\pmb{x}_t, \tilde{\pmb{y}}_t)$ , CRTM computes the gradient of the loss function as
$$
\nabla \tilde {\ell} _ {t} (\boldsymbol {\theta}) = (- \tilde {y} _ {t} + \mu \left(\boldsymbol {x} _ {t} ^ {\top} \boldsymbol {\theta}\right)) \boldsymbol {x} _ {t}, \tag {11}
$$
where  $\tilde{\ell}_t(\cdot)$  is the negative log-likelihood of the generalized linear model (8). After that, CRTM employs a variant of online Newton step (ONS) to update its estimator, given by
$$
\hat {\boldsymbol {\theta}} _ {t + 1} = \operatorname * {a r g m i n} _ {\| \boldsymbol {\theta} \| _ {2} \leq S} \frac {\| \boldsymbol {\theta} - \hat {\boldsymbol {\theta}} _ {t} \| _ {\mathbf {V} _ {t + 1}} ^ {2}}{2} + \langle \boldsymbol {\theta} - \hat {\boldsymbol {\theta}} _ {t}, \nabla \tilde {\ell} _ {t} (\hat {\boldsymbol {\theta}} _ {t}) \rangle .
$$
Equipped with above estimation, CRTM constructs the confidence region  $\mathcal{C}_{t + 1}$  where the inherent parameters  $\theta_{*}$  lies in with high probability, such that
where  $\gamma = \widetilde{O}(T^{\frac{1 - \epsilon}{1 + \epsilon}})$  denotes the width of the confidence region, and details are shown in Theorem 1. Given the confidence region  $\mathcal{C}_{t+1}$ , the most promising arm  $x_{t+1}$  can be obtained through the following maximize operation,
$$
(\boldsymbol{x}_{t + 1},\tilde{\boldsymbol{\theta}}_{t + 1}) = \operatorname *{argmax}_{\boldsymbol {x}\in \mathcal{D}_{t + 1},\boldsymbol {\theta}\in \mathcal{C}_{t + 1}}\langle \boldsymbol {x},\boldsymbol {\theta}\rangle
$$
since  $\mu (\cdot)$  is monotonically increasing according to Assumption 2.
Although there exists several heavy-tailed linear bandit algorithms based on the truncation strategy, such as TOFU [Shao et al., 2018] and BTC [Xue et al., 2020], CRTM differs from them in two aspects. Firstly, both TOFU and BTC have to store the historical rewards and truncate them at each epoch, resulting in a computational complexity of  $O(T^2)$ . In contrast, CRTM achieves online learning by processing only the reward of current round, whose computational complexity is  $O(T)$ . Secondly, while TOFU and BTC are designed for SLB model and calculate the estimator via least-squares estimation, CRTM is designed for the GLB model and updates the estimator using the ONS method, which makes the analytical techniques fundamentally different. Theorem 1 provides a tight confidence region, and its proof relies on the induced method because ONS is an iteratively updated method. Due to the page limit, we provide the detailed proof in the Appendix A.2.
Theorem 1 If the rewards satisfy (9) and (10), then with probability as least  $1 - \delta$ , the confidence region in CRTM is
$$
\left\| \boldsymbol {\theta} - \hat {\boldsymbol {\theta}} _ {t + 1} \right\| _ {\mathbf {V} _ {t + 1}} ^ {2} \leq \gamma , \forall t \geq 0
$$
Algorithm 2 Confidence Region with Mean of Medians (CRMM)
Input:  $\delta, \epsilon, v, \kappa, S, \lambda = \max\{1, \kappa/2\}$  and  $T \in \mathbb{N}_+$
1: Initialize  $\hat{\theta}_{1} = \mathbf{0},\mathbf{V}_{1} = \lambda \mathbf{I}_{d}$  and  $\gamma_{1} = \lambda S^{2}$
2:  $r = \left\lceil 16 \ln \frac{4T}{\delta} \right\rceil$  and  $T_0 = \lfloor T / r \rfloor$
3: for  $t = 1,2,\ldots ,T_0$  do
4:  $(\pmb{x}_t,\tilde{\pmb{\theta}}_t) = \operatorname{argmax}_{\pmb{x}\in \mathcal{D},\pmb{\theta}\in \mathcal{C}_t}\langle \pmb{x},\pmb{\theta}\rangle$
5: Play the arm  $x_{t}r$  times and observe the rewards  $\{y_t^1,y_t^2,\ldots ,y_t^r\}$
6: Take the median of  $\{y_t^1, y_t^2, \ldots, y_t^r\}$  as  $\bar{y}_t$
7: Compute the gradient  $\nabla \bar{\ell}_t(\hat{\pmb{\theta}}_t) = (-\bar{y}_t + \mu (\pmb{x}_t^\top \hat{\pmb{\theta}}_t))\pmb{x}_t$
8: Update  $\mathbf{V}_{t + 1} = \mathbf{V}_t + \frac{\kappa}{2}\pmb {x}_t\pmb{x}_t^t$
9: Compute the center of confidence region
$$
\hat {\boldsymbol {\theta}} _ {t + 1} = \operatorname * {a r g m i n} _ {\| \boldsymbol {\theta} \| _ {2} \leq S} \frac {\| \boldsymbol {\theta} - \hat {\boldsymbol {\theta}} _ {t} \| _ {\mathbf {V} _ {t + 1}} ^ {2}}{2} + \langle \boldsymbol {\theta} - \hat {\boldsymbol {\theta}} _ {t}, \nabla \bar {\ell} _ {t} (\hat {\boldsymbol {\theta}} _ {t}) \rangle
$$
$$
\mathcal {C} _ {t + 1} = \left\{\boldsymbol {\theta} \in \mathbb {R} ^ {d} | \| \boldsymbol {\theta} - \hat {\boldsymbol {\theta}} _ {t + 1} \| _ {\mathbf {V} _ {t + 1}} ^ {2} \leq \gamma_ {t + 1} \right\}
$$
$$
\gamma = 2 2 4 u ^ {\frac {2}{1 + \epsilon}} \ln (4 T / \delta) ^ {\frac {2 \epsilon}{1 + \epsilon}} T ^ {\frac {1 - \epsilon}{1 + \epsilon}} \frac {4 d}{\kappa} \ln \left(1 + \frac {\kappa T}{2 \lambda d}\right) + 2 \lambda S ^ {2} + \frac {4 8 U ^ {2} d}{\kappa} \ln \left(1 + \frac {\kappa T}{2 \lambda d}\right).
$$
With above confidence region, the regret bound of CRTM is explicitly given as follows.
Theorem 2 If the rewards satisfy (9) and (10), then with probability at least  $1 - \delta$ , the regret of CRTM satisfies
$$
R (T) \leq O \left(d (\log T) ^ {\frac {1 + 2 \epsilon}{1 + \epsilon}} T ^ {\frac {1}{1 + \epsilon}}\right).
$$
Remark: The above theorem establishes a  $\widetilde{O}(dT^{\frac{1}{1+\epsilon}})$  regret bound with the assumption that the  $(1+\epsilon)$ -th moment of the rewards is bounded for some  $\epsilon \in (0,1]$ . Existing algorithms based on truncation are time-consuming because they need to store the learning history and truncate all historical rewards at each epoch [Shao et al., 2018; Xue et al., 2020]. Unlike the recently proposed mean of medians method which is limited in symmetric rewards [Zhong et al., 2021], CRTM expands it to asymmetric and achieves an improved regret bound by a factor of  $O((\log T)^{\frac{1}{2\alpha}})$  for some  $\alpha \in (0,1)$  if  $\epsilon = 1$ . Furthermore, CRTM is almost optimal as the lower bound is  $\Omega(dT^{\frac{1}{1+\epsilon}})$  [Shao et al., 2018].
3.3 Mean of Medians
In this section, we present our second algorithm, referred to as Confidence Region with Mean of Medians (CRMM), which shares a similar framework with CRTM but uses a different mean of medians estimator. The complete procedure is outlined in Algorithm 2. CRMM requires that for some  $\epsilon \in (0,1]$ , the  $1 + \epsilon$  central moment of the rewards is bounded, and the distribution of rewards is symmetric. Precisely, for some  $\epsilon \in (0,1]$ , there exists a constant  $v > 0$  such that the rewards satisfy
$$
\mathrm {E} \left[ | \eta_ {t} | ^ {1 + \epsilon} \mid \mathcal {G} _ {t - 1} \right] \leq v \text {a n d} p (\eta_ {t}) = p (- \eta_ {t}). \tag {12}
$$
At each epoch  $t$ , CRMM plays the selected arm  $x_{t}$ $r$  times, generating rewards  $\{y_t^1,\dots ,y_t^r\}$  with  $r = O(\log T)$ . To obtain a robust estimation using mean of medians strategy, CRMM first takes the median of  $\{y_t^1,\dots ,y_t^r\}$ , denoted by  $\bar{y}_t$ . Subsequently, CRMM computes the gradient with the arm-reward pair  $(x_{t},\bar{y}_{t})$  through the operation similar to (11). Then, CRMM employs a variant of ONS to update the estimator and construct the confidence region  $C_{t + 1}$  centered on the new estimator. The details about the constructed confidence region is given in Theorem 3.
Compared to existing bandit algorithms that utilize the median of means strategy, the primary difference lies in the item chosen as the "means". As we have introduced in (7), MENU of Shao et al. [2018] uses the distance between different estimators as the "means". BMM of Xue et al. [2020] calculates multiple estimated rewards for each arm and treats them as the "means". Both MENU and BMM require  $O(\log T)$  estimators during each round, whereas CRMM only requires one estimator. Moreover, compared to the mean of medians approach [Zhong et al., 2021], CRMM plays each selected arm fewer times, leading to more model updates, which is critical based on experimental results. Since the chosen arm has to be played multiple times, we assume the arm set for CRMM is static, such that  $\mathcal{D}_t = \mathcal{D}$  for  $t > 0$ , which is a common assumption [Medina and Yang, 2016; Zhang et al., 2016; Lu et al., 2019]. The following theorem guarantees a tight confidence region.
Theorem 3 If the rewards satisfy (9) and (12), then with probability as least  $1 - 2\delta$ , the confidence region in CRMM is
$$
\left\| \boldsymbol {\theta} - \hat {\boldsymbol {\theta}} _ {t + 1} \right\| _ {\mathbf {V} _ {t + 1}} ^ {2} \leq \gamma_ {t + 1}, \forall t \geq 0
$$
$$
\begin{array}{l} \gamma_ {t + 1} = \left(4 U ^ {2} + C \rho t ^ {\frac {1 - \epsilon}{1 + \epsilon}}\right) \frac {4 d}{\kappa} \ln \left(1 + \frac {\kappa t}{2 \lambda d}\right) + \lambda S ^ {2} + \frac {2 \rho^ {2}}{\kappa} t ^ {\frac {1 - \epsilon}{1 + \epsilon}}, \\ \rho = 2 C \ln (4 T / \delta) + 2 C ^ {- \epsilon} r v, C = (4 v) ^ {\frac {1}{1 + \epsilon}}. \\ \end{array}
$$
With above confidence region, we prove the following regret bound for CRMM.
Theorem 4 If the rewards satisfy (9) and (12), then with probability at least  $1 - 2\delta$ , the regret of CRMM satisfies
$$
R (T) \leq O \left(d (\log T) ^ {\frac {3}{2} + \frac {\epsilon}{1 + \epsilon}} T ^ {\frac {1}{1 + \epsilon}}\right).
$$
Remark: Theorem 3 clarifies that if the rewards have a finite  $1 + \epsilon$  central moment for some  $\epsilon \in (0,1]$ , CRMM achieves a regret bound of  $\widetilde{O}(dT^{\frac{1}{1 + \epsilon}})$ . This bound reduces to  $\widetilde{O}(d\sqrt{T})$  when  $\epsilon = 1$ , indicating that CRMM achieves the same order as the bounded rewards assumption regarding both  $d$  and  $T$  [Zhang et al., 2016; Jun et al., 2017]. Compared to the approach of Zhong et al. [2021], CRMM enhances the bound by an order of  $O((\log T)^{\frac{1}{2\alpha} - \frac{1}{2}})$  for a fixed  $\alpha \in (0,1)$  if  $\epsilon = 1$ .
This section demonstrates the improvement of our algorithms by numerical experiments. Firstly, we show the effectiveness of our algorithms in dealing with heavy-tailed problems by comparing their regret to that of existing generalized linear bandit algorithms. Secondly, we evaluate the efficiency of our algorithms by comparing their time consumption to other existing algorithms designed for heavy-tailed bandit problems. All algorithms are implemented using PyCharm 2022 and tested on a laptop with a 2.5GHz CPU and 32GB of memory.
4.1 Regret Comparison
To assess the enhancement of our algorithms in handling heavy-tailed problems, we utilize the vanilla GLB algorithms, specifically  $\mathrm{OL}^2\mathbf{M}$  [Zhang et al., 2016] and GLOC [Jun et al., 2017], as baselines. Additionally, we incorporate the mean of medians method proposed by Zhong et al. [2021] into  $\mathrm{OL}^2\mathbf{M}$  and GLOC, resulting in another two baselines  $\mathrm{OL}^2\mathbf{M}_{-}$  mom and GLOC_mom, respectively. All algorithms are configured with  $\epsilon = 1$ $\delta = 0.01$  ,and  $T = 10^{6}$
Let  $\pmb{\theta}_{*} = \mathbf{1} / \sqrt{d}\in \mathbb{R}^{d}$ , where  $\mathbf{1}$  is an all-1 vector and  $\| \pmb{\theta}_{*}\|_{2} = 1$ . The number of arms is set to  $K = 20$ , and the feature dimension is  $d = 10$ . Each component of the contextual vector  $\pmb{x}_t$  is uniformly sampled from the interval [0, 1], and then normalized to be unit norm, i.e.,  $\| \pmb{x}_t\|_2 = 1$ . We tune the width of the confidence region following the common practice in bandit learning [Zhang et al., 2016; Jun et al., 2017]. Precisely, with  $c$  being a tuning parameter searched within  $[1e^{-4}, 1]$ , the width of the confidence region for  $\mathrm{OL}^2\mathrm{M}$  and GLOC are set as  $\gamma_t = cd\ln (t / \lambda + 1)$  and  $\gamma_t = c\sum_{\tau=1}^{t}(\mu(\pmb{x}_{\tau}^\top\hat{\pmb{\theta}}_{\tau}) - y_{\tau})^2\|\pmb{x}_{\tau}\|_{\mathbf{V}_{\tau}^{-1}}^2$ , respectively. In addition, the radius of the confidence region is set as  $cd\ln (4T/\delta)^{\frac{2\epsilon}{1+\epsilon}}\ln (T/(d\lambda) + 1)T^{\frac{1-\epsilon}{1+\epsilon}}$  for CRTM, and  $cd\ln (t/(d\lambda) + 1)t^{\frac{1-\epsilon}{1+\epsilon}}$  for
(a) Student's  $t$  -noises (symmetric)
Figure 1: Regret comparison
(b) Pareto noises (asymmetric)
CRMM. For  $\mathrm{OL}^2\mathrm{M}$  _mom and GLOC_mom, the chosen arm is played  $\bar{r} = (16\ln (2T / \delta))^{1 / \alpha}$  times per round, and  $\alpha = 0.62$  is close to optimal according to the experiments of Zhong et al. [2021].
We run 10 repetitions for each algorithm and display the average regret with time evolution. According to the generalized linear bandit model, the observed reward  $y_{t}$  is given by
where  $\mu(x) = \frac{1}{1 + e^{-x}}$  is the logit model and  $\eta_t$  is the heavy-tailed noises. To evaluate the algorithms performance under both symmetric and asymmetric rewards,  $\eta_t$  fits the following two distributions,
(i) Student's  $t$ -Noise:  $\eta_t \sim \frac{G(2)}{\sqrt{3\pi G(1.5)}} \left(1 + \frac{x^2}{3}\right)^{-2}$  where  $G(\cdot)$  is the Gamma function;
(ii) Pareto Noise:  $\eta_t \sim \frac{s x_m^s}{x^{s+1}} \mathbb{I}_{x \geq x_m}$  where  $s = 3$  and  $x_m = 0.01$ .
Fig. 1 compares our algorithms against two vanilla GLB algorithms (OL $^2$ M and GLOC), as well as these two algorithms exploiting mean of medians estimators (OL $^2$ M_mom and GLOC_mom). Fig. 1(a) shows that CRTM and CRMM outperform the other four algorithms. CRTM provides the best performance, which is consistent with the theoretical guarantees. OL $^2$ M_mom and GLOC_mom appear ineffective at handling heavy-tailed problems, because they update estimator only 100 times with the chosen arm played  $\bar{r}$  times [Zhong et al., 2021]. Fig. 1(b) presents the cumulative regrets under asymmetric noises, with CRTM still having the lowest regret curve, demonstrating its generality and effectiveness in handling heavy-tailed bandit problems. On the other hand, CRMM, GLOC_mom, and OL $^2$ M_mom performs poorly in Fig. 1(b), as they can not deal with the asymmetric rewards.
4.2 Runtime Comparison
To demonstrate the efficiency improvement of our algorithms, we compare them with existing heavy-tailed bandit algorithms such as CRT and MoM [Medina and Yang, 2016], TOFU and MENU [Shao et al., 2018], and SupBTC and SupBMM [Xue et al., 2020]. Among them, CRT, TOFU and SupBTC employ truncation strategy, while MoM, MENU and SupBMM utilize the median of means strategy.
The experimental settings are the same as described in Regret Comparison section, except for the time horizon and feature dimension. We use a smaller time horizon  $T = 10^{4}$  since TOFU is time-consuming. The feature dimension is increased to  $d = 100$  to highlight the difference between SupBTC and TOFU. The computational runtimes are shown in Table 2.
Table 2: Runtime comparison
Algorithm	Time(s)	Algorithm	Time(s)
CRT	3.1737	MoM	0.0630
TOFU	3931.9963	MENU	24.1990
SupBTC	1187.1863	SupBMM	0.0685
CRTM	2.2909	CRMM	0.0514
For the truncation-based algorithms, CRTM
consumes the least time, while TOFU and SupBTC takes over a hundred times longer to execute than CRTM, representing a significant improvement. CRT takes only slightly longer than CRTM as both algorithms update the model online, but the regret bound of CRT is  $\widetilde{O}(dT^{\frac{3}{4}})$ , which is  $\widetilde{O}(T^{\frac{1}{4}})$  worse
than the bound of CRTM. For median of means algorithms, CRMM has the shortest runtime. MENU takes significantly longer than the other algorithms because MENU needs to calculate the distance between  $O(\log T)$  estimators.
We present two algorithms, CRTM and CRMM, for the generalized linear bandit model with heavily-tailed rewards, which utilize the truncation and mean of medians strategies, respectively. Both algorithms achieve the regret bound of  $\widetilde{O}(dT^{\frac{1}{1+\epsilon}})$  conditioned on a bounded  $(1+\epsilon)$ -th moment of rewards, where  $\epsilon \in (0,1]$ . This bound is almost optimal since the lower bound of the stochastic linear bandit problem is  $\Omega(dT^{\frac{1}{1+\epsilon}})$  [Shao et al., 2018]. CRTM is the first truncation-based online algorithm for the heavy-tailed bandit problem that handles both symmetric and asymmetric rewards and approaches the optimal regret bound. CRMM enhances the regret bound of the most related work by a logarithmic factor [Zhong et al., 2021]. However, CRMM is limited to symmetric rewards and we will investigate to overcome this restriction in the future.
This work was partially supported by the National Key R&D Program of China (2022ZD0114801), the Key Basic Research Foundation of Shenzhen (JCYJ20220818100005011), NSFC (62122037, 61921006) and the major key project of PCL (PCL2021A12).
Yasin Abbasi-yadkori, David Pál, and Csaba Szepesvári. Improved algorithms for linear stochastic bandits. In Advances in Neural Information Processing Systems 24, pages 2312-2320, 2011.
Ayya Alieva, Ashok Cutkosky, and Abhimanyu Das. Robust pure exploration in linear bandits with limited budget. In Proceedings of the 38th International Conference on Machine Learning, pages 187-195, 2021.
Jean-Yves Audibert and Olivier Catoni. Robust linear least squares regression. The Annals of Statistics, 39(5):2766-2794, 2011.
Peter Auer. Using confidence bounds for exploitation-exploration trade-offs. Journal of Machine Learning Research, 3(11):397-422, 2002.
Kazuoki Azuma. Weighted sums of certain dependent random variables. Tohoku Math. J. (2), 19(3):357-367, 1967.
Viktor Bengs, Aadirupa Saha, and Eyke Hüllermeier. Stochastic contextual dueling bandits under linear stochastic transitivity models. In Proceedings of the 39th International Conference on Machine Learning, pages 1764-1786, 2022.
Sebastien Bubeck and Nicolò Cesa-Bianchi. Regret analysis of stochastic and nonstochastic multiarmed bandit problems. Foundations and Trends in Machine Learning, 5(1):1-122, 2012.
Sebastien Bubeck, Nicolò Cesa-Bianchi, and Gábor Lugosi. Bandits with heavy tail. IEEE Transactions on Information Theory, 59(11):7711-7717, 2013.
Rama Cont and Jean-Philippe Bouchaud. Herd behavior and aggregate fluctuations in financial markets. *Macroeconomic Dynamics*, 4(2):170–196, 2000.
Varsha Dani, Thomas P. Hayes, and Sham M. Kakade. Stochastic linear optimization under bandit feedback. In Proceedings of the 21st Annual Conference on Learning, pages 355-366, 2008.
Ilias Diakonikolas, Daniel Kane, Jasper Lee, and Ankit Pensia. Outlier-robust sparse mean estimation for heavy-tailed distributions. In Advances in Neural Information Processing Systems 35, pages 5164-5177, 2022.
Gábor Lugosi and Shahar Mendelson. Robust multivariate mean estimation: The optimality of trimmed mean. The Annals of Statistics, 49(1):393-410, 2021.
Andres Munoz Medina and Scott Yang. No-regret algorithms for heavy-tailed linear bandits. In Proceedings of the 33rd International Conference on International Conference on Machine Learning, pages 1642-1650, 2016.
William Mendenhall, Robert.J. Beaver, and Barara.M. Beaver. Introduction to Probability and Statistics. Cengage Learning, 2012.
John A. Nelder P. McCullagh. Generalized linear models. 1989.
Sayak Ray Chowdhury and Aditya Gopalan. Bayesian optimization under heavy-tailed payoffs. In Advances in Neural Information Processing Systems 32, pages 13790-13801, 2019.
Herbert Robbins. Some aspects of the sequential design of experiments. Bull. Amer. Math. Soc., 58(5):527-535, 1952.
Yevgeny Seldin, François Laviolette, Nicolò Cesa-Bianchi, John Shawe-Taylor, and Peter Auer. Pac-bayesian inequalities for martingales. IEEE Transactions on Information Theory, 58(12):7086-7093, 2012.
Han Shao, Xiaotian Yu, Irwin King, and Michael R. Lyu. Almost optimal algorithms for linear stochastic bandits with heavy-tailed payoffs. In Advances in Neural Information Processing Systems 31, pages 8430-8439, 2018.
Sofia S. Villar, Jack Bowden, and James Wason. Multi-armed Bandit Models for the Optimal Design of Clinical Trials: Benefits and Challenges. Statistical Science, 30(2):199-215, 2015.
Bo Xue, Guanghui Wang, Yimu Wang, and Lijun Zhang. Nearly optimal regret for stochastic linear bandits with heavy-tailed payoffs. In Proceedings of the 29th International Joint Conference on Artificial Intelligence, pages 2936-2942, 2020.
Shuo Yang, Tongzheng Ren, Sanjay Shakkottai, Eric Price, Inderjit S. Dhillon, and Sujay Sanghavi. Linear bandit algorithms with sublinear time complexity. In Proceedings of the 39th International Conference on Machine Learning, pages 25241-25260, 2022.
Lijun Zhang and Zhi-Hua Zhou.  $\ell_1$ -regression with heavy-tailed distributions. In Advances in Neural Information Processing Systems 31, pages 1084-1094, 2018.
Lijun Zhang, Tianbao Yang, Rong Jin, Yichi Xiao, and Zhi-Hua Zhou. Online stochastic linear optimization under one-bit feedback. In Proceedings of the 33rd International Conference on Machine Learning, pages 392-401, 2016.
Han Zhong, Jiayi Huang, Lin Yang, and Liwei Wang. Breaking the moments condition barrier: No-regret algorithm for bandits with super heavy-tailed payoffs. In Advances in Neural Information Processing Systems 34, pages 15710-15720, 2021.
Zhengyuan Zhou, Renyuan Xu, and Jose Blanchet. Learning in generalized linear contextual bandits with stochastic delays. In Advances in Neural Information Processing Systems 32, pages 5197-5208, 2019.