SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Remark 5.2 (Comparison with existing literature). For private MABs (without contamination), the state-of-the-art also builds upon the idea of batching and forgetting [19, 20, 24] to achieve optimal regret. For robust MABs (without privacy), existing works take different robust mean estimations. For example, both [31, 30] use a trimmed mean estimator for sub-Gaussian inlier distributions while [17] adopts Huber's estimator to handle inlier distributions with only bounded variance. We are the first to study privacy and robustness simultaneously, via a simple truncation-based estimator, which in turn reveals the close relationship between privacy and robustness in MABs. This complements the recent advances in capturing the connection between these two in (high-dimensional) statistics [25, 27].
6 Upper Bounds
In this section, we establish the regret upper bounds for two specific instantiations of our meta-algorithm - one for the finite raw moment case and another for the finite central moment case. The results could match our lower bound up to a log factor on  $T$ , demonstrating their near-optimality.
One can use the tree-based mechanism [44] to reduce it to  $O(\log T)$ , but it is still sub-optimal [19].
6.1 Finite Raw Moment Case
In this section, we will focus on private and robust MABs where the inlier distributions have a finite  $k$ -th raw moment as given by Definition 3.1. In particular, we first introduce the choice of PRM in this case (see Algorithm 2) and establish its concentration property, which plays a key role in our implementation of meta-algorithm.
The PRM in Algorithm 2 is simply a truncation-based Laplace mechanism. That is, it first truncates all the received data with the threshold  $M$  (line 3). Then, Laplace noise is added to the empirical mean to preserve privacy (line 5). We highlight again that truncation here helps with both robustness (via removing outliers) and privacy (via bounding the sensitivity of empirical mean).
As in the standard algorithm design of MABs, the key is to utilize the concentration of the mean estimator. To this end, we first give the following high-probability concentration result for the mean estimate returned by PRM in Algorithm 2.
Theorem 6.1 (Concentration of Mean Estimate). Given a collection of Huber-contaminated data  $\{x_{i}\}_{i = 1}^{n}$  where the inlier distribution satisfies Definition 3.1 with mean  $\mu$ , let  $\widetilde{\mu}$  be the mean estimate by Algorithm 2. Then, for any privacy budget  $\epsilon >0$  and  $\delta \in (0,1)$ , the following results hold:
Uncontaminated case. For  $\alpha = 0$ , we have  $|\widetilde{\mu} - \mu| = O\left(\sqrt{\frac{\log(1/\delta)}{n}} + \frac{M\log(1/\delta)}{n\epsilon} + \frac{1}{M^{k-1}}\right)$ , with probability at least  $1 - \delta$ . Thus, choosing the truncation threshold  $M = \Theta\left(\frac{n\epsilon}{\log(1/\delta)}\right)^{\frac{1}{k}}$  yields  $|\widetilde{\mu} - \mu| = O\left(\sqrt{\frac{\log(1/\delta)}{n}} + \left(\frac{\log(1/\delta)}{n\epsilon}\right)^{1 - \frac{1}{k}}\right)$ .
Contaminated case. For  $0 < \alpha \leq \alpha_{1} \in (0, 1/2)$  and  $n = \Omega\left(\frac{\log(1/\delta)}{\alpha_{1}}\right)$ , we have the following with probability at least  $1 - \delta$ ,  $|\widetilde{\mu} - \mu| = O\left(\sqrt{\frac{\log(1/\delta)}{n}} + \frac{M\log(1/\delta)}{n\epsilon} + \frac{1}{M^{k-1}} + \alpha_{1}M\right)$ . Therefore, choosing the truncation threshold  $M = \Theta\left(\min\left\{\left(\frac{n\epsilon}{\log(1/\delta)}\right)^{\frac{1}{k}}, \alpha_{1}^{-\frac{1}{k}}\right\}\right)$ , yields  $|\widetilde{\mu} - \mu| \leq \beta$ , where  $\beta = O\left(\sqrt{\frac{\log(1/\delta)}{n}} + \left(\frac{\log(1/\delta)}{n\epsilon}\right)^{1 - \frac{1}{k}} + \alpha_{1}^{1 - \frac{1}{k}}\right)$ .
With the above result, several remarks are in order. First, for the uncontaminated case, our concentration result consists of the standard sub-Gaussian term and a new one due to privacy and heavy-tailed data. It can be translated into a sample complexity bound, i.e., to guarantee  $|\widetilde{\mu} - \mu| \leq \eta$  for any  $\eta \in (0,1)$ , it requires the sample size to be  $n \geq O\left(\frac{\log(1 / \delta)}{\eta^2} + \frac{\log(1 / \delta)}{\epsilon \eta^{k / (k - 1)}}\right)$ , which is optimal since it matches the lower bound for private heavy-tail mean estimation (cf. Theorem 7.2 in [46]). Second, for the contaminated case, it has an additional bias term  $O(\alpha_1^{1 - 1 / k})$ , which is also known to be information-theoretically optimal [47]. Thus, via truncation, the PRM given by Algorithm 2 achieves the optimal trade-off between accuracy, privacy and robustness, which in turn shows its potential to be integrated into our meta-algorithm. Note that  $\alpha_1$  here is any upper bound (e.g., estimate) on the true contamination level  $\alpha$ .
Now, based on the concentration result, we can set other missing parameters in our meta-algorithm accordingly. In particular, we have the following theorem that states the specific instantiation along with its performance guarantees.
Theorem 6.2 (Performance Guarantees). Consider a private and robust MAB with inlier distributions satisfying Definition 3.1 and  $0 < \alpha \leq \alpha_{1} \in (0,1/2)$ . Let Algorithm 1 be instantiated with Algorithm 2 and  $M_{\tau}$ ,  $\beta_{\tau}$  be given by Theorem 6.1 with  $n$  replaced by  $B_{\tau}$ . Set  $\mathcal{T} = \Omega\left(\frac{\log(1/\delta)}{\alpha_{1}}\right)$  and  $\delta = 1/T$ . Then Algorithm 1 is  $\epsilon$ -DP with its regret upper bound
$$
\mathcal {R} _ {T} = O \left(\sqrt {K T \log T} + \left(\frac {K \log T}{\epsilon}\right) ^ {\frac {k - 1}{k}} T ^ {\frac {1}{k}} + T \alpha_ {1} ^ {1 - \frac {1}{k}} + \frac {K \log T}{\alpha_ {1}}\right).
$$
The above theorem presents the first achievable regret guarantee for private and robust bandits. The first three terms match our lower bound in Theorem 4.1 up to  $\log T$  factor. The last additive term is
Algorithm 3 PRM for the finite central moment case
1: Input: A collection of data  $\{x_{i}\}_{i = 1}^{2n}$ , truncation parameter  $M$ , additional parameters  $\Phi = \{\epsilon, D, r\}, r \in \mathbb{R}$ .
2: // First step: initial estimate
3:  $B_{j} = [j,j + r),j\in \mathcal{J} = \{-D, - D + r,\dots ,D - r\}$
4: Compute private histogram using the first fold of data:  $\widetilde{p}_j = \frac{\sum_{i=1}^n \mathbb{1}_{\{X_i \in B_j\}}}{n} + \mathrm{Lap}\left(\frac{2}{n\epsilon}\right)$ .
5: Get the initial estimate  $J = \arg \max_{j\in \mathcal{J}}\hat{p}_j$
6: // Second step: final estimate
7: Get final estimator using the second fold of data:  $\widetilde{\mu} = J + \frac{1}{n}\sum_{i=n+1}^{2n}(X_i - J)\mathbb{1}_{\{|X_i - J|\leq M\}} + \mathrm{Lap}\left(\frac{2M}{n\epsilon}\right)$ .
mainly due to the fact that the mean concentration result only holds when the sample size is larger than  $\mathcal{T} = \Omega\left(\frac{\log(1 / \delta)}{\alpha_1}\right)$ . As a result, each sub-optimal has to be played at least  $\Omega\left(\frac{\log(1 / \delta)}{\alpha_1}\right)$  times. However, for a sufficiently large  $T$  and a constant  $\alpha$ , the last term is dominated by other terms. The last term is also exactly the reason that we choose an upper bound  $\alpha_{1}$  on the actual contamination level  $\alpha$  and state the upper bound results in terms of  $\alpha_{1}$  rather than  $\alpha$ . That is, for a very small but non-zero  $\alpha$ , one can choose a larger  $\alpha_{1}$  to balance the regret. This subtle issue is also mentioned in one nice related work [42], see the remark after Theorem 7.4 and Remark 5.4 in their work.
Remark 6.3. For the uncontaminated case with  $\alpha = 0$ , using the uncontaminated concentration bound in Theorem 6.1 and the same analysis, we achieve a regret upper bound  $O(\sqrt{KT\log T} + \left(\frac{K\log T}{\epsilon}\right)^{\frac{k - 1}{k}}T^{\frac{1}{k}})$ , which also matches the lower bound in Theorem 4.1 up to a factor of  $O(\log T)$ .
6.2 Finite Central Moment Case
The setting in the last section for the finite raw moment case may not be entirely satisfactory as it essentially assumes that the mean of arms is bounded within a small range (hence the sub-optimal gaps). Thus, in this section, we turn to private and robust MABs where the inlier distributions have a finite  $k$ -th central moment as given by Definition 3.2 for a reward distribution with large mean but small variability around the mean. To this end, we first need a new PRM, since now simply truncating around zero as in Algorithm 2 will not work well (as discussed in detail in Remark 6.13).
Our new PRM is presented in Algorithm 3, which consists of two steps. The intuition is simple: the first step aims to have a rough estimate of the mean, which is necessary since now the mean could be far away from zero. Then, in the second step, it truncates around the initial estimate to return the final result. More specifically, in the first step, we first construct bins over the range  $[-D, D]$ , which is assumed to contain the true mean by Definition 3.2. Then, we compute the private histogram via the Laplace mechanism. The initial estimate is given by the left endpoint of the bin that has the largest empirical mass. Next, in the second step, it simply truncates around the initial estimate and again adds Laplace noise for privacy.
Remark 6.4. It is worth noting that a similar idea of two-step estimation has been used in previous work on robust mean estimation in the one-dimensional heavy-tailed case [33, 38, 40]. However, there are several differences in our algorithm design and analysis. In particular, while [33] considers mean estimation under Huber's model without privacy constraints, we further impose differential privacy requirements. As a result, the estimates for both two steps are in different forms in our case compared to [33], though they share the same high-level intuition. On the other hand, while [38] considers mean estimation under differential privacy, there is no consideration of Huber contamination as in our case. Moreover, our second estimate is based on truncation while their method is via medians-of-means. In fact, as will be shown later (see Remark 6.7), when our result reduces to the uncontaminated case, it achieves improvement over the one in [38]. Finally, [40] considers both Huber contamination and local differential privacy, and establishes the corresponding mean square error (MSE). In contrast, we consider the central differential privacy and aim to establish a high-probability tail concentration. To this end, we take a different truncation method (i.e., using an indicator function in Line 7) compared to the one in [40] (Section 3.2 in its first arxiv version).
As before, we first present the concentration property of our new PRM, which will manifest in the specific instantiation of our meta-algorithm. In particular, we first give the following general theorem and then state two more detailed corollaries.
Theorem 6.5 (Concentration of Mean Estimate). Given a collection of Huber-contaminated data  $\{x_{i}\}_{i = 1}^{2n}$  where the inlier distribution satisfies Definition 3.2 with mean  $\mu$ , let  $\widetilde{\mu}$  be the mean estimate by Algorithm 3. For any  $\alpha \leq \alpha_{1} \in (0, \alpha_{\max})$ ,  $\epsilon \in (0, 1]$  and  $\delta \in (0, 1)$ , there exist some constants  $\mathcal{T}(\alpha_{1}, \epsilon, \delta)$ ,  $r, M$  and  $D \geq 2r$  such that for all  $n \geq \mathcal{T}(\alpha_{1}, \epsilon, \delta)$ , with probability at least  $1 - \delta$
$$
| \widetilde {\mu} - \mu | \leq O \left(\sqrt {\frac {\log (1 / \delta)}{n}} + \frac {M \log (1 / \delta)}{n \epsilon} + \frac {1}{M ^ {k - 1}} + \alpha_ {1} M\right),
$$
where  $\alpha_{\mathrm{max}} < 1 / 2$  is the breakdown point.
The above theorem follows the same pattern as the one for the raw moment case (Theorem 6.1). The key differences are the threshold value  $\mathcal{T}(\alpha, \epsilon, \delta)$  and the breakdown point  $\alpha_{\mathrm{max}}$ , which are summarized in the following results.
Corollary 6.6 (Mean Concentration,  $\alpha = 0$ ). Let the same assumptions in Theorem 6.5 hold. For any  $\epsilon \in (0,1]$ , setting  $r = 10^{1 / k}$  and  $M = \Theta \left(\frac{n\epsilon}{\log(1 / \delta)}\right)^{1 / k}$ , then for all  $n \geq \Omega (\log (D / \delta) / \epsilon)$  and  $D \geq 2r$ , we have that for any  $\delta \in (0,1)$ , with probability at least  $1 - \delta$ ,  $|\tilde{\mu} - \mu| \leq \beta$  where  $\beta = O\left(\sqrt{\frac{\log(1 / \delta)}{n}} + \left(\frac{\log(1 / \delta)}{n\epsilon}\right)^{1 - \frac{1}{k}}\right)$ . In other words, taking number of samples  $n$  such that  $n \geq O\left(\frac{\log(1 / \delta)}{\eta^2} + \frac{\log(1 / \delta)}{\epsilon\eta^{\frac{k}{k - 1}}} + \frac{\log(D / \delta)}{\epsilon}\right)$  we have  $|\tilde{\mu} - \mu| \leq \eta$  with probability at least  $1 - \delta$ .
Remark 6.7. The above lemma strictly improves the result in [38, Theorem 3.5]. In particular, it uses the method of medians-of-means and achieves  $\frac{\log D\cdot\log(1 / \delta)}{\epsilon}$  for the third term. In contrast, our third term is additive rather than multiplicative. In fact, our concentration is optimal, which matches the lower bound for the one-dimensional case (see [46, Theorem 7.2]).
Corollary 6.8 (Mean Concentration,  $\alpha >0$ ). Let the same assumptions in Theorem 6.5 hold. For any  $\epsilon \in (0,1]$  and  $0 < \alpha \leq \alpha_{1} \in (0,0.133)$ , we let  $r = \iota^{1 / k}$  where  $\iota = \frac{1 - \alpha}{0.249 - \alpha}$  and  $M = \Theta (\min \left\{\left(\frac{n\epsilon}{\log(1 / \delta)}\right)^{1 / k},(\alpha_1)^{-1 / k}\right\})$ . Then, there exists constant  $c_{1}$ , for all  $n$  such that  $n \geq \mathcal{T} = \Omega (\max \{\frac{\iota\log(1 / \delta)}{\epsilon},\frac{c_1\log(D / \delta)}{\epsilon},\frac{\log(1 / \delta)}{\alpha_1^2}\})$  and  $D \geq 2r$ , we have that for any  $\delta \in (0,1)$ , with probability at least  $1 - \delta$ ,  $|\widetilde{\mu} -\mu |\leq \beta$  with  $\beta = O\left(\sqrt{\frac{\log(1 / \delta)}{n}} +\left(\frac{\log(1 / \delta)}{n\epsilon}\right)^{1 - \frac{1}{k}} + \alpha_1^{1 - \frac{1}{k}}\right)$ .
Remark 6.9. The above concentration has the same form as the one in Theorem 6.1. Specifically, for a large sample size  $n$ , it has the optimal concentration (for small  $\alpha$ ). The threshold  $\mathcal{T}$  on  $n$  depends on both  $\alpha, \epsilon$  now. We note that even for the sub-Gaussian inlier distributions without privacy protection, the existing concentration also has a threshold  $\mathcal{T} = \frac{\log(1 / \delta)}{\alpha_1^2}$  (see Lemma 4.1 in [30]). The finite central moment assumption will help us get logarithmic results for sample complexity on  $D$  of  $n \geq \Omega (\log (D / \delta) / \epsilon)$  which is better than the polynomial results on  $D$  from the finite raw moment. Actually, finding a private estimator that achieves an error on the logarithmic of the range of the parameter is significant in the topic of DP estimation (see [48-50] on the importance of the problem).
Now, we are left to leverage the above two concentration results to design specific instantiations of our meta-algorithm and establish their performance guarantees. Our first instantiation is for the uncontaminated case, i.e.,  $\alpha = 0$ . Therefore, robustness is then only with respect to heavy-tailed rewards while privacy is still preserved.
Theorem 6.10 (Performance Guarantees,  $\alpha = 0$ ). Consider a private and robust MAB with inlier distributions satisfying Definition 3.2 and  $\alpha = 0$ . Let Algorithm 1 be instantiated with Algorithm 3, and  $r$ ,  $M_{\tau}$ ,  $\beta_{\tau}$  be given by Corollary 6.6 with  $n$  replaced by  $B_{\tau}$ . Set  $\mathcal{T} = \Omega \left( \frac{\log(D / \delta)}{\epsilon} \right)$  and  $\delta = 1 / T$ . Then, Algorithm 1 is  $\epsilon$ -DP with its regret upper bound
$$
\mathcal {R} _ {T} = O \left(\sqrt {K T \log T} + (K \log T / \epsilon) ^ {\frac {k - 1}{k}} T ^ {\frac {1}{k}} + \gamma\right),
$$
where  $\gamma \coloneqq O(KD\log (DT) / \epsilon)$ .
We also note that the main focus of [38] is not on achieving the optimal estimate though.
Remark 6.11. To the best of our knowledge, this is the first result on private and heavy-tailed bandits with the finite central moment assumption. The state-of-the-art result only focuses on the simpler case, i.e., the finite raw moment assumption [22].
Finally, armed with Corollary 6.8, we have the second instantiation of our meta-algorithm that deals with the contaminated case.
Theorem 6.12 (Performance Guarantees,  $\alpha >0$ ). Consider a private and robust MAB with inlier distributions satisfying Definition 3.2 and  $\alpha \leq \alpha_{1}\in (0,0.133)$ . Let Algorithm 1 be instantiated with Algorithm 3, and  $r,T,M_{\tau},\beta_{\tau}$  be given by Corollary 6.8 with  $n$  replaced by  $B_{\tau}$ . Set  $\delta = 1 / T$ , then Algorithm 1 is  $\epsilon$ -DP with its regret upper bound
$$
\mathcal {R} _ {T} = O \left(\sqrt {K T \log T} + (K \log T / \epsilon) ^ {\frac {k - 1}{k}} T ^ {\frac {1}{k}} + T \alpha_ {1} ^ {1 - \frac {1}{k}} + \hat {\gamma}\right),
$$
where  $\hat{\gamma} \coloneqq O\left(\frac{DK\log T}{\alpha_1^2} + \frac{\iota DK\log T}{\epsilon} + \frac{DK\log(DT)}{\epsilon}\right)$  and  $\iota = \frac{1 - \alpha}{0.249 - \alpha}$ .
Remark 6.13. The above upper bound also matches our lower bound up to log factor and  $O(\hat{\gamma})$ , which is dominated by other terms for a sufficiently large  $T$  and small  $\alpha, D$ . A comparison between the results in Theorem 6.2 (Remark 6.3) and Theorem 6.12 (Theorem 6.10) for contaminated case (uncontaminated case), respectively, allows us to address a fundamental question: why we do not first transform the central moment condition to the raw moment condition, and then simply apply Algorithms 1 and 2 to get a regret upper bound? This is because the central moment condition implies  $\mathbb{E}[|X|^k] = O(D^k)$ . By employing a simple scaling technique, Algorithms 1 and 2 would yield a regret bound of  $O(D \cdot \text{Reg})$ , where Reg corresponds to the bound stated in Theorem 6.2. Thus, a significant contrast arises when comparing this result with Theorem 6.12, where the dependence on  $D$  is only an additive linear term rather than a multiplicative factor of  $O(D)$ . Consequently, the bounds presented in Theorem 6.12 and Theorem 6.10 prove to be substantially tighter when  $D$  assumes larger values, hence highlighting the contributions of our new PRM module in Algorithm 3.
7 Simulations and Conclusion
We refer readers to Appendix A for our simulation results. In this paper, we investigated private and robust multi-armed bandits with heavy-tailed rewards under Huber's contamination model as well as differential privacy constraints. We proposed a meta-algorithm that builds on a private and robust mean estimation sub-routine PRM for different heavy-tailed assumptions of rewards, i.e. finite  $k$ -th raw moment and finite  $k$ -th central moment with  $k \geq 2$ . Moreover, we also established regret upper bounds for these algorithms, which nearly match our derived minimax lower bound.
There remain many open questions in this direction. First, the problem-dependent lower bound is unclear for private and robust bandits but some work has been done for corrupted heavy-tailed bandits (see Theorem 1 in [17]). One interesting future direction is to study how to leverage both the insights in [17] and the lower bound under privacy to derive a problem-dependent lower bound for private and robust bandits. Second, how to design algorithms and analyze the theoretical results for private and robust bandits under other corrupted models such as "bandits with total corruption budget" in [15]. Third, throughout the paper  $T$  is known in advance, and it remains to see how to handle the case where  $T$  is unknown and infinity.
We thank Mengchu Li for the insightful discussions and for pointing out the first arxiv version of [40]. And we also thank the anonymous NeurIPS reviewers for their feedback. Xingyu Zhou is supported in part by NSF grant CNS-2153220. Di Wang and Yulian Wu were supported in part by the baseline funding BAS/1/1689-01-01, funding from the CRG grand URF/1/4663-01-01, FCC/1/1976-49-01 from CBRC of King Abdullah University of Science and Technology (KAUST). Di Wang was also supported by the funding of the SDAIA-KAUST Center of Excellence in Data Science and Artificial Intelligence (SDAIA-KAUST AI)
[40] Mengchu Li, Thomas B Berrett, and Yi Yu. On robustness and local differential privacy. arXiv preprint arXiv:2201.00751, 2022.
[41] Nikita Mishra and Abhradeep Thakurta. (Nearly) optimal differentially private stochastic multi-arm bandits. In Proceedings of the 31st Conference on Uncertainty in Artificial Intelligence (UAI), pages 592-601, 2015.
[42] Sitan Chen, Frederic Koehler, Ankur Moitra, and Morris Yau. Online and distribution-free robustness: Regression and contextual bandits with huber contamination. In 2021 IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS), pages 684-695. IEEE, 2022.
[43] Eyal Even-Dar, Shie Mannor, Yishay Mansour, and Sridhar Mahadevan. Action elimination and stopping conditions for the multi-armed bandit and reinforcement learning problems. Journal of Machine Learning Research, 7(6), 2006.
[44] T-H Hubert Chan, Elaine Shi, and Dawn Song. Private and continual release of statistics. ACM Transactions on Information and System Security (TISSEC), 14(3):1-24, 2011.
[45] Bingshan Hu and Nidhi Hegde. Near-optimal thompson sampling-based algorithms for differentially private stochastic bandits. In Uncertainty in Artificial Intelligence, pages 844-852. PMLR, 2022.
[46] Samuel B Hopkins, Gautam Kamath, and Mahbod Majid. Efficient mean estimation with pure differential privacy via a sum-of-squares exponential mechanism. In Proceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing, pages 1406-1417, 2022.
[47] Ilias Diakonikolas. Algorithmic high-dimensional robust statistics. Webpage http://www.iliasdiakonikolas.org/simons-tutorial-robust.html, 2018.
[48] Gautam Kamath, Jerry Li, Vikrant Singhal, and Jonathan Ullman. Privately learning high-dimensional distributions. In Conference on Learning Theory, pages 1853-1902. PMLR, 2019.
[49] Vishesh Karwa and Salil Vadhan. Finite sample differentially private confidence intervals. arXiv preprint arXiv:1711.03908, 2017.
[50] Gautam Kamath, Xingtu Liu, and Huanyu Zhang. Improved rates for differentially private stochastic convex optimization with heavy-tailed data. In International Conference on Machine Learning, pages 10633-10660. PMLR, 2022.
[51] Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sci., 9(3-4):211-407, 2014.
[52] Joseph P Near and Chike Abuah. Programming differential privacy. URL: https://uvm, 2021.
[53] Roman Vershynin. High-dimensional probability: An introduction with applications in data science, volume 47. Cambridge university press, 2018.
[54] Mengjie Chen, Chao Gao, and Zhao Ren. Robust covariance and scatter matrix estimation under huber's contamination model. The Annals of Statistics, 46(5):1932-1960, 2018.
[55] Tor Lattimore and Csaba Szepesvári. Bandit algorithms. Cambridge University Press, 2020.