SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Karim Lounici
Ecole Polytechnique
karim.lounici@polytechnique.edu
Gregoire Pacreau
gregoire.pacreau@polytechnique.edu
Large datasets are often affected by cell-wise outliers in the form of missing or erroneous data. However, discarding any samples containing outliers may result in a dataset that is too small to accurately estimate the covariance matrix. Moreover, the robust procedures designed to address this problem require the invertibility of the covariance operator and thus are not effective on high-dimensional data. In this paper, we propose an unbiased estimator for the covariance in the presence of missing values that does not require any imputation step and still achieves near minimax statistical accuracy with the operator norm. We also advocate for its use in combination with cell-wise outlier detection methods to tackle cell-wise contamination in a high-dimensional and low-rank setting, where state-of-the-art methods may suffer from numerical instability and long computation times. To complement our theoretical findings, we conducted an experimental study which demonstrates the superiority of our approach over the state of the art both in low and high dimension settings.
Outliers are a common occurrence in datasets, and they can significantly affect the accuracy of data analysis. While research on outlier detection and treatment has been ongoing since the 1960s, much of it has focused on cases where entire samples are outliers (Huber's contamination model) [11,41,13]. While sample-wise contamination is a common issue in many datasets, modern data analysis often involves combining data from multiple sources. For example, data may be collected from an array of sensors, each with an independent probability of failure, or financial data may come from multiple companies, where reporting errors from one source do not necessarily impact the validity of the information from the other sources. Discarding an entire sample as an outlier when only a few features are contaminated can result in the loss of valuable information, especially in high-dimensional datasets where samples are already scarce. It is important to identify and address the specific contaminated features, rather than simply treating the entire sample as an outlier. In fact, if each dimension of a sample has a contamination probability of  $\varepsilon$ , then the probability of that sample containing at least one outlier is given by  $1 - (1 - \varepsilon)^p$ , where  $p$  is the dimensionality of the sample. In high dimension, this probability can quickly exceed  $50\%$ , surpassing the breakdown point of many robust estimators designed for the Huber sample-wise contamination setting. Hence, it is crucial to develop robust methods that can handle cell-wise contaminations and still provide accurate results.
The issue of cell-wise contamination, where individual cells in a dataset may be contaminated, was first introduced in [3]. However, the issue of missing data due to outliers was studied much earlier, dating back to the work of [35]. Although missing values in a dataset are much easier to detect than outliers, they can lead to errors in estimating the location and scale of the underlying distribution [25]
Figure 1: Left: Estimation error of the covariance matrix for  $n = 100$ ,  $p = 50$ ,  $\mathbf{r}(\Sigma) = 2$  under a Dirac contamination (tailMV and DDCMV are our methods). Here  $\varepsilon = 1$  and  $\delta$  varies in  $(0,1)$ . Right: For each method, mean computation time (in seconds) over 20 repetitions and whether it uses matrix inversion. For  $p = 100$ , we had to raise  $r(\Sigma)$  to 10 otherwise both DI and TSGS would fail due to numerical instability.
Estimator	Computation time	Matrix inversion
p=50	p=100
tailMV	10-3± 10-4	10-2± 10-3	no
DDCMV	0.6± 10-3	0.7± 0.007	no
DI	6± 0.5	74± 4	yes
TSGS	20± 0.2	200± 10	yes
and can negatively affect the performance of supervised learning algorithms [17]. This motivated the development of the field of data imputation. Several robust estimation methods have been proposed to handle missing data, including Expectation Maximization (EM)-based algorithms [7], maximum likelihood estimation [14] and Multiple Imputation [25], among which we can find k-nearest neighbor imputation [39] and iterative imputation [42]. Recently, sophisticated solutions based on deep learning, GANs [47, 28, 9], VAE [27] or Diffusion schemes [48] have been proposed to perform complex tasks like artificial data generation or image inpainting. The aforementioned references focus solely on minimising the entrywise error for imputed entries. Noticeably, our practical findings reveal that applying state-of-the-art imputation methods to complete the dataset, followed by covariance estimation on the completed dataset, does not yield satisfactory results when evaluating the covariance estimation error using the operator norm.
In comparison to data missingness or its sample-wise counterpart, the cell-wise contamination problem is less studied. The Detection Imputation (DI) algorithm of [32] is an EM type procedure combining a robust covariance estimation method with an outlier detection method to iteratively update the covariance estimation. Other methods include adapting methodology created for Huber contamination for the cell-wise problem, such as in [6] or [2]. In high dimensional statistics, however, most of these methods fail due to high computation time and numerical instability. Or they are simply not designed to work in this regime since they are based on the Mahalanobis distance, which requires an inversion of the estimated covariance matrix. This is a major issue since classical covariance matrix estimators have many eigenvalues close to zero or even exactly equal to zero in high-dimension. To the best of our knowledge, no theoretical result exists concerning the statistical accuracy of these methods in the cell-wise contamination setting contrarily to the extensive literature on Huber's contamination [1].
Contributions. In this paper we address the problem of high-dimensional covariance estimation in the presence of missing observations and cell-wise contamination. To formalize this problem, we adopt and generalize the setting introduced in [10]. We propose and investigate two different strategies, the first based on filtering outliers and debiasing and the second based on filtering outliers followed by imputation and standard covariance estimation. We propose novel computationally efficient and numerically stable procedures that avoid matrix inversion, making them well-suited for high-dimensional data. We derive non-asymptotic estimation bounds of the covariance with the operator norm and minimax lower bounds, which clarify the impact of the missing value rate and outlier contamination rate. Our theoretical results also improve over [26] in the MCAR and no contamination. Next, we conduct an experimental study on synthetic data, comparing our proposed methods to the state-of-the-art (SOTA) methods. Our results demonstrate that SOTA methods fail in the high-dimensional regime due to matrix inversions, while our proposed methods perform well in this regime, highlighting their effectiveness. Then we demonstrate the practical utility of our approach by applying it to real-life datasets, which highlights that the use of existing estimation methods significantly alters the spectral properties of the estimated covariance matrices. This implies
that cell-wise contamination can significantly impact the results of dimension reduction techniques like PCA by completely altering the computed principal directions. Our experiments demonstrate that our methods are more robust to cell-wise contamination than SOTA methods and produce reliable estimates of the covariance.
2 Missing values and cell-wise contamination setting
Let  $X_{1}, \ldots, X_{n}$  be  $n$  i.i.d. copies of a zero mean random vector  $X$  admitting unknown covariance operator  $\Sigma = \mathbb{E}[X \otimes X]$ , where  $\otimes$  is the outer product. Denote by  $X_{i}^{(j)}$  the  $j$ th component of vector  $X_{i}$  for any  $j \in [p]$ . All our results are non-asymptotic and cover a wide range of configurations for  $n$  and  $p$  including the high-dimensional setting  $p \gg n$ . In this paper, we consider the following two realistic scenarios where the measurements are potentially corrupted.
Missing values. We assume that each component  $X_{i}^{(j)}$  is observed independently from the others with probability  $\delta \in (0,1]$ . Formally, we observe the random vector  $Y \in \mathbb{R}^p$  defined as follows:
$$
Y _ {i} ^ {(j)} = d _ {i, j} X _ {i} ^ {(j)}, 1 \leq i \leq n, 1 \leq j \leq p \tag {1}
$$
where  $d_{ij}$  are independent realisations of a bernoulli random variable of parameter  $\delta$ . This corresponds the Missing Completely at Random (MCAR) setting of [35]. Our theory also covers the more general Missing at Random (MAR) setting in Theorem 2.
Cell-wise contamination. Here we assume that some missing components  $X_{i}^{(j)}$  can be replaced with probability  $\varepsilon$  by some independent noise variables, representing either a poisoning of the data or random mistakes in measurements. The observation vector  $Y$  then satisfies:
$$
Y _ {i} ^ {(j)} = d _ {i, j} X _ {i} ^ {(j)} + \left(1 - d _ {i, j}\right) e _ {i, j} \xi_ {i} ^ {(j)}, 1 \leq i \leq n, 1 \leq j \leq p \tag {2}
$$
where  $\xi_i^{(j)}$  are independent erroneous measurements and  $e_{i,j}$  are i.i.d. bernoulli random variables with parameter  $\varepsilon$ . We also assume that all the variables  $X_{i},\xi_{i}^{(j)},d_{i,j},e_{i,j}$  are mutually independent. In this scenario, a component  $X_{i}^{(j)}$  is either perfectly observed with probability  $\delta$ , replaced by a random noise with probability  $\varepsilon^{\prime} = \varepsilon (1 - \delta)$  or missing with probability  $(1 - \delta)(1 - \varepsilon)$ . Cell-wise contamination as introduced in [3] corresponds to the case where  $\varepsilon = 1$ , and thus  $\varepsilon^{\prime} = 1 - \delta$ .
In both of these settings, the task of estimating the mean of the random vectors  $X_{i}$  is well-understood, as it reduces to the classical Huber setting for component-wise mean estimation. One could for instance apply the Tuker median on each component separately [3]. However, the problem becomes more complex when we consider non-linear functions of the data, such as the covariance operator. Robust covariance estimators originally designed for the Huber setting may not be suitable when applied in the presence of missing values or cell-wise contaminations.
We study a simple estimator based on a correction of the classical covariance estimator on  $Y_{1},\ldots ,Y_{n}$  as introduced in [26] for the missing values scenario. The procedure is based on the following observation, linking  $\Sigma^{Y}$  the covariance of the data with missing values and  $\Sigma$  the true covariance:
$$
\Sigma = \left(\delta^ {- 1} - \delta^ {- 2}\right) \operatorname {d i a g} \left(\Sigma^ {Y}\right) + \delta^ {- 2} \Sigma^ {Y} \tag {3}
$$
Note that this formula assumes the knowledge of  $\delta$ . In the missing values scenario,  $\delta$  can be efficiently estimated by a simple count of the values exactly set to 0 or equal to NaN (not a number). In the contamination setting (2), the operator  $\Sigma^{Y} = \mathbb{E}\left(Y\otimes Y\right)$  satisfies, for  $\Lambda = \mathbb{E}\left[\xi \otimes \xi \right]$ :
$$
\Sigma^ {Y} = \delta^ {2} \Sigma + (\delta - \delta^ {2}) \mathrm {d i a g} (\Sigma) + \varepsilon (1 - \delta) \Lambda .
$$
In this setting, as one does not know the exact location and number of outliers we propose to estimate  $\delta$  by the proportion of data remaining after the application of a filtering procedure.
Notations. We denote by  $\odot$  the Hadamard (or term by term) product of two matrices and by  $\otimes$  the outer product of vectors, i.e.  $\forall x, y \in \mathbb{R}^d, x \otimes y = xy^\top$ . We denote by  $\|.\|$  and  $\|.\|_F$  the operator and Frobenius norms of a matrix respectively. We denote by  $\|.\|_2$  the vector  $l_2$ -norm.
3 Estimation of covariance matrices with missing values
We consider the scenario outlined in [1] where the matrix  $\Sigma$  is of approximately low rank. To quantify this, we use the concept of effective rank, which provides a useful measure of the inherent complexity of a matrix. Specifically, the effective rank of  $\Sigma$  is defined as follows
$$
\boldsymbol {r} (\Sigma) := \frac {\mathbb {E} \| X \| _ {2} ^ {2}}{\| \Sigma \|} = \frac {\operatorname {t r} (\Sigma)}{\| \Sigma \|} \tag {4}
$$
We note that  $0 \leq r(\Sigma) \leq \mathrm{rank}(\Sigma)$ . Furthermore, for approximately low rank matrices with rapidly decaying eigenvalues, we have  $r(\Sigma) \ll \mathrm{rank}(\Sigma)$ . This section presents a novel analysis of the estimator defined in equation (3), which yields a non-asymptotic minimax optimal estimation bound in the operator norm. Our findings represent a substantial enhancement over the suboptimal guarantees reported in [26, 18]. Similar results could be established for the Frobenius norm using more straightforward arguments, as those in [4] or [31]. We give priority to the operator norm since it aligns naturally with learning tasks such as PCA. See [19, 21, 22] and the references cited therein.
We need the notion of Orlicz norms. For any  $\alpha \geq 1$ , the  $\psi_{\alpha}$ -norms of a real-valued random variable  $V$  are defined as:  $\| V\|_{\psi_\alpha} = \inf \{u > 0, \mathbb{E}\exp(|V|^{\alpha} / u^{\alpha}) \leq 2\}$ . A random vector  $X \in \mathbb{R}^p$  is sub-Gaussian if and only if  $\forall x \in \mathbb{R}^p$ ,  $\| \langle X, x \rangle \|_{\psi_2} \lesssim \| \langle X, x \rangle \|_{L^2}$ .
Minimax lower-bound. We now provide a minimax lower bound for the covariance estimation with missing values problem. Let  $S_{p}$  the set of  $p \times p$  symmetric semi-positive matrices. Then, define  $\mathcal{C}_{\overline{r}} = \{S \in S_{p} : r(S) \leq \overline{r}\}$  the set of matrices of  $S_{p}$  with effective rank at most  $\overline{r}$ .
Theorem 1. Let  $p, n, \overline{r}$  be strictly positive integers such that  $p \geq \max \{n, 2\overline{r}\}$ . Let  $X_1, \ldots, X_n$  be i.i.d. random vectors in  $\mathbb{R}^p$  with covariance matrix  $\Sigma \in \mathcal{C}_{\overline{r}}$ . Let  $(d_{i,j})_{1 \leq i < n, 1 \leq j \leq p}$  be an i.i.d. sequence of Bernoulli random variables with probability of success  $\delta \in (0, 1]$ , independent from the  $X_1, \ldots, X_n$ . We observe  $n$  i.i.d. vectors  $Y_1, \ldots, Y_n \in \mathbb{R}^p$  such that  $Y_i^{(j)} = d_{i,j}X_i^{(j)}, i \in [n], j \in [p]$ . Then there exists two absolute constants  $C > 0$  and  $\beta \in (0, 1)$  such that:
$$
\inf  _ {\widehat {\Sigma}} \max  _ {\Sigma \in \mathcal {C} _ {\overline {{r}}}} \mathbb {P} _ {\Sigma} \left(\left\| \widehat {\Sigma} - \Sigma \right\| \geq C \frac {\| \Sigma \|}{\delta} \sqrt {\frac {\boldsymbol {r} (\Sigma)}{n}}\right) \geq \beta \tag {5}
$$
where  $\inf_{\widehat{\Sigma}}$  represents the infimum over all estimators  $\widehat{\Sigma}$  of matrix  $\Sigma$  based on  $Y_{1},\ldots ,Y_{n}$ .
Sketch of proof. We first build a sufficiently large test set of hard-to-learn covariance operators exploiting entropy properties of the Grassmann manifold such that the distance between any two distinct covariance operator is at least of the order  $\frac{\|\Sigma\|}{\delta}\sqrt{\frac{r(\Sigma)}{n}}$ . Next, in order to control the Kullback-Leibler divergence of the observations with missing values, we exploit in particular interlacing properties of the eigenvalues of the perturbed covariance operators [37].
This lower bound result improves upon [26, Theorem 2] as it relaxes the hypotheses on  $n$  and  $\overline{r}$ . More specifically, the lower bound in [26] requires  $n \geq 2\overline{r}^2/\delta^2$  while we only need the mild assumption  $p \geq \max \{n, 2\overline{r}\}$ . Our proof leverages the properties of the Grassmann manifold, which has been previously utilized in different settings such as sparse PCA without missing values or contamination [45] and low-rank covariance estimation without missing values or contamination [23]. However, tackling missing values in the Grassmann approach adds a technical challenge to these proofs as they modify the distribution of observations. Our proof requires several additional nontrivial arguments to control the distribution divergences, which is a crucial step in deriving the minimax lower bound.
Non-asymptotic upper-bound in the operator norm. We provide an upper bound of the estimation error in operator norm. We write  $Y_{i} = d_{i}\odot X_{i}$ . Let  $\widehat{\Sigma}^{Y} = n^{-1}\sum_{i = 1}^{n}Y_{i}\otimes Y_{i}$  be the classical covariance estimator of the covariance of  $Y$ . When the dataset contains missing values and corruptions,  $\widehat{\Sigma}^{Y}$  is a biased estimator of  $\Sigma$ . Exploiting Equation (3), [26] proposed the following unbiased estimator of the covariance matrix  $\Sigma$ :
$$
\widehat {\Sigma} = \delta^ {- 2} \widehat {\Sigma} ^ {Y} + \left(\delta^ {- 1} - \delta^ {- 2}\right) \operatorname {d i a g} \left(\widehat {\Sigma} ^ {Y}\right). \tag {6}
$$
The following result is from [18, Theorem 4.2].
Lemma 1. Let  $X_{1},\ldots ,X_{n}$  be i.i.d. sub-Gaussian random variables in  $\mathbb{R}^p$  with covariance matrix  $\Sigma$  and let  $d_{ij},i\in [1,n],j\in [1,p]$  be i.i.d bernoulli random variables with probability of success  $\delta >0$ . Then there exists an absolute constant  $C$  such that, for  $t > 0$ , with probability at least  $1 - e^{-t}$ :
$$
\left\| \widehat {\Sigma} - \Sigma \right\| \leq C \| \Sigma \| \left(\sqrt {\frac {\boldsymbol {r} (\Sigma) \log \boldsymbol {r} (\Sigma)}{\delta^ {2} n}} \vee \sqrt {\frac {t}{\delta^ {2} n}} \vee \frac {\boldsymbol {r} (\Sigma) (t + \log \boldsymbol {r} (\Sigma))}{\delta^ {2} n} \log (n)\right) \tag {7}
$$
This result uses a recent unbounded version of the non-commutative Bernstein inequality, thus yielding some improvement upon the previous best known bound of [26]. Theorem and Lemma provide some important insights on the minimax rate of estimation in the missing values setting. In the high-dimensional regime  $p \geq \max \{n, 2\overline{r}\}$  and  $n \geq \delta^{-2}\mathbf{r}(\Sigma)(\log \mathbf{r}(\Sigma))\log^2 n$ , we observe that the two bounds coincide up to a logarithmic factor in  $\mathbf{r}(\Sigma)$ , hence clarifying the impact of missing data on the estimation rate via the parameter  $\delta$ .
Heterogeneous missingness. We can extend the correction to the more general case where each feature has a different missing value rate known as the Missing at Random (MAR) setting in [35]. We denote by  $\delta_j \in (0,1]$  the probability to observe feature  $X^{(j)}$ ,  $1 \leq j \leq p$  and we set  $\delta \coloneqq (\delta_j)_{j \in [p]}$ . As in the MCAR setting, the probabilities  $(\delta_j)_{j \in [p]}$  can be readily estimated by tallying the number of missing entries for each feature. Hence they will be assumed to be known for the sake of brevity. Let  $\delta_{\mathrm{inv}} = (\delta_j^{-1})_{j \in [p]}$  be the vector containing the inverse of the observing probabilities and  $\Delta_{\mathrm{inv}} = \delta_{\mathrm{inv}} \otimes \delta_{\mathrm{inv}}$ . In this case, the corrected estimator becomes:
$$
\widehat {\Sigma} = \Delta_ {\text {i n v}} \odot \widehat {\Sigma} ^ {Y} + (\operatorname {d i a g} (\delta_ {\text {i n v}}) - \Delta_ {\text {i n v}}) \odot \operatorname {d i a g} (\widehat {\Sigma} ^ {Y}) \tag {8}
$$
Let  $\bar{\delta} = \max_{j}\{\delta_{j}\}$  and  $\underline{\delta} = \min_{j}\{\delta_{j}\}$  be the largest and smallest probabilities to observe a feature. Theorem 2. (i) Let  $X_{1},\ldots ,X_{n}$  be i.i.d. sub-Gaussian random variables in  $\mathbb{R}^p$  with covariance matrix  $\Sigma$ . We consider the MAR setting described above. Then the estimator (8) satisfies, for any  $t > 0$ , with probability at least  $1 - e^{-t}$
$$
\left\| \widehat {\Sigma} - \Sigma \right\| \leq C \| \Sigma \| \frac {\bar {\delta}}{\bar {\delta} ^ {2}} \left(\sqrt {\frac {\boldsymbol {r} (\Sigma) \log \boldsymbol {r} (\Sigma)}{n}} \vee \sqrt {\frac {t}{n}} \vee \frac {\boldsymbol {r} (\Sigma) (t + \log \boldsymbol {r} (\Sigma))}{\bar {\delta} n} \log n\right) \tag {9}
$$
(ii) Let  $p, n, \overline{r}$  be strictly positive integers such that  $p \geq \max\{n, 2\overline{r}\}$ . Let  $X_1, \ldots, X_n$  be i.i.d. random vectors in  $\mathbb{R}^p$  with covariance matrix  $\Sigma \in \mathcal{C}_{\overline{r}}$ . Then,
$$
\inf  _ {\widehat {\Sigma}} \max  _ {\Sigma \in \mathcal {C} _ {\overline {{r}}}} \mathbb {P} _ {\Sigma} \left(\left\| \widehat {\Sigma} - \Sigma \right\| \geq C \frac {\| \Sigma \|}{\bar {\delta}} \sqrt {\frac {\boldsymbol {r} (\Sigma)}{n}}\right) \geq \beta . \tag {10}
$$
If  $\bar{\delta} \asymp \underline{\delta}$  then the rates for the MCAR and MAR settings match. The proof is a straightforward adaptation of the proof in the MCAR setting.
4 Optimal estimation of covariance matrices with cell-wise contamination
In this section, we consider the cell-wise contamination setting (2). We derive both an upper bound on the operator norm error of the estimator (6) and a minimax lower bound for this specific setting. Let us assume that the  $\xi_1, \ldots, \xi_n$  are sub-Gaussian r.v. Note also that  $\Lambda := \mathbb{E}[\xi_1 \otimes \xi_1]$  is diagonal in the cell-wise contamination setting (2).
Minimax lower-bound. The lower bound for missing values still applies to the contaminated case as missing values are a particular case of cell-wise contamination. But we want a more general lower bound that also covers the case of adversarial contaminations.
Theorem 3. Let  $p, n, \overline{r}$  be strictly positive integers such that  $p \geq \max \{n, 2\overline{r}\}$ . Let  $X_1, \ldots, X_n$  be i.i.d. random vectors in  $\mathbb{R}^p$  with covariance matrix  $\Sigma \in \mathcal{C}_{\overline{r}}$ . Let  $(d_{i,j})_{1 \leq i \leq n, 1 \leq j \leq p}$  be i.i.d. sequence of bernoulli random variables of probability of success  $\delta \in (0,1]$ , independent to the  $X_1, \ldots, X_n$ . We observe  $n$  i.i.d. vectors  $Y_1, \ldots, Y_n \in \mathbb{R}^p$  satisfying (2) where  $\xi_i$  are i.i.d. of arbitrary distribution  $Q$ . Then there exists two absolute constants  $C > 0$  and  $\beta \in (0,1)$  such that:
$$
\inf  _ {\hat {\Sigma}} \max  _ {\Sigma \in \mathcal {C} _ {\bar {r}}} \max  _ {Q} \mathbb {P} _ {\Sigma , Q} \left(\left\| \widehat {\Sigma} - \Sigma \right\| \geq C \frac {\| \Sigma \|}{\delta} \sqrt {\frac {\boldsymbol {r} (\Sigma)}{n}} \bigvee \frac {\varepsilon (1 - \delta)}{\delta}\right) \geq \beta \tag {11}
$$
where  $\inf_{\Sigma}$  represents the infimum over all estimators of matrix  $\Sigma$  and  $\max_Q$  is the maximum over all contamination  $Q$ .
The proof of this theorem adapts an argument developed to derive minimax lower bounds in the Huber contamination setting. See App. G.3 for the full proof.
Non-asymptotic upper-bound in the operator norm. Note that the term  $\varepsilon (1 - \delta)\Lambda$  in the cell-wise contamination setting is negligible when  $\delta \approx 1$  or  $\varepsilon \approx 0$ . Using the DDC detection procedure of [32], we can detect the contaminations and make  $\varepsilon$  smaller without decreasing  $\delta$  too much. For simplicity, we assume from now on that the  $\xi_i^{(j)}$  are i.i.d. with common variance  $\sigma_{\xi}^{2}$ . Hence  $\Lambda = \sigma_{\xi}^{2}I_{p}$ . We further assume that the  $\xi_i^{(j)}$  are sub-Gaussian since we observed in our experiments that filtering removed all the large-valued contaminated cells and only a few inconspicuous contaminated cells remained. Our procedure satisfies the following result.
Theorem 4. Let the assumptions of Theorem be satisfied. We assume in addition that the observations  $Y_{1},\ldots ,Y_{n}$  satisfy (2) with  $\varepsilon \in [0,1)$  and  $\delta \in (0,1]$  and i.i.d. sub-Gaussian  $\xi_i^{(j)}$ s. Then, for any  $t > 0$ , with probability at least  $1 - e^{-t}$ :