SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Youguang Chen George Biros
Oden Institute for Computational Engineering and Sciences
We investigate theory and algorithms for pool-based active learning for multiclass classification using multinomial logistic regression. Using finite sample analysis, we prove that the Fisher Information Ratio (FIR) lower and upper bounds the excess risk. Based on our theoretical analysis, we propose an active learning algorithm that employs regret minimization to minimize the FIR. To verify our derived excess risk bounds, we conduct experiments on synthetic datasets. Furthermore, we compare FIRAL with five other methods and found that our scheme outperforms them: it consistently produces the smallest classification error in the multiclass logistic regression setting, as demonstrated through experiments on MNIST, CIFAR-10, and 50-class ImageNet.
Active learning is of interest in applications with large pools of unlabeled data for which labeling is expensive. In pool active learning, we're given a set of unlabeled points  $U$ , an initial set of labeled points  $S_0$ , and a budget of new points  $b$ , our goal is to algorithmically select  $b$  new points to label in order to minimize the log-likelihood error over the unlabeled points. Equivalently instead of selecting points directly, we seek to find a probability density function that we can use to sample the  $b$  points. Informally (precise formulation in §2), let  $x$  denote a data point and  $p(x)$  denote the distribution density of unlabeled points. Let  $q(x)$  be the sampling distribution we will use to select the new  $b$  points for labeling. We will choose  $q(x)$  in order to minimize the generalization error (or excess risk) of the classifier over  $p(x)$ . Our theory is classifier specific: it assumes multinomial logistic regression with parameters  $\theta$ . The expectations of the Hessian—with respect to  $\theta$ —of the classifier loss function over  $p(x)$  and  $q(x)$  distributions are denoted by  $\mathbf{H}_p$  and  $\mathbf{H}_q$  respectively. Using finite sample analysis, our first result (Theorem 3 in §3) is to show that the unlabeled data excess risk is bounded below and above by the Fisher information ratio  $\mathrm{Trace}(\mathbf{H}_q^{-1}\mathbf{H}_p)$ , subject to the assumption of  $p$  being a sub-Gaussian distribution. Our second result (Theorem 4 in §4) is to propose and analyze a point selection algorithm based on regret minimization that allows us to bound the generalization error.
There is a large body of work on various active learning methods based on uncertainty estimation ([1, 2, 3]), sample diversity ([4, 5]), Bayesian inference ([6, 7]), and many others ([8]). Here we just discuss the papers closest to our scheme. Zhang and Oles [9] claimed without proof that FIR is asymptotically proportional to the log-likelihood error of unlabeled data. Sourati et al. [10] proved that FIR is an upper bound of the expected variance of the asymptotic distribution of the log-likelihood error. Chaudhuri et al. [11] proved non-asymptotic results indicating that FIR is closely related to the expected log-likelihood error of an Maximum Likelihood Estimation-based classifier in bounded domain. In this work, we use finite sample analysis to establish FIR-based bounds for the excess risk in the case of multinomial logistic regression with sub-Gaussian assumption for the point distributions.
Alge. rically finding points to minimize FIR is an NP-hard combinatorial optimization problem. There have been several approximate algorithms proposed for this problem. Hoi et al. [12] studied the binary classification problem and approximated the FIR using a submodular function and then used a greedy optimization algorithm. Chaudhuri et al. [11] proposed an algorithm that first solves a relaxed continuous convex optimization problem, followed by randomly sampling from the weights. Although they derived a performance guarantee for their approach, it needs a substantial number of samples to approach near-optimal performance solely through random sampling from the weights, and no numerical experiments results were provided using such approach. Ash et al. [13] adopted a forward greedy algorithm to initially select an excess of points and then utilized a backward greedy algorithm to remove surplus points. But such approach has no performance guarantee. Hence, there is still a need for computationally efficient algorithms that can optimize FIR in a multi-class classification context while providing provable guarantees.
Our proposed algorithm, FIRAL, offers a locally near-optimal performance guarantee in terms of selecting points to optimize FIR. In our algorithm we have two steps: first we solve a continuous convex relaxation of the original problem in which we find selection weights for all points in  $U$ . Then given these weights, we select  $b$  points for labeling by a regret minimization approach. This two-step scheme is inspired by Allen-Zhu et al. [14] where a similar approach was used selecting points for linear regression. Extending this approach to active learning for multinomial logistic regression has two main challenges. Firstly, we need to incorporate the information from previously selected points in each new round of active learning. Additionally, while the original approach selects features of individual points, in logistic regression, we need to select a Fisher information matrix  $(\mathbf{H}_q)$ , which complicates the computation and derivation of theoretical performance guarantees. In Section 4 we present our approach in addressing these challenges.
Our Contributions. 1 In §3 we prove that FIR is a lower and upper bound of the excess risk for multinomial logistic regression under sub-Gaussian assumptions. 2 In §4 we detail our FIR Active Learning algorithm (FIRAL) and prove it selects  $b$  points that lead to a bound to the excess risk. 3 In §5 we evaluate our analysis empirically on synthetic and real world datasets: MNIST, CIFAR-10, and ImageNet using a subset of 50 classes. We compare FIRAL with several other methods for pool-based active learning.
We denote a labeled sample as a pair  $(x,y)$ , where  $x\in \mathbb{R}^d$  is a data point,  $y\in \{1,2,\dots ,c\}$  is its label, and  $c$  is the number of classes. Let  $\theta \in \mathbb{R}^{(c - 1)\times d}$  be the parameters of a  $c$ -class logistic regression classifier. Given  $x$  and  $\theta$ , the likelihood of the label  $y$  is defined by
$$
p (y | x, \theta) = \left\{ \begin{array}{l l} \frac {\exp \left(\theta_ {y} ^ {\top} x\right)}{1 + \sum_ {l \in [ c - 1 ]} \exp \left(\theta_ {l} ^ {\top} x\right)}, & y \in [ c - 1 ] \\ \frac {1}{1 + \sum_ {l \in [ c - 1 ]} \exp \left(\theta_ {l} ^ {\top} x\right)}, & y = c. \end{array} \right. \tag {1}
$$
We use the negative log-likelihood as the loss function:  $\ell_{(x,y)}(\theta) \triangleq -\log p(y|x,\theta)$ . To simplify notation we define  $\widetilde{d} = d(c - 1)$ . We derive standard expressions for the gradient  $\nabla \ell_{(x,y)}(\theta) \in \mathbb{R}^{(c - 1)\times d}$  and Hessian  $\nabla^2\ell_{(x,y)}(\theta) \in \mathbb{R}^{\widetilde{d}\times \widetilde{d}}$  in the Appendix B.1 (Proposition 23). We assume there exists  $\theta_*$  such that  $p(y|x) = p(y|x,\theta_*)$ . Then, given  $p(x)$ , the joint  $(x,y)$  distribution is given by
$$
\pi_ {p} (x, y) = p (y \mid x, \theta_ {*}) p (x). \tag {2}
$$
Then, the expected loss at  $\theta$  is defined by
$$
L _ {p} (\theta) \triangleq \mathbb {E} _ {(x, y) \sim \pi_ {p}} [ \ell_ {(x, y)} (\theta) ] = \mathbb {E} _ {x \sim p (x)} \mathbb {E} _ {y \sim p (y | x, \theta_ {*})} [ \ell_ {(x, y)} (\theta) ]. \tag {3}
$$
The excess risk of  $p(x)$  at  $\theta$  is defined as  $R_{p}(\theta) = L_{p}(\theta) - L_{p}(\theta_{*})$ . Note that  $R_{p}(\theta) \geq 0$ .
Notation: The inner product between two matrices is  $\mathbf{A} \cdot \mathbf{B} = \operatorname{Trace}(\mathbf{A}^{\top}\mathbf{B})$ . For a matrix  $\mathbf{A} \in \mathbb{R}^{m \times n}$ , let  $\| \mathbf{A} \|$  be the spectral norm of  $\mathbf{A}$ , let  $\operatorname{vec}(\mathbf{A}) \in \mathbb{R}^{mn}$  be the vectorization of  $\mathbf{A}$  by stacking all rows together, i.e.  $\operatorname{vec}(\mathbf{A}) = (\mathbf{A}_1^{\top}, \dots, \mathbf{A}_m^{\top})^{\top}$  where  $\mathbf{A}_i$  is  $i$ -th row of  $\mathbf{A}$ . Given a positive definite matrix  $\mathbf{A} \in \mathbb{R}^{d \times d}$ , we define norm  $\| \cdot \|_{\mathbf{A}}$  for  $x \in \mathbb{R}^d$  by  $\| x \|_{\mathbf{A}} = \sqrt{x^{\top} \mathbf{A}x}$ . For
integer  $k \geq 1$ , we denote by  $\mathbf{I}_k$  the  $k$ -by-  $k$  identity matrix. For any point distribution  $p(x)$  we define  $\mathbf{V}_p \triangleq \mathbb{E}_{x \sim p(x)}[xx^\top]$  to be its covariance matrix,  $\mathbf{H}_p(\theta) \triangleq \nabla^2 L_p(\theta)$  be the Hessian matrix of  $L_p(\theta)$ , define  $\mathbf{H}_p \triangleq \mathbf{H}_p(\theta_*)$ .
Active learning. Let  $U = \{x_{i}\}_{i=1}^{m}$ , be the set of unlabeled points and  $S_{0}$  be the set of  $n_{0}$  initially labeled samples. In particular, we denote the set of points in  $S_{0}$  as  $X_{0}$ . Let  $\theta_{0}$  be the solution of training a classifier with  $S_{0}$ , i.e.,  $\theta_{0} \in \arg \min_{\theta} \frac{1}{n_{0}} \sum_{(x,y) \in S_{0}} \ell_{(x,y)}(\theta)$ . We select a set of  $b$  points  $X \subset U$ , query their labels  $y \sim p(y|x,\theta_{*})$ ,  $\forall x \in X$ , and train a new classifier  $\theta_{n} \in \arg \min_{\theta} \frac{1}{n} \sum_{(x,y) \in S} \ell_{(x,y)}(\theta)$ , where  $S$  is the set of  $S_{0}$  with new labeled points and  $n = n_{0} + b$ .
Our goal is to optimize the selection of  $X$  so that we can minimize the excess risk on the original unlabeled set  $U$ , i.e.  $L_{p}(\theta_{n}) - L_{p}(\theta_{*})$ . In this context, we define two problems:
Problem 1: Given  $X$  or equivalently  $q(x)$ , can we bound  $L_{p}(\theta_{n}) - L_{p}(\theta_{*})$ ?
Problem 2: Can we construct an efficient algorithm for finding  $X$  that minimizes  $L_{p}(\theta_{n}) - L_{p}(\theta_{*})$ ?
3 Excess Risk Bounds
In this section, we develop our theory to address Problem 1. Our plan is to endow  $p(x)$  and  $q(x)$  with certain properties (sub-Gaussianity or finite support) and derive FIR bounds for  $L_{p}(\theta_{n}) - L_{p}(\theta_{*})$ . Let  $\theta_{n}$  be the empirical risk minimizer (ERM) obtained from  $n$  i.i.d. samples drawn from  $\pi_q(x,y)$ :
$$
\theta_ {n} \in \arg \min  _ {\theta} \frac {1}{n} \sum_ {i = 1} ^ {n} \ell_ {\left(x _ {i}, y _ {i}\right)} (\theta), \quad \forall i \in [ n ], \quad \left(x _ {i}, y _ {i}\right) \stackrel {\text {i.i.d.}} {\sim} \pi_ {q} (x, y). \tag {4}
$$
We assume that both  $p(x)$  and  $q(x)$  are sub-Gaussian distributions. Appendix A.1 gives a brief review of definitions and basic properties of sub-Gaussian random variables (vector). We define the  $\psi_{2}$ -norm of a sub-Gaussian random variable  $x\in \mathbb{R}$  as  $\| x\|_{\psi_2}\triangleq \inf \{t > 0:\mathbb{E}\exp (x^2 /t^2)\leq 2\}$ . For a sub-Gaussian random vector  $x\in \mathbb{R}^d$ $\| x\|_{\psi_2} = \sup \{\| u^\top x\|_{\psi_2}:\| u\| _2\leq 1\}$ . We formalize our assumption for  $p(x)$  and  $q(x)$  in Assumption [1]. Based on this assumption, we can derive some properties for the gradient and Hessian of  $\ell_{(x,y)}(\theta)$  shown in Lemma [2] (proof can be found in Appendix D). We present the results for  $q$  (thus the subscript in the  $K$  constants); exactly the same results, with different constants hold for  $p$ .
Assumption 1. Let  $q(x)$  be a sub-Gaussian distribution for  $x \in \mathbb{R}^b$ , we assume that  $\mathbf{V}_q$  is positive definite. We assume that there exists  $r \gtrsim 1$  such that for any  $\theta \in \mathcal{B}_r(\theta_*) = \{\theta : \| \theta - \theta_* \|_{2,\infty} \leq r\}$ ,  $\mathbf{H}_q(\theta)$  is positive definite, where  $\| \cdot \|_{2,\infty}$  denotes the maximum row norm of a matrix.
Lemma 2. If Assumption holds for  $q(x)$ , then for  $(x,y) \sim \pi_q(x,y)$ :
(1) There exists  $K_{0,q} > 0$  s.t.  $\| \mathbf{V}_q^{-1 / 2}x\|_{\psi_2}\leq K_{0,q}$
(2) There exists  $K_{1,q} > 0$  s.t.  $\| \mathbf{H}_q^{-1 / 2}\mathrm{vec}(\nabla \ell_{(x,y)}(\theta_*))\|_{\psi_2}\leq K_{1,q}$
(3) There exists  $K_{2,q}(r) > 0$  s.t. for any  $\theta$  in the ball  $\mathcal{B}_r(\theta_*) = \{\theta : \| \theta - \theta_* \|_{2,\infty} \leq r\}$ ,
$$
\sup  _ {u \in \mathcal {S} ^ {\tilde {d} - 1}} \| u ^ {\top} \mathbf {H} _ {q} (\theta) ^ {- 1 / 2} \nabla^ {2} \ell_ {(x, y)} (\theta) \mathbf {H} _ {q} (\theta) ^ {- 1 / 2} u \| _ {\psi_ {1}} \leq K _ {2, q} (r), \tag {5}
$$
where  $S^{\widetilde{d} -1}$  is the unit sphere in  $\mathbb{R}^{\widetilde{d}}$ , norm  $\| \cdot \|_{\psi_1}$  is the norm defined for a sub-exponential random variable  $z\in \mathbb{R}$  by  $\| z\|_{\psi_1} = \inf \{t > 0:\mathbb{E}\exp (|z| / t)\leq 2\}$ .
Our main result of this section is Theorem 3. Under the sample bound given by Eq. (6), we derive high probability bounds for the excess risk in Eq. (7). Details and the proof of Theorem 3 can be found in Appendix C.
Theorem 3. Suppose Assumption  $\boxed{I}$  holds for both  $p(x)$  and  $q(x)$ . Let  $\sigma$  and  $\rho > 0$  be constants such that  $\mathbf{H}_p \preceq \sigma \mathbf{H}_q$  and  $\mathbf{I}_{c-1} \otimes \mathbf{V}_p \preceq \rho \mathbf{H}_p(\theta_*)$  hold. There exit constants  $C_1, C_2$  and  $C_3 > 0$ , such that for any  $\delta \in (0,1)$ , whenever
$$
n \geq \max  \left\{C _ {1} \widetilde {d} \log (e d / \delta), C _ {2} \sigma \rho (\widetilde {d} + \sqrt {\widetilde {d}} \log (e / \delta)) \right\}, \tag {6}
$$
where  $\widetilde{d} \triangleq d(c - 1)$ , we have with probability at least  $1 - \delta$ ,
$$
\frac {e ^ {- \alpha} + \alpha - 1}{\alpha^ {2}} \frac {\mathbf {H} _ {q} ^ {- 1} \cdot \mathbf {H} _ {p}}{n} \lesssim \mathbb {E} \left[ L _ {p} \left(\theta_ {n}\right) \right] - L _ {p} \lesssim \frac {e ^ {\alpha} - \alpha - 1}{\alpha^ {2}} \frac {\mathbf {H} _ {q} ^ {- 1} \cdot \mathbf {H} _ {p}}{n}. \tag {7}
$$
Here  $\mathbf{H}_p = \mathbf{H}_p(\theta_*)$  and  $\mathbf{H}_q = \mathbf{H}_q(\theta_*)$ ; and  $\mathbb{E}$  is the expectation over  $\{y_i \sim p(y_i | x_i, \theta_*)\}_{i=1}^n$ . Furthermore,
$$
\alpha = C _ {3} \sqrt {\sigma \rho} \sqrt {\left(\tilde {d} + \sqrt {\tilde {d}} \log (e / \delta)\right) / n}. \tag {8}
$$
From Eq. (7), we observe that FIR  $(\mathbf{H}_q^{-1} \cdot \mathbf{H}_p)$  appears in both the lower and upper bounds for  $R(\theta_n)$ . In other words, it is essential for controlling the excess risk. To the right we show how the prefactors  $\frac{e^{\alpha} - \alpha - 1}{\alpha^2}$  and  $\frac{e^{-\alpha} + \alpha - 1}{\alpha^2}$  change as a function of  $\alpha$ . Constants  $C_1, C_2$  and  $C_3$  depend on constants defined in Lemma 2 for both  $p(x)$  and  $q(x)$ . In Appendix D, we derive bounds for  $K_{1,p}$  and  $K_{2,p}(r)$  in Proposition 35. For a Gaussian design  $x \sim \mathcal{N}(0, \mathbf{V}_p)$ , we derive bounds for  $\rho, K_{0,p}, K_{1,p}$  and  $K_{2,p}(r)$  in Proposition 37.
Figure 1: FIR prefactors in Eq. (7).
Bounded domain. If the domain of  $x$  is bounded, Chaudhuri et al. [11] provided lower and upper bounds for the excess risk of  $p(x)$  (Lemma 1 in [11]). Their conclusion is similar to ours, namely that FIR is crucial in controlling the excess risk of  $p(x)$ . It is worth noting that when the domain is bounded, both  $p(x)$  and  $q(x)$  are always sub-Gaussian. Thus, our assumption is more general. For the sake of completeness, we provide a detailed derivation of the excess risk bounds for  $p(x)$  in Theorem 40 when  $x$  is bounded with Assumption 38.
4 Active Learning via Minimizing Fisher Information Ratio
We now discuss the FIRAL algorithm that addresses Problem 2. We can use the theoretical analysis derived in the previous section to guide us for the point selection. Let  $p(x)$  be the empirical distribution on unlabeled pool  $U$  with  $|U| = m$ , and  $q$  the distribution for the  $n = n_0 + b$  labeled points. Eq. (7) inspires us to select points to label such that we can minimize the FIR  $\mathbf{H}_q^{-1} \cdot \mathbf{H}_p$ , where  $\mathbf{H}_q = \mathbf{H}_q(\theta_*)$ ,  $\mathbf{H}_p = \mathbf{H}_p(\theta_*)$ . However, we cannot directly use this as  $\theta_*$  is unknown. Instead, we will use  $\theta_0$ , the solution by training the classifier with the initial labeled set  $S_0$ . That is, we will find  $q$  by minimizing  $\mathbf{H}_q(\theta_0)^{-1} \cdot \mathbf{H}_p(\theta_0)$ .
In § 4.1, we formalize our optimization objective in Eq. (13). Solving Eq. (13) exactly is NP-hard [15]. Inspired by [14], we approximate the solution in two steps: (1) we solve a continuous convex optimization problem in Eq. (14) (\$ 4.2), (2) and use the results in a regret minimization algorithm to select points by Eq. (19) (\$ 4.3). In Algorithm I we summarize the scheme.
We state theoretical guarantees for the algorithm in §4.4 where we prove that it achieves  $(1 + \epsilon)$ -approximation of the optimal objective value in Eq.(13) with sample complexity  $b = \mathcal{O}(\tilde{d}/\epsilon^2)$ , as stated in Theorem 10. Finally, we obtain the excess risk bound for unlabeled points  $U$  by accounting for the fact that we use  $\theta_0$  instead of  $\theta_*$  in the objective function. The overall result is summarized in the following theorem.
Theorem 4. Suppose that Assumption  $\boxed{I}$  holds. Let  $\epsilon \in (0,1)$ ,  $\delta \in (0,1)$ , and  $b$  the number of points to label. Then with probability at least  $1 - \delta$ , the  $\theta_{n}$ -computed by fitting a multinomial logistic regression classifier on the labeled points selected using FIRAL (Algorithm  $\boxed{I}$ ) with learning rate  $\eta = 8\sqrt{\widetilde{d}}/\epsilon$ ,  $b \geq 32\widetilde{d}/\epsilon^2 + 16\sqrt{\widetilde{d}}/\epsilon^2$ , and  $n = n_0 + b$  satisfying Eq. (6)—results in
$$
\mathbb {E} \left[ L _ {p} \left(\theta_ {n}\right) \right] - L _ {p} \left(\theta_ {*}\right) \lesssim (1 + \epsilon) 2 e ^ {2 \alpha_ {0}} \frac {e ^ {\alpha_ {n}} - \alpha_ {n} - 1}{\alpha_ {n} ^ {2}} \frac {O P T}{n}. \tag {9}
$$
Here OPT is the minimal  $\mathbf{H}_q^{-1} \cdot \mathbf{H}_p$ , attained by selecting the best  $b$  points from  $U$ ;  $\mathbb{E}$  is the expectation over  $\{y_i \sim p(y_i | x_i, \theta_*)\}_{i=1}^n$ ; and  $\alpha_0$  and  $\alpha_n$  are constants.
Such a set can be generated by an alternative method that only uses  $p(x)$  to select points, e.g., K-means.
4.1 Optimization objective
First we define the precise expression for  $\mathbf{H}_q(\theta_0)^{-1} \cdot \mathbf{H}_p(\theta_0)$ . We define the Fisher information matrix  $\mathbf{H}(x, \theta) = \nabla^2 \ell_{(x,y)}(\theta)$ . By Eq. (40) in Proposition 23 (Appendix B), we find that for multinomial logistic regression
$$
\mathbf {H} (x, \theta) = \left[ \operatorname {d i a g} (\mathbf {h} (x, \theta)) - \mathbf {h} (x, \theta) \mathbf {h} (x, \theta) ^ {\top} \right] \otimes (x x ^ {\top}), \tag {10}
$$
where  $\otimes$  represents the matrix Kronecker product,  $\mathbf{h}(x,\theta)$  is a  $(c - 1)$ -dimensional vector whose  $k$ -th component is  $\mathbf{h}_k(x,\theta) = p(y = k|x,\theta)$ . In Eq. (10) we can see that the Hessian of  $\ell_{(x,y)}(\theta)$  does not depend on the class label  $y$ . Following our previous definitions,  $\mathbf{H}_p(\theta_0) = \nabla^2 L_p(\theta_0) = \frac{1}{m}\sum_{x\in U}\mathbf{H}(x,\theta_0)$  and  $\mathbf{H}_q(\theta_0) = \nabla^2 L_q(\theta_0) = \frac{1}{n}\sum_{x\in X_0\cup X}\mathbf{H}(x,\theta_0)$ . For notational simplicity we also define
$$
\mathbf {H} (x) \triangleq \mathbf {H} \left(x, \theta_ {0}\right) + \frac {1}{b} \sum_ {x ^ {\prime} \in X _ {0}} \mathbf {H} \left(x ^ {\prime}, \theta_ {0}\right) \tag {11}
$$
$$
\boldsymbol {\Sigma} (z) \triangleq \sum_ {i \in [ m ]} z _ {i} \mathbf {H} \left(x _ {i}\right), \quad z _ {i} \text {s c a l a r}. \tag {12}
$$
Then minimizing  $\mathbf{H}_q(\theta_0)^{-1} \cdot \mathbf{H}_p(\theta_0)$  is equivalent to
$$
\min  _ {\substack {z \in \{0,1 \} ^ {m} \\ \| z \| _ {1} = b}} f (z) \triangleq f (\boldsymbol {\Sigma}(z)) \triangleq (\boldsymbol {\Sigma}(z)) ^ {-1} \cdot \mathbf{H}_{p}(\theta_{0}). \tag{13}
$$
We define  $z_*$  be the optimal solution of Eq. (13) and  $f_* \triangleq f(z_*)$ . In the following, with some abuse of notation, we will consider  $f$  being a function of either a vector  $z$  or a positive semidefinite matrix  $f(\Sigma)$ , depending on the context. Lemma 5 lists key properties of  $f$  when viewed as a matrix function; we use them in §4.2 to prove the optimality of FIRAL.
Lemma 5.  $f:\{\mathbf{A}\in \tilde{\mathbb{R}^{d\times \tilde{d}}}: \mathbf{A}\succeq \mathbf{0}\} \to \mathbb{R}$  defined in Eq. (13) satisfies the following properties:
(1) convex:  $f(\lambda \mathbf{A} + (1 - \lambda)\mathbf{B}) \leq \lambda f(\mathbf{A}) + (1 - \lambda)f(\mathbf{B})$  for all  $\lambda \in [0,1]$  and  $\mathbf{A},\mathbf{B} \in \mathbb{S}_{++}^{\tilde{d}}$ .
(2) monotonically non-increasing: if  $\mathbf{A} \preceq \mathbf{B}$  then  $f(\mathbf{A}) \geq f(\mathbf{B})$ ,
(3) reciprocally linear: if  $t > 0$  then  $f(t\mathbf{A}) = t^{-1}f(\mathbf{A})$ .
4.2 Relaxed problem
As a first step in solving Eq. (13) we relax the constraint  $z \in \{0, 1\}^m$  to  $z \in [0, 1]^m$ . Then we obtain the following convex programming problem:
$$
\begin{array}{l} z _ {\diamond} = \arg \min  f (\boldsymbol {\Sigma} (z)). \tag {14} \\ z \in [ 0, 1 ] ^ {m} \\ \| z \| _ {1} = b \\ \end{array}
$$
Since both the objective function and the constraint set are convex, conventional convex programming algorithm can be used to solve Eq. (14). We choose to use a mirror descent algorithm in our implementation (outlined in the Appendix, Algorithm 2). Since the integrality constraint is a subset of the relaxed constraint we obtain the following result.
Proposition 6.  $f(z_{\diamond}) \leq f_{*}$ .
In what follows, we use matrices  $\pmb{\Sigma}_{\diamond}$  and  $\widetilde{\mathbf{H}}(x_i)$  ( $i \in [m]$ ) defined by
$$
\boldsymbol {\Sigma} _ {\diamond} \triangleq \boldsymbol {\Sigma} \left(z _ {\diamond}\right) \quad \text {a n d} \quad \widetilde {\mathbf {H}} \left(x _ {i}\right) \triangleq \boldsymbol {\Sigma} _ {\diamond} ^ {- 1 / 2} \mathbf {H} \left(x _ {i}\right) \boldsymbol {\Sigma} _ {\diamond} ^ {- 1 / 2}, \quad i \in [ m ]. \tag {15}
$$
4.3 Solving Sparsification problem via Regret Minimization
Goal of sparsification. Now we introduce our method of sparsifying  $z_{\diamond}$  (optimal solution to Eq. (14)) into a valid integer solution to Eq. (13). To do so, we use an online optimization algorithm in which we select one point at a time in sequence until we have  $b$  points. Notice that alternative techniques like thresholding  $z_{\diamond}$  could be used but it was unclear to us how to provide error estimates for such a scheme. Instead, we use an alternative scheme that we describe below.
Let  $i_t \in [m]$  be the point index selected at step  $t \in [b]$ . We can observe that if  $\lambda_{\min} \left( \sum_{t \in [b]} \widetilde{\mathbf{H}}(x_{i_t}) \right) \geq \tau$  for some  $\tau > 0$ , then  $\sum_{t \in [b]} \mathbf{H}(x_{i_t}) \succeq \tau \Sigma_\diamond$ . By Lemma 5-Item (3) and Proposition 6, we obtain the following result.
Proposition 7. Given  $\tau \in (0,1)$ , we have
$$
\lambda_ {\min } \left(\sum_ {t \in [ b ]} \tilde {\mathbf {H}} \left(x _ {i _ {t}}\right)\right) \geq \tau \Longrightarrow f \left(\sum_ {t \in [ b ]} \mathbf {H} \left(x _ {i _ {t}}\right)\right) \leq \tau^ {- 1} f _ {*}. \tag {16}
$$
From Eq. (16), a larger  $\tau$  value indicates that  $f$  is closer to  $f_{*}$ . Therefore, our objective is to choose points in such a way that  $\lambda_{\min} \left( \sum_{t \in [b]} \widetilde{\mathbf{H}}(x_{i_t}) \right)$  is maximized.