SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Composable Coresetts for Determinant Maximization: Greedy is Almost Optimal*
Siddharth Gollapudi
sgollapu@berkeley.edu
Sepideh Mahabadi
smahabadi@microsoft.com
Varun Sivashankar
varunsiva@princeton.edu
Given a set of  $n$  vectors in  $\mathbb{R}^d$ , the goal of the determinant maximization problem is to pick  $k$  vectors with the maximum volume. Determinant maximization is the MAP-inference task for determinantal point processes (DPP) and has recently received considerable attention for modeling diversity. As most applications for the problem use large amounts of data, this problem has been studied in the relevant composable coreset setting. In particular, [IMGR20] showed that one can get composable coresets with optimal approximation factor of  $\tilde{O}(k)^k$  for the problem, and that a local search algorithm achieves an almost optimal approximation guarantee of  $O(k)^{2k}$ . In this work, we show that the widely-used Greedy algorithm also provides composable coresets with an almost optimal approximation factor of  $O(k)^{3k}$ , which improves over the previously known guarantee of  $C^{k^2}$ , and supports the prior experimental results showing the practicality of the greedy algorithm as a coreset. Our main result follows by showing a local optimality property for Greedy: swapping a single point from the greedy solution with a vector that was not picked by the greedy algorithm can increase the volume by a factor of at most  $(1 + \sqrt{k})$ . This is tight up to the additive constant 1. Finally, our experiments show that the local optimality of the greedy algorithm is even lower than the theoretical bound on real data sets.
In the determinant maximization problem, we are given a set  $P$  of  $n$  vectors in  $\mathbb{R}^d$ , and a parameter  $k \leq d$ . The objective is to find a subset  $S = \{v_1, \ldots, v_k\} \subseteq P$  consisting of  $k$  vectors such that the volume squared of the parallelepiped spanned by the points in the subset  $S$  is maximized. Equivalently, the volume squared of a set  $S$ , denoted by  $\operatorname{vol}(S)$ , is equal to the determinant of the Gram matrix of the vectors in  $S$ . Determinant maximization is the MAP-inference of determinantal point processes, and both of these problems as well as their variants have found numerous applications in data summarization, machine learning, experimental design, and computational geometry. In particular, the determinant of a subset of points is one way to measure the diversity of the subset, and thus they have been studied extensively over the last decade in this context [MJK17, GCGS14, KT+12, CGGS15, KT11, YFZ+16, LCYO16].
The best approximation factor for the problem in this regime is due to the work of [Nik15] who shows a factor of  $e^k$ , and it is known that an exponential dependence on  $k$  is necessary [CMI13] unless  $\mathrm{P} = \mathrm{NP}$ . However, the most common algorithm used for this problem in practical applications is a natural greedy algorithm. In this setting, the algorithm first picks the vector with the largest norm,
*This is an equal contribution paper
and then greedily picks the vector with largest perpendicular component to the subspace spanned by the current set of picked vectors, thus maximizing the volume greedily in each iteration. This algorithm is known to have an approximation factor of  $(k!)^2[\mathsf{CMI09}]$ .
As in most applications of determinant maximization one needs to work with large amounts of data, there has been an increasing interest in studying determinant maximization in large data models of computation [MJK17, WIB14, PJG $^{+}$ 14, MKSK13, MKBK15, MZ15, BENW15]. One such model that we focus on in this work is the composable coreset setting [IMMM14]. Intuitively, composable coresets are small "summaries" of a data set with the composability property: for the summaries of multiple datasets, the union of the summaries should make a good summary for the union of the datasets. More precisely, in this setting, instead of a single set of vectors  $P$ , there are  $m$  sets  $P_{1}, \ldots, P_{m} \subseteq \mathbb{R}^{d}$ . In this context, a mapping function  $c$  that maps a point set to one of its subsets is called  $\alpha$ -composable coreset for determinant maximization, if for any collection of point sets  $P_{1}, \ldots, P_{m}$ ,
$$
\operatorname {M A X D e t} _ {k} \left(\cup_ {i = 1} ^ {m} c \left(P _ {i}\right)\right) \geq \frac {1}{\alpha} \cdot \operatorname {M A X D e t} _ {k} \left(\cup_ {i = 1} ^ {m} P _ {i}\right) \tag {1}
$$
where  $\mathrm{MAXDET}_k$  is used to denote the maximum achievable determinant with parameter  $k$ . (Similarly,  $\mathrm{MAXVOL}_k$  is used to denote the maximum volume, with  $\mathrm{MAXVOL}_k^2 = \mathrm{MAXDET}_k$ .) For clarity, we note that the mapping function  $c$  can only view its input data set  $P_i$  and has no knowledge of other data sets while constructing  $c(P_i)$ . [IMMM14] showed that a composable coreset for a task automatically gives an efficient distributed and an efficient streaming algorithm for the same task.
Indeed, composable coresets have been used for determinant maximization. In particular, [IMGR20, MIGR19], presented a composable coreset of size  $O(k \log k)$  with approximation factor of  $\tilde{O}(k)^k$  using spectral spanners, which they showed to be almost tight. In particular, the best approximation factor one can get is  $\Omega(k^{k - o(k)})$  (Theorem 1.4). As the above algorithm is LP-based and does not provide the best performance in practice, they proposed to use the greedy algorithm followed by a local search procedure, and showed that this simple algorithm also yields a coreset of size  $k$  with an almost optimal approximation guarantee of  $O(k)^{2k}$ . They also proved that the greedy algorithm alone yields a  $C^{k^2}$  guarantee for composable coresets, which is far larger than the optimal approximation of  $\tilde{O}(k)^k$  for this problem.
Since the greedy algorithm provides a very good performance in practice [MIGR19, MKSK13], an improved analysis of the greedy algorithm in the coreset setting is very desirable. Furthermore, both of these prior work implied that greedy performs well in practice in the context of distributed and composable coreset settings [MKSK13], and in particular its performance is comparable to that of the local search algorithm for the problem [MIGR19].
Our contribution. In this paper, we close this theoretical gap: we prove that the greedy algorithm provides a  $O(k)^{3k}$ -composable coreset for the determinant maximization problem (Theorem 4). This explains the very good performance of this algorithm on real data previously shown in [MIGR19, MKSK13]. We achieve this by proving an elegant linear algebra result on the local optimality of the greedy algorithm: swapping a single point from the greedy solution with a vector that was not picked by the greedy algorithm can increase the volume by a factor of at most  $(1 + \sqrt{k})$ . We further show that this is tight up to the additive constant 1. As an application of our result, we give a proof that the locality property can recover and in fact marginally improve the  $k!$  guarantee of the greedy algorithm of [CMI09] for the offline volume maximization problem.
Finally, in Section 4, we run experiments to measure the local optimality of the greedy algorithm on real data, and show that this number is much smaller in practice than the worst case theoretically guaranteed bound. In fact, in our experiments this number is always less than 1.5 even for  $k$  even as large as 300. Again this explains the practical efficiency of the greedy algorithm as a coreset shown in [MIGR19, MKSK13].
1.1.1 The Greedy Algorithm
Recall the standard offline setting for determinant maximization, where one is required to pick  $k$  vectors out of the  $n$  vectors in  $P$  of maximum volume. Here, [CMI09] showed that greedily picking
the vector with the largest perpendicular distance to the subspace spanned by the current solution (or equivalently, greedily picking the vector that maximizes the volume as in Algorithm 1) outputs a set of vectors that is within  $k!$  of the optimal volume. Formally, if  $\mathrm{Greedy}(\bar{P})$  is the output of Algorithm 1, then
$$
\operatorname {v o l} (\operatorname {G r e e d y} (P)) \geq \frac {\operatorname {M A X V O L} _ {k} (P)}{k !} \tag {2}
$$
1.1.2 Local Search for Composable Coresets
In [MIGR19], the authors show that the greedy algorithm followed by the local search procedure with parameter  $\epsilon$  (as described in Algorithm 2) provides a  $(2k(1 + \epsilon))^{2k}$ -composable coreset for determinant maximization. A locally optimal solution can thus be naturally defined as follows:
Definition 1  $((1 + \epsilon)$ -Locally Optimal Solution). Given a point set  $P \subseteq \mathbb{R}^d$  and  $c(P) \subseteq P$  with  $|c(P)| = k$ , we say  $c(P)$  is a  $(1 + \epsilon)$ -locally optimal solution for volume maximization if for any  $v \in c(P)$  and any  $w \in P \setminus c(P)$ ,
$$
\operatorname {v o l} (c (P) - v + w) \leq (1 + \epsilon) \operatorname {v o l} (c (P)) \tag {3}
$$
Given the output of the greedy algorithm Greedy  $(P)$ , one can obtain a locally optimal solution using a series of swaps: if the volume of the solution can be increased by a factor of  $(1 + \epsilon)$  by swapping a vector in the current solution with a vector in the point set  $P$  that has not been included, we make the swap. Since  $\operatorname{vol}(\operatorname{Greedy}(P))$  is within a factor of  $k!$  of the optimal, we will make at most  $\frac{k\log k}{\log(1 + \epsilon)}$  swaps. This is precisely the local search algorithm (Algorithm 2). For any point set  $P$ , we denote the output of Algorithm 2 by  $\mathsf{LS}(P)$ .
In [MIGR19], the authors prove that local search yields a  $O(k)^{2k}$ -composable coreset for determinant maximization. Formally, they prove the following.
Theorem 2. Let  $P_{1}, \ldots, P_{m} \subseteq \mathbb{R}^{d}$ . For each  $i = 1, \ldots, m$ , let  $\mathrm{LS}(P_{i})$  be the output of the local search algorithm (Algorithm 2) with parameter  $\epsilon$ . Then
$$
\operatorname {M A X D e t} _ {k} \left(\cup_ {i = 1} ^ {m} P _ {i}\right) \leq (2 k (1 + \epsilon)) ^ {2 k} \operatorname {M A X D e t} _ {k} \left(\cup_ {i = 1} ^ {m} \mathrm {L S} \left(P _ {i}\right)\right) \tag {4}
$$
Remark 3. Even though [MIGR19] treats  $\epsilon$  as a small constant in  $[0,1]$ , the proof for Theorem 2 above holds for any non-negative  $\epsilon$ .
1.2 Outline of our approach
In [MIGR19], the authors prove Theorem 2 for local search using a reduction to a related problem called  $k$ -directional height. The authors then use similar ideas to prove that the output of the greedy algorithm is also a composable coreset for determinant maximization. However, since we do not know a priori whether greedy is  $(1 + \epsilon)$ -locally optimal, the guarantee they obtain is significantly weaker: they only prove that the greedy algorithm yields a  $((2k) \cdot 3^k)^{2k} = C^{k^2}$ -composable coreset for determinant maximization. This is clearly far from the desired bound of  $k^{O(k)}$ .
To improve the analysis of the greedy algorithm in the coreset setting, we ask the following natural question:
Can we prove that the output of the greedy algorithm is already locally optimal?
We answer this question positively. Our main result is Theorem 5, where we show that for any point set  $P$ , Greedy(P) is a  $(1 + \sqrt{k})$ -locally optimal solution. In other words, the greedy algorithm has the same guarantee as local search with the parameter  $\epsilon = \sqrt{k}$ . This circumvents the loose reduction from greedy to the  $k$ -directional height problem and directly implies the following improved guarantee for the greedy algorithm in the coreset setting:
Theorem 4.
$$
\operatorname {M A X D e t} _ {k} \left(\cup_ {i = 1} ^ {m} P _ {i}\right) \leq \left(2 k \left(1 + \sqrt {k}\right)\right) ^ {2 k} \operatorname {M A X D e t} _ {k} \left(\cup_ {i = 1} ^ {m} \operatorname {G r e e d y} (P _ {i})\right) \tag {5}
$$
Thus, the greedy algorithm also provides a  $(2k(1 + \sqrt{k}))^{2k} = k^{O(k)}$ -composable coreset for determinant maximization, which is near the optimal  $\Omega(k^{k - o(k)})$ .
Section 2 is dedicated to proving that greedy is  $(1 + \sqrt{k})$ -locally optimal (Theorem 5). We also show that this local optimality result of  $(1 + \sqrt{k})$  for the greedy algorithm is tight up to the additive constant 1. In Section 4 we show that on real and random datasets, the local optimality constant  $\epsilon$  is much smaller than the bound of  $1 + \sqrt{k}$ , which serves as an empirical explanation for why greedy performs much better in practice than what the theoretical analysis suggests.
Algorithm 1 Greedy Algorithm
Input: A point set  $P\subset \mathbb{R}^d$  and integer  $k$
Output: A set  $\mathcal{C} \subset P$  of size  $k$ .
Initialize  $\mathcal{C} = \emptyset$
Add  $\arg \max_{p\in P\backslash \mathcal{C}}\mathrm{vol}(\mathcal{C} + p)$  to  $\mathcal{C}$ .
Return  $\mathcal{C}$
Algorithm 2 Local Search Algorithm
Input: A point set  $P\subset \mathbb{R}^d$  , integer  $k$  , and  $\epsilon >0$
Add  $\arg \max_{p\in P\backslash \mathcal{C}}\operatorname {vol}(\mathcal{C} + p)$  to  $\mathcal{C}$
If there are points  $q\in P\setminus \mathcal{C}$  and  $p\in \mathcal{C}$  such that
$$
\operatorname {v o l} (\mathcal {C} + q - p) \geq (1 + \epsilon) \operatorname {v o l} (\mathcal {C})
$$
replace  $p$  with  $q$ .
until No such pair exists.
2 Greedy is Locally Optimal
Theorem 5 (Local Optimality). Let  $V \coloneqq \mathrm{Greedy}(P) = \{v_1, \ldots, v_k\} \subseteq P$  be the output of the greedy algorithm. Let  $v_{k+1} \in P \setminus V$  be a vector not chosen by the greedy algorithm. Then for all  $i = 1, \ldots, k$ ,
$$
\operatorname {v o l} (V - v _ {i} + v _ {k + 1}) \leq (1 + \sqrt {k}) \operatorname {v o l} (V) \tag {6}
$$
Proof. If  $\operatorname{rank}(P) < k$ , then the result is trivial. So we may assume  $\operatorname{rank}(P) \geq k$  and  $V$  is linearly independent. Fix any  $v_{i} \in V$ . Our goal is to show that  $\mathsf{vol}(V - v_{i} + v_{k+1}) \leq (1 + \sqrt{k}) \mathsf{vol}(V)$ . This trivially holds when  $i = k$  by the property of the greedy algorithm, so assume  $1 \leq i \leq k-1$ .
Let  $\{v_1',\ldots ,v_k',v_{k + 1}'\}$  be the set of orthogonal vectors constructed by performing the Gram-Schmidt algorithm on  $\{v_{1},\dots,v_{k},v_{k + 1}\}$ . Formally, let  $\mathcal{G}_t = \operatorname {span}\{v_1,\dots,v_t\}$ . Define  $v_{1}^{\prime} = v_{1}$  and  $v_{t}^{\prime} = v_{t} - \Pi (\mathcal{G}_{t - 1})(v_{t})$  for  $t = 2,\ldots ,k,k + 1$ , where  $\Pi (\mathcal{G})(v)$  denotes the projection of the vector  $v$  onto the subspace  $\mathcal{G}$ . Note that
$$
\operatorname {v o l} (V) = \prod_ {j = 1} ^ {k} \| v _ {j} ^ {\prime} \| _ {2}
$$
For each  $j = i + 1,\ldots ,k,k + 1$  , write
$$
\begin{array}{l} v _ {j} = \Pi (\mathcal {G} _ {i - 1}) (v _ {j}) + \sum_ {l = i} ^ {j} \alpha_ {l} ^ {j} v _ {l} ^ {\prime} \\ := \Pi (\mathcal {G} _ {i - 1}) (v _ {j}) + w _ {j} \\ \end{array}
$$
We must have that  $|\alpha_j^l| \leq 1$  by the greedy algorithm because if  $|\alpha_l^j| > 1$ , the vector  $v_j$  would have been chosen before  $v_l$ . Further,  $\alpha_j^j = 1$  by definition of Gram-Schmidt. The vector  $w_j$  is what remains of  $v_j$  once we subtract its projection onto the first  $i - 1$  vectors.
We are interested in bounding the following quantity:
$$
\begin{array}{l} \operatorname {v o l} (V - v _ {i} + v _ {k + 1}) = \operatorname {v o l} (v _ {1}, \dots , v _ {i - 1}, v _ {i + 1}, \dots , v _ {k}, v _ {k + 1}) \\ = \operatorname {v o l} \left(v _ {1} ^ {\prime}, \dots , v _ {i - 1} ^ {\prime}, v _ {i + 1}, \dots , v _ {k}, v _ {k + 1}\right) \\ = \operatorname {v o l} \left(v _ {1} ^ {\prime}, \dots , v _ {i - 1} ^ {\prime}, w _ {i + 1}, \dots , w _ {k}, w _ {k + 1}\right) \\ = \operatorname {v o l} \left(v _ {1} ^ {\prime}, \dots , v _ {i - 1} ^ {\prime}\right) \cdot \operatorname {v o l} \left(w _ {i + 1}, \dots , w _ {k}, w _ {k + 1}\right) \\ = \left(\prod_ {j = 1} ^ {i - 1} \| v _ {j} ^ {\prime} \| _ {2}\right) \cdot \operatorname {v o l} \left(w _ {i + 1}, \dots , w _ {k}, w _ {k + 1}\right) \\ \end{array}
$$
Therefore, it suffices to prove the following:
$$
\operatorname {v o l} \left(w _ {i + 1}, \dots , w _ {k}, w _ {k + 1}\right) \leq (1 + \sqrt {k}) \prod_ {j = i} ^ {k} \| v _ {j} ^ {\prime} \| _ {2} \tag {7}
$$
To establish this, we consider two cases. Recall that  $v_{k+1}' = v_{k+1} - \Pi(\mathcal{G}_k)(v_k)$ . We analyze the cases where  $v_{k+1}' \neq 0$  and  $v_{k+1}' = 0$  separately, although the ideas are similar. In Claim 7 and Claim 8 below, we establish the desired bound stated in Eq. (7) for  $v_{k+1}' \neq 0$  and  $v_{k+1}' = 0$  respectively. Theorem 5 then follows immediately.
To prove Claim 7 and Claim 8, the following well-known lemma will be useful. A proof can be found in [DZ07].
Lemma 6 (Matrix Determinant Lemma). Suppose  $M$  is an invertible matrix. Then
$$
\det  (M + u v ^ {T}) = \left(1 + v ^ {T} M ^ {- 1} u\right) \det  (M) \tag {8}
$$
Claim 7. Suppose  $v_{k+1}^{\prime} \neq 0$ . Then
$$
\operatorname {v o l} \left(w _ {i + 1}, \dots , w _ {k}, w _ {k + 1}\right) \leq \left(\sqrt {k + 1}\right) \prod_ {j = i} ^ {k} \| v _ {j} ^ {\prime} \| _ {2} \tag {9}
$$
Proof. Define the matrix  $B = [w_{i+1}|\dots|w_k|w_{k+1}]$ . Note that  $\det(B^T B) = \operatorname{vol}(w_{i+1},\ldots,w_k,w_{k+1})^2$  is the quantity we are interested in bounding. For clarity,
$$
B ^ {T} = \left[ \begin{array}{c} \alpha_ {i} ^ {i + 1} v _ {i} ^ {\prime} + v _ {i + 1} ^ {\prime} \\ \alpha_ {i} ^ {i + 2} v _ {i} ^ {\prime} + \alpha_ {i + 1} ^ {i + 2} v _ {i + 1} ^ {\prime} + v _ {i + 2} ^ {\prime} \\ \vdots \\ \alpha_ {i} ^ {k + 1} v _ {i} ^ {\prime} + \dots + \alpha_ {k} ^ {k + 1} v _ {k} ^ {\prime} + v _ {k + 1} ^ {\prime} \end{array} \right]
$$
We define the matrix  $A$  by just removing the  $v_{i}^{\prime}$  terms from  $B$  as follows:
$$
A ^ {T} = \left[ \begin{array}{c} v _ {i + 1} ^ {\prime} \\ \alpha_ {i + 1} ^ {i + 2} v _ {i + 1} ^ {\prime} + v _ {i + 2} ^ {\prime} \\ \vdots \\ \alpha_ {i + 1} ^ {k + 1} v _ {i + 1} ^ {\prime} + \dots + \alpha_ {k} ^ {k + 1} v _ {k} ^ {\prime} + v _ {k + 1} ^ {\prime} \end{array} \right]
$$
Since  $\langle v_i', v_j' \rangle = 0$  for all  $j \neq i$ , we have that
$$
B ^ {T} B = A ^ {T} A + u u ^ {T}
$$
where the column vector  $u$  is given by
$$
u = \| v _ {i} ^ {\prime} \| \cdot \left( \begin{array}{c c c c} \alpha_ {i} ^ {i + 1} & \alpha_ {i} ^ {i + 2} & \dots & \alpha_ {i} ^ {k + 1} \end{array} \right)
$$
Since  $|\alpha_i^j |\leq 1$  for  $j = i + 1,\ldots ,k + 1$  by the nature of the greedy algorithm, we have that
$$
\| u \| _ {2} ^ {2} \leq (k - i + 1) \| v _ {i} ^ {\prime} \| _ {2} ^ {2} \leq k \| v _ {i} ^ {\prime} \| _ {2} ^ {2} \tag {10}
$$
We now bound the desired volume quantity. Let  $M = A^T A$ .  $M$  is clearly a positive semi-definite matrix. In fact, because we assumed that  $v_{k+1}' \neq 0$ , it will turn out that  $M$  is positive definite and thus invertible. For now, assume that  $M^{-1}$  exists. We will compute the inverse explicitly later.
$$
\begin{array}{l} \left. \operatorname {v o l} \left(w _ {i + 1}, \dots , w _ {k}, w _ {k + 1}\right) ^ {2} = \det  \left(B ^ {T} B\right) \right. \\ = \det  \left(A ^ {T} A + u u ^ {T}\right) \\ = \left(1 + u ^ {T} M ^ {- 1} u\right) \det (M) \tag {by Lemma 6} \\ \leq \left(1 + k \| v _ {i} ^ {\prime} \| ^ {2} \lambda_ {\max } (M ^ {- 1})\right) \det  (M) \\ \end{array}
$$
where  $\lambda_{\mathrm{max}}(M^{-1})$  is the largest eigenvalue of  $M^{-1}$ . We will now show that  $M^{-1}$  does in fact exist and bound  $\lambda_{\mathrm{max}}(M^{-1})$ . Consider the matrix  $E$  and  $W$  defined as follows:
$$
E = \left[ \begin{array}{c c c c c} 1 & 0 & & \dots & 0 \\ \alpha_ {i + 1} ^ {i + 2} & 1 & 0 & \dots & 0 \\ \vdots & & & & \\ \alpha_ {i + 1} ^ {k + 1} & & \dots & \alpha_ {k} ^ {k + 1} & 1 \end{array} \right] \qquad \qquad W ^ {T} = \left[ \begin{array}{c} v _ {i + 1} ^ {\prime} \\ v _ {i + 2} ^ {\prime} \\ \vdots \\ v _ {k + 1} ^ {\prime} \end{array} \right]
$$
It is easy to check that  $E W^{T} = A^{T}$ . Therefore,
$$
M = A ^ {T} A = E W ^ {T} W E ^ {T} = E D E ^ {T}
$$
where  $D$  is the diagonal matrix given by
$$
D = \operatorname {d i a g} \left(\left\| v _ {i + 1} ^ {\prime} \right\| _ {2} ^ {2}, \dots , \left\| v _ {k + 1} ^ {\prime} \right\| _ {2} ^ {2}\right)
$$
It is easy to see that  $E$  has all eigenvalues equal to 1, and so must be invertible with determinant 1. The same is obviously true for  $E^T$ ,  $E^{-1}$  and  $(E^T)^{-1}$  as well. It follows that
$$
\det  (M) = \det  (D) = \| v _ {i + 1} ^ {\prime} \| ^ {2} \dots \| v _ {k + 1} ^ {\prime} \| ^ {2}
$$
Since  $v_{k+1}' \neq 0$ , we have that  $\| v_j' \|_2 > 0$  for all  $j$ , so  $D^{-1}$  clearly exists. It follows that
$$
M ^ {- 1} = \left(A ^ {T} A\right) ^ {- 1} = \left(E ^ {T}\right) ^ {- 1} D ^ {- 1} E ^ {- 1}
$$
$$
\lambda_ {\max } (M ^ {- 1}) \leq \lambda_ {\max } (D ^ {- 1}) = \frac {1}{\| v _ {k + 1} ^ {\prime} \| ^ {2}}
$$
$$
\begin{array}{l} \operatorname {v o l} \left(w _ {i + 1}, \dots , w _ {k}, w _ {k + 1}\right) ^ {2} \leq \left(1 + k \| v _ {i} ^ {\prime} \| ^ {2} \lambda_ {\max } (M ^ {- 1})\right) \det  (M) \\ \leq \left(1 + \frac {k \| v _ {i} ^ {\prime} \| ^ {2}}{\| v _ {k + 1} ^ {\prime} \| ^ {2}}\right) \det  (M) \\ = \det (M) + k \prod_ {j = i} ^ {k} \| v _ {j} ^ {\prime} \| _ {2} ^ {2} \\ \leq (1 + k) \prod_ {j = i} ^ {k} \| v _ {j} ^ {\prime} \| _ {2} ^ {2} \\ \end{array}
$$
Claim 8. Suppose  $v_{k+1}^{\prime} = 0$ . Then
$$
\operatorname {v o l} \left(w _ {i + 1}, \dots , w _ {k}, w _ {k + 1}\right) \leq \left(1 + \sqrt {k}\right) \prod_ {j = i} ^ {k} \| v _ {j} ^ {\prime} \| _ {2} \tag {11}
$$
Proof. The idea for this proof is similar to the previous claim. However, the main catch is that decomposing  $B^T B$  into  $A^T A + uu^T$  (as defined in the proof of Claim 7) is no longer helpful because  $v_{k+1}' = 0$  implies that  $A^T A$  is not invertible. However, there is a simple workaround.
Define the matrix  $B' = [w_{k+1}|w_{i+1}|\dots|w_k]$ . Note that  $\det((B')^T B') = \operatorname{vol}(w_{i+1},\ldots,w_k,w_{k+1})^2$  is the quantity we are interested in bounding. Recall that  $v_{k+1}' = 0$  by assumption. For clarity,
$$
(B ^ {\prime}) ^ {T} = \left[ \begin{array}{c} \alpha_ {i} ^ {k + 1} v _ {i} ^ {\prime} + \dots + \alpha_ {k} ^ {k + 1} v _ {k} ^ {\prime} \\ \alpha_ {i} ^ {i + 1} v _ {i} ^ {\prime} + v _ {i + 1} ^ {\prime} \\ \alpha_ {i} ^ {i + 2} v _ {i} ^ {\prime} + \alpha_ {i + 1} ^ {i + 2} v _ {i + 1} ^ {\prime} + v _ {i + 2} ^ {\prime} \\ \vdots \\ \alpha_ {i} ^ {k} v _ {i} ^ {\prime} + \dots + v _ {k} ^ {\prime} \end{array} \right]
$$
Note that  $(B')^T$  is the same as  $B^T$  from the proof of Claim 7 except for moving the last row to the position of the first row. This change is just for convenience in this proof.
Define the following coefficients matrix  $C \in \mathbb{R}^{(k - i + 1) \times (k - i + 1)}$ :
$$
\begin{array}{l} C = \left[ \begin{array}{c c c c c} \alpha_ {i} ^ {k + 1} & \dots & & & \alpha_ {k} ^ {k + 1} \\ \alpha_ {i} ^ {i + 1} & 1 & 0 & 0 & 0 \\ \alpha_ {i} ^ {i + 2} & \alpha_ {i + 1} ^ {i + 2} & 1 & 0 & 0 \\ \vdots & & & & \\ \alpha_ {i} ^ {k} & \alpha_ {i + 1} ^ {k} & & \dots & 1 \end{array} \right] = \left[ \begin{array}{c c c c c} 1 & 0 & & & 0 \\ \alpha_ {i} ^ {i + 1} & 1 & 0 & 0 & 0 \\ \alpha_ {i} ^ {i + 2} & \alpha_ {i + 1} ^ {i + 2} & 1 & 0 & 0 \\ \vdots & & & & \\ \alpha_ {i} ^ {k} & \alpha_ {i + 1} ^ {k} \end{array} \right] + \left[ \begin{array}{c} 1 \\ 0 \\ \vdots \\ \vdots \\ 0 \end{array} \right] \left[ (\alpha_ {i} ^ {k + 1} - 1) \quad \alpha_ {i + 1} ^ {k + 1} \quad \dots \quad \alpha_ {k} ^ {k + 1} \right] \\ := C ^ {\prime} + e _ {1} x ^ {T} \\ \end{array}
$$
Define  $W' = [v_i'] \cdots |v_k']$ . By construction,  $(B')^T = C(W')^T$ . Therefore
$$
\left(W ^ {\prime}\right) ^ {T} W := D ^ {\prime} = \operatorname {d i a g} \left(\| v _ {i} ^ {\prime} \| _ {2} ^ {2}, \dots , \| v _ {k} ^ {\prime} \| _ {2} ^ {2}\right)
$$