Title: Composable Coresets for Determinant Maximization: Greedy is Almost Optimal
Venue: NeurIPS 2023
Accepted: yes
Keywords: Determinant Maximization

Composable Coresets

Greedy Algorithm

DPP
Abstract: Given a set of $n$ vectors in $\mathbb{R}^d$, the goal of the \emph{determinant maximization} problem is to pick $k$ vectors with the maximum volume.
 Determinant maximization is the MAP-inference task for determinantal point processes (DPP) and has recently received considerable attention for modeling diversity.
 As most applications for the problem use large amounts of data, this problem has been studied in the relevant \textit{composable coreset} setting.
In particular, [Indyk-Mahabadi-OveisGharan-Rezaei--SODA'20, ICML'19] showed that one can get composable coresets with optimal approximation factor of $\tilde O(k)^k$ for the problem, and that a local search algorithm achieves an almost optimal approximation guarantee of $O(k)^{2k}$.
In this work, we show that the widely-used Greedy algorithm also provides composable coresets with an almost optimal approximation factor of $O(k)^{3k}$, which improves over the previously known guarantee of $C^{k^2}$, and supports the prior experimental results showing the practicality of the greedy algorithm as a coreset.
Our main result follows by showing a local optimality property for Greedy:
swapping a single point from the greedy solution with a vector that was not picked by the greedy algorithm can increase the volume by a factor of at most $(1+\sqrt{k})$. This is tight up to the additive constant $1$. Finally, our experiments show that the local optimality of the greedy algorithm is even lower than the theoretical bound on real data sets.

Composable Coresets for Determinant Maximization: Greedy is Almost Optimal*

Abstract

Given a set of n vectors in R d , the goal of the determinant maximization problem is to pick k vectors with the maximum volume. Determinant maximization is the MAP-inference task for determinantal point processes (DPP) and has recently received considerable attention for modeling diversity. As most applications for the problem use large amounts of data, this problem has been studied in the relevant composable coreset setting. In particular, [IMGR20] showed that one can get composable coresets with optimal approximation factor of ˜ O ( k ) k for the problem, and that a local search algorithm achieves an almost optimal approximation guarantee of O ( k ) 2 k . In this work, we show that the widely-used Greedy algorithm also provides composable coresets with an almost optimal approximation factor of O ( k ) 3 k , which improves over the previously known guarantee of C k 2 , and supports the prior experimental results showing the practicality of the greedy algorithm as a coreset. Our main result follows by showing a local optimality property for Greedy: swapping a single point from the greedy solution with a vector that was not picked by the greedy algorithm can increase the volume by a factor of at most (1 + √ k ) . This is tight up to the additive constant 1 . Finally, our experiments show that the local optimality of the greedy algorithm is even lower than the theoretical bound on real data sets.

1 Introduction

In the determinant maximization problem, we are given a set P of n vectors in R d , and a parameter k ≤ d . The objective is to find a subset S = { v 1 , . . . , v k } ⊆ P consisting of k vectors such that that the volume squared of the parallelepiped spanned by the points in the subset S is maximized. Equivalently, the volume squared of a set S , denoted by vol ( S ) , is equal to the determinant of the Gram matrix of the vectors in S . Determinant maximization is the MAP-inference of determinantal point processes, and both of these problems as well as their variants have found numerous applications in data summarization, machine learning, experimental design, and computational geometry. In particular, the determinant of a subset of points is one way to measure the diversity of the subset, and thus they have been studied extensively over the last decade in this context [MJK17, GCGS14, KT + 12, CGGS15, KT11, YFZ + 16, LCYO16].

The best approximation factor for the problem in this regime is due to the work of [Nik15] who shows a factor of e k , and it is known that an exponential dependence on k is necessary [CMI13] unless P = NP. However, the most common algorithm used for this problem in practical applications is a natural greedy algorithm. In this setting, the algorithm first picks the vector with the largest norm,

and then greedily picks the vector with largest perpendicular component to the subspace spanned by the current set of picked vectors, thus maximizing the volume greedily in each iteration. This algorithm is known to have an approximation factor of ( k !) 2 [ÇMI09].

As in most applications of determinant maximization one needs to work with large amounts of data, there has been an increasing interest in studying determinant maximization in large data models of computation [MJK17, WIB14, PJG + 14, MKSK13, MKBK15, MZ15, BENW15]. One such model that we focus on in this work is the composable coreset setting [IMMM14]. Intuitively, composable coresets are small 'summaries' of a data set with the composability property: for the summaries of multiple datasets, the union of the summaries should make a good summary for the union of the datasets. More precisely, in this setting, instead of a single set of vectors P , there are m sets P 1 , . . . , P m ⊆ R d . In this context, a mapping function c that maps a point set to one of its subsets is called α -composable coreset for determinant maximization, if for any collection of point sets P 1 , . . . , P m ,

where MAXDET k is used to denote the maximum achievable determinant with parameter k . (Similarly, MAXVOL k is used to denote the maximum volume, with MAXVOL 2 k = MAXDET k .) For clarity, we note that the mapping function c can only view its input data set P i and has no knowledge of other data sets while constructing c ( P i ) . [IMMM14] showed that a composable coreset for a task automatically gives an efficient distributed and an efficient streaming algorithm for the same task.

Indeed, composable coresets have been used for determinant maximization. In particular, [IMGR20, MIGR19], presented a composable coreset of size O ( k log k ) with approximation factor of ˜ O ( k ) k using spectral spanners, which they showed to be almost tight. In particular, the best approximation factor one can get is Ω( k k -o ( k ) ) (Theorem 1.4). As the above algorithm is LP-based and does not provide the best performance in practice, they proposed to use the greedy algorithm followed by a local search procedure, and showed that this simple algorithm also yields a coreset of size k with an almost optimal approximation guarantee of O ( k ) 2 k . They also proved that the greedy algorithm alone yields a C k 2 guarantee for composable coresets, which is far larger than the optimal approximation of ˜ O ( k ) k for this problem.

Since the greedy algorithm provides a very good performance in practice [MIGR19, MKSK13], an improved analysis of the greedy algorithm in the coreset setting is very desirable. Furthermore, both of these prior work implied that greedy performs well in practice in the context of distributed and composable coreset settings [MKSK13], and in particular its performance is comparable to that of the local search algorithm for the problem [MIGR19].

Our contribution. In this paper, we close this theoretical gap: we prove that the greedy algorithm provides a O ( k ) 3 k -composable coreset for the determinant maximization problem (Theorem 4). This explains the very good performance of this algorithm on real data previously shown in [MIGR19, MKSK13]. We achieve this by proving an elegant linear algebra result on the local optimality of the greedy algorithm: swapping a single point from the greedy solution with a vector that was not picked by the greedy algorithm can increase the volume by a factor of at most (1 + √ k ) . We further show that this is tight up to the additive constant 1 . As an application of our result, we give a proof that the locality property can recover and in fact marginally improve the k ! guarantee of the greedy algorithm of [ÇMI09] for the offline volume maximization problem.

Finally, in Section 4, we run experiments to measure the local optimality of the greedy algorithm on real data, and show that this number is much smaller in practice than the worst case theoretically guaranteed bound. In fact, in our experiments this number is always less than 1 . 5 even for k even as large as 300 . Again this explains the practical efficiency of the greedy algorithm as a coreset shown in [MIGR19, MKSK13].

1.1 Preliminaries

1.1.1 The Greedy Algorithm

Recall the standard offline setting for determinant maximization, where one is required to pick k vectors out of the n vectors in P of maximum volume. Here, [ÇMI09] showed that greedily picking

the vector with the largest perpendicular distance to the subspace spanned by the current solution (or equivalently, greedily picking the vector that maximizes the volume as in Algorithm 1) outputs a set of vectors that is within k ! of the optimal volume. Formally, if Greedy ( P ) is the output of Algorithm 1, then

1.1.2 Local Search for Composable Coresets

In [MIGR19], the authors show that the greedy algorithm followed by the local search procedure with parameter ϵ (as described in Algorithm 2) provides a (2 k (1 + ϵ )) 2 k -composable coreset for determinant maximization. A locally optimal solution can thus be naturally defined as follows:

Definition 1 ( (1 + ϵ ) -Locally Optimal Solution) . Given a point set P ⊆ R d and c ( P ) ⊆ P with | c ( P ) | = k , we say c ( P ) is a (1 + ϵ ) -locally optimal solution for volume maximization if for any v ∈ c ( P ) and any w ∈ P \ c ( P ) ,

Given the output of the greedy algorithm Greedy ( P ) , one can obtain a locally optimal solution using a series of swaps: if the volume of the solution can be increased by a factor of (1 + ϵ ) by swapping a vector in the current solution with a vector in the point set P that has not been included, we make the swap. Since vol ( Greedy ( P )) is within a factor of k ! of the optimal, we will make at most k log k log(1+ ϵ ) swaps. This is precisely the local search algorithm (Algorithm 2). For any point set P , we denote the output of Algorithm 2 by LS ( P ) .

In [MIGR19], the authors prove that local search yields a O ( k ) 2 k -composable coreset for determinant maximization. Formally, they prove the following.

Theorem 2. Let P 1 , . . . , P m ⊆ R d . For each i = 1 , . . . , m , let LS ( P i ) be the output of the local search algorithm (Algorithm 2) with parameter ϵ . Then

Remark 3. Even though [MIGR19] treats ϵ as a small constant in [0 , 1] , the proof for Theorem 2 above holds for any non-negative ϵ .

1.2 Outline of our approach

In [MIGR19], the authors prove Theorem 2 for local search using a reduction to a related problem called k -directional height. The authors then use similar ideas to prove that the output of the greedy algorithm is also a composable coreset for determinant maximization. However, since we do not know a priori whether greedy is (1 + ϵ ) -locally optimal, the guarantee they obtain is significantly weaker: they only prove that the greedy algorithm yields a ((2 k ) · 3 k ) 2 k = C k 2 -composable coreset for determinant maximization. This is clearly far from the desired bound of k O ( k ) .

To improve the analysis of the greedy algorithm in the coreset setting, we ask the following natural question:

Can we prove that the output of the greedy algorithm is already locally optimal?

We answer this question positively. Our main result is Theorem 5, where we show that for any point set P , Greedy ( P ) is a (1+ √ k ) -locally optimal solution. In other words, the greedy algorithm has the same guarantee as local search with the parameter ϵ = √ k . This circumvents the loose reduction from greedy to the k -directional height problem and directly implies the following improved guarantee for the greedy algorithm in the coreset setting:

Theorem 4.

Thus, the greedy algorithm also provides a (2 k (1 + √ k )) 2 k = k O ( k ) -composable coreset for determinant maximization, which is near the optimal Ω( k k -o ( k ) ) .

Section 2 is dedicated to proving that greedy is (1 + √ k )-locally optimal (Theorem 5). We also show that this local optimality result of (1 + √ k ) for the greedy algorithm is tight up to the additive constant 1 . In Section 4 we show that on real and random datasets, the local optimality constant ϵ is much smaller than the bound of 1 + √ k , which serves as an empirical explanation for why greedy performs much better in practice than what the theoretical analysis suggests.

2 Greedy is Locally Optimal

Theorem 5 (Local Optimality) . Let V := Greedy ( P ) = { v 1 , . . . , v k } ⊆ P be the output of the greedy algorithm. Let v k +1 ∈ P \ V be a vector not chosen by the greedy algorithm. Then for all i = 1 , . . . , k ,

Proof. If rank( P ) < k , then the result is trivial. So we may assume rank( P ) ≥ k and V is linearly independent. Fix any v i ∈ V . Our goal is to show that vol ( V -v i + v k +1 ) ≤ (1 + √ k ) vol ( V ) . This trivially holds when i = k by the property of the greedy algorithm, so assume 1 ≤ i ≤ k -1 .

Let { v ′ 1 , . . . , v ′ k , v ′ k +1 } be the set of orthogonal vectors constructed by performing the Gram-Schmidt algorithm on { v 1 , . . . , v k , v k +1 } . Formally, let G t = span { v 1 , . . . , v t } . Define v ′ 1 = v 1 and v ′ t = v t -Π( G t -1 )( v t ) for t = 2 , . . . , k, k +1 , where Π( G )( v ) denotes the projection of the vector v onto the subspace G . Note that

For each j = i +1 , . . . , k, k +1 , write

We must have that | α l j | ≤ 1 by the greedy algorithm because if | α j l | > 1 , the vector v j would have been chosen before v l . Further, α j j = 1 by definition of Gram-Schmidt. The vector w j is what remains of v j once we subtract its projection onto the first i -1 vectors.

We are interested in bounding the following quantity:

Therefore, it suffices to prove the following:

̸

To establish this, we consider two cases. Recall that v ′ k +1 = v k +1 -Π( G k )( v k ) . We analyze the cases where v ′ k +1 = 0 and v ′ k +1 = 0 separately, although the ideas are similar. In Claim 7 and Claim 8 below, we establish the desired bound stated in Eq. (7) for v ′ k +1 = 0 and v ′ k +1 = 0 respectively. Theorem 5 then follows immediately.

To prove Claim 7 and Claim 8, the following well-known lemma will be useful. A proof can be found in [DZ07].

Lemma 6 (Matrix Determinant Lemma) . Suppose M is an invertible matrix. Then

Claim 7. Suppose v ′ k +1 = 0 . Then

Proof. Define the matrix B = [ w i +1 | · · · | w k | w k +1 ] . Note that det( B T B ) = vol ( w i +1 , . . . , w k , w k +1 ) 2 is the quantity we are interested in bounding. For clarity,

We define the matrix A by just removing the v ′ i terms from B as follows:

̸

where the column vector u is given by

Since | α j i | ≤ 1 for j = i +1 , . . . , k +1 by the nature of the greedy algorithm, we have that

Since ⟨ v ′ i , v ′ j ⟩ = 0 for all j = i , we have that

̸

̸

Therefore,

Claim 8. Suppose v ′ k +1 = 0 . Then

Proof. The idea for this proof is similar to the previous claim. However, the main catch is that decomposing B T B into A T A + uu T (as defined in the proof of Claim 7) is no longer helpful because v ′ k +1 = 0 implies that A T A is not invertible. However, there is a simple workaround.

̸

̸

We now bound the desired volume quantity. Let M = A T A . M is clearly a positive semi-definite matrix. In fact, because we assumed that v ′ k +1 = 0 , it will turn out that M is positive definite and thus invertible. For now, assume that M -1 exists. We will compute the inverse explicitly later.

where λ max ( M -1 ) is the largest eigenvalue of M -1 . We will now show that M -1 does in fact exist and bound λ max ( M -1 ) . Consider the matrix E and W defined as follows:

It is easy to check that EW T = A T . Therefore,

where D is the diagonal matrix given by

It is easy to see that E has all eigenvalues equal to 1 , and so must be invertible with determinant 1 . The same is obviously true for E T , E -1 and ( E T ) -1 as well. It follows that

Since v ′ k +1 = 0 , we have that ∥ v ′ j ∥ 2 > 0 for all j , so D -1 clearly exists. It follows that

Define the matrix B ′ = [ w k +1 | w i +1 | · · · | w k ] . Note that det(( B ′ ) T B ′ ) = vol ( w i +1 , . . . , w k , w k +1 ) 2 is the quantity we are interested in bounding. Recall that v ′ k +1 = 0 by assumption. For clarity,

Note that ( B ′ ) T is the same as B T from the proof of Claim 7 except for moving the last row to the position of the first row. This change is just for convenience in this proof.

Define the following coefficients matrix C ∈ R ( k -i +1) × ( k -i +1) :

Define W ′ = [ v ′ i | · · · | v ′ k ] . By construction, ( B ′ ) T = C ( W ′ ) T . Therefore

It remains to show that | det( C ) | ≤ (1 + √ k ) . We may assume that α k +1 i ≥ 0 by taking the negative of the first column if necessary. This does not affect the magnitude of the determinant. Note that all eigenvalues of C ′ and ( C ′ ) -1 are 1 . Further,

We now provide a simple example where the output of the greedy algorithm is at best √ k -locally optimal, thus demonstrating that the locality result for greedy is optimal up to the constant 1 .

Theorem 9 (Tightness of Local Optimality) . There exists a point set P = { v 1 , . . . , v k , v k +1 } from which the greedy algorithm picks V = { v 1 , . . . , v k } , and

It follows that

Proof. Let P = { v 1 , . . . , v k , v k +1 } where v 1 ∈ R k is the vector of all ones and v i = √ ke i -1 for i = 2 , . . . , k +1 . Since the magnitude of every vector in P is √ k , the greedy algorithm could start by picking v 1 . The greedy algorithm will then pick any k -1 of the remaining k vectors. Without loss in generality, assume that the algorithm picks V = { v 1 , . . . , v k } . Then vol ( V ) = ( √ k ) k -1 . On the other hand, vol ( V -v 1 + v k +1 ) = ( √ k ) k . The result follows.

3 Application to Standard Determinant Maximization

The greedy algorithm for volume maximization was shown to have an approximation factor of k ! in [ÇMI09]. We provide a completely new proof for this result with a slightly improved approximation factor.

Theorem 10. Let P be a point set, Greedy ( P ) = { v 1 , . . . , v k } the output of the greedy algorithm, and MAXVOL k ( P ) the maximum volume of any subset of k vectors from P . Then

Proof. Let S ⊆ P be the set of k vectors with maximum volume. Without loss of generality and for simplicity of exposition, we assume that Greedy ( P ) ∩ S = ∅ (the proof still goes through if this is not the case). We will order S in a convenient manner.

Consider the set W 1 = { v 1 } ∪ S with k +1 elements. Perform the greedy algorithm on W 1 with k steps. Clearly, greedy will choose v 1 first and then some k -1 of the remaining vectors. Label the left out vector w 1 .

Inductively define W i +1 = { v 1 , . . . , v i , v i +1 } ∪ ( S -{ w 1 , . . . , w i } ) , which has size k +1 . Perform greedy on W i +1 with k steps. The first i +1 vectors chosen will be v 1 , . . . , v i , v i +1 by definition. Call the left out vector w i +1 . We now have an ordering for S = { w 1 , . . . , w k } .

Starting with the greedy solution, we will now perform k swaps to obtain the optimal solution. Each swap will increase the volume by a factor of at most 1 + √ k . Initially, our solution starts with Greedy ( P ) = { v 1 , . . . , v k } . Note that this is also the output of greedy when applied to the set Greedy ( P ) ∪ { w k } = W k . Swapping in w k in place of v k increases our volume by a factor of at most 1 + √ k .

Our current set of vectors is now { v 1 , . . . , v k -1 , w k } . By the ordering on S , this is also the greedy output on the set W k -1 = { v 1 , . . . , v k -1 , w k -1 , w k } . Therefore, we may swap in w k -1 in place of v k -1 in our current set of vectors by increasing the volume by at most a factor of (1 + √ k ) . Proceeding in this manner, we can perform k swaps to obtain the optimal solution from the greedy solution by increasing our volume by a factor of at most (1 + √ k ) k .

To obtain the slightly better approximation factor in the theorem statement, we observe that in the proof of Theorem 5, swapping out the i th vector from the greedy solution for a vector that was not chosen increases the volume only by a factor of (1 + √ k +1 -i ) ≤ 1 + √ k (Eq. (10),Eq. (12)), and that swapping out the k th vector does not increase the volume at all. Therefore, the approximation factor of greedy is at most

Remark 11. Note that ∏ k i =2 (1 + √ i ) < 2 k √ k ! for k ≥ 7 , which is ( k !) 1 2 + o (1) . While the improvement in the approximation factor is quite small, we emphasize that the proof idea is very different from the k ! guarantee obtained in [ÇMI09].

4 Experiments

In this section, we measure the local optimality parameter for the greedy algorithm empirically. We use two real world datasets, both of which were used as benchmarks for determinant maximization in immediately related work ([MIGR19, LJS16]:

MNIST [LBBH98], which has 60000 elements, each representing a 28 -by28 bitmap image of a hand-drawn digit;

GENES [BQK + 14], which has 10000 elements, with each representing the feature vector of a gene. The data set was initially used in identifying a diverse set of genes to predict breast cancer tumors. After removing the elements with some unknown values, we have around 8000 points.

We measure the local optimality parameter both as a function of k , and as a function of the data set size as explained in the next two subsections.

4.1 Local Optimality for Real and Random Datasets as a Function of k

Experiment Setup: For both MNIST and GENES, we consider a collection of m = 10 data sets, each with n = 3000 points chosen uniformly at random from the full dataset. We ran the greedy algorithm for k from 1 to 20 and measured the local optimality value (1 + ϵ ) as a fucntion of k = 2 , 4 , . . . , 20 for each of the 10 data sets in the collection. More precisely, for each such k , we took the maximum value of (1 + ϵ ) over every data set in the collection. The reason we take the worst value of (1 + ϵ ) , is that in the context of composable coresets, we require the guarantee to hold for each individual data set to be (1 + ϵ ) -locally optimal. We repeated this process for 5 iterations and took the average. We plot this value as a function of k .

Further, to compare against a random data set, for both MNIST and GENES, we repeated the above experiment against a set of random points of the same dimension sampled uniformly at random from the unit sphere.

Results: As shown in Fig. 1, while the real world data sets have local optimality value (1 + ϵ ) higher than the random data sets, they are both significantly lower than (less than 1 . 4 ) the theoretical bound of (1 + √ k ) . This suggests that real world data sets behave much more nicely and are closer to random than the worst case analysis would suggest, which explains why greedy does so well in practice.

For the purpose of diversity maximization, the regime of interest is when k ≪ n . However, we wanted to verify that the local optimality value does not increase much even when k is much larger and closer to n . Since measuring local optimality is expensive when both k and n are large, we ran the same experiment again, except with n = 300 points per point set, and measuring the local optimality at k = 1 , 50 , 100 , . . . , 300 in steps of 50 . Again, as seen in Fig. 2, local optimality stays much below 1 + √ k (in fact less than 1 . 5 ) for larger values of k as well.

Dataset: GENES

Real Dataset

Random Dataset

5

Theoretical Bound: 1 + sqrt(k)

Local Optimality: 1 + Epsilon

4

m

2

2.5

5.0

7.5

10.0

12.5

15.0

17.5

20.0

K

Dataset:MNIST

Real Dataset

Random Dataset

5

Theoretical Bound: 1 + sqrt(k)

Local Optimality: 1 + Epsilon

4

2

2.5

5.0

7.5

10.0

12.5

15.0

17.5

20.0

K

Dataset:GENES

Dataset: MNIST

Real Dataset

Real Dataset

1.20

Random Dataset

1.4

Random Dataset

+ Epsilon

Local Optimality: 1 + Epsilon

1.15

1.3

Local Optimality: 1

1.10

1.2

1.05

1.1

1.00

1.0

0

50

100

150

200

250

300

0

50

100

150

200

250

300

K

K

4.2 Local Optimality as a Function of the Size of Point Sets

Experiment Setup: Here, we fix the value of k ∈ { 5 , 10 , 15 , 20 } and compute the local optimality value (1 + ϵ ) while increasing the size of the point sets. The point set size is varied from 500 to 4000 in intervals of 500 . For each point set size, we chose a stream of 10 random point sets from the dataset and took the maximum value over 10 iterations. Once again, we did this on MNIST and GENES and took the average of 5 iterations.

Results: As shown in Fig. 3, the local optimality parameter remains very low (lower than 1 . 2 ) regardless of the number of points in the data set, which is much smaller than (1 + √ k ) .

Dataset:GENES

Dataset:MNIST

1.200

k = 5

k =5

k =10

1.25

k =10

1.175

k = 15

k = 15

Epsilon

k =20

+ Epsilon

k = 20

1.150

1.20

Local Optimality:

+

1

1.125

Local Optimality:1

1.15

1.100

1.10

1.075

1.050

1.05

1.025

1.00

500

1000

1500

2000

2500

3000

3500

4000

500

1000

1500

2000

2500

3000

3500

4000

Number of Points in P

Number of Points in P

5 Conclusion

In this work, we provided an almost tight analysis of the greedy algorithm for determinant maximization in the composable coreset setting: we improve upon the previous known bound of C k 2 to O ( k ) 3 k , which is optimal upto the factor 3 in the exponent. We do this by proving a result on the local optimality of the greedy algorithm for volume maximization. We also support our theoretical analysis by measuring the local optimality of greedy over real world data sets. It remains an interesting question to tighten the constant in the exponent or otherwise provide a lower bound showing that 3 is in fact optimal.

References

[BENW15] Rafael Barbosa, Alina Ene, Huy Nguyen, and Justin Ward. The power of randomization: Distributed submodular maximization on massive datasets. In International Conference on Machine Learning , pages 1236-1244, 2015.

[BQK + 14] Nematollah Kayhan Batmanghelich, Gerald Quon, Alex Kulesza, Manolis Kellis, Polina Golland, and Luke Bornn. Diversifying sparsity using variational determinantal point processes. arXiv preprint arXiv:1411.6307 , 2014.

[CGGS15] Wei-Lun Chao, Boqing Gong, Kristen Grauman, and Fei Sha. Large-margin determinantal point processes. In UAI , pages 191-200, 2015.

[ÇMI09] Ali Çivril and Malik Magdon-Ismail. On selecting a maximum volume sub-matrix of a matrix and related problems. Theoretical Computer Science , 410(47-49):4801-4811, 2009.

[CMI13] Ali Civril and Malik Magdon-Ismail. Exponential inapproximability of selecting a maximum volume sub-matrix. Algorithmica , 65(1):159-176, 2013.

[DZ07] Jiu Ding and Aihui Zhou. Eigenvalues of rank-one updated matrices with some applications. Applied Mathematics Letters , 20(12):1223-1226, 2007.

[GCGS14] Boqing Gong, Wei-Lun Chao, Kristen Grauman, and Fei Sha. Diverse sequential subset selection for supervised video summarization. In Advances in Neural Information Processing Systems , pages 2069-2077, 2014.

[IMGR20] Piotr Indyk, Sepideh Mahabadi, Shayan Oveis Gharan, and Alireza Rezaei. Composable core-sets for determinant maximization problems via spectral spanners. In Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms , pages 16751694. SIAM, 2020.

[IMMM14] Piotr Indyk, Sepideh Mahabadi, Mohammad Mahdian, and Vahab S Mirrokni. Composable core-sets for diversity and coverage maximization. In Proceedings of the 33rd ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems , pages 100-108. ACM, 2014.

[KT11] Alex Kulesza and Ben Taskar. Learning determinantal point processes. 2011.

[KT + 12] Alex Kulesza, Ben Taskar, et al. Determinantal point processes for machine learning. Foundations and Trends® in Machine Learning , 5(2-3):123-286, 2012.

[LBBH98] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE , 86(11):2278-2324, 1998.

[LCYO16] Donghoon Lee, Geonho Cha, Ming-Hsuan Yang, and Songhwai Oh. Individualness and determinantal point processes for pedestrian detection. In European Conference on Computer Vision , pages 330-346. Springer, 2016.

[LJS16] Chengtao Li, Stefanie Jegelka, and Suvrit Sra. Efficient sampling for k-determinantal point processes. In Artificial Intelligence and Statistics , pages 1328-1337. PMLR, 2016.

[MIGR19] Sepideh Mahabadi, Piotr Indyk, Shayan Oveis Gharan, and Alireza Rezaei. Composable core-sets for determinant maximization: A simple near-optimal algorithm. In International Conference on Machine Learning , pages 4254-4263. PMLR, 2019.

[MJK17] Baharan Mirzasoleiman, Stefanie Jegelka, and Andreas Krause. Streaming nonmonotone submodular maximization: Personalized video summarization on the fly. arXiv preprint arXiv:1706.03583 , 2017.

[MKBK15] Baharan Mirzasoleiman, Amin Karbasi, Ashwinkumar Badanidiyuru, and Andreas Krause. Distributed submodular cover: Succinctly summarizing massive data. In Advances in Neural Information Processing Systems , pages 2881-2889, 2015.

[MKSK13] Baharan Mirzasoleiman, Amin Karbasi, Rik Sarkar, and Andreas Krause. Distributed submodular maximization: Identifying representative elements in massive data. In Advances in Neural Information Processing Systems , pages 2049-2057, 2013.

[MZ15] Vahab Mirrokni and Morteza Zadimoghaddam. Randomized composable core-sets for distributed submodular maximization. In Proceedings of the forty-seventh annual ACM symposium on Theory of computing , pages 153-162. ACM, 2015.

[Nik15] Aleksandar Nikolov. Randomized rounding for the largest simplex problem. In Proceedings of the forty-seventh annual ACM symposium on Theory of computing , pages 861-870. ACM, 2015.

[PJG + 14] Xinghao Pan, Stefanie Jegelka, Joseph E Gonzalez, Joseph K Bradley, and Michael I Jordan. Parallel double greedy submodular maximization. In Advances in Neural Information Processing Systems , pages 118-126, 2014.

[WIB14] Kai Wei, Rishabh Iyer, and Jeff Bilmes. Fast multi-stage submodular maximization. In International conference on machine learning , pages 1494-1502, 2014.

[YFZ + 16] Jin-ge Yao, Feifan Fan, Wayne Xin Zhao, Xiaojun Wan, Edward Y Chang, and Jianguo Xiao. Tweet timeline generation with determinantal point processes. In AAAI , pages 3080-3086, 2016.

Peer reviews:
comments: SUMMARY:
The paper investigates the composable coresets for the determinant maximization problem that aims to pick $k$ vectors with the maximum volume. The authors prove that the widely-used greedy algorithm also provides composable coresets with an approximation factor of $O(k)^{3k}$, followed by showing a local optimality property for greedy. Empirical results show that the local optimality of the greedy algorithm is even lower.

STRENGTHS:
- The topic of DPP is relevant.
- The technical result is solid.
- The observation of local optimality for greedy is of independent interest.

WEAKNESSES:
- The paper is a follow-up work of [IMGR20]. The improved analysis for the greedy algorithm is interesting but not the first result of a composable coreset.
- The studied problem is limited, e.g., only consider unconstrained DPP in which every set $P_i$ selects exactly $k$ vectors.

QUESTIONS:
- Can the theoretical result extend to other settings of DPP, e.g., different $P_i$ selects a different number of vectors, or certain constrained DPP?
- The observation of local optimality for greedy is interesting, does it have other applications?

LIMITATIONS:
No. I suggest discussing some social impact, e.g., whether the greedy is better or worse for fairness issues than local search.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-06-23

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work studies the determinant maximization problem, in which the input is a set of $n$ vectors in $d$ dimensions, and the goal is to select a subset of $k\\leq d$ of these vectors which maximizes the determinant of the Gram matrix of the $k$ vectors, which is also the volume of the parallelepiped spanned by the vectors. This is an important classical problem with connections to diversity. \n\nA popular heuristic for this problem is the greedy algorithm, which iteratively builds a set of vectors one at a time by selecting the vector that maximizes the improvement in the current volume. In prior work, an analysis of the greedy algorithm was given, which showed that it gives a $C^{k^2}$ factor approximation. However, the best known lower bound is $\\Omega(k^{k-o(k)})$ (for composable coresets? not sure what the exact setting is here). This work shows that the analysis of the greedy algorithm can in fact be tightened to approximately match this upper bound, giving an approximation factor of $O(k)^{3k}$. This result is shown by proving that the greedy algorithm is approximately locally optimal, which in turn implies the approximation guarantee from results shown in prior work.\n\nSTRENGTHS:\nThe analysis of greedy is an extremely important and old question (the prior bound is over 10 years old), and the authors provide a very nice improvement to this result, and achieves a tight approximation over some class of algorithms. The analysis is nontrivial to carry out.\n\nWEAKNESSES:\nWhile the result is very important, the techniques might be viewed as incremental over the analysis of local search given in prior work, and the paper doesn't give much intuition or discussion on why this result might have been missed in prior work.\n\nQUESTIONS:\n* I could not find the referenced lower bound of $\\Omega(k^{k-o(k)})$ in prior work (I checked https://arxiv.org/abs/1907.03197 and https://arxiv.org/abs/1807.11648), I would appreciate a pointer to a specific theorem stating this. In particular, I would like to understand the exact setting in which this lower bound applies.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

comments: SUMMARY:
The paper gives a tighter analysis for the greedy algorithm for constructing composable coreset for the determinant maximization problem. The greedy algorithm performs well in practice; however, the previous analysis was giving an approximation factor much farther from the lower bound. The authors in the paper bring the approximation factor for greedy algorithm much closer to the lower bound. this is achieved by showing that by just swapping a single point from the greedy solution with a vector obtained using local search approach does not increase the volume (determinant) by much. The authors also perform some experiments to show that in practice this local optimality property of greedy algorithm is actually much better than theoretical bounds.

STRENGTHS:
The paper is well written and clear. 
I tried to go through the proofs and unless I have missed something, the proofs are sound. Infact the proofs look quite elegant using techniques from simple linear algebra.
Code is provided for the empirical results. 
By closing the gap between the lower bound and the approximation factor given by the greedy algorithm, the paper is able to give better explanation for the good performance of greedy algorithm in practice.

WEAKNESSES:
The only weakness I could think of is that the contribution is only theoretical and does not really have much practical implication as greedy algorithm was already known to perform nicely in practice. The simplicity of the proof techniques while not really a weakness may not excite the community much from the novelty perspective.

QUESTIONS:
Please refer to weaknesses.

LIMITATIONS:
Please refer to weaknesses.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper considers the problem of picking k among n vectors that maximize the volume of the parallelepiped spanned by the selected vectors, which is equal to the squared determinant of the corresponding Gram matrix. In particular, the authors focus on the composable coreset setting of the problem where given a large dataset split into multiple subsets, the aim is to find small summaries (coreset) of each subset such that the union of the summaries is a good summary of the full dataset. Existing work showed that the best approximation possible for this problem is $\\Omega(k)^{k - o(k)}$ and that an LP-based algorithm achieves almost optimal $\\tilde{O}(k)^k$-approximation. Another Greedy algorithm followed by local search, which works better in practice than the LP-based one, was also shown to achieve ${O}(k)^{2k}$-approximation, while the Greedy algorithm alone is only shown to achieve $C^{k^2}$ for some constant $C$. This paper provides an improved approximation guarantee of ${O}(k)^{3k}$ for the Greedy algorithm, by providing an almost tight bound on the local optimality of the solution outputted by Greedy. The presented numerical results also show that the local optimality of the Greedy solution is significantly lower than the theoretical upper bound on real and random datasets.\n\nSTRENGTHS:\n- The paper provides a new analysis to the greedy algorithm which significantly improves upon its known approximation guarantee. Even though this approximation is still worse than the guarantees achieved by the LP-based and the Greedy + local search algorithms, this result is interesting as in practice the greedy algorithm performs better than the LP-based one, and skipping the local search stage saves time.\n \n- The theoretical results are correct and experimental results illustrate nicely that the local optimality of the Greedy solution in practice is significantly better than the theoretical bound.\n\nWEAKNESSES:\n- The paper is not well self-contained; several details are omitted in the discussion and proofs which affect clarity (see questions for details).\n\nQUESTIONS:\nSuggestions to improve clarity:\n- The Greedy algorithm description in the preliminaries Section 1.1.1 does not match the pseudocode in Algorithm 1 with no mention of the relation between them. It is good to explicitly discuss the relation between the two, even if this is possibly well known.\n- In the proof of Theorem 5, the vectors in V are implicitly assumed to be linearly independent. This should be stated explicitly, explaining why this can be assumed without loss of generality.\n- Provide details for the expression of $\\mathrm{vol}(V)$ given in Theorem 5 and for why $|a^j_l| \\leq 1$, or at least a clear reference. Similarly for other steps in this proof and other ones.\n- Provide a reference for Lemma 6.\n\n- In the experiments, do you use the data points directly as feature vectors, or you apply a kernel as in prior work?\n\nMinor comments/suggestions:\n- using $\\mathrm{vol}(S)$ to denote the square volume is a bit confusing. I propose to use $\\mathrm{vol}^2(S)$.\n- typo line 87: $(2 k ( 1 + \\epsilon))^k \\rightarrow (2 k ( 1 + \\epsilon))^{2k}$\n- typo line 154: $a^j_i \\rightarrow |a^j_i|$\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}