Title: Random Cuts are Optimal for Explainable k-Medians
Venue: NeurIPS 2023
Accepted: yes
Keywords: Clustering

k-medians

Decision Tree

Explainability
Abstract: We show that the RandomCoordinateCut algorithm gives the optimal competitive ratio for explainable $k$-medians in $\ell_1$. The problem of explainable $k$-medians was introduced by Dasgupta, Frost, Moshkovitz, and Rashtchian in 2020. Several groups of authors independently proposed a simple polynomial-time randomized algorithm for the problem and showed that this algorithm is $O(\log k \log\log k)$ competitive.  We provide a tight analysis of the algorithm and prove that its competitive ratio is upper bounded by $2\ln k+2$. This bound matches the $\Omega(\log k)$ lower bound by Dasgupta et al (2020).

Random Cuts are Optimal for Explainable k -Medians

Abstract

We show that the RANDOMCOORDINATECUT algorithm gives the optimal competitive ratio for explainable k -medians in ℓ 1 . The problem of explainable k -medians was introduced by Dasgupta, Frost, Moshkovitz, and Rashtchian in 2020. Several groups of authors independently proposed a simple polynomial-time randomized algorithm for the problem and showed that this algorithm is O (log k log log k ) competitive. We provide a tight analysis of the algorithm and prove that its competitive ratio is upper bounded by 2 ln k +2 . This bound matches the Ω(log k ) lower bound by Dasgupta et al (2020).

1 Introduction

Machine learning is being increasingly used to make decisions for critical applications, such as healthcare, finance, and public policy. Considering the profound impact of algorithmic decisions on individuals and society, it is essential to understand the underlying logic behind these decisions. In this paper, we explore an explainable k -medians clustering algorithm (called RANDOMCOORDINATECUT). The algorithm's aim is to cluster data sets and present results in a manner easily understood and visualized by humans.

Clustering is a fundamental task in unsupervised learning. Among many clustering methods, k -means, k -medians, and k -medoids are particularly popular. These are centroid-based methods that choose k centers and assign each data point to the center nearest to it. As a result, each cluster is a V oronoi cell in the V oronoi partition of the space. Since these cells may have a complicated boundary (see Figure 1 for an example of k -medians), it is not always easy for humans to comprehend and visualize such clustering.

To address this problem, Dasgupta, Frost, Moshkovitz, and Rashtchian [2020] introduced explainable k -means and k -medians clustering. They argued that decision trees are easy to understand and interpret. Therefore, in order to make clustering more explainable, we need to use threshold decision trees to define clusters. A threshold decision tree is a binary space partitioning tree with k leaves. Each internal node of the threshold decision tree splits the data into two groups using a threshold cut ( j, θ ) : on the one side of the cut, we have points x with x j ≤ θ and, on the other side, points x with x j > θ . Thus, every node of the tree corresponds to a rectangular region of the space. A decision tree with k leaves partitions data set X into k clusters, P 1 , . . . , P k . See Figure 1 for an example. Dasgupta et al. [2020] suggested that we use the standard k -medians and k -means objectives to measure the cost of the threshold decision tree. For k -medians, the cost of a threshold decision tree T equals

where P 1 , . . . , P k is the partitioning of X produced by T ; and ˆ c 1 , . . . , ˆ c k are the medians of clusters P 1 , . . . , P k . We denote the ℓ 1 -norm by ∥ · ∥ 1 . Note that each P i is a rectangular region of the space. Thus, generally speaking, every x is not assigned to the closest center ˆ c 1 , . . . , ˆ c k like in unconstrained k -medians or k -means.

Liren Shan ∗ TTIC

y

8

.

6

1

x

≤ -

1

.

9

2

3

20.0

17.5

15.0

12.5

10.0

7.5

5.0

2.5

0.0

10.0-7.5-5.0-2.5

0.0

2.5

5.0

7.5

10.0

20.0

17.5

15.0

12.5

10.0

7.5

5.0

2.5

0.0

10.0-7.5-5.0-2.5

0.0

2.5

5.0

7.5

10.0

Dasgupta, Frost, Moshkovitz, and Rashtchian [2020] defined the price of explainability as the ratio of the k -medians cost of explainable clustering to the optimal cost of unconstrained k -medians clustering. They showed that the cost of explainability for k -means and k -medians (somewhat surprisingly) does not depend on the number of points in the data set X and only depends on k . Specifically, they provided a greedy algorithm that given k reference centers c 1 , c 2 , · · · , c k of any unconstrained k -medians as input, outputs a threshold decision tree of cost at most O ( k ) times the cost of original unconstrained k -medians with centers c 1 , c 2 , · · · , c k . We call such an algorithm O ( k ) competitive. To get an explainable k -medians clustering, we first obtain reference centers c 1 , c 2 , · · · , c k using an off-the-shelf approximation algorithm for k -medians and then run an α -competitive algorithm for explainable k -medians with centers c 1 , c 2 , · · · , c k given as input. This algorithm produces the desired threshold decision tree. Dasgupta et al. [2020] also gave an O ( k 2 ) competitive algorithm for k -means and showed Ω(log k ) lower bounds on the price of explainability for both k -medians and k -means.

The notion of explainable clustering immediately got a lot of attention in the field (Laber and Murtinho [2021], Makarychev and Shan [2021], Gamlath et al. [2021], Charikar and Hu [2022], Esfandiari et al. [2022]). Particularly, Makarychev and Shan [2021], Esfandiari, Mirrokni, and Narayanan [2022] provided almost optimal algorithms for explainable k -medians, and Makarychev and Shan [2021], Esfandiari, Mirrokni, and Narayanan [2022], Gamlath, Jia, Polak, and Svensson [2021] provided almost optimal algorithms for k -means. The competitive ratios of these algorithms are ˜ O (log k ) for k -medians and ˜ O ( k ) for k -means.

The algorithms for explainable k -medians by Makarychev and Shan [2021], Esfandiari, Mirrokni, and Narayanan [2022], Gamlath, Jia, Polak, and Svensson [2021] are variants of the same simple algorithm, which we call RANDOMCOORDINATECUT. This algorithm receives a set of k reference centers c 1 , . . . , c k as input and then builds a threshold decision tree with k leaves. It works as follows. It recursively partitions d -dimensional space until every cell contains exactly one reference center c i . The algorithm starts with a tree consisting of one node, the root. Initially, all k reference centers are assigned to that root. At every step, the algorithm picks a random threshold cut ( j, θ ) and splits centers in every cell using this cut. If this cut does not separate any centers in a cell u (i.e., all centers in u are located on one side of the cut), then the algorithm does not split u into two regions at this step. Finally, for every leaf u of the constructed tree, the unique center that belongs to the cell corresponding to u is assigned to u . We provide pseudo-code for this algorithm in Figure 2.

Makarychev and Shan [2021], Esfandiari et al. [2022] showed that the competitive ratio of RANDOMCOORDINATECUT is at most O (log k log log k ) . That is, for every data set X and set of centers c 1 , . . . , c k ,

Note that the running time of this algorithm is ˜ O ( kd ) . Gamlath, Jia, Polak, and Svensson [2021] provided a slightly worse bound of O (log 2 k ) on the competitive ratio of this algorithm. They also

conjectured that this algorithm is optimal and its competitive ratio is O (log k ) , more specifically, H k -1 + 1 , where H k is the k -th harmonic number. They provided some justification for their conjecture by proving this bound for a very special set of centers and data points (corresponding to the case of completely disjoint sets in our Set Elimination Game).

Our Results. In this work, we show that indeed the competitive ratio of RANDOMCOORDINATECUT is at most 2 ln k +2 , and, therefore, this algorithm has the optimal competitive ratio which matches the lower bound of Dasgupta, Frost, Moshkovitz, and Rashtchian [2020]. Our analysis is not only tight but also fairly simple. To get our result we define a game, the Set Elimination Game, which was also implicitly analyzed in previous works on this topic. We show that the cost of this game is at most 2 ln k +2 .

Related Work. The unconstrained k -medians clustering has been extensively studied. Charikar, Guha, Tardos, and Shmoys [1999] gave the first constant factor approximation algorithm for the problem in general metric spaces. Li and Svensson [2013] provided a 1 + √ 3 + ε approximation algorithm. Byrka, Pensyl, Rybicki, Srinivasan, and Trinh [2017] improved the approximation factor to 2 . 675 + ε . Cohen-Addad, Esfandiari, Mirrokni, and Narayanan [2022] recently improved the approximation factor to 2 . 406 for Euclidean k -medians. Megiddo and Supowit [1984] showed that the k -medians in ℓ 1 problem is NP-hard. Cohen-Addad and Lee [2022] showed that it is also NP-hard to approximate k -medians in ℓ 1 within a factor of 1 . 06 .

As we discuss above, Gamlath, Jia, Polak, and Svensson [2021], Esfandiari, Mirrokni, and Narayanan [2022], Makarychev and Shan [2021], independently proposed the RANDOMCOORDINATECUT algorithm. They also gave an ˜ O ( k ) algorithm for explainable k -means and showed a lower bound of ˜ Ω( k ) for the problem. Charikar and Hu [2022] provided an O ( k 1 -2 /d · poly( d, log k )) competitive algorithm for explainable k -means, whose competitive ratio depends on the dimension d of the instance. For small d ≪ log k/ log log k , their bound is better than O ( k ) . They showed an almost matching Ω( k 1 -2 /d / ploy log k ) lower bound for explainable k -means. Esfandiari et al. [2022] gave an upper bound of O ( d log 2 d ) on the competitive ratio of RANDOMCOORDINATECUT for explainable k -medians. This bound is better than O (log k ) for small d ≪ log k/ log log k . Laber and Murtinho [2021] gave O ( d log k ) and O ( dk log k ) competitive algorithms for explainable k -medians and k -means, respectively. Frost, Moshkovitz, and Rashtchian [2020] provided some empirical evidence that bi-criteria algorithms for explainable k -means (that partition the data set into (1 + δ ) k clusters) can give a much better competitive ratio than O ( k ) . Then, Makarychev and Shan [2022] gave a ˜ O ( 1 δ log 2 k ) competitive bi-criteria algorithm for explainable k -means. Bandyapadhyay, Fomin, Golovach, Lochet, Purohit, and Simonov [2022] provided an algorithm

that computes the optimal explainable k -medians and k -means clustering in time n 2 d + O (1) and (4 nd ) k + O (1) , respectively. Laber, Murtinho, and Oliveira [2023] proposed to use shallow decision trees for explainable clustering.

Independently and concurrently with our work, Gupta, Pittu, Svensson, and Yuan [2023] proved a O (log k ) bound on the price of explainability for k -medians. They showed that the competitive ratio of RANDOMCOORDINATECUT is 1 + H k -1 , where H k is the k -th harmonic number. Their work answers the open question raised by Gamlath, Jia, Polak, and Svensson [2021]. They also proved a hardness of approximation result for explainable k -medians clustering and improved the competitive ratio for explainable k -means from O ( k log k ) to O ( k log log k ) .

2 Set Elimination Game

In this section, we define the set elimination game. Consider a discrete finite measure space (Ω , µ ) . In this space, each element ω ∈ Ω has a measure of µ ( ω ) , and the measure of every set S ⊆ Ω equals µ ( S ) = ∑ ω ∈ S µ ( ω ) . Let S 1 , S 2 , . . . , S k ⊂ Ω be k distinct sets which may overlap with each other. The set elimination game proceeds in a series of rounds. Initially, all sets S 1 , . . . , S k enter the competition. Formally, they belong to the set of remaining sets R 0 = { S 1 , . . . , S k } . At every round n , the host picks a random ω n ∈ Ω with probability Pr( ω n = ω ) = µ ( ω ) /µ (Ω) . Then, all sets S i that contain ω n are eliminated from the game unless all remaining sets contain ω n , in which case, no set gets eliminated. That is, for n ≥ 1 ,

The last remaining set is declared the winner. We denote that winner by winner . We say that the cost of the game is the measure of the winning set, µ (winner) .

We remark that R n cannot get empty (in which case, the winner would not be defined) because of the 'otherwise' clause in the definition (1). We shall always assume that all sets S 1 , . . . , S k are not only distinct and non-empty but also (a) for every i , µ ( S i ) > 0 , and (b) for all i and j , µ ( S i △ S j ) > 0 (here, S i △ S j denotes the symmetric difference of sets S i and S j ). Then, in every game, there is a unique winner with probability 1 .

We similarly define the set elimination game for arbitrary finite measure spaces: For an arbitrary finite measure space (Ω , µ ) , element ω n is chosen with probability function Pr( ω n ∈ S ) = µ ( S ) /µ (Ω) .

Our main result is the following theorem, which, as we discuss later in Section 2.1, implies that the competitive ratio of the explainable clustering algorithm is 2 ln k +2 .

Theorem 2.1. Consider a set elimination game with the finite measure space (Ω , µ ) and k distinct sets S 1 , S 2 , . . . , S k (as above). The expected cost of the game is at most

To simplify the exposition, we will prove this theorem for discrete finite measure sets. If Ω is not a discrete measure space, we first replace it with a quotient space: We say that ω ′ ∈ Ω and ω ′′ ∈ Ω are equivalent ( ω ′ ∼ ω ′′ ) if they are contained in exactly the same set of sets S 1 , . . . , S k . This equivalence relation partitions Ω into at most 2 k different equivalence classes. We replace Ω with the quotient space Ω / ∼ whose elements are equivalence classes. In other words, we merge all equivalent ω 's. The measure of a new element ˜ ω equals to the measure of the corresponding equivalence class.

Organization. In Section 2.1, we discuss the connection between explainable k -medians and set elimination games. We define a set elimination game in a set system I ⊂ { S 1 , . . . , S k } in Section 2.2. Then, we define the hitting and elimination time in Section 2.3. In Section 3, we first illustrate our proof strategy by showing Theorem 2.1 for the case when the smallest set S 1 does not overlap with S 2 , . . . , S k . An important ingredient of our proof is the notion of surprise sets , which we discuss in Section 3.1. Finally, we complete the proof of Theorem 2.1 in Section 3.2.

2.1 Explainable k -Medians via Set Elimination Game

In this section, we show how to use Theorem 2.1 to obtain a bound of 2 ln k +2 on the competitive ratio of the RANDOMCOORDINATECUT algorithm.

Theorem 2.2. The competitive ratio of the RANDOMCOORDINATECUT algorithm for Explainable k -Medians is at most 2 ln k +2 . That is, for every set of centers C = { c 1 , . . . , c k } and data set X , the algorithm finds a random decision tree T such that

The pseudo-code for the RANDOMCOORDINATECUT algorithm is provided in Figure 2.

Theorem 2.2 shows that given any k centers C = { c 1 , . . . , c k } , RANDOMCOORDINATECUT finds a decision tree T with cost at most 2 ln k +2 times the cost of unconstrained k -medians with centers C = { c 1 , . . . , c k } . By using k centers given by any constant approximation algorithm for k -medians, RANDOMCOORDINATECUT finds a decision tree with cost at most O (log k ) times the optimal unconstrained k -medians cost. This implies an O (log k ) upper bound on the price of explainability.

Proof of Theorem 2.2. Consider an arbitrary data set X ⊂ R d and set of k centers C ⊂ R d . We assume that all points in X and all centers in C are in the cube [ -M,M ] d . The threshold decision tree obtained by the RANDOMCOORDINATECUT algorithm partitions the space into k cells. Each cell contains a single reference center c i . The center c i is not necessarily optimal for cluster P i (cluster P i is the intersection of the data set X and i -th cell). However, we will use it as a proxy for the optimal center. In other words, we will upper bound the cost of the threshold decision tree as follows:

Let Ω be the set of all coordinate cuts: Ω = { ( j, θ ) : j ∈ [ d ] , θ ∈ [ -M,M ] } . We define a measure µ on Ω as follows. For every subset S ⊂ Ω , we set

where µ L is the Lebesgue measure on R . Thus, we have µ (Ω) = 2 dM , which implies (Ω , µ ) is a finite measure space.

Consider any data point x ∈ X . Define k sets S 1 , S 2 , . . . , S k for the set elimination game. For every i ∈ { 1 , . . . , k } , let S i be the set of all threshold cuts that separate x and center c i , i.e.,

̸

Note that the ℓ 1 distance from x to center c i equals the measure of S i : ∥ x -c i ∥ 1 = µ ( S i ) . We now examine the set elimination game with sets S 1 , . . . , S k , measure space (Ω , µ ) , and random sequence of draws ω 1 , ω 2 , . . . (each ω n ∈ Ω is the threshold cut chosen by the RANDOMCOORDINATECUT algorithm at step n ). We claim that S i belongs to R n if and only if center c i lies in the same cell as point x after step n of the algorithm. This is the case for n = 0 , since R 0 contains all sets S 1 , . . . , S k and the root of the threshold tree contains all centers c 1 , . . . , c k . Then, whenever we pick cut ω n , all centers separated from x by ω n are removed from the cell of x . The only exception from this rule occurs when all centers in that cell lie on the same side of the cut ω n . That is exactly the same rule as we have for the set elimination game (note that center c i is separated from x by ω n if and only if ω n ∈ S i ). Therefore, the same sets S i remain in the game as center c i in the cell of x (namely, sets S i and centers c i have the same indices).

The RANDOMCOORDINATECUT algorithm stops when all leaves of the decision tree contain exactly one center. At this step, the set elimination game contains one set, S i . This set corresponds to the center c i assigned to point x . The cost of the game µ ( S i ) equals the distance from x to c i . By Theorem 2.1, we have

We sum this bound over all data points x in X

and get the desired result.

2.2 Local Competitions

We now revisit the definition of the set elimination game and define competitions in subsets of { S 1 , . . . , S k } . For the rest of the proof, we assume (Ω , µ ) is a discrete finite measure space. We remind the reader that every set elimination game is determined by an infinite sequence of i.i.d. random variables ω 1 , ω 2 , . . . . In each round n , we sample an element ω n from Ω with probability Pr( ω n = ω ) = µ ( ω ) /µ (Ω) .

Definition 2.3. Consider a finite measure space (Ω , µ ) . Let I be a set of subsets of Ω . We say that I is a valid set system if (a) for every S ∈ I , µ ( S ) > 0 , and (b) for every S ′ , S ′′ ∈ I , µ ( S ′ △ S ′′ ) > 0 .

The reader may assume that µ ( ω ) > 0 for all ω in Ω . Then, the definition above says that in a valid set system I , all sets are non-empty and distinct.

Definition 2.4. Consider a finite measure space (Ω , µ ) . Let ω 1 , ω 2 , . . . be i.i.d. random variables as described above and I be a valid set system. We define a set elimination game in I . Initially, R 0 ( I ) = I . Then, for every n ≥ 1 ,

The winner of the game in I , denoted by winner( I ) , is the only element remaining, or, formally, the unique element in ∩ n ≥ 0 R n ( I ) . If ∩ n ≥ 0 R n ( I ) contains more than one element, then the winner is not defined. The cost of the game is the measure of the winner, µ (winner( I )) .

We remark that ∩ n ≥ 0 R n ( I ) contains exactly one element with probability 1 . Thus, the winner and cost of the game are defined with probability 1 .

Consider sets S 1 , . . . , S k from Theorem 2.1. Denote K = { S 1 , . . . , S k } . The definition of the competition among sets S 1 , . . . , S k (given in the beginning of Section 2) is exactly the same as the definition of competition in K . Our goal is to show that E [ µ (winner( K ))] ≤ 2(ln k + 1) · min S i ∈ K µ ( S i ) . In the proof of Theorem 2.1, we will consider competitions in different set systems I ⊆ K . We show the following key lemma. We defer the proof of Lemma 2.5 to Appendix A.

Lemma 2.5. Consider a partitioning of the set system K = { S 1 , . . . , S k } into m sets I 1 , . . . , I m . Then, winner( K ) ∈ { winner( I 1 ) , . . . , winner( I m ) } .

2.3 Set Elimination with Exponential Clock

Consider a set elimination game on sets S 1 , . . . , S k . It is determined by the sequence of random i.i.d. draws ω 1 , ω 2 , . . . . Random variable ω n is chosen in round n . We assign every round a random time τ n . Let the time between two consecutive rounds be an exponential random variable with parameter µ (Ω) . Specifically, let ∆ τ 1 , ∆ τ 2 , . . . be a sequence of i.i.d. exponential random variables with parameter µ (Ω) and each τ n = τ n -1 +∆ τ n = ∆ τ 1 + · · · +∆ τ n . Note that all ∆ τ n are positive and τ 1 , τ 2 , . . . is an increasing sequence with probability 1 . The number of draws that occur by time t (i.e., N t (Ω) = |{ n : τ n ≤ t }| ) is a Poisson process with parameter µ (Ω) . We now can think of the set elimination game as follows: The host of the game observes a Poisson process with parameter µ (Ω) . Whenever the process jumps (at time τ n ), the host picks an element ω n in Ω with probability Pr( ω n = ω ) = µ ( ω ) /µ (Ω) and eliminates some sets according to the rules of the game discussed above. Note that by assigning every round some time τ n , we do not change the game, the winner, and the cost of the game (because the sequence of random draws ω 1 , ω 2 , . . . remains the same as before). This interpretation of the game allows us to introduce a hitting time h ( S ) of every subset S ⊂ Ω with the following properties: (a) each h ( S ) is an exponential random variable with rate µ ( S ) ; (b) hitting times of disjoint sets are mutually independent random variables.

Definition 2.6. For every subset X ⊂ Ω , the hitting time h ( X ) is the time τ n when the first ω n is drawn from X : h ( X ) = min { τ n : ω n ∈ X } . When the set contains one element ω , we will write h ( ω ) instead of h ( { ω } ) .

We also define the elimination time of each set S i .

Definition 2.7. Consider any set elimination game with the measure space (Ω , µ ) and k sets S 1 , S 2 , . . . , S k in Ω . The elimination time e ( S i ) of set S i is the time when set S i is eliminated from the game, i.e., e ( S i ) = min { τ n : S i / ∈ R n ( K ) } . If S i is the winner, then we let e ( S i ) = ∞ (because the winner is never eliminated).

Note that e ( S i ) ≥ h ( S i ) . Sometimes, e ( S i ) may be equal to h ( S i ) , but e ( S i ) and h ( S i ) are not always the same. We now prove that hitting times for disjoint sets are independent. To this end, we split the Poisson process N t (Ω) = |{ n : τ n ≤ t }| . Let N t ( ω ) = |{ n : τ n ≤ t and ω n = ω }| . It is easy to see that N t (Ω) = ∑ ω ∈ Ω N t ( ω ) for every t . It is also true that each N t ( ω ) is a Poisson process with parameter µ ( ω ) and all N t ( ω ) (for ω ∈ Ω ) are mutually independent. This fact follows from the Coloring Theorem (see e.g., Kingman [1992], Coloring Theorem, page 53).

Theorem 2.8 (Coloring Theorem) . Let Π t be a Poisson process on the real line with rate λ . We color each event of the Poisson process randomly with one of M colors: The probability that a point receives the i -th color is p i . The colors of different points are independent. Let Π t ( i ) be the number of events of color i in the interval (0 , t ] . Then, Π t (1) , . . . , Π t ( M ) are independent Poisson processes. The rate of process Π t ( i ) is λp i .

Lemma 2.9. For every ω ∈ Ω , h ( ω ) is an exponential random variable with parameter µ ( ω ) , and all random variables h ( ω ) (for ω ∈ Ω ) are mutually independent.

Proof. Observe that h ( ω ) = min { t : N t ( ω ) ≥ 1 } . Thus, h ( ω ) is an exponential random variable (the time of the first jump of a Poisson process) with rate µ ( ω ) . Also, since all N t ( ω ) (for ω ∈ Ω ) are mutually independent, all h ( ω ) are also mutually independent.

Note that the set elimination game depends only on the hitting times for elements ω in Ω . This is the case because it matters only when every ω is drawn the first time. At that time - the hitting time of ω - all sets that contain ω are eliminated unless all remaining sets contain this ω . When the same ω is drawn again, it does not eliminate any new sets. Also, note that for any set S ⊂ Ω , the hitting time h ( S ) = min ω ∈ S h ( ω ) . Thus, h ( S ) is an exponential random variable with parameter µ ( S ) = ∑ ω ∈ S µ ( ω ) .

3 Proof of Main Result

We now present the proof of our main result, Theorem 2.1. We assume without loss of generality that S 1 is the smallest set i.e., µ ( S 1 ) ≤ µ ( S i ) for all i . Then, the expected cost of the game is at most:

We first provide some intuition for the proof by considering the case when S 1 does not intersect with sets S 2 , . . . , S k , i.e. sets S 1 and S i are disjoint for all i = 2 , 3 , . . . , k . We split all sets into two groups S 1 and the rest of the sets S 2 , . . . , S k . We know from Lemma 2.5 that the winner among all sets S 1 , . . . , S k is either S 1 or winner ( { S 2 , . . . , S k } ) . Denote I -= { S 2 , . . . , S k } . Each set S i is eliminated at time e ( S i ) . The set S 1 is eliminated at its hitting time h ( S 1 ) unless it is the only remaining set at time h ( S 1 ) (because we are considering the case when S 1 does not overlap with other sets). Thus,

When the winner among S 1 , . . . , S k is not S 1 , we consider two cases of the winner S i : (1) S i is a surprise set; (2) S i is a non-surprise set.

Definition 3.1. We say that S i is a surprise set if e ( S i ) ≥ h ( S 1 ) ≥ L/µ ( S i ) , where L = ln k .

We call S i a surprise set because the probability of the event e ( S i ) ≥ h ( S 1 ) ≥ L/µ ( S i ) is small. We give a bound on the probability of e ( S i ) ≥ h ( S 1 ) ≥ L/µ ( S i ) in Lemma 3.3. Here, we provide some intuition. By Lemma 2.9, the hitting time h ( S i ) is an exponential random variable with parameter µ ( S i ) . Thus, the expected hitting time for S i is 1 /µ ( S i ) . Consider a set S i with a small measure ( µ ( S i ) is close to µ ( S 1 ) ). If the hitting time h ( S 1 ) ≥ L/µ ( S i ) , then h ( S 1 ) is much larger than its expected value 1 /µ ( S 1 ) , which happens with a small probability. Consider a set S i with a large measure µ ( S i ) ≫ µ ( S 1 ) . Then, the expected hitting time for S i is 1 /µ ( S i ) , which is much smaller than the expected hitting time of S 1 . Thus, the event e ( S i ) ≥ h ( S 1 ) occurs with a small probability.

Let us examine bound (3). Let Surprise be the set of all surprise sets. Note that Surprise is a random set. Then,

We show in the next section (Lemma 3.3) that the second sum is upper bounded by µ ( S 1 ) . We now bound the first sum. For every winner S i which is not a surprise set, we have e ( S i ) ≥ h ( S 1 ) (because S i is the winner) and h ( S 1 ) ≤ L/µ ( S i ) (because S i is not a surprise set). We also have S i = winner( I -) , thus

By Lemma 2.9, all hitting times h ( S i ) = min ω ∈ S i h ( ω ) for i ≥ 2 are independent from h ( S 1 ) . Thus, winner( I -) is also independent of h ( S 1 ) ( winner( I -) depends only on the hitting times for sets S i ∈ I -). Therefore,

We combine all bounds on terms of (5) and get the following bound on the expected cost of the game:

This concludes the proof of the theorem for the case when S 1 does not overlap with S 2 , . . . , S k . We now analyze surprise sets.

3.1 Surprise Sets

In this section, we prove a bound on the probability that a set S i is a surprise set. We no longer assume that S 1 does not intersect with other sets S i . We first show a lemma about exponential random variables.

Lemma3.2. Let X and Y be two independent exponential random variables with positive parameters λ X and λ Y , respectively. Then, for every T ≥ 0 , we have

Proof. The desired probability can be easily found by computing ∫ ∞ T ( F X ( t ) -F X ( T )) f Y ( t ) dt , where F X ( t ) = 1 -e -λ X t is the cumulative distribution function of X , and f Y ( t ) = λ Y · e -λ Y t is the probability density function of Y . Here, we give an alternative proof. Write,

We have Pr ( min( X,Y ) ≥ T ) = e -( λ X + λ Y ) T , because the minimum of two independent exponential random variables with parameters λ X and λ Y is an exponential random variable with parameter λ X + λ Y . Then, Pr ( X ≤ Y | min( X,Y ) ≥ T ) = Pr ( X ≤ Y ) because the exponential distribution is memoryless; and Pr ( X ≤ Y ) = λ X / ( λ X + λ Y ) .

Lemma 3.3. For every set S i , we have

Proof. First, we show that min( e ( S i ) , h ( S 1 )) ≤ h ( S i \ S 1 ) .

Claim 3.4. We always have min( e ( S i ) , h ( S 1 )) ≤ h ( S i \ S 1 ) .

Proof. Consider an arbitrary realization of the game and the time t = h ( S i \ S 1 ) when S i \ S 1 is hit. If by this time, S 1 has already been hit then h ( S 1 ) < t . Similarly, if by this time, S i has already been eliminated then e ( S i ) < t . Otherwise, both S 1 and S i are still remaining in the game at time t . Therefore, when we pick ω ∈ S i \ S 1 at time t , set S i gets eliminated (since ω ∈ S i ; ω / ∈ S 1 ; both S 1 and S i are remaining in the game). Thus, in this case, e ( S i ) = t . This concludes the proof.

If S i is a surprise set, then min( e ( S i ) , h ( S 1 )) = h ( S 1 ) ≥ L/µ ( S i ) . By Claim 3.4, we have

Thus, Pr( S i is surprise set ) ≤ Pr ( h ( S i \ S 1 ) ≥ h ( S 1 ) ≥ L/µ ( S i ) ) . By Lemma 3.2 applied to the independent exponential random variables h ( S 1 ) , h ( S i \ S 1 ) , and time T = L/µ ( S i ) , we have

3.2 General Case

Proof of Theorem 2.1. We upper bound the expected cost of the game for arbitrary sets S 1 , . . . , S k . As before, we assume that S 1 is the smallest set. We remind the reader that each hitting time h ( S i ) is an exponential random variable with parameter µ ( S i ) . In the proof, we will use the definitions of surprise sets (see Definitions 3.1). We also set L = ln k . We define all sets S i for i = 1 that are not a surprise set to be non-surprise sets.

̸

We separately upper bound the cost of the winner depending on whether the winner is (a) set S 1 , (b) surprise set, (c) non-surprise set. Write

Term (a) is upper bounded by µ ( S 1 ) . We bound term (b) using Lemma 3.3: The probability that a set is a surprise set is at most 1 / k · µ ( S 1 ) /µ ( S i ) . Thus, the expected total measure of all sets (not only the surprise winner) is upper bounded by 1 k ∑ k i =2 µ ( S 1 ) µ ( S i ) µ ( S i ) < µ ( S 1 ) .

We now bound term (c). Define a new random variable: Let cost( ω ) be the cost of the winner (i.e., µ ( S i ) , where S i is the winner) if (1) the winner is a non-surprise set, and (2) ω is the first element that was chosen in S 1 . We let cost( ω ) = 0 , otherwise. If ω is the first element that was chosen in S 1 , then h ( S 1 ) = h ( ω ) . So, the definition of cost( ω ) can be written as follows:

Since the hitting time h ( S 1 ) is finite with probability 1 , the term (c) equals

Lemma 3.5, which we prove below, gives a bound of 2 Lµ ( S 1 ) on the expression above. Combining upper bounds on terms (a), (b), and (c), we get

Lemma 3.5. For every ω ∈ S 1 , we have E [cost( ω )] ≤ 2 Lµ ( ω ) .

Proof. We have

If S i is a non-surprise set, then h ( S 1 ) < L/µ ( S i ) or e ( S i ) < h ( S 1 ) . If S i is the winner, then e ( S i ) ≥ h ( S 1 ) . Thus, if S i is a non-surprise winner, then h ( S 1 ) < L/µ ( S i ) . This observations gives us the following upper bound on (7):

Define two set systems I -ω and I + ω of sets S i containing and not containing ω :

Note that K ≡ { S 1 , . . . , S k } = { S 1 } ∪ I -ω ∪ I + ω . By Lemma 2.5,

Observe that if S i with i ≥ 2 is the winner, then S i = winner( I -ω ) or S i = winner( I + ω ) . We replace the condition S i = winner( K ) with S i ∈ { winner( I -ω ) , winner( I + ω ) } in (8) and get bound:

The key observation now is that sets winner( I -ω ) and winner( I + ω ) are independent of h ( ω ) . This is the case, because sets remaining in the competitions R n ( I -ω ) and R n ( I + ω ) do not change when we select ω . The set R n ( I -ω ) does not change in the round n when ω is chosen because all sets S i in R n ( I -ω ) ⊂ I -ω do not contain ω . The set R n ( I + ω ) does not change in this round because all sets S i in R n ( I + ω ) ⊂ I + ω contain ω and consequently when ω is chosen, none of these sets is removed from R n ( I + ω ) (otherwise, R n ( I + ω ) would become empty). Thus,

Using that h ( ω ) is an exponential random variable with parameter µ ( ω ) , we get (for every i )

Hence,

The sum on the right hand side is at most 2 . Thus, E [cost( ω )] ≤ 2 Lµ ( ω )

.

Acknowledgments and Disclosure of Funding

The authors are supported by NSF Awards CCF-1955351, CCF-1934931, EECS-29 2216970.

References

Sayan Bandyapadhyay, Fedor Fomin, Petr A Golovach, William Lochet, Nidhi Purohit, and Kirill Simonov. How to find a good explanation for clustering? In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 3904-3912, 2022.

Jarosław Byrka, Thomas Pensyl, Bartosz Rybicki, Aravind Srinivasan, and Khoa Trinh. An improved approximation for k-median and positive correlation in budgeted optimization. ACM Transactions on Algorithms (TALG) , 13(2):1-31, 2017.

Moses Charikar and Lunjia Hu. Near-optimal explainable k-means for all dimensions. In Proceedings of the 2022 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA) , pages 2580-2606. SIAM, 2022.

Moses Charikar, Sudipto Guha, Éva Tardos, and David B Shmoys. A constant-factor approximation algorithm for the k-median problem. In Proceedings of the thirty-first annual ACM symposium on Theory of computing , pages 1-10, 1999.

Vincent Cohen-Addad and Euiwoong Lee. Johnson coverage hypothesis: Inapproximability of k-means and k-median in lp-metrics. In Proceedings of the 2022 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA) , pages 1493-1530. SIAM, 2022.

Vincent Cohen-Addad, Hossein Esfandiari, Vahab Mirrokni, and Shyam Narayanan. Improved approximations for euclidean k-means and k-median, via nested quasi-independent sets. In Proceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing , pages 16211628, 2022.

Sanjoy Dasgupta, Nave Frost, Michal Moshkovitz, and Cyrus Rashtchian. Explainable k-means and k-medians clustering. In Proceedings of the 37th International Conference on Machine Learning , pages 7055-7065, 2020.

Hossein Esfandiari, Vahab Mirrokni, and Shyam Narayanan. Almost tight approximation algorithms for explainable clustering. In Proceedings of the 2022 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA) , pages 2641-2663. SIAM, 2022.

Nave Frost, Michal Moshkovitz, and Cyrus Rashtchian. Exkmc: Expanding explainable k -means clustering. arXiv preprint arXiv:2006.02399 , 2020.

Buddhima Gamlath, Xinrui Jia, Adam Polak, and Ola Svensson. Nearly-tight and oblivious algorithms for explainable clustering. Advances in Neural Information Processing Systems , 34:28929-28939, 2021.

Anupam Gupta, Madhusudhan Reddy Pittu, Ola Svensson, and Rachel Yuan. The price of explainability for clustering. arXiv preprint arXiv:2304.09743 , 2023.

John Frank Charles Kingman. Poisson processes , volume 3. Clarendon Press, 1992.

Eduardo Laber, Lucas Murtinho, and Felipe Oliveira. Shallow decision trees for explainable k-means clustering. Pattern Recognition , 137:109239, 2023.

Eduardo S Laber and Lucas Murtinho. On the price of explainability for some clustering problems. In International Conference on Machine Learning , pages 5915-5925. PMLR, 2021.

Shi Li and Ola Svensson. Approximating k-median via pseudo-approximation. In proceedings of the forty-fifth annual ACM symposium on theory of computing , pages 901-910, 2013.

Konstantin Makarychev and Liren Shan. Near-optimal algorithms for explainable k-medians and k-means. In International Conference on Machine Learning , pages 7358-7367. PMLR, 2021.

Konstantin Makarychev and Liren Shan. Explainable k-means: don't be greedy, plant bigger trees! In Proceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing , pages 1629-1642, 2022.

Nimrod Megiddo and Kenneth J Supowit. On the complexity of some common geometric location problems. SIAM journal on computing , 13(1):182-196, 1984.

A Proof of Lemma 2.5

Lemma 2.5. Consider a partitioning of the set system K = { S 1 , . . . , S k } into m sets I 1 , . . . , I m . Then,

The proof of Lemma 2.5 relies on the following observarion.

Lemma A.1. Let X and Y be two subsets of K . If X ⊂ Y , then for every n , we always have

Proof. We prove that (9) holds by induction on n . Initially, when n = 0 , we have R 0 ( X ) = X and R 0 ( Y ) = Y . Therefore, R 0 ( Y ) ∩ X = X ∩ Y = X = R 0 ( X ) . Suppose (9) holds for n , we prove that (9) also holds for n ′ = n +1 . If R n ( Y ) ∩ X = ∅ , then R n ( Y ) ∩ X remains empty for all n ′ ≥ n . Therefore, (9) holds for n +1 . So, let us assume that R n ( Y ) ∩ X = R n ( X ) . Consider three cases:

If ω n +1 belongs to all sets in R n ( Y ) , then it also belongs to all sets in R n ( X ) = R n ( Y ) ∩ X . Thus, in this case, no set is eliminated in X or Y . That is, R n +1 ( X ) = R n ( X ) and R n +1 ( Y ) = R n ( Y ) .

If ω n +1 belongs to all sets in R n ( X ) , but not all sets in R n ( Y ) , then, at step n + 1 , we remove all sets that contain ω n +1 and, particularly, all sets in R n ( X ) , from R n ( Y ) . Consequently, R n +1 ( Y ) ∩ X = ∅ .

If not all sets in R n ( X ) and not all sets in R n ( Y ) contain ω n +1 , then we remove exactly the same sets from both R n ( X ) and R n ( Y ) ∩ X . Namely, we remove sets S i ∈ R n ( Y ) that contain ω n +1 .

We conclude that (9) holds for n ′ = n +1 .

Proof of Lemma 2.5. Consider an arbitrary realization of the game ω 1 , ω 2 , . . . . Let n be the round when all sets but the winner are eliminated from the competition i.e., R n contains only one set, the winner. Since K is the union of I 1 , . . . , I k , the winner must belong to some I j . Now, by Lemma A.1 for X = I j and Y = K , we have R n ( K ) ∩ I j = R n ( I j ) or R n ( K ) ∩ I j = ∅ . We know that R n ( K ) = { winner( K ) } and winner( K ) ∈ I j . Thus, R n ( K ) ∩ I j = { winner( K ) } ̸ = ∅ , and

We conclude that at round n , R n ( I j ) contains only one set - the winner in K . Consequently, it is also the winner in I j i.e., winner( I j ) = winner( K ) . This finishes the proof.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the optimal upper bound of explainable k-median and shows a tight analysis for the random coordinate cut algorithm. The explainable k-clustering model was first introduced by Dasgupta et al. [ICML’20] and was followed by a flurry of work to understand the optimal approximation factor to achieve explainability. For the k-median task, previous work has shown that an $\\Omega(\\log k)$ factor is necessary, and algorithms that follow a random coordinate cut scheme can achieve $O(\\log k \\cdot \\log\\log k)$-approximation. This paper shows that by picking appropriate distributions, the random coordinate cut can achieve the asymptotically tight approximation of $O(\\log k)$.\n\nAt the heart of the analysis of the paper is a newly-defined *set elimination game* (SEG), where elements are split into multiple (potentially overlapping) sets, and the referee takes an element $e$ at random and eliminates all the sets that contain $e$ unless every remaining set contain $e$. The final value of the game is defined as the measure of the final surviving set. The paper establishes the upper bound for explainable k-median clustering by: 1). showing a ‘’reduction’’ from the cost of explainable k-median of a single point $x$ to the value of the SEG, 2). proving the expected value of the game as $O(\\log k)\\cdot \\mu(S^*)$, where $S^*$ is the set with the smallest measure.\n\nIn my opinion, the paper is generally well-written, and it settles an important question in explainable k-median clustering. The proof idea is novel, and it is presented in a clean manner (minus some minor exposition issues, see the weakness for details). Therefore, I believe the merits of the paper are sufficient for acceptance.\n\nSTRENGTHS:\nExplainable clustering has been a popular topic in the recent few years, and the paper settles an open problem in the area. The idea of set elimination games is novel and interesting, and the paper is well-written. The organization helps the reader a lot in understanding the paper.\n\nWEAKNESSES:\nI do not see any major technical weaknesses in the paper. On the exposition part, I have some minor comments:\n- The paper is written in relatively advanced technical languages, and they might not be readily accessible to the broader NeurIPS audiences. To make the readers’ task easier, I think it makes sense to add some more intuitions on why using the set elimination game, why the value should be small, and the high-level intuitions for the analysis. \n- Conversely, the paper spent little effort discussing why explainability in k-clustering is important and why shaving off the exponentially-small $\\log\\log k$ factor is significant. I would recommend adding some discussion on this front for later versions.\n- Some notation problems. In the proof of Theorem 2.2, there is a slight overload of notation in the construction of $\\Omega$, and $S$: $S$ was used as a *variable* for the sets in the definition of the SEG, but in the proof it becomes the *realization*, i.e., fixed sets. I would recommend using a different notation instead. Also, $x_j$ (which I guess means the j-th coordinate of the point $x$) is used before it is defined.\n\nQUESTIONS:\n- You never mentioned the initialization of the reference sets, but judging from the proof of Theorem 2.2, you’re assuming the optimal k-median centers as the reference centers, right? Something like this should be mentioned and discussed.\n- You never mentioned the *distribution* of the random coordinate cut in the paper; from the requirement of Theorem 2.1, it seems uniform at random works. Is this right? You should explicitly define the distribution, either way (‘’pick at random’’ $\\neq$ ‘’pick uniformly at random’’ – the former only means picking something from *a* distribution).\n- What is the intuition for the 'surprising sets'? I can read that the name 'surprising' may come from the fact that for $h(S_1) \\cdot \\mu(S_i) \\geq \\log k$, either $S_1$ should be hit very late or $S_i$ should have a high volume, which means it should have been hit/eliminated way earlier. If this is true, please also add a discussion about it, as it'll help readers understand your proof.\n- Do you see any other applications for the Set Elimination Game?\n\nLIMITATIONS:\nThe work is of theoretical nature, so the (non-technical) limitations are on the model side, e.g., the explainable clustering model might not provide easy-to-understand explanations in real life. However, I do not think this affects the significance of the contribution.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-02"}

comments: SUMMARY:
The paper provides a tight analysis of the random-coordinate-cut (RCC shortly) algorithm for explainable k-medians in the $ \ell_1 $ metric. To be precise, the RCC algorithm was independently proposed  by several works, and for the explainable k-medians in $ \ell_1 $ , the tightest proved competitive ratio is $ O(\log k \log\log k) $, which has a small gap between its lower bound $ \Omega(\log k) $. This paper shows that the competitive ratio is no larger than $ 2\ln k+2 $, thereby concluding that the RCC algorithm is optimal for explainable k-medians.

STRENGTHS:
- This paper completely answers the problem of whether there exists an algorithm with a competitive ratio that matches the lower bound $ \Omega(\log k) $ for the explainable k-medians problem.  Its contribution is significant and solid. The proofs are simple and elegant.
- The idea behind the analysis method set elimination game is ingenious. Actually, it effectively summarizes a series of prior works in a concise and abstract manner.

WEAKNESSES:
i don’t find obvious weaknesses

QUESTIONS:
- I have no questions but the independent work of this paper. I am interested in learning more about the comparison between these two works, particularly with regard to the differences in the techniques and high-level ideas underlying them.

LIMITATIONS:
None
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-03

comments: SUMMARY:
The paper considers the problem of explainable $k$-medians which was recently introduced by Dasgupta et al, and analyzes the RandomCoordinateCut algorithm. The paper proves an upper bound of $2\ln k + 2$ on the competitive ratio of RandomCoordinateCut. This matches a previously known lower bound of $\Omega(\log k)$, and hence the paper proves that RandomCoordinateCut for explainable $k$-medians is optimal up to constant factors.

Edit: I've read the authors' rebuttals. As I mentioned in my first review, I believe that this is a strong accept paper.

STRENGTHS:
The paper proves optimality of RandomCoordinateCut for $k$-medians with a remarkably simple and elegant proof. Furthermore, the paper is generally very well written and easy to follow.

WEAKNESSES:
I can't see any significant weakness in the paper.

Here are few minor comments/typos:
- Page 4, line 116: The equation $Pr(\omega_n = \omega) = \mu(\omega)/\mu(\Omega)$  is not strictly formal for spaces that are not discrete.
- Page 5: In the displayed equation right after line 156, the summation should be over $j$.
- Page 5, line 185: "all sets are non-empty and disjoint" -> "all sets are non-empty and distinct".
- Page 8: In the displayed equation after line 296, I think that in (c) we should also specify that the winner is not $S_1$, because technically $S_1$ does not satisfy the definition of a surprise set.

QUESTIONS:
- Can the authors comment about the proof of Gupta et al. (mentioned on page 4, line 105)? Is it very different from the approach that is presented in the current paper?

LIMITATIONS:
No concerns regarding potential societal impact of this work.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers the Explainable k-Medians problem, which is a more interpretable model for k-clustering than traditional models such as k-median. An explainable clustering partitions space based on coordinate threshold cuts, so that instead of a Voronoi diagram as in k-median / k-means, we end up with rectangular partitions. More specifically, the partitions are determined by a binary decision tree where each node contains a coordinate and a threshold, and the left child corresponds to points that fall below the threshold in that coordinate, and the right child corresponds to points that fall above the threshold in that coordinate. An algorithm is f(k)-competitive for explainable k-median if it produces an explainable clustering that has cost at most f(k) times the cost of the optimal (unconstrainted) k-median clustering. This paper shows that Random Coordinate Cut algorithm (studied by several authors before this paper) has the optimal competitive ratio: specifically, they prove it is (2 \\log k + 2)-competitive--improving on the previously best known bound of O(log log k)--and matching a previously known \\Omega(log k) lower bound. \n\nThe technique is to reduce the problem to a game which the authors call the Set Elimination Game, which is syntactically similar to the Random Coordinate Cut algorithm (and indeed, the authors note that other authors have implicitly analyzed this game). In it, there is some ground set \\Omega and an associated probability measure \\mu, along with a set of subsets. At each round, an element of \\Omega is chosen according to \\mu, and any remaining subset that contains this element is eliminated (unless all remaining subsets contain this element). The last remaining subset is the winner, and the cost of the game is \\mu(winner). The main result, which directly translates to the competitive ratio bound for explainable k-median, is that the expected cost of the game is at most (2 log k + 2) times the cost of any subset. \n\nThe idea behind the analysis is to associate each round with an arrival in a Poisson process, so that they may then view the times at which elements first are chosen in the game as hitting times with exponential distributions.  The hitting times have the nice property that they are independent for different elements. Notably, hitting times single-handedly determine the behavior of the game, so the analysis of the game can be reduced to the analysis of hitting times.\n\nSTRENGTHS:\n- This paper resolves the question of whether the Random Coordinate Cut algorithm is optimal via a tight analysis. \n\n- The analysis of the Set Elimination Game using Poisson processes is creative and novel (based on my understanding from the related work section, other papers implicitly analyze this game, but the method here is new). It is also quite clean. \n\n- The formulation of the Set Elimination Game and the main result on its expected cost (Theorem 2.1) may be of independent interest.\n\nWEAKNESSES:\nThe result is slightly weaker (by a factor of 2) than a concurrent result of Gupta, Pittu, Svensson, and Yuan (2023), which gives a bound of 1+H_{k-1}.\n\nQUESTIONS:\nCan any of the methods in this paper potentially be adapted to explainable k-means?\n\nLIMITATIONS:\nNot applicable.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-07"}