SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Random Cuts are Optimal for Explainable  $k$ -Medians
Konstantin Makarychev*
Liren Shan*
We show that the RANDOMCOORDINATECUT algorithm gives the optimal competitive ratio for explainable  $k$ -medians in  $\ell_1$ . The problem of explainable  $k$ -medians was introduced by Dasgupta, Frost, Moshkovitz, and Rashtchian in 2020. Several groups of authors independently proposed a simple polynomial-time randomized algorithm for the problem and showed that this algorithm is  $O(\log k\log \log k)$  competitive. We provide a tight analysis of the algorithm and prove that its competitive ratio is upper bounded by  $2\ln k + 2$ . This bound matches the  $\Omega (\log k)$  lower bound by Dasgupta et al (2020).
Machine learning is being increasingly used to make decisions for critical applications, such as healthcare, finance, and public policy. Considering the profound impact of algorithmic decisions on individuals and society, it is essential to understand the underlying logic behind these decisions. In this paper, we explore an explainable  $k$ -medians clustering algorithm (called RANDOMCOORDINATE-CUT). The algorithm's aim is to cluster data sets and present results in a manner easily understood and visualized by humans.
Clustering is a fundamental task in unsupervised learning. Among many clustering methods,  $k$ -means,  $k$ -medians, and  $k$ -medoids are particularly popular. These are centroid-based methods that choose  $k$  centers and assign each data point to the center nearest to it. As a result, each cluster is a Voronoi cell in the Voronoi partition of the space. Since these cells may have a complicated boundary (see Figure 1 for an example of  $k$ -medians), it is not always easy for humans to comprehend and visualize such clustering.
To address this problem, Dasgupta, Frost, Moshkovitz, and Rashtchian [2020] introduced explainable  $k$ -means and  $k$ -medians clustering. They argued that decision trees are easy to understand and interpret. Therefore, in order to make clustering more explainable, we need to use threshold decision trees to define clusters. A threshold decision tree is a binary space partitioning tree with  $k$  leaves. Each internal node of the threshold decision tree splits the data into two groups using a threshold cut  $(j,\theta)$ : on the one side of the cut, we have points  $x$  with  $x_{j} \leq \theta$  and, on the other side, points  $x$  with  $x_{j} > \theta$ . Thus, every node of the tree corresponds to a rectangular region of the space. A decision tree with  $k$  leaves partitions data set  $X$  into  $k$  clusters,  $P_{1},\ldots,P_{k}$ . See Figure 1 for an example. Dasgupta et al. [2020] suggested that we use the standard  $k$ -medians and  $k$ -means objectives to measure the cost of the threshold decision tree. For  $k$ -medians, the cost of a threshold decision tree  $\mathcal{T}$  equals
$$
\operatorname {c o s t} (X, \mathcal {T}) = \sum_ {i = 1} ^ {k} \sum_ {x \in P _ {i}} \| x - \hat {c} ^ {i} \| _ {1},
$$
where  $P_{1},\ldots ,P_{k}$  is the partitioning of  $X$  produced by  $\mathcal{T}$ ; and  $\hat{c}^1,\dots ,\hat{c}^k$  are the medians of clusters  $P_{1},\ldots ,P_{k}$ . We denote the  $\ell_1$ -norm by  $\| \cdot \| _1$ . Note that each  $P_{i}$  is a rectangular region of the space. Thus, generally speaking, every  $x$  is not assigned to the closest center  $\hat{c}^1,\dots ,\hat{c}^k$  like in unconstrained  $k$ -medians or  $k$ -means.
Figure 1: The unconstrained  $k$ -medians clustering and explainable  $k$ -medians clustering. The left diagram shows the Voronoi partition of the plane w.r.t. three centers in  $\ell_1$  distance. The Voronoi cell for each center consists of all points that are closer (in  $\ell_1$  distance) to this center than to any other center (the boundaries between cells are not straight lines because we use the  $\ell_1$  distance). The middle diagram shows an explainable partition. The right diagram shows the corresponding decision tree for explainable clustering.
Dasgupta, Frost, Moshkovitz, and Rashtchian [2020] defined the price of explainability as the ratio of the  $k$ -medians cost of explainable clustering to the optimal cost of unconstrained  $k$ -medians clustering. They showed that the cost of explainability for  $k$ -means and  $k$ -medians (somewhat surprisingly) does not depend on the number of points in the data set  $X$  and only depends on  $k$ . Specifically, they provided a greedy algorithm that given  $k$  reference centers  $c^1, c^2, \dots, c^k$  of any unconstrained  $k$ -medians as input, outputs a threshold decision tree of cost at most  $O(k)$  times the cost of original unconstrained  $k$ -medians with centers  $c^1, c^2, \dots, c^k$ . We call such an algorithm  $O(k)$  competitive. To get an explainable  $k$ -medians clustering, we first obtain reference centers  $c^1, c^2, \dots, c^k$  using an off-the-shelf approximation algorithm for  $k$ -medians and then run an  $\alpha$ -competitive algorithm for explainable  $k$ -medians with centers  $c^1, c^2, \dots, c^k$  given as input. This algorithm produces the desired threshold decision tree. Dasgupta et al. [2020] also gave an  $O(k^2)$  competitive algorithm for  $k$ -means and showed  $\Omega(\log k)$  lower bounds on the price of explainability for both  $k$ -medians and  $k$ -means.
The notion of explainable clustering immediately got a lot of attention in the field (Laber and Murtinho [2021], Makarychev and Shan [2021], Gamlath et al. [2021], Charikar and Hu [2022], Esfandiari et al. [2022]). Particularly, Makarychev and Shan [2021], Esfandiari, Mirrokni, and Narayanan [2022] provided almost optimal algorithms for explainable  $k$ -medians, and Makarychev and Shan [2021], Esfandiari, Mirrokni, and Narayanan [2022], Gamlath, Jia, Polak, and Svensson [2021] provided almost optimal algorithms for  $k$ -means. The competitive ratios of these algorithms are  $\tilde{O} (\log k)$  for  $k$ -medians and  $\tilde{O}(k)$  for  $k$ -means.
The algorithms for explainable  $k$ -medians by Makarychev and Shan [2021], Esfandiari, Mirrokni, and Narayanan [2022], Gamlath, Jia, Polak, and Svensson [2021] are variants of the same simple algorithm, which we call RANDOMCOORDINATECUT. This algorithm receives a set of  $k$  reference centers  $c^1, \ldots, c^k$  as input and then builds a threshold decision tree with  $k$  leaves. It works as follows. It recursively partitions  $d$ -dimensional space until every cell contains exactly one reference center  $c^i$ . The algorithm starts with a tree consisting of one node, the root. Initially, all  $k$  reference centers are assigned to that root. At every step, the algorithm picks a random threshold cut  $(j, \theta)$  and splits centers in every cell using this cut. If this cut does not separate any centers in a cell  $u$  (i.e., all centers in  $u$  are located on one side of the cut), then the algorithm does not split  $u$  into two regions at this step. Finally, for every leaf  $u$  of the constructed tree, the unique center that belongs to the cell corresponding to  $u$  is assigned to  $u$ . We provide pseudo-code for this algorithm in Figure 2.
Makarychev and Shan [2021], Esfandiari et al. [2022] showed that the competitive ratio of RANDOMCOORDINATECUT is at most  $O(\log k\log \log k)$ . That is, for every data set  $X$  and set of centers  $c^1,\ldots ,c^k$ ,
$$
\mathbf {E} [ \operatorname {c o s t} (X, \mathcal {T}) ] \leq O (\log k \log \log k) \cdot \sum_ {x \in X} \min  _ {c \in \{c ^ {1}, \dots , c ^ {k} \}} \| x - c \| _ {1}.
$$
Note that the running time of this algorithm is  $\tilde{O}(kd)$ . Gamlath, Jia, Polak, and Svensson [2021] provided a slightly worse bound of  $O(\log^2 k)$  on the competitive ratio of this algorithm. They also
Input: a data set  $X \subset \mathbb{R}^d$  and set of centers  $C = \{c^1, c^2, \ldots, c^k\} \subset \mathbb{R}^d$
Output: a threshold tree  $\mathcal{T}$
Create tree  $\mathcal{T}_0$  containing a root node  $r$ . Assign  $C_r = \{c^1, c^2, \dots, c^k\}$  to the root. Let  $t = 0$ . Let  $M = \max_{ij} |c_j^i|$ .
while  $\mathcal{T}_n$  contains a leaf with at least two distinct centers do
Pick a coordinate  $j$  and threshold  $\theta \in (-M, M)$  uniformly at random. Let  $\omega_{n} = (j, \theta)$ .
For every leaf node  $u$  in  $\mathcal{T}_n$ , split the set  $C_u$  into two sets:
$$
\text {L e f t} = \{c \in C _ {u}: c _ {j} \leq \theta \} \quad \text {a n d} \quad \text {R i g h t} = \{c \in C _ {u}: c _ {j} > \theta \}.
$$
If both sets are not empty, then create two children of  $u$  in tree  $\mathcal{T}_t$ . The left child corresponds to the subregion of  $u$  with  $x_j \leq \theta$ , and the right child corresponds to the subregion of  $u$  with  $x_j > \theta$ . Assign sets Left and Right to the left and right child, respectively.
Denote the updated tree by  $\mathcal{T}_{t + 1}$
Update  $t = t + 1$ .
Figure 2: RANDOMCOORDINATECUT algorithm
conjectured that this algorithm is optimal and its competitive ratio is  $O(\log k)$ , more specifically,  $H_{k-1} + 1$ , where  $H_k$  is the  $k$ -th harmonic number. They provided some justification for their conjecture by proving this bound for a very special set of centers and data points (corresponding to the case of completely disjoint sets in our Set Elimination Game).
Our Results. In this work, we show that indeed the competitive ratio of RANDOMCOORDINATECUT is at most  $2\ln k + 2$ , and, therefore, this algorithm has the optimal competitive ratio which matches the lower bound of Dasgupta, Frost, Moshkovitz, and Rashtchian [2020]. Our analysis is not only tight but also fairly simple. To get our result we define a game, the Set Elimination Game, which was also implicitly analyzed in previous works on this topic. We show that the cost of this game is at most  $2\ln k + 2$ .
Related Work. The unconstrained  $k$ -medians clustering has been extensively studied. Charikar, Guha, Tardos, and Shmoys [1999] gave the first constant factor approximation algorithm for the problem in general metric spaces. Li and Svensson [2013] provided a  $1 + \sqrt{3} + \varepsilon$  approximation algorithm. Byrka, Pensyl, Rybicki, Srinivasan, and Trinh [2017] improved the approximation factor to  $2.675 + \varepsilon$ . Cohen-Addad, Esfandiari, Mirrokni, and Narayanan [2022] recently improved the approximation factor to 2.406 for Euclidean  $k$ -medians. Megiddo and Supowit [1984] showed that the  $k$ -medians in  $\ell_1$  problem is NP-hard. Cohen-Addad and Lee [2022] showed that it is also NP-hard to approximate  $k$ -medians in  $\ell_1$  within a factor of 1.06.
As we discuss above, Gamlath, Jia, Polak, and Svensson [2021], Esfandiari, Mirrokni, and Narayanan [2022], Makarychev and Shan [2021], independently proposed the RANDOMCOORDINATECUT algorithm. They also gave an  $\tilde{O}(k)$  algorithm for explainable  $k$ -means and showed a lower bound of  $\tilde{\Omega}(k)$  for the problem. Charikar and Hu [2022] provided an  $O(k^{1 - 2/d} \cdot \mathrm{poly}(d, \log k))$  competitive algorithm for explainable  $k$ -means, whose competitive ratio depends on the dimension  $d$  of the instance. For small  $d \ll \log k / \log \log k$ , their bound is better than  $O(k)$ . They showed an almost matching  $\Omega(k^{1 - 2/d} / \mathrm{ploy} \log k)$  lower bound for explainable  $k$ -means. Esfandiari et al. [2022] gave an upper bound of  $O(d \log^2 d)$  on the competitive ratio of RANDOMCOORDINATECUT for explainable  $k$ -medians. This bound is better than  $O(\log k)$  for small  $d \ll \log k / \log \log k$ . Laber and Murtinho [2021] gave  $O(d \log k)$  and  $O(dk \log k)$  competitive algorithms for explainable  $k$ -medians and  $k$ -means, respectively. Frost, Moshkovitz, and Rashtchian [2020] provided some empirical evidence that bi-criteria algorithms for explainable  $k$ -means (that partition the data set into  $(1 + \delta)k$  clusters) can give a much better competitive ratio than  $O(k)$ . Then, Makarychev and Shan [2022] gave a  $\tilde{O}\left(\frac{1}{\delta} \log^2 k\right)$  competitive bi-criteria algorithm for explainable  $k$ -means. Bandyapadhyay, Fomin, Golovach, Lochet, Purohit, and Simonov [2022] provided an algorithm
that computes the optimal explainable  $k$ -medians and  $k$ -means clustering in time  $n^{2d + O(1)}$  and  $(4nd)^{k + O(1)}$ , respectively. Laber, Murtinho, and Oliveira [2023] proposed to use shallow decision trees for explainable clustering.
Independently and concurrently with our work, Gupta, Pittu, Svensson, and Yuan [2023] proved a  $O(\log k)$  bound on the price of explainability for  $k$ -medians. They showed that the competitive ratio of RANDOMCOORDINATECUT is  $1 + H_{k-1}$ , where  $H_k$  is the  $k$ -th harmonic number. Their work answers the open question raised by Gamlath, Jia, Polak, and Svensson [2021]. They also proved a hardness of approximation result for explainable  $k$ -medians clustering and improved the competitive ratio for explainable  $k$ -means from  $O(k \log k)$  to  $O(k \log \log k)$ .
2 Set Elimination Game
In this section, we define the set elimination game. Consider a discrete finite measure space  $(\Omega, \mu)$ . In this space, each element  $\omega \in \Omega$  has a measure of  $\mu(\omega)$ , and the measure of every set  $S \subseteq \Omega$  equals  $\mu(S) = \sum_{\omega \in S} \mu(\omega)$ . Let  $S_1, S_2, \ldots, S_k \subset \Omega$  be  $k$  distinct sets which may overlap with each other. The set elimination game proceeds in a series of rounds. Initially, all sets  $S_1, \ldots, S_k$  enter the competition. Formally, they belong to the set of remaining sets  $\mathcal{R}_0 = \{S_1, \ldots, S_k\}$ . At every round  $n$ , the host picks a random  $\omega_n \in \Omega$  with probability  $\operatorname*{Pr}(\omega_n = \omega) = \mu(\omega) / \mu(\Omega)$ . Then, all sets  $S_i$  that contain  $\omega_n$  are eliminated from the game unless all remaining sets contain  $\omega_n$ , in which case, no set gets eliminated. That is, for  $n \geq 1$ ,
$$
\mathcal {R} _ {n} = \left\{ \begin{array}{l l} \mathcal {R} _ {n - 1} \backslash \left\{S _ {i} \in \mathcal {R} _ {n - 1}: \omega_ {n} \in S _ {i} \right\}, & \text {i f f o r s o m e} S _ {i} \in \mathcal {R} _ {n - 1}, \omega_ {n} \notin S _ {i}; \\ \mathcal {R} _ {n - 1}, & \text {o t h e r w i s e .} \end{array} \right. \tag {1}
$$
The last remaining set is declared the winner. We denote that winner by winner. We say that the cost of the game is the measure of the winning set,  $\mu(\text{winner})$ .
We remark that  $\mathcal{R}_n$  cannot get empty (in which case, the winner would not be defined) because of the "otherwise" clause in the definition (1). We shall always assume that all sets  $S_1,\ldots ,S_k$  are not only distinct and non-empty but also (a) for every  $i$ ,  $\mu (S_i) > 0$ , and (b) for all  $i$  and  $j$ ,  $\mu (S_i\triangle S_j) > 0$  (here,  $S_{i}\triangle S_{j}$  denotes the symmetric difference of sets  $S_{i}$  and  $S_{j}$ ). Then, in every game, there is a unique winner with probability 1.
We similarly define the set elimination game for arbitrary finite measure spaces: For an arbitrary finite measure space  $(\Omega, \mu)$ , element  $\omega_{n}$  is chosen with probability function  $\operatorname{Pr}(\omega_n \in S) = \mu(S) / \mu(\Omega)$ .
Our main result is the following theorem, which, as we discuss later in Section 2.1, implies that the competitive ratio of the explainable clustering algorithm is  $2\ln k + 2$ .
Theorem 2.1. Consider a set elimination game with the finite measure space  $(\Omega, \mu)$  and  $k$  distinct sets  $S_1, S_2, \ldots, S_k$  (as above). The expected cost of the game is at most
$$
\mathbf {E} \left[ \mu (\text {w i n n e r}) \right] \leq (2 \ln k + 2) \cdot \min  _ {i \in [ k ]} \mu \left(S _ {i}\right).
$$
To simplify the exposition, we will prove this theorem for discrete finite measure sets. If  $\Omega$  is not a discrete measure space, we first replace it with a quotient space: We say that  $\omega' \in \Omega$  and  $\omega'' \in \Omega$  are equivalent ( $\omega' \sim \omega''$ ) if they are contained in exactly the same set of sets  $S_1, \ldots, S_k$ . This equivalence relation partitions  $\Omega$  into at most  $2^k$  different equivalence classes. We replace  $\Omega$  with the quotient space  $\Omega / \sim$  whose elements are equivalence classes. In other words, we merge all equivalent  $\omega$ 's. The measure of a new element  $\tilde{\omega}$  equals to the measure of the corresponding equivalence class.
Organization. In Section 2.1, we discuss the connection between explainable  $k$ -medians and set elimination games. We define a set elimination game in a set system  $I \subset \{S_1, \ldots, S_k\}$  in Section 2.2. Then, we define the hitting and elimination time in Section 2.3. In Section 3, we first illustrate our proof strategy by showing Theorem 2.1 for the case when the smallest set  $S_1$  does not overlap with  $S_2, \ldots, S_k$ . An important ingredient of our proof is the notion of surprise sets, which we discuss in Section 3.1. Finally, we complete the proof of Theorem 2.1 in Section 3.2.
2.1 Explainable  $k$ -Medians via Set Elimination Game
In this section, we show how to use Theorem 2.1 to obtain a bound of  $2\ln k + 2$  on the competitive ratio of the RANDOMCOORDINATECUT algorithm.
Theorem 2.2. The competitive ratio of the RANDOMCOORDINATECUT algorithm for Explainable  $k$ -Medians is at most  $2\ln k + 2$ . That is, for every set of centers  $C = \{c^1,\dots ,c^k\}$  and data set  $X$ , the algorithm finds a random decision tree  $\mathcal{T}$  such that
$$
\mathbf {E} [ \mathrm {c o s t} (X, \mathcal {T}) ] \leq (2 \ln k + 2) \cdot \sum_ {x \in X} \min _ {c \in \{c ^ {1}, \dots , c ^ {k} \}} \| x - c \| _ {1}.
$$
The pseudo-code for the RANDOMCOORDINATECUT algorithm is provided in Figure 2.
Theorem 2.2 shows that given any  $k$  centers  $C = \{c^1,\dots ,c^k\}$ , RANDOMCOORDINATECUT finds a decision tree  $\mathcal{T}$  with cost at most  $2\ln k + 2$  times the cost of unconstrained  $k$ -medians with centers  $C = \{c^{1},\ldots ,c^{k}\}$ . By using  $k$  centers given by any constant approximation algorithm for  $k$ -medians, RANDOMCOORDINATECUT finds a decision tree with cost at most  $O(\log k)$  times the optimal unconstrained  $k$ -medians cost. This implies an  $O(\log k)$  upper bound on the price of explainability.
Proof of Theorem 2.2. Consider an arbitrary data set  $X \subset \mathbb{R}^d$  and set of  $k$  centers  $C \subset \mathbb{R}^d$ . We assume that all points in  $X$  and all centers in  $C$  are in the cube  $[-M, M]^d$ . The threshold decision tree obtained by the RANDOMCOORDINATECUT algorithm partitions the space into  $k$  cells. Each cell contains a single reference center  $c^i$ . The center  $c^i$  is not necessarily optimal for cluster  $P_i$  (cluster  $P_i$  is the intersection of the data set  $X$  and  $i$ -th cell). However, we will use it as a proxy for the optimal center. In other words, we will upper bound the cost of the threshold decision tree as follows:
$$
\operatorname {c o s t} (X, \mathcal {T}) \equiv \min  _ {\hat {c} ^ {1}, \dots , \hat {c} ^ {k}} \sum_ {i = 1} ^ {k} \sum_ {x \in P _ {i}} \| x - \hat {c} ^ {i} \| _ {1} \leq \sum_ {i = 1} ^ {k} \sum_ {x \in P _ {i}} \| x - c ^ {i} \| _ {1}.
$$
Let  $\Omega$  be the set of all coordinate cuts:  $\Omega = \{(j,\theta):j\in [d],\theta \in [-M,M]\}$ . We define a measure  $\mu$  on  $\Omega$  as follows. For every subset  $S\subset \Omega$ , we set
$$
\mu (S) = \sum_ {j = 1} ^ {d} \mu_ {L} (\{\theta : (j, \theta) \in S \}),
$$
where  $\mu_L$  is the Lebesgue measure on  $\mathbb{R}$ . Thus, we have  $\mu(\Omega) = 2dM$ , which implies  $(\Omega, \mu)$  is a finite measure space.
Consider any data point  $x \in X$ . Define  $k$  sets  $S_{1}, S_{2}, \ldots, S_{k}$  for the set elimination game. For every  $i \in \{1, \ldots, k\}$ , let  $S_{i}$  be the set of all threshold cuts that separate  $x$  and center  $c^{i}$ , i.e.,
$$
S _ {i} = \left\{\left(j, \theta\right) \in \Omega : \operatorname {s i g n} \left(x _ {j} - \theta\right) \neq \operatorname {s i g n} \left(c _ {j} ^ {i} - \theta\right) \right\}.
$$
Note that the  $\ell_1$  distance from  $x$  to center  $c^i$  equals the measure of  $S_i$ :  $\| x - c^i\| _1 = \mu (S_i)$ . We now examine the set elimination game with sets  $S_{1},\ldots ,S_{k}$ , measure space  $(\Omega ,\mu)$ , and random sequence of draws  $\omega_{1},\omega_{2},\ldots$  (each  $\omega_{n}\in \Omega$  is the threshold cut chosen by the RANDOMCOORDINATE CUT algorithm at step  $n$ ). We claim that  $S_{i}$  belongs to  $\mathcal{R}_n$  if and only if center  $c^i$  lies in the same cell as point  $x$  after step  $n$  of the algorithm. This is the case for  $n = 0$ , since  $\mathcal{R}_0$  contains all sets  $S_{1},\dots ,S_{k}$  and the root of the threshold tree contains all centers  $c^{1},\ldots ,c^{k}$ . Then, whenever we pick cut  $\omega_{n}$ , all centers separated from  $x$  by  $\omega_{n}$  are removed from the cell of  $x$ . The only exception from this rule occurs when all centers in that cell lie on the same side of the cut  $\omega_{n}$ . That is exactly the same rule as we have for the set elimination game (note that center  $c^i$  is separated from  $x$  by  $\omega_{n}$  if and only if  $\omega_{n}\in S_{i}$ ). Therefore, the same sets  $S_{i}$  remain in the game as center  $c^i$  in the cell of  $x$  (namely, sets  $S_{i}$  and centers  $c^i$  have the same indices).
The RANDOMCOORDINATECUT algorithm stops when all leaves of the decision tree contain exactly one center. At this step, the set elimination game contains one set,  $S_{i}$ . This set corresponds to the center  $c^{i}$  assigned to point  $x$ . The cost of the game  $\mu(S_{i})$  equals the distance from  $x$  to  $c^{i}$ . By Theorem 2.1, we have
$$
\mathbf {E} [ \mathrm {c o s t} (x, \mathcal {T}) ] = \mathbf {E} [ \mu (\mathrm {w i n n e r}) ] \leq (2 \ln k + 2) \cdot \min _ {i} \mu (S _ {i}) = (2 \ln k + 2) \cdot \min _ {i} \| x - c ^ {i} \| _ {1}.
$$
We sum this bound over all data points  $x$  in  $X$  and get the desired result.
2.2 Local Competitions
We now revisit the definition of the set elimination game and define competitions in subsets of  $\{S_1,\ldots ,S_k\}$ . For the rest of the proof, we assume  $(\Omega ,\mu)$  is a discrete finite measure space. We remind the reader that every set elimination game is determined by an infinite sequence of i.i.d. random variables  $\omega_{1},\omega_{2},\ldots$ . In each round  $n$ , we sample an element  $\omega_{n}$  from  $\Omega$  with probability  $\operatorname*{Pr}(\omega_n = \omega) = \mu (\omega) / \mu (\Omega)$ .
Definition 2.3. Consider a finite measure space  $(\Omega, \mu)$ . Let  $I$  be a set of subsets of  $\Omega$ . We say that  $I$  is a valid set system if (a) for every  $S \in I$ ,  $\mu(S) > 0$ , and (b) for every  $S', S'' \in I$ ,  $\mu(S' \triangle S'') > 0$ .
The reader may assume that  $\mu (\omega) > 0$  for all  $\omega$  in  $\Omega$ . Then, the definition above says that in a valid set system  $I$ , all sets are non-empty and distinct.
Definition 2.4. Consider a finite measure space  $(\Omega, \mu)$ . Let  $\omega_1, \omega_2, \ldots$  be i.i.d. random variables as described above and  $I$  be a valid set system. We define a set elimination game in  $I$ . Initially,  $\mathcal{R}_0(I) = I$ . Then, for every  $n \geq 1$ ,
$$
\mathcal {R} _ {n} (I) = \left\{ \begin{array}{l l} \mathcal {R} _ {n - 1} (I) \backslash \{S \in \mathcal {R} _ {n - 1} (I): \omega_ {n} \in S \}, & i f f o r s o m e S ^ {\prime} \in \mathcal {R} _ {n - 1} (I), \omega_ {n} \notin S ^ {\prime}; \\ \mathcal {R} _ {n - 1} (I), & o t h e r w i s e. \end{array} \right. \tag {2}
$$
The winner of the game in  $I$ , denoted by winner(I), is the only element remaining, or, formally, the unique element in  $\cap_{n\geq 0}\mathcal{R}_n(I)$ . If  $\cap_{n\geq 0}\mathcal{R}_n(I)$  contains more than one element, then the winner is not defined. The cost of the game is the measure of the winner,  $\mu (\mathrm{winner}(I))$ .
We remark that  $\cap_{n\geq 0}\mathcal{R}_n(I)$  contains exactly one element with probability 1. Thus, the winner and cost of the game are defined with probability 1.