SlowGuess/NeurIPS_2023_OCR huggingface dataset:
This brings us to the following problem: given a colored point set  $P$  and a set of at most  $k$  balls  $B$ , how to pick the largest subset of balls  $B' \subseteq B$ , such that enough points exist outside of  $B'$ . (Note that  $B$  is basically the set of balls with radius  $O(d_j)$ , that are centered at the  $j$  points returned by the first  $j$  iterations of GMM). This problem can be solved exactly by an exhaustive search and spending exponential in  $k$  time. One can also get a polynomial time  $O(m)$ -approximate solution for the problem by iteratively excluding half of the remaining balls from the solution, and at the same time satisfying the condition for half of the colors. Thus, after  $O(\log m)$  iterations, all color constraints are satisfied, and a  $1/m$  fraction of the balls remain. There are further technical details needed for the proof to go through which we need to take care of.
3.1 Algorithm Description
Given a colored point set  $P = P_{1} \cup \dots \cup P_{m}$ , and  $k_{1}, \ldots, k_{m}$ , the goal is to find a solution  $\mathrm{SOL} = \mathrm{SOL}_{1} \cup \dots \cup \mathrm{SOL}_{m}$  where  $\mathrm{SOL}_{i} \subseteq P_{i}$ , and  $|\mathrm{SOL}_{i}| = k_{i}$ , and that  $\operatorname{div}(\mathrm{SOL}) \geq \frac{1}{\alpha} \cdot \operatorname{div}_{k_{1}, \ldots, k_{m}}(P)$ . The approximation algorithm is shown in Algorithm 3. The only unspecified part of the algorithm is in Line 8, where we need to specify how we find the subset  $B'$  given the set of balls  $B$ . One can take different approaches resulting in different trade-offs described below.
Algorithm 3 Approximation Algorithm for FDM under SUM-NN DIST Notion of Diversity
Input a colored point set  $P = P_{1} \cup \dots \cup P_{m}$ , and  $k_{1},\ldots ,k_{m}$   
Output a solution SOL  $\subseteq P$  where  $|\mathrm{SOL}\cap P_i| = k_i$ , with approximately maximum diversity
1: SOL  $\leftarrow$  an arbitrary solution satisfying the color constraints, i.e.,  $|\mathrm{SOL}\cap P_i| = k_i$
2: for  $i \in [m]$  do
3:  $G_{i} = \{p_{1},\dots ,p_{k}\} \leftarrow \mathbf{GMM}(P_{i},k)$  , and let  $r_1,\ldots ,r_k$  be their corresponding radii.
4: for  $j = 2$  to  $k$  do
5:  $S_{1},\ldots ,S_{m}\gets \emptyset$
6: Let  $t(j) \geq j$  be the largest iteration where  $r_{t(j)} \geq r_j / 2$  and let  $t(j) = k$  if no such iteration exists.
7: Let  $B = \{B_1, \ldots, B_{t(j)}\}$  be the balls of radius  $r_{t(j)} / 2$  around the points  $p_1, \ldots, p_{t(j)}$ .
8: Find an (approximately) largest subset of balls  $B' \subseteq B$  such that  $P \setminus B'$  contains at least  $k_i - |B'|$  points from color  $i$ , and at least  $k_\ell$  points from color  $\ell$  for each color  $\ell \neq i$ .
9: Add centers of  $B'$  to  $S_i$
0: Add an arbitrary set of  $k_{i} - |B^{\prime}|$  points from  $P_{i}\setminus B^{\prime}$  to  $S_{i}$
1: for  $\ell \neq i$  do
2: Add an arbitrary set of  $k_{\ell}$  points from  $P_{\ell} \setminus B'$  to  $S_{\ell}$
3: end for
4: if  $S = \bigcup_{i} S_{i}$  is a valid solution and  $\operatorname{div}(S) > \operatorname{div}(\operatorname{SOL})$  then
5:  $\mathrm{SOL}\gets S$
6: end if
7: end for
8: end for
9: return SOL
3.1.1 Finding an optimal subset of balls (Line 8 of Algorithm 3)
Given a colored point set  $P$  and a set of disjoint balls  $B$ , the goal here is to find an (approximately) largest subset of balls  $B' \subseteq B$ , such that
- For a pre-specified color  $i \in [m]$ , we have  $|P_i \setminus B'| \geq k_i - |B'|$
- For all other colors  $\ell \in [m] \setminus \{i\}$ , we have  $|P_{\ell} \setminus B'| \geq k_{\ell}$ .
Moreover, we may assume that  $|B| \leq t(j) \leq k$ , and that we are interested only in subsets  $B'$  of size up to  $|B'| \leq k_i$ .
Approach 1. If the algorithm is allowed to spend exponential time in  $k$ , one can find the optimal subset  $B'$  in time  $k^{k_i} \cdot m$  as stated below.
Observation 3.1 (Approach 1). There is an algorithm that finds the optimal largest subset  $B' \subseteq B$  in time  $\binom{t_j}{j} \cdot m \cdot j = k^{O(k_i)}$ . (Proof in Appendix A.2).
Approach 2. One can get an approximate solution in polynomial time if the number of points of each color is sufficiently large, i.e., for each  $\ell \in [m]$ , we have  $n_{\ell} = |P_{\ell}| \geq 2 \cdot k_{\ell}$ . Note that this is a realistic assumption (also appeared in [11]), as we expect the number of data points to be much larger than the target size for the summary. Further, this assumption can be relaxed to  $n_{\ell}$  being larger than a constant times  $k_{\ell}$  similar to [11].
Lemma 3.2. There is an algorithm that finds an  $O(m)$ -approximate largest subset  $B' \subseteq B$  in polynomial time, under the assumption that for each  $\ell \in [m]$ , we have  $n_{\ell} \geq 2 \cdot k_{\ell}$ . (Proof in Appendix A.3).
3.2 Algorithm Analysis
Let us now describe the intuition behind Algorithm 3 along with defining a set of notations. Let  $\mathrm{OPT} \subseteq P$  be the optimal solution and let  $\mathrm{OPT}_i = \mathrm{OPT} \cap P_i$  be the points of color  $i$  in the optimal solution. Moreover let  $d_1^i, \ldots, d_{k_i}^i$  be the values of the optimal solution, i.e.,  $d_j^i = \mathrm{dist}(p_j^i, \mathrm{OPT} \setminus \{p_j^i\})$ , where  $p_j^i$ 's are the points in  $\mathrm{OPT}_i$ . Therefore, we have that  $\mathrm{div}_{k_1, \ldots, k_m}(P) = \mathrm{div}(\mathrm{OPT}) = \sum_{i \leq m} \sum_{j \leq k_i} d_j^i$ .
Let  $i^{*} \leq m$  be the color with the maximum contribution in the optimal solution, i.e.,  $i^{*} = \operatorname{argmax}_{i \in [m]} \sum_{j \leq k_i} d_j^i$ . Clearly,  $\sum_{j \leq k_{i*}} d_j^{i*} \geq \operatorname{div}(\mathrm{OPT}) / m$ . Moreover, WLOG assume that for each  $i$ ,  $d_j^{i*}$ 's are sorted, i.e.,  $d_1^i \geq \dots \geq d_{k_i}^i$ . Now, let  $j^{*}(i)$  be defined as  $j^{*}(i) = \operatorname{argmax}_{j \leq k_i} j \cdot d_j^i$  and define  $j^{*} = j^{*}(i^{*})$ .
Claim 3.3. Let  $i \in [m]$  be any color. Then we have  $j^{*}(i) \cdot d_{j^{*}(i)}^{i} \geq \frac{1}{\lceil \log(k_{i} + 1) \rceil} \sum_{j \leq k_{i}} d_{j}^{i}$ . (Proof in Appendix A.4)
Theorem 3.4. Algorithm 3 using Approach 2, runs in polynomial time and produces a solution with an approximation factor of  $O(m^2 \cdot \log k)$ . (Proof in Appendix A.5)
Similarly, running Algorithm 3 with Approach 1 would give us the following theorem.
Theorem 3.5. There exists an  $O(m \cdot \log k)$  approximation algorithm that runs in time  $O(k^k \cdot \text{poly}(m, k))$ .
4 Core-set for FDM under SUM-NN DIST
In this section we show how to get a composable core-set for FDM with respect to the SUM-NN DIST notion of diversity.
Overview of the algorithm. In order to construct a core-set for FDM under the SUM-NN DIST notion of diversity, we show that it is enough to construct a subset of the points such that the solution to the problem of finding the maximum number of balls such that enough points exist outside of them (discussed in Section 3) remains unchanged. We show that the GMM algorithm has this property. Intuitively, since GMM chooses the points that are pairwise-far from each other, not too many points are picked inside individual balls and thus the solution to the balls problem remains relatively unchanged. Therefore, if we run the algorithm of Section 3 on the core-sets produced by GMM, the solution is similar as if we had run it on the whole data.
Algorithm description. In this section, the goal is to give a summarization algorithm  $\mathcal{A}$  that processes each dataset  $P_{i}$  independently of other color datasets  $P_{j}$  and produces a summary  $S_{i} = \mathcal{A}(P_{i})$  such that  $\mathrm{div}_{k_1,\dots,k_m}(S)\geq \frac{1}{\alpha}\cdot \mathrm{div}_{k_1,\dots,k_m}(P)$  where again  $S = \bigcup_{i}S_{i}$ . The core-set construction algorithm is simple and is shown in Algorithm 4. The algorithm proceeds by running the GMM algorithm  $k$  times, and each time for  $k + 1$  iterations, and without replacement.
Algorithm 4 Core-set Construction Algorithm for SUM-NN
Input a point set  $P_{i}$  , together with parameters  $k_{i}$  and  $k$  (where  $k = k_{1} + \dots +k_{m})$    
Output a subset  $S_{i}\subseteq P_{i}$    
1:  $S_{i}\gets \emptyset$    
2: for  $j = 1$  to  $k$  do   
3:  $G_{i} = \{p_{1},\ldots ,p_{k + 1}\} \leftarrow \mathrm{GMM}(P_{i},k + 1)$    
4:  $S_{i}\gets S_{i}\cup G_{i}$    
5:  $P_{i}\gets P_{i}\backslash G_{i}$    
6: end for   
7: return  $S_{i}$
Theorem 4.1. Algorithm 4 returns a composable core-set of size  $O(k^2)$  for each of the  $m$  colors, with an approximation factor of  $O(m \cdot \log k)$ . (Proof in Appendix A.6)
To demonstrate the effectiveness of our algorithms, we run simulations on public and timed datasets.
5.1 Tasks and Datasets
FDM as a tool to account for recency in a summary. Here the task is to produce a summary of a set of timed messages with a property that (i) the summary is a diverse subset of the messages, and (ii) there are more recent messages shown in the summary. We model this task by partitioning the messages into groups / colors based on their creation times and assign a desired budget to each group.
We use a Reddit dataset [42] of text messages that are semantically embedded using BERT [12] into a metric space. We also utilize message creation time stamps as we aim to show diverse, yet timely and relevant messages to a user created across different time-window intervals within a certain period and thus assign a color to each message based on to which time-window interval (e.g., a week, thus having 4 colors in total) the message creation time belongs to.
FDM as a tool for controlling the desired fairness contribution of each group. We further use FDM to control the contribution of each genre (besides time) in a movie dataset. Apart from the Reddit dataset, for this task we also use the MovieLens dataset [18] where the movie titles are semantically embedded into a metric space. For this dataset, we have assigned a movie to a group represented by a color based on two criteria: (i) movie genre, and (ii) the time-window interval the movie review creation time belongs to. We include the result of this dataset in Appendix D.2.
More details on the datasets preparation are given in Appendix D.1 and additional result figures are provided in Appendix D.2. We have also made the code of the algorithms publicly available<sup>2</sup>.
(a) Reddit (SUM-PAIRWISE)
(b) Reddit (SUM-NN)
Figure 1: DM algorithm outcomes for all colors ( $m = 4$ ) with equidistant time period as fairness colors in the Reddit dataset ( $k = 20$  items).
(c) Reddit (MIN-PAIRWISE)
5.2 Summary of the experiments and results
We run the following experiments using all three diversity measures.
Need for FDM. We first show why we need to resort to FDM. In particular, we show that if we run DM on the data, the results are not as fairly balanced as we want, as depicted in Figure 1 and Figures 2 & 3 (Appendix D.2.1). In the case of time periods as colors dividing the data into  $m = 4$  colors (i.e., quarters within a month), for  $k = 20$  DM algorithms give: (i) a certain color that is clearly dominantly present in the outcome (for all diversity distances), and (ii) this color is not the one from the most recent messages (i.e., from the last color), see Figure 1 for the Reddit dataset.
Price of fairness (balancedness). Our next set of experiments show that when using FDM the diversity is not decreased by a lot compared to DM. We use the loss of diversity (\% Div. loss) expressed as a relative change of diversity distances as a measure. This is shown in Table 2, and Tables 4 & 5 (Appendix D.2.2). For most of these experiments we use the state-of-the-art algorithms. However for SUM-NN DIST we use our proposed deterministic algorithm of Section 3 as an alternative to the LP-based randomized algorithm of [6]. When we run FDM, we gain in fairness (or balancedness), but we lose in diversity over the DM, expressed in different metrics. Experiments from Table 2 show that we lose around  $1\%$  for SUM-PAIRWISE DIST; from few percent up to no more than  $20\%$  for SUM-NN DIST and around  $50\%$  for MIN-PAIRWISE DIST (due to its fragility) for various color distributions per group, while achieving the desired per group distribution, in both cases where we enforce balancedness (i.e., uniform  $k_{i}$ ), and when we enforce recency (i.e.,  $k_{i}$  increasing). The results also show a similar trend if we use alternative message embeddings (Table 7 in Appendix D.2.4).
Effectiveness of our core-sets. We then show the effectiveness of the core-sets. We run our core-set construction algorithms (of Section 2 and Section 4) on each color independently, to get a smaller size dataset. We then run the FDM optimization once on the union of the core-sets and once on the whole data. We also measure the loss of diversity and runtime improvement achieved by the use of core-sets. The results for all three measures are given in Table 3 and Table 6 (Appendix D.2.3). In
$^{2}$ https://github.com/microsoft/coresets-fair-diverse
Table 2: The loss of diversity (% Div. loss) between DM vs. FDM on the full data, expressed as a relative change, of the concerned SUM-PAIRWISE, SUM-NN and MIN-PAIRWISE distances for uniform (upper part) or increasing (lower part) color values  $k_{i}$  for the Reddit dataset.
DM vs. FDM	SUM-PAIRWISE	SUM-NN	MIN-PAIRWISE
colors \( {k}_{i} \)	\( \sum {k}_{i} \)% Div. loss	% Div. loss	% Div. loss
\( \left\lbrack {2,2,2,2}\right\rbrack \)	8	1.22%	9.66%	51.57%
\( \left\lbrack {3,3,3,3}\right\rbrack \)	12	0.98%	14.27%	49.99%
\( \left\lbrack {4,4,4,4}\right\rbrack \)	16	0.50%	13.72%	48.78%
\( \left\lbrack {5,5,5,5}\right\rbrack \)	20	0.47%	18.96%	48.05%
\( \left\lbrack {6,6,6,6}\right\rbrack \)	24	0.19%	9.48%	47.20%
\( \left\lbrack {2,4,6,8}\right\rbrack \)	20	0.42%	15.40%	48.05%
\( \left\lbrack {3,6,9,{12}}\right\rbrack \)	30	0.29%	13.29%	46.34%
\( \left\lbrack {4,8,{12},{16}}\right\rbrack \)	40	0.25%	1.98%	45.52%
\( \left\lbrack {5,{10},{15},{20}}\right\rbrack \)	50	0.16%	9.62%	44.48%
\( \left\lbrack {6,{12},{18},{24}}\right\rbrack \)	60	0.12%	3.98%	43.60%
terms of diversity loss, the results show the diversity values are on-par or with marginal difference if we apply the FDM to the union of core-sets, compared to FDM applied to the full data. (In the case of SUM-NN DIST, there are cases where one is marginally better than the other, due to higher approximation factor.) Our experiments show that using core-sets, the runtime of our algorithm improves on average by factor of few  $10 \times$  or  $100 \times$ , while only losing diversity by few percent. We remark that using core-sets in this context has an additional benefit: it removes the need to recompute the summary on the whole data when new messages arrive: once we summarize a batch of old message, we no longer need to process the batch and only need to work with the core-set that is computed once.
Table 3: The loss of diversity (\% Div. loss) expressed as a relative change of diversity distances, and the running time gains ( $\times$  times faster) of the FDM when applied to the union of core-sets compared to FDM applied to the full data for uniform or increasing color values  $k_{i}$  for the Reddit dataset. We remark that the results for MIN-PAIRWISE have  $\%$  Div. loss being  $0\%$  as for this data the two points connected with the minimum distance always end up in the union of the core-sets.
FDM full data vs. core-sets	SUM-PAIRWISE	SUM-NN	MIN-PAIRWISE
colorski	∑ki% Div. loss	Time gain (×)	% Div. loss	Time gain (×)	% Div. loss	Time gain (×)	% Div. loss
[2,2,2,2]	8	1.35%	196.24	2.22%	1769.70	0.00%	208.64
[3,3,3,3]	12	0.67%	333.13	0.29%	888.55	0.00%	152.48
[4,4,4,4]	16	1.21%	539.69	-1.59%	474.26	0.00%	122.29
[5,5,5,5]	20	1.17%	432.68	-0.44%	294.23	0.00%	89.08
[6,6,6,6]	24	0.94%	130.87	-3.03%	183.28	0.00%	63.69
[2,4,6,8]	20	1.50%	845.98	-1.80%	285.68	0.00%	91.44
[3,6,9,12]	30	1.06%	134.76	2.27%	110.36	0.00%	53.05
[4,8,12,16]	40	1.02%	182.06	-0.88%	57.88	0.00%	36.51
[5,10,15,20]	50	1.16%	194.36	0.71%	34.90	0.00%	26.97
[6,12,18,24]	60	1.27%	172.25	-0.49%	23.71	0.00%	20.53
Comparison to state-of-the-art core-set algorithm for SUM-PAIRWISE DIST. Finally, we compare the core-set construction algorithm in [8] for SUM-PAIRWISE DIST to our algorithm of Section 2. The experiments in Appendix D.2.5 show that, the size of the core-set obtained by the algorithm in [8] for getting a factor 2 approximation is close to the data size in practice. In addition, the final diversity loss of our algorithm is negligible (less than  $1\%$  to few percent) even though it produces a much smaller core-set. Further, our core-set construction algorithm is also significantly faster.
[35] K. Song, X. Tan, T. Qin, J. Lu, and T.-Y. Liu. MPNet: Masked and Permuted Pre-training for Language Understanding. In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems (NeurIPS), volume 33, pages 16857-16867. Curran Associates, Inc., 2020.
[36] N. Thakur, N. Reimers, J. Daxenberger, and I. Gurevych. Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Online, June 2021. Association for Computational Linguistics.
[37] M. Volske, M. Potthast, S. Syed, and B. Stein. TL;DR: Mining Reddit to Learn Automatic Summarization. In Proceedings of the Workshop on New Frontiers in Summarization, Copenhagen, Denmark, Sept. 2017. Association for Computational Linguistics.
[38] W. Wang, F. Wei, L. Dong, H. Bao, N. Yang, and M. Zhou. MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers. In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems (NeurIPS'20), volume 33, pages 5776–5788, 2020.
[39] M. J. Welch, J. Cho, and C. Olston. Search Result Diversity for Informational Queries. In Proceedings of the 20th International Conference on World Wide Web (WWW'11), pages 237-246, 2011.
[40] C. Yu, L. V. Lakshmanan, and S. Amer-Yahia. Recommendation Diversification using Explanations. In 2009 IEEE 25th International Conference on Data Engineering, pages 1299-1302. IEEE, 2009.
[41] S. Zadeh, M. Ghadiri, V. Mirrokni, and M. Zadimoghaddam. Scalable feature selection via distributed diversity maximization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017.
[42] H. Zhu, F. Nan, Z. Wang, R. Nallapati, and B. Xiang. Who did They Respond to? Conversation Structure Modeling using Masked Hierarchical Transformer. arXiv preprint arXiv:1911.10666 (AAAI'20), 2019.
[43] C.-N. Ziegler, S. M. McNee, J. A. Konstan, and G. Lausen. Improving Recommendation Lists Through Topic Diversification. In Proceedings of the 14th International Conference on World Wide Web (WWW'05), pages 22-32, 2005.
Proof. First note that it is easy to see that the size of each  $S_{i}$  is at most  $k_{i}^{2}$ .
We will then show that the approximation factor is  $O(1)$ . Let  $\mathrm{OPT}_i = \mathrm{OPT} \cap P_i$  be the points of color  $i$  in the optimal solution of  $P$ . For each color  $i \in [m]$ , let us use  $r_i$  to denote the minimum pairwise distance between the set of  $k_i$  points returned by  $\mathrm{GMM}(P_i, k_i)$  in Line 1 of the algorithm. Further let us abuse the notation, and for a point  $x \in P$ , use  $r_x$  to denote  $r_{c_x}$  where  $c_x \in [m]$  is the color of the point  $x$ .
Now let us also divide the optimal value of the diversity into two parts.
$$
\operatorname {div}(\mathrm{OPT}) = \sum_{x,y\in \mathrm{OPT}:  \operatorname {dist}(x,y) <   5  \max \{r_{x},r_{y}\}}\operatorname {dist}(x,y) + \sum_{x,y\in \mathrm{OPT}:  \operatorname {dist}(x,y)\geq 5  \max \{r_{x},r_{y}\}}\operatorname {dist}(x,y)
$$
Let  $A := \sum_{x,y \in \mathrm{OPT}} \, \mathrm{dist}(x,y) < 5 \max \{r_x, r_y\} \, \mathrm{dist}(x,y)$  be the first term in the above summation and  $B := \sum_{x,y \in \mathrm{OPT}} \, \mathrm{dist}(x,y) \geq 5 \max \{r_x, r_y\} \, \mathrm{dist}(x,y)$  denote the second term. Now we consider two cases separately.
Case 1:  $A \geq B$ . In this case we have that
$$
\operatorname {div} (\mathrm {O P T}) \leq 2 \cdot A \leq 2 \sum_ {x, y \in \mathrm {O P T}} 5 \cdot \max  \left\{r _ {x}, r _ {y} \right\} \leq 2 0 \sum_ {x, y \in \mathrm {O P T}} r _ {x} \leq 2 0 \cdot k \sum_ {i \in [ m ]} k _ {i} r _ {i}
$$
Now let us define a solution as follows. Let  $\mathrm{SOL}_i\subseteq S_i$  be the first set of  $k_{i}$  points added to  $S_{i}$  in Line 1 of the algorithm using GMM. Then clearly the overall diversity of this solution is at least
$$
\begin{array}{l} \operatorname {div}(\operatorname {SOL}) = \sum_{i\in [m]}\sum_{x,y\in \operatorname{SOL}_{i}}\operatorname {dist}(x,y) + \frac{1}{2}\cdot \sum_{i\in [m]}\sum_{y\in \operatorname {SOL}\backslash \operatorname{SOL}_{i}}\sum_{x\in \operatorname{SOL}_{i}}\operatorname {dist}(x,y) \\ \geq \sum_ {i \in [ m ]} \binom {k _ {i}} {2} r _ {i} + \frac {1}{2} \cdot \sum_ {i \in [ m ]} \sum_ {y \in \text {S O L} \backslash \text {S O L} _ {i}} \frac {\binom {k _ {i}} {2}}{(k _ {i} - 1)} r _ {i} \quad \text {b y} \\ \geq \sum_ {i \in [ m ]} r _ {i} (\binom {k _ {i}} {2} + (k - k _ {i}) \cdot \frac {k _ {i}}{4}) \\ \geq \frac {k}{4} \sum_ {i \in [ m ]} k _ {i} r _ {i} + \sum_ {i \in [ m ]} r _ {i} (\frac {k _ {i} ^ {2}}{4} - \frac {k _ {i}}{2}) \\ \geq \frac {k}{4} \sum_ {i \in [ m ]} k _ {i} r _ {i} \\ \end{array}
$$
where to get the last inequality, we used the fact that for  $k_{i} > 1$  the second term is non-negative. Therefore  $\mathrm{div}(\mathrm{SOL}) \geq \frac{1}{80} \cdot \mathrm{div}(\mathrm{OPT})$ .
Case 2:  $A < B$ . In this case we have that  $\mathrm{div}(\mathrm{OPT}) \leq 2B$ . We will show how to choose the solution SOL. Let  $\mathrm{OPT}_i = \mathrm{OPT} \cap P_i$  as before.
Observation A.1. There exists a one-to-one mapping  $\mu : \mathrm{OPT}_i \to S_i$  s.t. for each  $o \in \mathrm{OPT}_i$ ,  $\mathrm{dist}(o, \mu(o)) \leq 2r_i$ .
Proof. Let  $S_i' \subseteq S_i$  be the first set of  $k_i$  points added to  $S_i$  in Line 1 of the algorithm. For a point  $p \in P_i$  let  $n(p) = \operatorname*{argmin}_{s' \in S_i'} \mathrm{dist}(p, s')$  be the nearest center in  $S_i'$  to  $p$ . Also, for a center  $s' \in S_i'$ , let  $N(s') = \{ s \in S_i \colon n(s) = s' \}$  be the set of points in the core-set whose nearest center in  $S_i'$  is  $s'$ , i.e., the points that have been added to the core-set in Line 5 of the algorithm while processing  $s'$ . Finally let  $D(s') = \{ o \in \mathrm{OPT}_i \colon n(o) = s' \}$  be the set of points of color  $i$  in the optimal solution whose closest center in  $S'$  is  $s'$ . Note that because  $|\mathrm{OPT}_i| = k_i$ , we have  $|N(s')| \geq |D(s')|$ , as the algorithm keeps adding points to  $S_i$  as long as such point exists, and fewer than  $k_i$  points
have been added to  $N(s')$ . Therefore, we can always find a matching between  $D(s')$  and  $N(s')$  of size at least  $|D(s')|$ . Thus such a  $\mu$  exists that maps a point  $o \in D(s')$  to its match in  $N(s')$ . By Property 3 of GMM we know that  $\mathrm{dist}(o, s') \leq r_i$  and  $\mathrm{dist}(s', \mu(o)) \leq r_i$ , and thus we get that  $\mathrm{dist}(o, \mu(o)) \leq 2r_i$ .
Now define  $\mathrm{SOL}_i\subseteq S_i$  to be  $\{\mu (o)\colon o\in \mathrm{OPT}_i\}$  and clearly  $|\mathrm{SOL}_i| = k_i$ . Then we have