SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Remark B.3. The assumed upper bound on  $k$  in Theorem B.2 is necessary (even in terms of  $\log n$ ). Indeed, consider a tree  $T$  which is a star  $S$  on  $n - k \log k$  vertices along with a path  $P$  of length  $k \log k$  having one endpoint at the center  $v$  of the star. With probability  $\Omega(1)$ , some color does not appear on  $P$ . We refer to this color as color 1. Now consider the color of the star center. With probability at least  $9/10$ , say, this color is different from 1 and appears  $\Omega(\log k)$  times on  $P$  with average distance  $\Omega(k \log k)$  to the star center  $v$ . Let the star center have color 2. With high probability, each color appears  $\Theta(n/k)$  times in  $S$ . Combining these bounds, we find that with constant probability, the average distance from  $v$  to vertices of color 1 is  $O(1)$ , whereas the average distance from  $v$  to vertices of color 2 is  $\Omega\left(1 + \frac{k^2 (\log k)^2}{n}\right)$ . In particular for the algorithm to give an  $O(1)$ -approximate IP-stable clustering, we need to assume that  $k = O\left(\frac{\sqrt{n}}{\log n}\right)$ .
C Additional Empirical Evaluations
We implement our  $O(1)$ -approximation algorithm for IP-clustering. These experiments extend those of [1] and confirm their experimental findings:  $k$ -means++ is a strong baseline for IP-stable clustering. Nevertheless, our algorithm is competitive with it while guaranteeing robustness against worst-case datasets, a property which  $k$ -means++ does not possess.
Our datasets are the following. There are three datasets from [11] used in [1], namely, Adult, Drug [14], and IndianLiver. We also add two additional datasets from UCI Machine Learning Repository [11], namely, BreastCancer and Car. For IP-clustering, we also consider a synthetic dataset which is the hard instance for  $k$ -means++ given in [1].
Our goal is to show that our IP-clustering algorithm is practical and in real world datasets, is competitive with respect to  $k$ -means++, which was the best algorithm in the experiments in [1]. Furthermore, our algorithm is robust and outperform  $k$ -means++ for worst case datasets.
As before, all experiments were performed in Python 3. We use the  $k$ -means++ implementation of Scikit-learn package [21]. We note that in the default implementation in Scikit-learn,  $k$ -means++ is initiated many times with different centroid seeds. The output is the best of 10 runs by default. As we want to have control of this behavior, we set the parameter n_init=1 and then compute the average of many different runs.
Additionally to the metrics used in the main experimental section, we also compute the number of unstable points, defined as the size of the set  $U = \{x \in M : x \text{ is not stable}\}$ . In terms of clustering qualities, we additionally measure three quantities. First, we measure "cost", which is the average within-cluster distances. Formally,  $Cost = \sum_{i=1}^{k} \frac{1}{\binom{|C_i|}{2}} \sum_{x,y \in C_i, x \neq y} d(x,y)$ . We then measure  $k$ -center costs, defined as the maximum distances from any point to its center. Here, centers are given naturally from  $k$ -means++ and our algorithm. Finally,  $k$ -means costs, defined as  $k$ -means-cost =  $\sum_{i=1}^{k} \frac{1}{|C_i|} \sum_{x,y \in C_i, x \neq y} d(x,y)^2$ .
C.1 Hard Instance for  $k$ -means++ for IP-Stability
We briefly describe the hard instance for  $k$ -means++ for the standard IP-stability formulation given in [1]; see their paper for full details. The hard instance consists of a gadget of size 4. In the seed-finding phase of  $k$ -means++, if it incorrectly picks two centers in the gadget, then the final clustering is not  $\beta$ -approximate IP-stable, where  $\beta$  is a configurable parameter. The instance for  $k$ -clustering is produced by concatenating these gadgets together. In such an instance, with a constant probability, the clustering returned by  $k$ -means++ is not  $\beta$ -approximate IP-stable and in particular. We remark that the proof of Theorem 2 in [1] easily implies that  $k$ -means++ cannot have an approximation factor better than  $n^c$  for some absolute constant  $c > 0$ , i.e., we can insure  $\beta = \Omega(n^c)$ . Here, we test both our algorithm and  $k$ -means++ in an instance with 8,000 points (for  $k = 2, 000$  clusters).
IP-Stability results We first discuss five real datasets. We tested the algorithms for the range of  $k$  up to 25. The result in Figures 2 and 3 is consistent with the experiments in the previous paper as we see that  $k$ -means++ is a very competitive algorithm for these datasets. For small number of clusters, our algorithm sometimes outperforms  $k$ -means++. We hypothesize that on these datasets, especially for large  $k$ , clusters which have low  $k$ -means cost separate the points well and therefore are good clusters for IP-stability.
Next we discuss the  $k$ -means++ hard instance. The instance used in Figure 3 was constructed with  $\beta = 50$ . We vary  $k$  but omit the results for higher  $k$  values since the outputs from both algorithms are stable. We remark that the empirical results with different  $\beta$  gave qualitatively similar results. For maximum and mean violation, our algorithm outperforms  $k$ -means++ (Figure 3).
D Future Directions
There are multiple natural open questions following our work.
- Note that in some cases,  $\alpha$ -IP stable clusterings for  $\alpha < 1$  may exist. On the other hand, in the hard example on  $n = 4$  from [1], we know that there some constant  $C > 1$  such that no  $C$ -IP stable clustering exists. For a given input, let  $\alpha^{*}$  be the minimum value such that a  $\alpha^{*}$ -IP stable clustering exists. Is there an efficient algorithm which returns an  $O(\alpha^{*})$ -IP stable clustering? Note that our algorithm satisfies this for  $\alpha = \Omega(1)$ . An even stronger result would be to find a PTAS which returns a  $(1 + \varepsilon)\alpha^{*}$ -IP stable clustering.
- For what specific metrics (other than the line or tree metrics with  $k = 2$ ) can we get 1-IP stable clusterings efficiently?
- In addition to stability, it is desirable that a clustering algorithm also achieves strong global welfare guarantee. Our algorithm gives constant approximation for  $k$ -center. What about other metrics, such as  $k$ -means?
Figure 2: Experiment results on three datasets used in [1].
OurAlg -k-means++
Figure 3: Additional experiment results on two real datasets and the synthetic dataset.