SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The upper bound simply comes from our stopping condition in the first nested while loop: we ensure  $n_{T'}(v) \leq \delta n$  before selecting it. The lower bound is slightly more complicated. Effectively, we start by noting that  $\max(\delta_1, \delta_2) > \epsilon$ , because otherwise the stopping condition for the outer loop would be met. Then, consider the total amount of "excess of large clusters", or more precisely, the sum over all deviations from  $n / h$  of clusters larger than  $n / h$  (note if all clusters were  $n / h$ , it would be perfectly balanced). This total excess must be matched in the "deficiency of small clusters", which is the sum of deviations of clusters smaller than  $n / h$ . Therefore, since there are at least  $h$  small or  $h$  large clusters, the largest deviation must be at most  $h$  times the smallest deviation, according to our accounting scheme. This allows us to bound  $\delta \geq \epsilon / (h - 1)$ . The tree that is inserted and deleted must have at least half this many leaves, since it is the larger child of a node with over  $\delta n$  leaves in its subtree. This gives our lower bound, showing we move at least a significant number of vertices each step.
Next, we aim to illustrate the relative balance. In addition to our analysis, we also obtain the runtime, which exhibits near-linearity when the condition  $h \ll n$  holds.
Lemma 2. SplitRoot (Algorithm  $\overline{I}$ ) yields a hierarchy whose root is  $\epsilon$ -relatively balanced with  $h$  children. In addition, it requires  $O(nh)$  time to terminate.
The root has  $h$  children by definition at the algorithm start and this remains invariant. The runtime comes from our aforementioned accounting scheme: the total excess and deficiency is reduced by the number of leaves in the subtree we move at each step, which we showed in Lemma [1] is  $n\epsilon/(2(h - 1))$  at least. This gives us a convergence time of  $O(h)$ , and each step can be bounded by  $O(n)$  time as we search for our insertion and deletion spots. Finally, the balance comes from the fact that our stopping condition is equivalent to the root being relatively balanced.
All that remains is to show the negative impact on the cost of edges that are separated by the algorithm. We bound this via the following lemma.
Lemma 3. In SplitRoot (Algorithm [7], for all  $e \in E$  that are separated:
$$
\operatorname {c o s t} _ {T ^ {\prime}} (e) \leq n \cdot w (e) \leq 2 (h - 1) \cdot \operatorname {c o s t} _ {T} (e) / \epsilon
$$
Lemma  $\boxed{1}$  gives us that moved subtrees are at least of size  $\epsilon n / (2(h - 1))$ , which is a lower bound on the size of the smallest cluster containing any edge separated by the algorithm. This is due to the fact that separated edges must have one endpoint in the deleted subtree and one outside, so their least common ancestor is an ancestor of the subtree. At worst, the final size of the smallest cluster containing such an edge is  $n$ , so the proportional increase is  $2(h - 1) / \epsilon$  at worst.
3.2 Fair Tree Folding
Next, we discuss how to achieve fairness by using MakeFair, as seen in Algorithm 2. This is our final recursive algorithm which utilizes SplitRoot. Assume we are given some hierarchical clustering. We start by running SplitRoot, to balance the split at the root and give it  $h$  children. Next we use a folding process similar to that of Knittel et al. [2023], but we use our shallow tree fold operator.
More specifically, we first sort the children of the root by the proportional representation of the first color (say, red). Then, we do a shallow fold across various  $k$ -sized sets, defined as follows: according to our ordering over the children, partition the vertices into  $k$  contiguous chunks starting from the first vertex. For each  $i \in [h / k]$ , we find the  $i$ -th vertex in each chunk and fold them together. Notice that this is a  $k$ -wise fold since there are  $k$  chunks, and we end up with  $h / k$  vertices. This is repeated on each color. After this, we simply recurse on the children. If a child is too small to be balanced by SplitRoot, then we stop and give it a trivial topology (a root with many leaf-children).
This completes our algorithm description. We now evaluate its runtime, degree of fairness, and approximation factor. To start, we show the degree of fairness achieved at the top level of the hierarchy.
Lemma 4. MakeFair (Algorithm) yields a hierarchy such that all depth 1 vertices satisfy fairness under  $\alpha_{i} \leq \frac{\lambda_{i}}{n} \cdot \frac{1 - \epsilon}{(1 + \epsilon)^{2}}\left(1 - \frac{k(1 + \epsilon)}{c_{i}h}\right)$  and  $\beta_{i} \geq \frac{\lambda_{i}}{n} \cdot \frac{1 + \epsilon}{(1 - \epsilon)^{2}}\left(1 + \frac{1 - \epsilon}{c_{i}k}\right)$ , where  $\lambda_{i} = c_{i}n$ .
This proof is quite in depth, and most details are deferred to Appendix B. At a high level, we are showing that the folding process guarantees a level of fairness. The parts in our partition are ordered by the density of the color (say, red). Since each final vertex is made by folding across one vertex in each part, meaning that the vertices have a relatively wide spread in terms of their density of red points. This means that red vertices are distributed relatively well across our final subtrees. This guarantees a degree of balance.
The problem is that the degree of fairness still exhibits a compounding affect as we recurse. More specifically, since the first children are not perfectly balanced, then in the next recursive step, the total data subset we are working on may now deviate from the true color proportions. This deviation is bounded by our result in Lemma 4, but it will increase proportionally at each step.
Lemma 5. In MakeFair (Algorithm 2), let  $\{\lambda_i\}_{i \in [\lambda]}$  be the proportion of each color and assume  $k^{\lambda} \ll h$ . At any recursive call, the proportion of any color is (where  $\lambda_i = 1 / c_i$  for constant  $c_i$ ):
$$
\lambda_ {i} \left(\frac {1 - \epsilon}{(1 + \epsilon) ^ {2}} \left(1 - \frac {k (1 + \epsilon)}{c _ {i} h}\right)\right) ^ {O (\log (n / h))} \leq \lambda_ {i} ^ {j} \leq \lambda_ {i} \left(\frac {1 + \epsilon}{(1 - \epsilon) ^ {2}} \left(1 + \frac {1 - \epsilon}{c _ {i} k}\right)\right) ^ {O (\log (n / h))}
$$
Moreover, the recursive depth is bounded above by  $O(\log (n / h))$ .
This result derives directly from Lemma 4. Effectively, we increase the proportion of each color by the same factor each recursive step. All that is left to do is bound the recursive depth. Notice we start with  $n$  vertices. After splitting, our subtrees have size at most  $(1 + \epsilon)n / h$ . After one fold, this is increased by a factor of  $k$ , and thus  $k^{\lambda}$  after all folds. Interestingly, this doesn't impact the final result significantly; it's fairly similar to turning an  $n$ -sized tree into an  $n / h$ -sized tree, giving an  $O(\log(n / h))$  recursive depth. This will be sufficient to show our fairness.
Next, we evaluate the cost incurred at each stage in the hierarchy.
Lemma 6. In MakeFair (Algorithm 2), for all  $e \in E$  that is separated before the recursive call:
$$
\operatorname {c o s t} _ {T ^ {\prime}} (e) \leq O \left(\frac {2 (h - 1)}{\epsilon} + \frac {1 + \epsilon}{1 - \epsilon} k ^ {\lambda}\right) \operatorname {c o s t} _ {T} (e)
$$
Algorithm 2 MakeFair
Input: A hierarchy tree  $T$  of size  $n \geq 1/2\epsilon$  over a graph  $G = (V, E, w)$  with vertices given one of  $\lambda$  colors, and parameters  $h \in [n]$ ,  $k \in [h / (\lambda - 1)]$ , and  $\epsilon \in (0, \min(1/6, 1/h))$ .
Output: A fair hierarchical clustering  $T'$ .
1:  $T^{\prime} = \mathrm{SplitRoot}(T,h,\epsilon)$
2:  $h^{\prime}\gets h$
3: for each color  $\ell \in [\lambda ]$  do
4: Order  $\{v_{i}\}_{i\in [h^{\prime}]} = \mathrm{children}(\mathrm{root}(T^{\prime}))$  decreasing by  $\frac{\ell(\mathrm{leaves}(v_i))}{n_{T'}(v_i)}$
5: For all  $i\in [k]$ $T^{\prime}\gets T^{\prime}.$  shallowfold  $\left(\{T^{\prime}[v_{i + (j - 1)k}]:j\in [h^{\prime} / k]\}\right)$
6:  $h^{\prime}\gets h^{\prime} / k$
7: end for
8: for each child  $v_{i}$  of root  $(T^{\prime})$  do
9: if  $n \geq \max(1/2\epsilon, h)$  then
0: Replace  $T'[v_i] \gets \text{MakeFair}(T'[v_i], h, k, \epsilon)$
11: else
12: Replace  $T'[v_i]$  with a tree of root  $v_i$ , leaves  $\text{leaves}(T'[v_i])$ , and depth 1.
13: end if
14: end for
As discussed before, the final cluster size should be  $(1 + \epsilon)nk^{\lambda} / h$ . Any separated edge must have a starting cluster size of at least  $(1 - \epsilon)n / h$ , as this is the size of the smallest cluster involved in tree folding. From this, it is straightforward to compute the proportional cost increase of a single recursive level, as well as the cost increase from the initial splitting in Lemma 3.
We additionally demonstrate that, whenever an edge is separated, its endpoints' least common ancestor will no longer be involved in any further recursive step. More formally:
Lemma 7. In MakeFair (Algorithm 2), any edge  $e \in E$  is separated at only one level of recursion.
Putting these two together pretty directly gives us our cost approximation.
Lemma 8. In MakeFair (Algorithm 2),  $\mathrm{cost}(T') \leq O\left(\frac{2(h - 1)}{\epsilon} + \frac{1 + \epsilon}{1 - \epsilon} k^{\lambda}\right) \mathrm{cost}(T)$ .
Finally, Theorem 1 comes directly from Lemmas 6 and 8.
This section validates the theoretical guarantees of Algorithm 2. Specifically, we demonstrate that modifying an unfair hierarchical clustering using the presented procedure yields a fair hierarchy that incurs only a modest increase in cost.
Datasets. We use two data sets, Census and Bank, from the UCI data repository Dua and Graff [2017]. Within each, we subsample only the features with numerical values. To compute the cost of a hierarchical clustering we set the similarity to be  $w(i,j) = \frac{1}{1 + d(i,j)}$  where  $d(i,j)$  is the Euclidean distance between points  $i$  and  $j$ . We color data based on binary (represented as blue and red) protected features: race for Census and marital status for Bank (both in line with the prior work of Ahmadian et al. [2020]). As a result, Census has a blue to red ratio of 1:7 while Bank has 1:3. We then subsample each color in each data set such that we retain (approximately) the data's original balance. We use samples of size 512 for the balance experiments, and vary the sample sizes when assessing cost. For each experiment we conduct 10 independent replications (with different random seeds for the subsampling), and report the average results. We vary the parameters  $(c,h,k,\varepsilon)^{\mathrm{T}}$  to experimentally assess their theoretical impact on the approximate guarantees of Section 3. Due to space constraints, we here present only the results for the Census dataset and defer the complimentary results on Bank to the appendix.
Note that the  $c$  parameter is used to scale  $\varepsilon$ .
Figure 3: Histogram of cluster balances after tree manipulation by Algorithm  $\boxed{2}$  on a subsample from the Census dataset of size  $n = 512$ . The four panels depict: (A) cluster balances after applying the (unfair) average-linkage algorithm, (B) the resultant cluster balances after running Algorithm  $\boxed{2}$  with parameters  $(c,h,k,\varepsilon) = (8,4,2,1/c \cdot \log_2 n)$ , (C) cluster balances after tuning  $c = 4$ , (D) cluster balances after further tuning  $c = 2$ . The vertical red line on each plot indicates the balance of the dataset itself.
Implementation. The Python code for the following experiments are available in the Supplementary Material. We start by running average-linkage, a popular hierarchical clustering algorithm. We then apply Algorithm 2 to modify this structure and induce a fair hierarchical clustering that exhibits a mild increase in the cost objective.
Metrics. In our results we track the approximate cost objective increase as follows: Let  $G$  be our given graph,  $T$  be average-linkage's output, and  $T'$  be Algorithm 2's output. We then measure the ratio  $\text{RATIO}_{\text{cost}} = \frac{\text{cost}_G(T')}{\text{cost}_G(T)}$ . We additionally quantify the fairness that results from application of our algorithm by reporting the balances of each cluster in the final hierarchical clustering, where true fairness would match the color proportions of the underlying dataset.
Results. We first demonstrate how our algorithm adapts an unfair hierarchy into one that achieves fair representation of the protected attributes as desired in the original problem formulation.
In Figure 3, we depict the cluster balances of an unfair hierarchical clustering algorithm, namely "average-linkage", and subsequently demonstrate that our algorithm effectively concentrates all clusters around the underlying data balance. In particular, we first apply the algorithm and then show how we the balance is further refined by tuning the parameters. The application of Algorithm 2 dramatically improves the representation of the protected attributes in the final clustering and, as such, firmly resolves the problem of achieving fairness.
While reaching this fair partitioning of the data is the overall goal, we further demonstrate that, in modifying the unfair clustering, we only increase the cost approximation by a modest amount. Figure 4 illustrates the change in relative cost as we increase the sample size  $n$ , the primary influence on our theoretical cost guarantees of Section 3. Specifically, we vary  $n$  in {128, 256, 512, 1024, 2048} and compute 10 replications (on different random seeds) of the fair hierarchical clustering procedure. Figure 4 depicts the mean relative cost of these replications with standard error bars. Notably, we see that the cost does increase with  $n$  as expected, but the increase relative to the unfair cost obtain by average linkage is only by a small multiplicative factor.
Figure 4: Relative cost of the fair hierarchical clustering resulting from Algorithm 2 compared to the unfair clustering as a function of the sample size  $n$ .
As demonstrated through this experimentation, the simplistic procedure of Algorithm 2 not only ensures the desired fairness properties absent in conventional (unfair) clustering algorithms but accomplishes this feat with a negligible rise in the overall cost. These results further highlight the immense value of our work.
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard S. Zemel. Fairness through awareness. In Shafi Goldwasser, editor, Innovations in Theoretical Computer Science 2012, Cambridge, MA, USA, January 8-10, 2012, pages 214-226. ACM, 2012.
Marina Knittel, Max Springer, John P. Dickerson, and MohammadTaghi Hajiaghayi. Generalized reductions: Making any hierarchical clustering fair and balanced with low cost, 2023.
Alexander Kraskov, Harald Stögbauer, Ralph G. Andrzejak, and Peter Grassberger. Hierarchical clustering using mutual information. CoRR, q-bio.QM/0311037, 2003.
Heidi Ledford. Millions of black people affected by racial bias in healthcare algorithms. Nature, 2019.
Benjamin Moseley and Joshua Wang. Approximation bounds for hierarchical clustering: Average linkage, bisecting  $k$ -means, and local search. In NIPS, pages 3094-3103, 2017.
Rohan Ramanath, Monjit Choudhury, and Kalika Bali. Entailment: An effective metric for comparing and evaluating hierarchical and non-hierarchical annotation schemes. In Stefanie Dipper, Maria Liakata, and Antonio Pareja-Lora, editors, Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse, LAW-ID@ACL 2013, August 8-9, 2013, Sofia, Bulgaria, pages 42-50. The Association for Computer Linguistics, 2013.
AN Selvan, LM Cole, L Spackman, S Naylor, and Wright C. Hierarchical cluster analysis to aid diagnostic image data visualization of ms and other medical imaging modalities. In Molecular Biotechnology, 2005.
Latanya Sweeney. Discrimination in online ad delivery. ACM Queue, 2013.