Paper: Demystifying Local & Global Fairness Trade-offs in Federated Learning Using Partial Information Decomposition (ICLR-2024)

Reviewer comment (Presentation):
The presentation is difficult to follow and the paper should be rewritten to give an example of the way the theorems and lemmas are proved.

Author response:
**Presentation of the paper**

We have rearranged the paper as follows:

Preliminary

* Background on Federated learning 

* Background on Partial Information Decomposition 

* Intuitive example to understand PID terms
* Formal PID definitions 

Main Results 

* Formalizing Global and Local Disparity in FL, Lemma 1, and 2 

* PID of Global and Local Disparity Proposition 1

* Canonical examples to understand decomposition 

* Fundamental limits on tradeoffs between Local and Global Disparity

* An optimization framework for exploring the accuracy-fairness trade-off

We have provided some intuitive examples in the preliminary section to facilitate understanding of PID. We have briefly described a few proofs in the main paper and referred to the location of full proofs in the Appendix. Section 3.2 on fundamental limits has been updated to highlight key results (Theorem 3 and previous Lemma 3 merged, Interaction information definition has been moved to Appendix B). We have further elaborated the section on convex optimization formulation (AFGLOP).

Reviewer comment (Presentation):
The general approach was confusing until I read the examples in Section 3.1. These examples aren't results, so they should be moved up to the preliminaries section to facilitate understanding.

Author response:
**Presentation of the paper**

We have rearranged the paper as follows:

Preliminary

* Background on Federated learning 

* Background on Partial Information Decomposition 

* Intuitive example to understand PID terms
* Formal PID definitions 

Main Results 

* Formalizing Global and Local Disparity in FL, Lemma 1, and 2 

* PID of Global and Local Disparity Proposition 1

* Canonical examples to understand decomposition 

* Fundamental limits on tradeoffs between Local and Global Disparity

* An optimization framework for exploring the accuracy-fairness trade-off

We have provided some intuitive examples in the preliminary section to facilitate understanding of PID. We have briefly described a few proofs in the main paper and referred to the location of full proofs in the Appendix. Section 3.2 on fundamental limits has been updated to highlight key results (Theorem 3 and previous Lemma 3 merged, Interaction information definition has been moved to Appendix B). We have further elaborated the section on convex optimization formulation (AFGLOP).

Reviewer comment (Theory):
Lemma 1 and Lemma 2 are results proved by the authors but they appear in the preliminaries section. This was confusing, especially because there was no discussion of how they were proved.

Author response:
**Presentation of the paper**

We have rearranged the paper as follows:

Preliminary

* Background on Federated learning 

* Background on Partial Information Decomposition 

* Intuitive example to understand PID terms
* Formal PID definitions 

Main Results 

* Formalizing Global and Local Disparity in FL, Lemma 1, and 2 

* PID of Global and Local Disparity Proposition 1

* Canonical examples to understand decomposition 

* Fundamental limits on tradeoffs between Local and Global Disparity

* An optimization framework for exploring the accuracy-fairness trade-off

We have provided some intuitive examples in the preliminary section to facilitate understanding of PID. We have briefly described a few proofs in the main paper and referred to the location of full proofs in the Appendix. Section 3.2 on fundamental limits has been updated to highlight key results (Theorem 3 and previous Lemma 3 merged, Interaction information definition has been moved to Appendix B). We have further elaborated the section on convex optimization formulation (AFGLOP).

Reviewer comment (Writing):
Unique information is used in the preliminaries before it is defined in Definition 3. This definition was not helpful, and there are no similar definitions for redundant information or synergistic information. These three quantities should be defined in a direct and similar way. As a possible approach, perhaps something like I(Z,Y|A ∩ B) could represent redundant information and I(Z,Y|A ∪ B) could represent synergistic information. The notation used was found excessive.

Author response:
**Definitions for redundant information or synergistic information** 

We have made some edits in the preliminary section on Page 3 to clarify this further.

We only provide the formal definition of $\text{Uni(Z:A|B)}$ because defining any one of the PID terms suffices to get the others. $\text{Red}(Z:A, B)$ is the sub-volume between $I(Z;A)$ and $I(Z;B)$ (see Fig.1). Hence,  $\text{Red}(Z:\hat{Y}, S) = I(Z;\hat{Y}) - \text{Uni}(Z: \hat{Y}| S)$ and $\text{Syn}(Z:\hat{Y},S) =  I(Z; \hat{Y}, S) - \text{Uni}(Z: \hat{Y}|S) -\text{Uni}(Z: \hat{Y}|S)-\text{Red}(Z:\hat{Y}, S)$ (from Equation 1).

Reviewer comment (Evaluation):
While the experiments seemed comprehensive, the review was missing a discussion about what was interesting in them. The paper could highlight interesting observations and findings from the experiments, and provide examples of how the mutual information perspective gives insight into local and global fairness.

Author response:
**On the interesting observations from our experiments**

We are grateful for the reviewer's recognition of the thoroughness of our experiments.  Our experimental results reinforces our findings and highlight the various scenarios in which unique, redundant, and masked disparities can occur in a practical setting. Our information-theoretic framework provides a nuanced understanding of the three sources of disparity in FL. Our experiments offer insights into the agreement and disagreement between local and global fairness under various data distributions. Our experiments and theoretical results show that depending on the data distribution achieving one (global or local fairness) can often come at the cost of the other (disagreement), emphasizing the need for global and local fairness to be considered together when addressing group fairness problems in FL. The way data is distributed across clients significantly impacts the type of disparity (unique, redundant, masked) that dominates. Our optimization framework establishes the accuracy-fairness tradeoffs for a given dataset and client distribution. We have updated the discussions on Page 9. 

---
 We summarize more observations here (also added to Appendix F on Page 25):

When data is uniformly distributed across clients, unique disparity is dominant and contributes to both global and local unfairness (see Figure 4 Scenario 1: model trained using FedAvg on the adult dataset and distributed uniformly across clients). In the trade-off Pareto Front  (Figure 3, Row 1), we see that both local and global fairness constraints have balanced tradeoffs with accuracy. The PID decomposition (Figure 4, Row 1, column 2,3,4) explains this as we see the disparity is mainly unique disparity, with little redundant or masked disparity. The unique disparity highlights where local and global disparity are in agreement. 

In the case with sensitive attribute heterogeneity (sensitive attribute imbalance across clients). The disparity observed is mainly redundant disparity (see Figure 4 scenario 2 and middle), this is a globally unfair but locally fair model (recall proposition 1). Observe in the tradeoff plot (see Figure 3, Row 2) that the accuracy trade-off is with mainly global fairness (an accurate model could have zero local disparity but be globally unfair). 

In the cases with sensitive-attribute synergy across clients. For example, in a two-client case (one client is more likely to have qualified women and unqualified men and vice versa at the other client). We observe that the mask disparity is dominant (see Figure 4 Scenario 3). The trade AGLFOP tradeoff plot (see Figure 3, Row 3) is characterized by Masked Disparity with trade-offs mainly between local fairness and accuracy (an accurate model could have zero global disparity but be locally unfair). 

The AGLFOP provides the theoretical boundaries trade-offs, capturing the optimal performance any model or FL technique can achieve for a specified dataset and client distribution. For example, say one wants a perfectly globally fair and locally fair model, i.e., ($\epsilon_g=0$, $\epsilon_l=0$). Under high sensitive attribute heterogeneity (see Figure 3 Row 2), they cannot have a model that does better than 64% accuracy on the dataset.

Reviewer comment (Theory):
Lemma 1 upper bounds statistical parity with mutual information, but how loose is the upper bound?

Author response:
**Regarding the tightness of this Lemma 1**: The only inequality used in the proof of Lemma 1 is the Pinsker’s inequality:  $TV(P,Q) \leq \sqrt{0.5KL(P,Q)}$ [6]. Pinsker’s inequality is known to be tight for $KL\leq 1$  (we refer to a short paper on the Tightness of Pinsker’s Bound [6]). We have added Remark 5 in Appendix B Page 15 to highlight this.

*Remark on Tightness of Lemma 1:*
Since our proof exclusively utilizes Pinsker's inequality, their tightness is equivalent. The mutual information (which can also be defined as a KL divergence) is upper bounded by the entropy $H(\hat{Y})$, that is $ I(Z;\hat{Y}) \leq \text{min}$ { $H(Z), H(\hat{Y})$}$ \leq H(\hat{Y})$. In binary classification  $ H(\hat{Y}) \leq 1 $. Hence, we have $ I(Z;\hat{Y}) \leq 1 $  which is aligned with the known tight regime of Pinsker's inequality (i.e., $KL \leq 1$) [6]. The inequality gets tighter with smaller mutual information $I(Z;\hat{Y})$ values [6].

Reviewer comment (Evaluation):
What are interesting observations from your experiments?

Author response:
**On the interesting observations from our experiments**

We are grateful for the reviewer's recognition of the thoroughness of our experiments.  Our experimental results reinforces our findings and highlight the various scenarios in which unique, redundant, and masked disparities can occur in a practical setting. Our information-theoretic framework provides a nuanced understanding of the three sources of disparity in FL. Our experiments offer insights into the agreement and disagreement between local and global fairness under various data distributions. Our experiments and theoretical results show that depending on the data distribution achieving one (global or local fairness) can often come at the cost of the other (disagreement), emphasizing the need for global and local fairness to be considered together when addressing group fairness problems in FL. The way data is distributed across clients significantly impacts the type of disparity (unique, redundant, masked) that dominates. Our optimization framework establishes the accuracy-fairness tradeoffs for a given dataset and client distribution. We have updated the discussions on Page 9. 

---
 We summarize more observations here (also added to Appendix F on Page 25):

When data is uniformly distributed across clients, unique disparity is dominant and contributes to both global and local unfairness (see Figure 4 Scenario 1: model trained using FedAvg on the adult dataset and distributed uniformly across clients). In the trade-off Pareto Front  (Figure 3, Row 1), we see that both local and global fairness constraints have balanced tradeoffs with accuracy. The PID decomposition (Figure 4, Row 1, column 2,3,4) explains this as we see the disparity is mainly unique disparity, with little redundant or masked disparity. The unique disparity highlights where local and global disparity are in agreement. 

In the case with sensitive attribute heterogeneity (sensitive attribute imbalance across clients). The disparity observed is mainly redundant disparity (see Figure 4 scenario 2 and middle), this is a globally unfair but locally fair model (recall proposition 1). Observe in the tradeoff plot (see Figure 3, Row 2) that the accuracy trade-off is with mainly global fairness (an accurate model could have zero local disparity but be globally unfair). 

In the cases with sensitive-attribute synergy across clients. For example, in a two-client case (one client is more likely to have qualified women and unqualified men and vice versa at the other client). We observe that the mask disparity is dominant (see Figure 4 Scenario 3). The trade AGLFOP tradeoff plot (see Figure 3, Row 3) is characterized by Masked Disparity with trade-offs mainly between local fairness and accuracy (an accurate model could have zero global disparity but be locally unfair). 

The AGLFOP provides the theoretical boundaries trade-offs, capturing the optimal performance any model or FL technique can achieve for a specified dataset and client distribution. For example, say one wants a perfectly globally fair and locally fair model, i.e., ($\epsilon_g=0$, $\epsilon_l=0$). Under high sensitive attribute heterogeneity (see Figure 3 Row 2), they cannot have a model that does better than 64% accuracy on the dataset.

Reviewer comment (Reproducibility):
Although this paper proposes an optimization framework with Definition 5, it does not give any solution or algorithm for solving the problem.

Author response:
**W1: Solution and algorithm for solving optimization**  

We show that the AGLFOP is a convex optimization problem and hence can be solved efficiently using standard convex solvers (which are well-established and robust, to find globally optimal solutions efficiently). This optimization problem can be computed in any FL environment. Specifically, their computation necessitates the characterization of the joint distribution $\Pr(Z{=}z,S{=}s,Y{=}y)= \Pr(S{=}s)\Pr(Z{=}z|S{=}s)\Pr(Y{=}y|Z{=}z,S{=}s)$, which can be acquired by aggregating pertinent statistics across all participating clients. For instance, $\Pr(S=s)$ denotes the proportion of data at client $s$, $\Pr(Z=z|S=s)$ signifies the fraction of individuals with sensitive attribute $z$ at client $s$, and $\Pr(Y=y|Z=z,S=s)$ represents the proportion of individuals labeled with true label $y$ among those with sensitive attribute $z$ at client $s$.

Reviewer comment (Experiments):
The experiments are somehow weak, both the baseline and dataset are rare. There are other works focusing on FL group fairness like [1,2], and also about fairness and accuracy tradeoffs, like [3], that should be compared.

Author response:
**Regarding the comparison with the works [1,2, 3]**

Our paper's aim diverges from these works. While [1,2] predominantly focuses on mitigating unfairness in FL, our work delves into theoretically exploring the fundamental limitations on the trade-offs between accuracy-global and local fairness. Our work establishes the theoretical limits of what any FL technique can achieve in terms of accuracy and fairness given a dataset and client distribution.

Similarly in [3], the definition and scope of **fairness** differ substantially from our work (we highlight the different notions of fairness in FL in the related works section and have cited [3]). [3] concentrates on **client fairness**, emphasizing model performance is consistent across clients. In contrast, our work is centered around **group fairness**, particularly addressing model discrimination against sensitive attributes (e.g, race, gender). 

---

**Clarification on Accuracy-Global and Local Fairness Tradeoff  (W2, W4, Q3, Q4)**

We introduce the Accuracy and Global-Local Fairness Optimality Problem (AGLFOP) as an optimization to delineate the theoretical boundaries of accuracy and fairness trade-offs, capturing the optimal performance any model or FL technique can achieve for a specified dataset and client distribution.

We aim to find the maximum achievable accuracy, quantified as $1 - c(Q)$, within the context of specific constraints on global and local disparities in fairness ($I(Z;\hat{Y}) \leq \epsilon_g$) for global disparity and \($I(Z;\hat{Y}|S) \leq \epsilon_l$) for local disparity. Here, $\epsilon_g$ and $\epsilon_l$ are fairness relaxation constants and not hyperparameters.
For example, consider the scenario where both $\epsilon_l = 0$ and $\epsilon_g = 0$. This represents a condition of perfect fairness both locally and globally. In such a case, our objective is to determine the highest level of accuracy that a model can achieve while adhering to these ideal global and local fairness constraints.

Reviewer comment (Presentation):
The visualization (table or figure) of accuracy and global-local fairness trade-off results is relatively insufficient relying solely on the Pareto Frontiers shown in Figure 3.

Author response:
**W3, Q3: Visualization of accuracy and global-local fairness trade-off using Pareto Frontiers**

The use of Pareto Frontiers is a well-established method for trade-off analysis in optimization, offering a clear and concise representation of the optimal trade-offs between competing objectives. In Figure 3, we provide the AGLFOP Pareto Frontiers showing maximum accuracy (shown as contour) that can be achieved on a dataset and client distribution for a given global and local fairness various relaxation $\epsilon_g$ ($x$-axis) and $\epsilon_l$ ($y$-axis) respectively. This helps visualize the effects of global and local fairness constraints on the accuracy of the model (i.e., Accuracy-global local fairness tradeoff). This method allows for a comprehensive overview of the optimal solutions, facilitating informed decision-making about the trade-offs that are inherent to the model's performance under fairness constraint.

Reviewer comment (Experiments):
It seems of vital importance to properly set the hyper-parameters $\epsilon_g$ and $\epsilon_L$ for the optimal trade-off. The acc-fairness Trade-off figure displayed in Figure 3 lacks discussion based on more experimental settings and datasets.

Author response:
**Clarification on Accuracy-Global and Local Fairness Tradeoff  (W2, W4, Q3, Q4)**

We introduce the Accuracy and Global-Local Fairness Optimality Problem (AGLFOP) as an optimization to delineate the theoretical boundaries of accuracy and fairness trade-offs, capturing the optimal performance any model or FL technique can achieve for a specified dataset and client distribution.

We aim to find the maximum achievable accuracy, quantified as $1 - c(Q)$, within the context of specific constraints on global and local disparities in fairness ($I(Z;\hat{Y}) \leq \epsilon_g$) for global disparity and \($I(Z;\hat{Y}|S) \leq \epsilon_l$) for local disparity. Here, $\epsilon_g$ and $\epsilon_l$ are fairness relaxation constants and not hyperparameters.
For example, consider the scenario where both $\epsilon_l = 0$ and $\epsilon_g = 0$. This represents a condition of perfect fairness both locally and globally. In such a case, our objective is to determine the highest level of accuracy that a model can achieve while adhering to these ideal global and local fairness constraints.

Reviewer comment (Novelty):
Could you explain the main difference from [4]? It seems it is a trivial improvement (Apply the PID analysis on FL) compared with this paper.

Author response:
**Q1: Difference between Survey Paper and our Paper**

It is important to note that [1] is a survey paper that surveys other papers that apply partial information decomposition (PID) in algorithmic fairness and explainability and provides a taxonomy. PID is a mathematical tool whose origin goes back to [2] and has been used previously in neuroscience. Our contribution is in leveraging PID specifically for the setup of group fairness in FL and using it to decompose disparities as well as formulate a novel convex optimization problem to characterize tradeoffs.

Reviewer comment (Experiments):
Could you provide experiments of different baselines and different datasets, in the FL setting (partial client participation of cross-device FL.)

Author response:
**Q2: On partial client participation of cross-device FL**

Our approach does not make specific assumptions about the training process of the federated learning model, which allows our results to be applicable to scenarios involving partial client participation in cross-device federated learning methods as well.  This flexibility is a significant aspect of our contribution, as it allows our methods to be adapted and applied in diverse federated learning contexts.

Reviewer comment (Experiments):
Could you provide more results for trade-offs on accuracy and global-local fairness?

Author response:
We thank the reviewer for appreciating our rebuttal!

*The goal of the AGLOP is to theoretically study the tradeoff between accuracy and global local fairness for a specified dataset and client distribution rather than provide a state-or-the-art algorithm*. To the best of our knowledge, there are no suitable baselines to compare with that quantify such fundamental tradeoffs between local and global fairness. Nonetheless, we ran some methods that attempt to minimize only the global fairness on the Adult dataset. For these methods, we measure the global and local disparities for various data splits and see that the accuracy achieved by the models is lower than the maximum accuracy limit for the given local and global disparity relaxation using our proposed optimization. We can include a remark regarding this observation in the paper as recommended by the reviewers.








| Method                | **Local Disparity** | **Global Disparity** | **Accuracy** |**AGLFOP**             |
|-----------------------------|---------------------|----------------------|------------------|-----------------------|
| FedFB (scenario 1)             | 0.0249     	   |   0.0241            | 81%                | 86%       	|
| FedFB (scenario 1)             | 0   		  |   0      	       | 75%         | 82%     		 |
| FairBatch FL (scenario 1)             | 0.0019              | 0.0019               | 81%            | 83%                  |
| FedAvg (scenario 2)     | 0.0180              | 0.0335               | 84%               | 86%                |
| FedAvg  (scenario 3)                 | 0.0148              | 0.0067     | 76%           | 78%      |

Scenario 1:  sensitive-attribute independently distributed across clients

Scenario 2: high sensitive-attribute heterogeneity across clients ($\alpha = 0.67$)

Scenario 3: high sensitive-attribute synergy level across clients ($\lambda = 0.90$)

**Methods**

FedAvg (+Local Fair Training): Communication-Efficient Learning of Deep Networks from Decentralized Data, https://proceedings.mlr.press/v54/mcmahan17a.html

FedFB: Improving Fairness via Federated Learning, https://arxiv.org/abs/2110.15545

FairBatch FL (FairBatch Training + Ensemble): FairBatch: Batch Selection for Model Fairness, https://arxiv.org/abs/2012.01696

Reviewer comment (Theory):
Overall this paper is well-written, but in Section 2, it would be great if the authors could provide some more justifications for Definitions 1 and 2, both mathematically and conceptually, even if the definitions are indeed fairly intuitive. This may be immensely helpful for readers especially those who do not have a strong background in information theory.

Author response:
**Justifications for Definitions 1 and 2 (mathematically and conceptually)**

Let's consider a case involving three banks training a federated model for credit scoring: Bank 1, predominantly serving men; Bank 2, catering to a diverse demographic; and Bank 3, primarily serving women. Global fairness means that the model doesn’t discriminate against any protected group when evaluated across all client (Bank) data. Meanwhile, local fairness means fairness within each client’s data (fairness at each Bank). 

In this work, we use mutual information as a measure of the unfairness or disparity of a model. 
Mutual Information has been interpreted as the dependence between sensitive attribute $Z$  and model prediction $\hat{Y}$ (captures correlation as well as all non-linear dependencies). Mutual information is zero if and only if $Z$ and $\hat{Y}$ are independent. This means that if the model’s predictions are highly correlated with sensitive attributes (like gender or race), that’s a sign of unfairness. Mutual information has been explored in fairness in the context of centralized machine learning in [1,2,3]. 

A recent work [5] provides another interpretation of mutual information $I(Z; \hat{Y})$ in fairness as the accuracy of predicting $Z$ from $\hat{Y}$  (or the expected probability of error in correctly guessing $Z$ from $\hat{Y}$). Even in information bottleneck literature [6], mutual information has been interpreted as a measure of how well one random variable predicts (or, aligns with) the other. 

For local fairness, we are interested in the dependence between model prediction $ \hat{Y} $ and sensitive attributes $Z$ at each and every client, i.e., the dependence between $\hat{Y}$ and $Z$ conditioned on the client $S$. For example, the disparity at client $S=1$ (Bank 1) is $I(Z; \hat{Y}|S=1)$ (the mutual information (dependence) between model prediction and sensitive attribute conditioned on client $S=1$ (considering data at client $S=1$). Our measure for local disparity (Definition 2) is the conditional mutual information (dependence) between $Z$ and $\hat{Y}$ conditioned on $S$, denoted as $I(Z;\hat{Y}|S)$. Local disparity $I(Z;\hat{Y}|S) =\sum_s p(s)I(Z; \hat{Y}|S=s)$, is an average of the disparity at each client weighted by the $p(s)$, the fraction of data at client $S=s$.  The local disparity is only zero if and only if all client has zero disparity in their local dataset.

While other works [3] have used mutual information measures to enforce statistical parity, we are the first to bring Prinsker’s Inequality [4] in the context of fairness to show that the statistical parity gap is actually upper bounded by the square root of mutual information (see Lemma 1). Though this is not the main result of the paper, it further justifies the use of mutual information measures to study global and local fairness in federated learning. 

We thank the reviewer for their valuable feedback on making our work accessible to readers without strong backgrounds in information theory. We have included a brief background on concepts and definitions in Appendix A on Page 14.




[1]Fairness-Aware Classifier with Prejudice Remover Regularizer https://link.springer.com/chapter/10.1007/978-3-642-33486-3_3

[2] A Fair Classifier Using Mutual Information, https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9174293

[3] INFOFAIR: Information-Theoretic Intersectional Fairness  https://arxiv.org/pdf/2105.11069.pdf

[4] https://blog.wouterkoolen.info/Pinsker/post.pdf

[5] Can Information Flows Suggest Targets for Interventions in Neural Circuits? https://arxiv.org/abs/2111.05299

[6]The Information Bottleneck Problem and Its Applications in Machine Learning  https://arxiv.org/pdf/2004.14941.pdf

Reviewer comment (Theory):
This was mentioned in the Weaknesses section, and I would be very interested in seeing more explanations for choosing those definitions.

Author response:
**Justifications for Definitions 1 and 2 (mathematically and conceptually)**

Let's consider a case involving three banks training a federated model for credit scoring: Bank 1, predominantly serving men; Bank 2, catering to a diverse demographic; and Bank 3, primarily serving women. Global fairness means that the model doesn’t discriminate against any protected group when evaluated across all client (Bank) data. Meanwhile, local fairness means fairness within each client’s data (fairness at each Bank). 

In this work, we use mutual information as a measure of the unfairness or disparity of a model. 
Mutual Information has been interpreted as the dependence between sensitive attribute $Z$  and model prediction $\hat{Y}$ (captures correlation as well as all non-linear dependencies). Mutual information is zero if and only if $Z$ and $\hat{Y}$ are independent. This means that if the model’s predictions are highly correlated with sensitive attributes (like gender or race), that’s a sign of unfairness. Mutual information has been explored in fairness in the context of centralized machine learning in [1,2,3]. 

A recent work [5] provides another interpretation of mutual information $I(Z; \hat{Y})$ in fairness as the accuracy of predicting $Z$ from $\hat{Y}$  (or the expected probability of error in correctly guessing $Z$ from $\hat{Y}$). Even in information bottleneck literature [6], mutual information has been interpreted as a measure of how well one random variable predicts (or, aligns with) the other. 

For local fairness, we are interested in the dependence between model prediction $ \hat{Y} $ and sensitive attributes $Z$ at each and every client, i.e., the dependence between $\hat{Y}$ and $Z$ conditioned on the client $S$. For example, the disparity at client $S=1$ (Bank 1) is $I(Z; \hat{Y}|S=1)$ (the mutual information (dependence) between model prediction and sensitive attribute conditioned on client $S=1$ (considering data at client $S=1$). Our measure for local disparity (Definition 2) is the conditional mutual information (dependence) between $Z$ and $\hat{Y}$ conditioned on $S$, denoted as $I(Z;\hat{Y}|S)$. Local disparity $I(Z;\hat{Y}|S) =\sum_s p(s)I(Z; \hat{Y}|S=s)$, is an average of the disparity at each client weighted by the $p(s)$, the fraction of data at client $S=s$.  The local disparity is only zero if and only if all client has zero disparity in their local dataset.

While other works [3] have used mutual information measures to enforce statistical parity, we are the first to bring Prinsker’s Inequality [4] in the context of fairness to show that the statistical parity gap is actually upper bounded by the square root of mutual information (see Lemma 1). Though this is not the main result of the paper, it further justifies the use of mutual information measures to study global and local fairness in federated learning. 

We thank the reviewer for their valuable feedback on making our work accessible to readers without strong backgrounds in information theory. We have included a brief background on concepts and definitions in Appendix A on Page 14.




[1]Fairness-Aware Classifier with Prejudice Remover Regularizer https://link.springer.com/chapter/10.1007/978-3-642-33486-3_3

[2] A Fair Classifier Using Mutual Information, https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9174293

[3] INFOFAIR: Information-Theoretic Intersectional Fairness  https://arxiv.org/pdf/2105.11069.pdf

[4] https://blog.wouterkoolen.info/Pinsker/post.pdf

[5] Can Information Flows Suggest Targets for Interventions in Neural Circuits? https://arxiv.org/abs/2111.05299

[6]The Information Bottleneck Problem and Its Applications in Machine Learning  https://arxiv.org/pdf/2004.14941.pdf

Reviewer comment (Reproducibility):
Under a concrete data set, how are Uni(), Red(), and Syn() efficiently computed? I might be wrong, but my first impression is that, since they are relevant with mutual information, such computation may be expensive?

Author response:
**On the Computation of Uni, Red, Syn**

The computation of Uni, Red, Syn disparity necessitates the characterization of the joint distribution $\Pr(Z{=}z,S{=}s,\hat{Y}{=}\hat{y})=\Pr(S{=}s)\Pr(Z{=}z|S{=}s)\Pr(\hat{Y}{=}\hat{y}|Z{=}z,S{=}s)$, which can be acquired by aggregating pertinent statistics across all participating clients.  For instance, $\Pr(S=s)$ denotes the proportion of data at client $s$, $\Pr(Z=z|S=s)$ signifies the fraction of individuals with sensitive attribute $z$ at client $s$, and $\Pr(\hat{Y}=\hat{y}|Z=z,S=s)$ represents the proportion of individuals with predictived label $\hat{y}$ among those with sensitive attribute $z$ at client $s$. 


The global and local disparity is first measured empirically using the classical definition of mutual information, i.e., $ I(Z; \hat{Y}) = \sum_{z , \hat{y} } p(z, \hat{y}) \log \frac{p(z, \hat{y})}{p(z)p(\hat{y})}$ and conditional mutual information, i.e.,  ${ I(Z; \hat{Y} | S) = \sum_{s, z , \hat{y}} p(s, z, \hat{y}) \log \frac{p(z, \hat{y} | s)}{p(z | s)p(\hat{y} | s)}}$. For the Unique disparity, we use Definition 3 which solves a convex optimization problem over the joint distribution $(Z,\hat{Y}, S)$ keeping the marginals over $(Z,\hat{Y}) $ and $(Z, S)$ fixed (we use an implementation from the Discrete Information Theory [1] python package) as shown in [2].

Defining any one of the PID terms suffices to get the others. $\text{Red}(Z:A, B)$ is the sub-volume between $I(Z;A)$ and $I(Z;B)$ (see Fig.1). Hence,  $\text{Red}(Z:\hat{Y}, S) = I(Z;\hat{Y}) - \text{Uni}(Z: \hat{Y}| S)$ and $\text{Syn}(Z:\hat{Y},S) =  I(Z; \hat{Y}, S) - \text{Uni}(Z: \hat{Y}|S) -\text{Uni}(Z: S|\hat{Y})-\text{Red}(Z:\hat{Y}, S)$ (from Equation 1). 

This estimation largely depends on: 

(i) the empirical estimators of the probability distributions; and

 (ii) the efficiency of the convex optimization algorithm used for calculating the unique information. 

Convex optimization problems are generally well-studied and have efficient solvers. In our experiments on the Adult dataset with $2$,$5$, and $10$ federated clients, estimating the disparity terms was not computationally expensive.  However, the method's computational cost can increase with the number of clients or sensitive attributes. We have highlighted this limitation in the paper (see Discussion on Page 9). There's also emergent interest in the estimation of these measures using Gaussian relaxations [3] or neural networks [4,5]. 


[1] dit: a Python package for discrete information theory, doi.org/10.21105/joss.00738

[2] Quantifying unique information, arxiv.org/abs/1311.2852

[3] Partial Information Decomposition via Deficiency for Multivariate Gaussians, arxiv.org/abs/2105.00769

[4] MINE: Mutual Information Neural Estimation, arxiv.org/abs/1801.04062

[5] Redundant Information Neural Estimation, www.mdpi.com/1099-4300/23/7/922