Paper: FedEBA+: Towards Fair and Effective Federated Learning via Entropy-based Model (ICLR-2024)

Reviewer comment (Experiments):
[Major] It is unclear which contributes to the improved performance, the optimization algorithm, or the entropy-based fairness regularization.

Author response:
Thank you for your comment.

We would like to clarify that 

- FedEBA+ is an optimization algorithm comprising two parts: the proposed entropy-based aggregation (derived from Eq 2) and the proposed alignment update (derived from Eq 6). The alignment update includes both model alignment (Sec 4.2.1, Line 45 of Algorithm 1) and gradient alignment (Sec 4.2.1, Line 45 of Algorithm 1). Although these are distinct components, they collectively form one part of our optimization algorithm. Thus, the optimization algorithm, integrating a new aggregation and a new alignment update process, contributes to improved performance.
- The ablation results of FedEBA+ demonstrate that each step, i.e., aggregation and alignment update, is beneficial to improved performance, as shown in Table 1 of the original submission. We present the ablation study here to illustrate that both aggregation and alignment updates contribute to the performance:
    - FedEBA corresponds to the algorithm that only uses entropy-based aggregation compared with FedAvg; FedEBA+ indicates that the algorithm uses both entropy-based aggregation and alignment update.
    - Comparing FedEBA with FedAvg, FedEBA improves accuracy by about 1.6 and fairness by about 14.3 on CIFAR-10, highlighting the proposed aggregation approach’s advantages.
    - comparing FedEBA+ and FedEBA, FedEBA+ improves accuracy by about 3.4 and fairness by about 20.8 on CIFAR-10, showing the effectiveness of the proposed alignment update approach.
    - Similar comparisons can be observed in FashionMNIST, CIFAR-100, and Tiny-ImageNet.

|  | CIFAR-10 |  |  |  | FashionMNIST |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Algorithm | Global Acc. | Var. | Worst 5% | Best 5% | Global Acc. | Var. | Worst 5% | Best 5% |
| FedAvg | 67.79±0.35 | 103.83±10.46 | 45.00±2.83 | 85.13±0.82 | 86.49 ± 0.09 | 62.44±4.55 | 71.27±1.14 | 95.84± 0.35 |
| FedEBA | 69.38±0.52   | 89.49±10.95 | 50.40±1.72  | 86.07±0.90 | 86.70±0.11 | 50.27±5.60 | 71.13±0.69 | 95.47±0.27 |
| FedEBA+ | 72.75±0.25  | 68.71±4.39  | 55.80±1.28  | 86.93±0.52 | 87.50±0.19 | 43.41±4.34 | 72.07±1.47 | 95.91±0.19 |
|  | **CIFAR-100** |  |  |  | **Tiny-ImageNet** |  |  |  |
| Algorithm | Global Acc. | Var. | Worst 5% | Best 5% | Global Acc. | Var. | Worst 5% | Best 5% |
| FedAvg | 30.94±0.04 | 17.24±0.08 | 0.20±0.00 | 65.90±1.48 | 61.99±0.17 | 19.62±1.12 | 53.60±0.06 | 71.18±0.13 |
| FedEBA | 32.38±0.13 | 17.09±0.06 | 0.75±0.22 | 66.40±0.47 | 63.34±0.25 | 15.29±1.36 | 54.17±0.04 | 70.98±0.10 |
| FedEBA+ | 33.39±0.22 | 16.92±0.04 | 0.95±0.15 | 68.51±0.21 | 64.05±0.09 | 14.91±1.85 | 54.32±0.09 | 71.27±0.04 |

The above Table has been added as Table 11 of the revised version.

Reviewer comment (Theory):
The communication overhead appears to be much higher than FedAvg in Algorithm 1. According to Line 3 and Line 9, clients need to communicate with the server twice at each round t. This contradicts the traditional FedAvg that only communicates once at each round.

Author response:
Thank you for your suggestions. We'd like to clarify the communication issue in Algorithm 1:

- **Communication time is less than twice that of FedAvg.** Apologies for the unclear description of Algorithm 1 in our original submission, which caused confusion that clients need to communicate with the server twice. We have carefully revised descriptions of Algorithm 1 in the revised manuscript. Actually, as shown in Algorithm 1 of the revised version, only the communication capacity is sufficient (Line 3) and the arccos between clients performance and $\mathbf{1}$ is larger than the threshold (Line 5), the server needs one additional communication (Line 6) compared with FedAvg, otherwise the communication cost is not doubled.
    - **Communication cost is limited and acceptable:** (1) In Line 3, the client updates only the loss, with negligible cost (for example, in our CIFAR-10 experiment, the communication cost of a loss is $e^{-6}$MB) compared to the model communication. (2) In Line 11, additional cost involves uploading one-step gradients, but, as per our $\theta$ ablation study, this cost is minimal and leads to satisfactory results. For instance, as shown in Appendix 11, Table 5, in the original submission, on FashionMNIST, when the additional communication ratio (additional communication as a percentage of total communications) is 0 ($\theta=90^\circ$) or 2.1% ($\theta=45^\circ$), FedEBA+ achieves the accuracy 87% comparable with best baselines, and improves the variance to minimal 48.91, far outperforming others.
- **FedEBA+ differs from FedAvg for a reason:** FedEBA+ aims to improve accuracy and ensure fairness simultaneously. FedEBA+ can be reverted to the FedAvg framework by setting $\theta$ large enough, making Line 3 unnecessary. Even so, FedEBA+ outperforms FedAvg in both accuracy and fairness, as demonstrated in our $\theta$ ablation study in Table 5 of Appendix 15, in the revised manuscript.

We've updated Algorithm 1 in our revised version.

Reviewer comment (Reproducibility):
In Line 9, $\tilde{g}^t$ is the fair gradient of the selected clients obtained using one local update. However, there is no step before line 9 that collects the one local update from clients. The authors should clarify this process.

Author response:
Thank you for your thoughtful reminder. Sorry for the confusion caused by our abbreviated description of Algorithm 1 in the original submission. In the revised manuscript, we have amended Line 6  
“Calculate $\tilde{g}^t$ by (9)” 
into 

“Sever collects the gradient $\nabla F_i(x_t)$, calculates the ideal fair gradient $\tilde{g}^{b,t}$ by (9) and sends $\tilde{g}^{b,t}$ to selected clients.”

Reviewer comment (Theory):
In Algorithm 1, Line 10 indicates that Eq 3 will use the local model $x_{t,K}^j$ as $x$ to calculate the $p_j$. However, using local model $x_{t,K}^j$ for Eq 3 leads to different denominators $\sum_{i=1}^N \exp[F_i(x_{t,K}^j)/\tau]$ for different client $j$. Then $p_j$ for different clients is not normalized by the same denominator. I am not sure how it satisfies the constraint $\sum_j p_j=1$ in Eq 2. An explanation of how this approach satisfies the constraint would be helpful.

Author response:
Thank you for your suggestions. We would like to explain how this approach satisfies the constraint:

1. In Line 10 of Algorithm 1, Eq 3 corresponds to $p_i = \frac{\exp({F_i(x_{t,K}^i)/\tau})}{\sum_{j=1}^N\exp({F_j(x_{t,K}^i)/\tau})}$. Despite different clients having different $F_i(x_{t,K}^i)$, the denominators among clients are the same, i.e., $\sum_{j=1}^N\exp({F_j(x_{t,K}^i)/\tau})$.
2. We verify $\sum_i p_i = 1$ by $\sum_{i}p_i=p_1+p_2+\dots+p_N = \frac{\exp({F_1(x_{t,K}^1)/\tau})}{\exp({F_1(x_{t,K}^1)/\tau})+\exp({F_2(x_{t,K}^2)/\tau})+\dots+\exp({F_N(x_{t,K}^N)/\tau})}+\frac{\exp({F_2(x_{t,K}^2)/\tau})}{\exp({F_1(x_{t,K}^1)/\tau})+\exp({F_2(x_{t,K}^2)/\tau})+\dots+\exp({F_N(x_{t,K}^N)/\tau})}+\cdots+\frac{\exp({F_N(x_{t,K}^N)/\tau})}{\exp({F_1(x_{t,K}^1)/\tau})+\exp({F_2(x_{t,K}^2)/\tau})+\dots+\exp({F_N(x_{t,K}^N)/\tau})} $
$= \frac{\exp({F_1(x_{t,K}^1)/\tau})+\exp({F_2(x_{t,K}^2)/\tau})+\dots+\exp({F_N(x_{t,K}^N)/\tau})}{\exp({F_1(x_{t,K}^1)/\tau})+\exp({F_2(x_{t,K}^2)/\tau})+\dots+\exp({F_N(x_{t,K}^N)/\tau})}=1$.
3. Moreover, using the same reasoning for $i\in S_t$ instead of $i \in [N]$, the result remains unchanged, i.e., $\sum_{i\in S_t}p_i = 1$.

Reviewer comment (Writing):
It would be clearer if the authors could connect Eq 8 and Eq 10 to Eq 6, given that they share a similar form and intuition. For example, Eq 6 is not referenced in Section 4.2.2, thus making the purpose of Equation 10 unclear.

Author response:
Thank you for your suggestions. We would like to connect (1) Eq 8 to Eq 6 and (2) Eq 10 to Eq 6.

- Eq 6 is  $\frac{\partial L\left(x,p_i, \lambda_0, \lambda_1\right)}{\partial x} = (1-\alpha)\sum_{i=1}^m p_i \nabla F_i(x) + \alpha \nabla \tilde{f}(x)$.
    
    According to model update and local SGD, $x_{t+1}= x_t-\eta\frac{\partial L\left(x,p_i, \lambda_0, \lambda_1\right)}{\partial x}$ and $\eta\Delta_t=x_{t+1}-x_{t}$, the final global model update can be written as:
    
    $\Delta_t=-\frac{\partial L\left(x,p_i, \lambda_0, \lambda_1\right)}{\partial x}=-(1-\alpha)\sum_{i=1}^m p_i \nabla F_i(x) - \alpha \nabla \tilde{f}(x)$.
    
- Eq 8 is  $\tilde{\Delta}^a_t = \frac{1}{|S_t|}\sum_{i\in S_t} \tilde{\Delta_{t}}^{a,i}= \frac{1}{|S_t|}\sum_{i\in S_t}(x_{t,1}^i - x_{t,0}^i)$ . 
Here, the ideal gradient $\nabla \tilde{f}(x)$ of Eq 6 is expressed by $\nabla \tilde{f}^a(x_t)=\tilde{\Delta}t^a$*,* and thus the final model update is expressed as \
$\Delta_t = (1-\alpha) \sum{i\in S_t}p_i\Delta_t^i +\alpha \tilde{\Delta}^a_t$, \
completing the connection to Eq 6.
- Eq 10 is $h_{t,k}^i \gets (1-\alpha)g_{t,k}^i + \alpha \tilde{g}^{b,t}$.
Here, the ideal gradient $\nabla \tilde{f}(x)$ of Eq 6 is expressed by $\nabla \tilde{f}^b(x_t)=\tilde{g}^{b,t}=\sum\nolimits_{i\in S_t}\tilde{p_i} g_i^t$, and thus the final model update is expressed as \
 $\Delta_t=-(1-\alpha)\sum_{i\in S_t}p_i\eta_L\sum_{k=0}^{K-1}g_{t,k}^i-\alpha K \eta_L\sum_{i\in S_t}\tilde{p}ig_i^t$ \
by $\Delta_t=\sum{i\in S_t}p_i\Delta_t^i,\Delta_t^i=x_{t,K}^i-x_{t,0}^i=-\eta_L\sum_{k=0}^{K-1} h_{t,k}^i$, finishing the connection to Eq 6.

We have added these connections in the revised manuscript of Sec 4.2.1 and Sec 4.2.2, indicated by red text.

Reviewer comment (Theory):
The motivation of maximum entropy for fairness may need further clarification. Though the authors discussed fairness in the introduction and mentioned the Shannon entropy in Section 4.1, the connection between entropy and fairness is not intuitively clear. More explicit reasoning or examples illustrating why higher entropy equates to greater fairness would be beneficial, especially in federated learning settings.

Author response:
Thank you for your suggestion, we would like to elaborate on why higher entropy equates to greater fairness would be beneficial in FL setting and give example to illustrate it:

- **Elaboration:** In FL, without constraints on the aggregation probability, maximum entropy results in a uniform distribution of aggregation probabilities, treating each client equally. However, when constraints are introduced, such as ensuring the aggregated loss is close to the ideal loss, maximum entropy is achieved by not making overly confident predictions while adhering to the constraints. Consequently, the aggregation becomes proportional to each client’s loss. The rationale is that a larger loss indicates poor model performance on a client, prompting the updated model to pay more attention to that client, resulting in equal performance across all clients.

Reviewer comment (Writing):
From Equation (2), the term “ideal loss” $\tilde{f}(x)$ is introduced without a clear definition or expression, leading to confusion that is not resolved until later sections (4.2.1 and 4.2.2). It would be beneficial if the authors could provide more discussion or examples for “ideal loss” here.

Author response:
Thank you for your suggestion. We would like to add an explanation of the ideal loss:

The ideal loss $\tilde{f}(x)$ serves as the target or objective for the aggregated losses. Its specific expression depends on the goal of the FL system. In Eq (2), as a robust constraint, $\tilde{f}(x)$ aims to ensure that the aggregated clients' losses are as close to the desired target loss as possible.

In this paper, for example, when the ideal loss acts as the ideal fair loss, the gradient of the ideal loss should be a reweighted aggregation of clients’ unbiased local updates, $\nabla \tilde{f_{fair}}(x)=\sum_{i\in S_t}p_i \nabla F_i(x)$, where $\nabla F_i(x)$ is the unbiased local update of the model $x$.

Motivated by your comments, we have added this discussion in our revised manuscript.

Reviewer comment (Theory):
The fairness of FedEBA+ is partly achieved based on the assumption that the FedSGD global gradient is an ideal fair gradient. Specifically, according to the end of Section 4.2, the authors view a FedSGD global gradient as the ideal global gradient and the ideal fair gradient. Clarity is needed on why this assumption is made and whether the FedSGD gradient truly encapsulates ideal fairness. Also, it would be helpful to consider FedSGD as one fairness baseline in the experiments, to demonstrate the effect of another component in FedEBA+, maximum entropy aggregation, for fairness.

Author response:
We would like to clarify that we employ the one-step gradient to obtain the ideal local gradient.

For the ideal global gradient, we use the average of the ideal local update as the ideal global gradient, which aligns with the FedSGD global gradient. However, it's crucial to note that even for the ideal global gradient, the FedSGD gradient is only utilized to correct the model update rather than being used as the updated model itself.

In contrast, for the ideal fair gradient, it differs from the FedSGD gradient. The fair gradient is the reweighted average of the ideal local update based on our fair aggregation probability $\sum_{i\in S_t}\tilde{p_i }\nabla F_i(x_t)$, instead of a direct average $\frac{1}{|S_t|}\sum_{i\in S_t}\nabla F_i(x_t)$. 

Clarification: **It is not FedSGD that encapsulates the ideal fairness, but the reweighted aggregated one-step local updates represent the ideal fairness.**

- [Why reweight] Based on the local loss with the global model on each client ($F_i(x_t)$), the server knows the global model’s performance on each client, allowing it to adjust the attention it pays to different clients: assigning higher weights to clients with larger loss (poor performance) for fairness. According to the analysis of entropy, to make the aggregated loss to be fairer, the aggregation should be reweighted based on Eq 3.
- [Why one-step update] The choice of using one-step local updates stems from the observation that multiple local updates can introduce local bias in FL[1,2,3]. Therefore, using the one-step gradient can better reflect the true global model's update on the clients without local bias. Combining this with the fair reweight aggregation strategy ensures that the model represents a more accurate fair gradient.

[1]Karimireddy S P, Kale S, Mohri M, et al. Scaffold: Stochastic controlled averaging for federated learning[C]//International conference on machine learning. PMLR, 2020: 5132-5143.

[2]Wang J, Liu Q, Liang H, et al. Tackling the objective inconsistency problem in heterogeneous federated optimization[J]. Advances in neural information processing systems, 2020, 33: 7611-7623.

[3]Mendieta M, Yang T, Wang P, et al. Local learning matters: Rethinking data heterogeneity in federated learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 8397-8406.

Reviewer comment (Theory):
The following statement is not clear: “The specific challenge lies in finding a way to introduce entropy into the FL training process while incorporating FL’s uniform performance as a constraint within the framework.” Isn’t maximum entropy encouraging uniform performance? (Intuitively, the outcomes with uniform probability give the highest entropy, i.e., uncertainty, based on the definition of entropy) Does maximum entropy go against the goal of uniform performance? The reason why these concepts are treated distinctly in this statement requires clarification.

Author response:
Sorry for the confusion by our previous expression. Thank you for pointing out the inaccurate statement, we have revised it into  “The challenge lies in modeling FL aggregation as entropy and incorporating that using aggregation method to make the performance distribution more uniform.”

Indeed, maximizing the entropy of performance aligns with the goal of achieving uniform performance. However, in our paper, the entropy is not over performance but aggregation probability $\mathbb{H}(p) := -\sum_{i=1}^m p_i \log(p_i)$. Simply maximizing the entropy $\mathbb{H}(p)$ corresponds to making each client have a uniform aggregation distribution, this is FedAvg. 

Recall that our aim is to make the distribution of performance uniform, thus it’s necessary to link the aggregation probability and model performance, and link them as a constraint to make the performance uniform. Thus, we introduce the ideal fair loss $\tilde{f}(x)$ as a constraint $\sum_{i}p_iF_i=\tilde{f}(x)$ of the constrained entropy as shown in Eq (2), this forces the loss of aggregation to be close to the loss under truly fair model performance to ensure that the aggregated model (global model) obtained with such an aggregation probability is fair. Consequently, the aggregation probability $p_i$ becomes proportional to each client’s loss $F_i(x)$. The rationale is that a larger loss indicates poor model performance on a client, prompting the updated model to pay more attention to that client, resulting in equal performance across all clients. 

The constrained entropy also yields the highest entropy after satisfying the condition, ensuring that the aggregation does not take other confidential conditions into consideration and giving the highest entropy.

Reviewer comment (Theory):
The claimed convergence rate is not appropriate because there is a non-vanishing constant error term, which means that the gradient of the algorithm will never approach zero. The convergence rate is O(1) instead. This finding is inconsistent with previous studies, such as FedAvg convergence under the non-iid setting [1], which does not have the constant error term.

Author response:
Thank you for your feedback. We'd like to clarify:

- **No-vanishing term is common in biased sampling/aggregation FL:** The inclusion of a no-vanishing term is a common requirement in biased sampling/aggregation methods, as demonstrated in various works[1,2,3,4,5]. The proposed entropy-based aggregation is a biased method, aiming to improve fairness, thus there is a no-vanishing term. Besides FL, this phenomenon is also commonly observed in optimization[6,7].
- **Biased sampling/aggregation has advantages in FL:** Previous research has shown that biased sampling can expedite convergence rates and enhance accuracy in FL [1], as well as mitigate local bias issues [2]. In our work, we employ biased aggregation methods to enhance the fairness of FL.

Reviewer comment (Theory):
The Theorem 5.3 claims that FedEBA+ achieves a smaller variance than FedAvg. However, the effect of FL data distributions is not reflected in Theorem 5.3. It would be clearer if the authors could justify how non-iid degree will affect the fairness results. Moreover, could the authors provide definitions for T() and A() in the variance analysis?

Author response:
Thank you for your suggestions. We'd like to justify the non-iid degree in Theorem 5.3:

As discussed in Appendix 11.2, specifically in the *Test Loss* paragraph, “the dataset on client $i$ is denoted as $\left(\boldsymbol{\Xi}_i, \mathbf{y}_i\right)$, where $\boldsymbol{\Xi}_i$ is fixed, and $\mathbf{y}_i$ follows a Gaussian distribution $\mathcal{N}\left(\boldsymbol{\Xi}_i \mathbf{w}_i, \sigma_2^2 \boldsymbol{I}_n\right)$. The heterogeneity across clients arises solely from the variability in $\mathbf{w}_i$“, where $\textbf{w}_i$ is the true parameter on client $i$.

In Theorem 5.3, we have demonstrated that $V^{\text{Avg}}=\operatorname{var}\left(f_i^{t e}\left(\mathbf{w}^{A v g}\right)\right)=\frac{b^2}{4} \operatorname{var}\left(\left\|\overline{\mathbf{w}}-\mathbf{w}_i\right\|_2^2\right)$ and $V^{\text{EBA+}}=\operatorname{var}\left(f_i^{t e}\left(\mathbf{w}^{EBA+}\right)\right)=\frac{b^2}{4} \operatorname{var}\left(\left\|\tilde{\mathbf{w}}-\mathbf{w}_i\right\|_2^2\right)$. This implies that a larger non-iid degree, i.e., greater data heterogeneity across clients, corresponds to a larger heterogeneity of $\mathbf{w}_i$. Consequently, larger heterogeneity of $\textbf{w}_i$ results in larger variance for both FedAvg and FedEBA+.

Moreover, Theorem 5.3 establishes that under the same heterogeneity degree, FedEBA+ consistently achieves smaller variance than FedAvg. 

Motivated by your comments, we have mentioned this explicitly in Theorem 5.3 of the revised manuscript.

Reviewer comment (Experiments):
The algorithms introduce a few additional hyperparameters: temperature $\tau$, angle threshold $\theta$, and $\alpha$. From Figure 4, it appears that the performance of the algorithm is quite sensitive to the hyperparameters, and different datasets have different optimal hyperparameters. It indicates a need for extensive hyperparameter tuning across different datasets. This requirement potentially reduces the algorithm's practicality compared to more straightforward federated learning algorithms like FedAvg.

Author response:
Thank you for your comments. We would like to clarify:

- **No extensive hyperparameter tuning is needed:** Our algorithm consistently **achieves the best performance across all four datasets (FashionMNIST, CIFAR-10, CIFAR-100, and Tiny-ImageNet) under the same set of parameters** (temperature $\tau=0.1$, $\alpha=0.9$), as reported in Table 1 and Table 2 of our original submission.
- **Fine-tuning hyperparameters can further improve algorithms, but common choices suffice:** Figure 4 demonstrates that different hyperparameter choices can enhance performance, even surpassing the best result in Table 1 and Table 2. However, no specific finetuned hyperparameter choice outperforms baselines, as indicated in the table below.

Reviewer comment (Experiments):
The influence of the angle threshold $\theta$ on FedEBA+'s performance is not demonstrated. It would be beneficial to provide an ablation study on $\theta$.

Author response:
Thank you for your suggestion. 

- We would like to clarify that due to page limits, we provided the ablation study of $\theta$ in Appendix 14 (Table 5 and Figures 9-12) on FashionMNIST in our original submission.
- Furthermore, we have presented new ablation results for $\theta$ on CIFAR-10. The table below illustrates that:
    - With sparse additional communication, like $\theta=40^\circ$, FedEBA+ achieves significantly better performance than baselines (Accuracy improves by 2.5, fairness improves by 24.3 compared to FedAvg on CIFAR-10).
    - Even with no additional communication ($\theta=90^\circ$), FedEBA+ promises the best fairness and comparable accuracy on both datasets.