# Entropy-Regularized Federated Optimization for Non-IID Data
## 4 Experimental Setup
### Baselines  
We compare our proposed method, _ERFO_, against the following federated learning algorithms that address data heterogeneity:  
* **FedAvg--the standard federated averaging algorithm that aggregates local model updates by weighted averaging with one local epoch and the Adam optimizer. * **FedProx--FedAvg augmented with a proximal term \(\frac{\mu}{\pi}\|w-w^{(t)}\|^{2}\) (with small \(\mu\)) to limit client drift under non-IID data. * **SCAFFOLD--a variance-reduction method using control variates exchanged between servers and clients each round to correct local update drift without additional tunable hyperparameters. * **FedNova--a normalized averaging scheme that scales each client's update by its number of local steps, which here reduces to FedAvg under equal single-epoch workloads. \begin{table}
\begin{tabular}{l c c c} \hline
**Method** & **Accuracy (\%)** & **Macro-F1** & **Macro-AUC** \\ \hline FedAvg & 79.5 \(\pm\) 0.30 (\(\pm\)0.21) & 0.774 \(\pm\) 0.005 (\(\pm\)0.004) & 0.85 \(\pm\) 0.02 (\(\pm\)0.01) \\ FedProx & 80.2 \(\pm\) 0.40 (\(\pm\)0.29) & 0.783 \(\pm\) 0.007 (\(\pm\)0.005) & 0.86 \(\pm\) 0.01 (\(\pm\)0.01) \\ SCAFFOLD & 78.9 \(\pm\) 0.50 (\(\pm\)0.36) & 0.767 \(\pm\) 0.010 (\(\pm\)0.007) & 0.84 \(\pm\) 0.03 (\(\pm\)0.02) \\ FedNova & 75.7 \(\pm\) 0.50 (\(\pm\)0.36) & 0.728 \(\pm\) 0.004 (\(\pm\)0.003) & 0.80 \(\pm\) 0.02 (\(\pm\)0.01) \\ FedCurv & 79.5 \(\pm\) 0.80 (\(\pm\)0.57) & 0.774 \(\pm\) 0.010 (\(\pm\)0.007) & 0.85 \(\pm\) 0.02 (\(\pm\)0.02) \\ FedDyn & 76.9 \(\pm\) 1.10 (\(\pm\)0.79) & 0.745 \(\pm\) 0.010 (\(\pm\)0.007) & 0.82 \(\pm\) 0.03 (\(\pm\)0.02) \\ FedAdam & 79.2 \(\pm\) 0.60 (\(\pm\)0.43) & 0.771 \(\pm\) 0.008 (\(\pm\)0.006) & 0.85 \(\pm\) 0.02 (\(\pm\)0.01) \\ FedYogi & 79.3 \(\pm\) 0.90 (\(\pm\)0.64) & 0.772 \(\pm\) 0.012 (\(\pm\)0.009) & 0.85 \(\pm\) 0.02 (\(\pm\)0.02) \\ ERFO (ours) & **81.1 \(\pm\) 0.64 (\(\pm\)0.46)** & **0.791 \(\pm\) 0.008 (\(\pm\)**0.006) & **0.95 \(\pm\) 0.01 (\(\pm\)0.01)** \\ Ditto & 76.7 \(\pm\) 0.10 (\(\pm\)0.07) & 0.752 \(\pm\) 0.009 (\(\pm\)0.006) & 0.83 \(\pm\) 0.02 (\(\pm\)0.01) \\ \hline \end{tabular}
\end{table} 
Table 2: Comparative performance on UNSW-NB15 (50 runs). Each entry reports the mean \(\pm\) standard deviation, with the 95 % confidence interval in parentheses; macro-averaged ROC-AUC is also shown. \begin{table}
\begin{tabular}{l c c c} \hline
**Method** & **Accuracy (\%)** & **Macro-F1** & **Macro-AUC** \\ \hline FedAvg & 87.5 \(\pm\) 1.60 (\(\pm\)1.14) & 0.856 \(\pm\) 0.020 (\(\pm\)0.014) & 0.92 \(\pm\) 0.02 (\(\pm\)0.01) \\ FedProx & 87.4 \(\pm\) 2.00 (\(\pm\)1.43) & 0.855 \(\pm\) 0.022 (\(\pm\)0.016) & 0.92 \(\pm\) 0.01 (\(\pm\)0.01) \\ SCAFFOLD & 65.6 \(\pm\) 11.0 (\(\pm\)7.87) & 0.461 \(\pm\) 0.154 (\(\pm\)0.11) & 0.65 \(\pm\) 0.05 (\(\pm\)0.03) \\ ERFO (ours) & **90.3 \(\pm\) 1.20 (\(\pm\)0.86)** & **0.878 \(\pm\) 0.014 (\(\pm\)0.01)** & **0.94 \(\pm\) 0.01 (\(\pm\)0.01)** \\ Ditto & 87.4 \(\pm\) 1.60 (\(\pm\)1.14) & 0.854 \(\pm\) 0.022 (\(\pm\)0.016) & 0.92 \(\pm\) 0.02 (\(\pm\)0.01) \\ FedNova & 86.4 \(\pm\) 1.50 (\(\pm\)1.07) & 0.842 \(\pm\) 0.021 (\(\pm\)0.015) & 0.91 \(\pm\) 0.02 (\(\pm\)0.01) \\ FedDyn & 81.5 \(\pm\) 1.20 (\(\pm\)0.86) & 0.776 \(\pm\) 0.017 (\(\pm\)0.012) & 0.87 \(\pm\) 0.02 (\(\pm\)0.01) \\ FedCurv & 87.0 \(\pm\) 0.20 (\(\pm\)0.14) & 0.849 \(\pm\) 0.003 (\(\pm\)0.002) & 0.92 \(\pm\) 0.01 (\(\pm\)0.01) \\ FedYogi & 84.2 \(\pm\) 1.00 (\(\pm\)0.72) & 0.812 \(\pm\) 0.017 (\(\pm\)0.012) & 0.89 \(\pm\) 0.02 (\(\pm\)0.01) \\ FedAdam & 84.3 \(\pm\) 0.70 (\(\pm\)0.50) & 0.814 \(\pm\) 0.010 (\(\pm\)0.007) & 0.89 \(\pm\) 0.01 (\(\pm\)0.01) \\ \hline \end{tabular}
\end{table} 
Table 3: Performance on PneumoniaMNIST (50 runs). Each entry reports the mean \(\pm\) standard deviation, with the 95 % confidence interval in parentheses; macro-averaged ROC-AUC is also shown. * **FedDyn--**a dynamically regularized FedAvg variant that adds a dual-variable adjustment \(\langle\gamma_{i}^{(t)},w\rangle\) to each client's loss to align local and global optima without extra hyperparameters.