## 3 Experiments and Results
### Experimental Setting  
The experiments utilize two datasets: MNIST and a synthetic dataset. The MNIST dataset, introduced by [PERSON] et al. (2019), comprises 70,000 grayscale images of handwritten digits (0-9), each \(28\times 28\) pixels, with 60,000 training and 10,000 testing samples. Following the approach suggested by [PERSON] et al. (2019), we generate non-IID settings across clients using a Dirichlet distribution with parameter \(\alpha\), where a smaller \(\alpha\) increases data heterogeneity. We evaluate two non-IID settings, each with 20 and 50 clients. The first setting employs a Dirichlet distribution with \(\alpha=0.05\). The second setting uses \(\alpha=0.5\) and selects the top two classes with the highest proportions, ensuring each client holds data for exactly two digits. Each client is randomly assigned a sample size ranging from 1665 to 3834. The synthetic dataset is generated to test generalization under controlled heterogeneity, following the methodology of [PERSON] et al. (2019). It consists of 100 clients, for each client \(i\), we create data \(\mathcal{D}_{i}=(X_{i},Y_{i})\) using the model \(y=\arg\max(\text{softmax}(Wx+b))\), with features \(x\in\mathbb{R}^{60}\), weights \(W\in\mathbb{R}^{10\times 60}\), and biases \(b\in\mathbb{R}^{10}\). We draw \(W_{i}\) and \(b_{i}\) from \(\mathcal{N}(u_{i},1)\), where \(u_{i}\sim\mathcal{N}(0,\alpha)\), and generate features \(x_{i}\sim\mathcal{N}(v_{i},\Sigma)\), with a diagonal covariance matrix \(\Sigma\) satisfying \(\Sigma_{ij}=j^{-1.2}\). The mean vector \(v_{i}\) has elements sampled from \(\mathcal{N}(B_{i},1)\), with \(B_{i}\sim\mathcal{N}(0,\beta)\). Thus, \(\alpha\) regulates the variation among local models, while \(\beta\) governs the diversity of data distributions across clients. Here we set \(\alpha=\beta=0.5\). The dataset of each client is partitioned into training and test sets, with 75% of the samples allocated to training and 25% to testing, using random permutation to ensure an unbiased split, and the data are converted to PyTorch tensors for compatibility with deep learning frameworks.  
Two models are employed to assess performance across varying complexities. The multinomial logistic regression (MLR) model is a simple linear classifier with an input layer of 784 units (for MNIST or 60 units for synthetic input) and an output layer of 10 classes, followed by a log-softmax activation to produce log-probabilities. The multilayer perceptron (MLP) is a neural network with an input layer of 784 units (for MNIST or 60 units for synthetic input), a hidden layer of 128 units with ReLU activation, and an output layer of 10 units with log-softmax activation. The MLP's additional capacity allows it to capture more complex patterns, while the MLR provides a lightweight baseline. Both models are trained to minimize the negative log-likelihood (NLL) loss, consistent with their log-softmax outputs and standard classification objectives in federated learning.  
We perform all experiments in this paper using a server node with four NVIDIA V100 GPUs with 16 GB memory, four Intel(R) Xeon(R) CPUs with 12 cores @ 2.40 GHz), and 360 GB memory. All experiments are performed with Python 3.12.7 and PyTorch 2.6.0 with CUDA 12.6.