## 7 Performance Evaluations
### Experiment Settings and Benchmarks  
To simulate real-world data distributions, two kinds of data distribution skews are applied to local datasets. Detailed settings are described as follows:1. Label distribution skew: For the HAR dataset, we assume each smartphone owns the same number of samples. Six human activities are labeled: sitting, lying down, walking, going upstairs, going downstairs and standing. A total of 34,440 data samples are assigned to \(N=10\) smartphones according to a Dirichlet distribution [45]. To match real-world data distributions, we set concentration parameter \(\alpha\) at 0.5 [46]. 2. Label and data quantity skew: For the CIFAR-10 and CIFAR-100 datasets, we consider that each smartphone owns at most 4 out of 10 labels for CIFAR-10 and at most 8 out of 100 labels for CIFAR-100. Note that we do not consider simple cases wherein labels or the quantity of training data are uniformly distributed. We process CIFAR-10 data samples (10 classes, with 6000 images per class) and CIFAR-100 data samples (100 classes, with 600 images per class) to form our synthetic non-IID datasets based on [46]. To be specific, we simulate label skew by assigning a random subset of classes to each smartphone. The number of classes per phone is generated using the function random.randint( ). Then, we create quantity skew by using a Dirichlet distribution to allocate different amounts of data for each class to different smartphones. The use of the function numpy.random.dirichlet( ) results in non-uniform data quantities across smartphones ([[https://github.com/IBM/probabilistic-federated-neural-matching/blob/master/experiment.py](https://github.com/IBM/probabilistic-federated-neural-matching/blob/master/experiment.py)]([https://github.com/IBM/probabilistic-federated-neural-matching/blob/master/experiment.py](https://github.com/IBM/probabilistic-federated-neural-matching/blob/master/experiment.py)) accessed on 30 June 2024). To simulate different degrees of skews, we set \(\alpha=10\) and \(\alpha=0.5\) for CIFAR-10 and CIFAR-100, respectively. We consider a fixed \(T_{n}\) for smartphones (i.e., \(R_{n,\text{max}}=200\)) and \(\lambda=1\) in experiments. For each smartphone, data samples are shuffled according to a discrete uniform distribution. Then, 70% of 60,000 samples are used for training and 15% of 60,000 samples are used for validating and testing, respectively. For benchmarks and model training, we follow similar parameter settings as the open-source code base ([[https://github.com/CharlieDinh/pFedMe](https://github.com/CharlieDinh/pFedMe)]([https://github.com/CharlieDinh/pFedMe](https://github.com/CharlieDinh/pFedMe)) accessed on 30 June 2024). To be specific, we set the local learning rate \(\eta_{r}\) to 0.005 for mini-batch SGD. The batch size and local iterations are set to 64 and 10, respectively. In our experiments, we use a DNN for classification. To be specific, a DNN is defined in [47] for HAR datasets. For CIFAR datasets, we use Swin transformer v2 [48] trained on Imagenet as the pre-trained model. Model heads are three-layer DNNs with input size 768, middle dimensions 500 and 100, and output size 10 for CIFAR-10 and 100 for CIFAR-100. We perform DML using PyTorch [49] version 2.3.0+computecanada. To run the simulation, an NVIDIA V100L GPU is used, and 24 CPU cores and 180 gigabytes of RAM are allocated. To assess our approaches on heterogeneous datasets, seven benchmarks are considered in our experiments: namely, FedProx [50], pFedMe [51], PerAvg [52], Curricula [41], Anti-Curricula, Random Weights and Addictive Noise [7; 8]. FedProx uses a regulation term between local and global models to mitigate deviations. Similarly, pFedMe uses Moreau envelopes as regularized loss functions. PerAvg is a meta-learning approach to handle the data heterogeneity problem. The superiority of our data scheduling is shown by comparing it with the state-of-the-art (SOTA) Curricula, for which our proposed adaptive window is missing.