## 5 Simulation
### Simulation Setup
#### 5.1.2 Machine-Learning Model and Training Datasets  
We will use two benchmark datasets to train the corresponding machine-learning model.  
1. CIFAR-10 [36] is an image classification dataset containing 10 labels/classes of images, each of which has 6000 images. Among the 60,000 images, 50,000 are used for model training and 10,000 for model testing.
2. MNIST [37] is a handwritten digit dataset that includes many 28 \(\times\) 28 pixel grayscale images of handwritten single digits between 0 and 9. The whole dataset has a training set of 60,000 examples and a test set of 10,000 examples.  
We apply the convolutional neural network (CNN) to classify the CIFAR-10 images. The CNN model has four 3 \(\times\) 3 convolution layers (where the first layer has 32 channels, and each of the following three layers has 64 channels. Also, only the first two layers are followed with 2 \(\times\) 2 max pooling), followed by a dropout layer with rate of 75%, a fully connected 256 units ReLU layer, and a 10-unit SoftMax output layer. There are a total of 1,144,650 parameters in this CNN model.  
With respect to MNIST, which is a much simpler image dataset than CIFAR-10, a smaller CNN model has been used. Specifically, the CNN model has two 5 \(\times\) 5 convolution layers (where the two layers have 6 and 16 channels, respectively, each of which is followed with a 2 \(\times\) 2 max pooling), followed with two fully connected ReLU layers with 120 and 84 units, respectively, and a 10-unit SoftMax output layer. There are a total of 61,706 parameters in this CNN model.  
\begin{table}
\begin{tabular}{l c} \hline \hline
**Parameter** & **Value** \\ \hline Noise and inter-cell interference (\(N_{0}\)) & \(-94\) dBm \\ Bandwidth \(B\) & \(30\) kHz \\ Transmission power & \(0.1\) watt \\ Size of the local model (\(s\)) & \(100\) kbit \\ Number of local iterations (\(\theta\log_{2}(\frac{1}{\varepsilon})\)) & \(1\times\log_{2}(\frac{1}{\log_{3}})\) \\ Number of local samples \(|\mathcal{D}_{i}|\) & \(1000\) \\ CPU cycles required for training one data sample \(C_{i}\) & \(\mathcal{U}(3,5)\times 10^{8}\) \\ CPU frequency \(f_{i}\) & \(\mathcal{U}(0.8,3)\) GHz \\ Number of local epochs & \(1\) \\ Number of local batch size & \(20\) \\ Non-IID Dirichlet distribution parameter \(\beta\) & \([0.1,1,10]\) \\ Client Learning Rate \(\delta\), & \(0.02\) \\ \hline \hline \end{tabular}
\end{table}
Table 1: Simulation parameters.  
Figure 3: Clients’ latency distribution.  
In addition, we partition the MNIST/CIFAR-10 dataset among the 50 clients based on the non-independent and identical distribution (Non-IID), and the probability of having \(\eta_{m}\) images in label class \(m\) at client \(i\) is assumed to follow a Dirichlet distribution [38], i.e.,  
\[f(\eta_{1},\eta_{2},\ldots,\eta_{M};\beta)=\frac{\Gamma(\beta M)}{\Gamma(\beta )^{M}}\prod_{m=1}^{M}\eta_{m}^{\beta-1}, \tag{13}\]  
where \(M\) is the total number of label classes (i.e., \(M=10\) for both CIFAR-10 and MNIST), \(\Gamma()\) is the gamma function (\(\Gamma(z)=\int_{0}^{\infty}x^{z-1}e^{-x}dx\)), and \(\beta\) is the concentration parameter that determines the level of label imbalance. A larger \(\beta\) results in a more balanced data partition among different labels within a client (i.e., much closer to IID) and vice versa. Figure 4 shows how different labels of images are distributed by varying \(\beta\). In addition, we assume each client would train its CNN model over \(|\mathbf{\mathcal{D}}_{i}|=20\) data samples locally based on stochastic gradient descent (SGD) with base learning rate \(\delta=0.1\)[39] and epoch equal to 1.