# FedRDS: Federated Learning on Non-IID Data via Regularization and Data Sharing
## 4 Experiments and Analysis  
### Datasets and Settings  
The datasets for the experiments include **MNIST**, **Fashion-MNIST**, **SVHN**, and **CIFAR-10**, and the network models in the experiments are all convolutional neural networks (CNNs).  
**MNIST**: Consists of grayscale images of handwritten digits classified into 10 categories. Consisting of 60,000 training samples and 10,000 test samples, the dataset features images that are \(28\times 28\) pixels in size. The neural network model for this dataset comprises two convolutional layers, each with a kernel size of \(5\times 5\), and the first layer has 32 channels. In contrast, the second layer has 64 channels, both featuring a max-pooling layer of \(2\times 2\). Additionally, it consists of a fully connected layer that employs 512 neurons, which are activated using the rectified linear unit (ReLU) mechanism, followed by a softmax output layer.  
**Fashion-MNIST**: an image dataset replacing the MNIST handwritten dataset, consisting of 10 classes of clothing items with (\(28\times 28\)) pixel grayscale images. The Fashion-MNIST dataset is divided into training and test sets in the same manner as MNIST, comprising 60,000 training samples and 10,000 test samples. Each image in this dataset measures \(28\times 28\). The neural network architecture for this dataset incorporates two convolutional layers and one fully connected layer.  
Figure 3: Data-sharing strategy.  
**SVHN**: Derived from Google Street View house numbers and consists of images comprising Arabic numerals '0-9'. It necessitates minimal data pre-processing and is comparable to the MNIST dataset, but it contains a more extensive collection of labeled data. The training set comprises 73,257 digits, while the test set comprises 26,032 digits and an additional 531,131 digits. We used the same neural network model as CIFAR-10 for this dataset.  
**CIFAR-10**: \((32\times 32)\) 3-channel pixel RGB images belonging to 10 categories of objects, with each image measuring \(32\times 32\). The dataset comprises five batches allocated for training and one batch for testing, each containing 10,000 images. For the test batch, 1000 images have been randomly drawn from each category. The remaining 50,000 images have been evenly distributed among the five training batches, resulting in around 5000 images per category. Due to the breakdown of the remaining 50,000 images into five training batches, the number of images per category may be different across batches. This leads to the possibility of some categories having more images than others. This dataset's neural network comprises two convolutional layers, two fully connected layers, and a linear transformation layer.  
For dividing the dataset, we used the method in [3]. For IID data, we shuffle the data and divide it into 100 clients with 600 shards per client. For non-IID data, we partitioned the training and test data into 200 shards based on their labels and randomly allocated two shards to each client.  
For the edge computing scenario, many data nodes often have the problem of non-IID data and this paper simulates the non-IID in the scenario. For the actual scenario simulated on the four datasets, we set the learning rate (\(\eta\)), the number of local epochs (E), batch size (B), the fraction of shared data (\(\beta\)), the regularization coefficient (\(\sigma\)), the fraction of shared data per client (\(\alpha\)), and the fraction of clients (C) as shown in Table 2. In order to assess the effectiveness of the algorithms, the experiments were run on a Windows computer with NVIDIA GeForce RTX 3070 Laptop GPU and 16 GB of memory.