## 4 Discussion
### Data Distribution (Statistical Data Heterogeneity) Challenge
#### 4.3.2 Non-IID Mitigation Methods  
Different non-IID characteristics may need different mitigating measures. There are three methods in the published FL for medical applications to improve the model performance with the non-IID dataset: (1) balancing the training dataset, (2) tuning the model hyperparameter in the FL algorithm, and (3) domain adaptation. **Balance the training dataset method. ** When dealing with quantity skew in non-IID characteristics, researchers balance the quantity of minority and majority classes in the training dataset with the synthetic data augmentation technique. It is important to note that the balancing method in the FL environment should keep the data secure and private. There are two methods to generate synthetic data augmentation in the FL environment: (1) local data augmentation and (2) server data sharing. 1. The healthcare node generates a synthetic sample to balance the training dataset in the local data augmentation method. The synthetic minority oversampling technique (SMOTE) [21; 49], generative adversarial method (GAN) [44], or geometric transformation [40; 48; 53] is employed to generate a synthetic sample in an FL environment. The SMOTE algorithm is an oversampling technique where the synthetic data are generated for the minority class. For instance, [PERSON] et al. [21] and [PERSON] et al. [49] employ SMOTE to balance the heavy imbalance in a fall detection and lung cancer training dataset, respectively. [PERSON] et al. [44] proposed secure synthetic COVID-19 data by combining the GAN and differential privacy method. [PERSON] et al. [40], [PERSON] et al. [48], and [PERSON] et al. [53] applied geometric transformations such as random flipping, random rotation, and random translation to balance the quantity of minority class in their training dataset for the data augmentation method. 2. The aggregate server securely shares a small portion of data to the healthcare node in the server data sharing method. For instance, [PERSON] et al. [23] proposed a global shared dataset partition to train non-IID data. The author demonstrated that by simply sharing 5% of data, they could get a 30% boost accuracy score. However, it raises model communication costs and is prone to data privacy attacks during the data sharing process. Figure 12: Non-IID from concept shift skew characteristic. (**a**) IID case; (**b**) non-IID same features but different label case; (**c**) non-IID same label but different features case. **Adaptive Hyperparameters Method. ** The adaptive hyperparameters method tries to find the proper FL hyperparameters values for each node during the training process. Each node can have different values of the FL hyperparameters, such as learning rate, loss score, and weighting coefficient. There are two published adaptive hyperparameters methods in the published FL studies for medical application: (1) weighting coefficient [16; 19; 20; 45], and (2) adaptive loss function [46]. 1. The weighting coefficient \(\alpha_{k}\) is a variable that indicates the relative influence of each node \(k\) on the aggregation equation in Equation (2) to update the global model. Initially, [PERSON] et al. [16] proposed FedAvg that the weighting coefficient is \(\alpha_{k}=\frac{n_{k}}{n}\) as shown in Equation (6), where \(n_{k}\) and \(n\) are the private data points hold by node \(k\) and the total data from all nodes that participated during training, respectively. In this case, a node with significant data points has a considerable effect on the global model. This method worked well when dealing with label distribution skew characteristics experimented in their studies [16; 20]. \[w_{t+1}^{g}\leftarrow\sum_{k=1}^{K}\frac{n_{k}}{n}\times w_{t+1}^{k}\] (6) In comparison, [PERSON] et al. [20] proposed that the weighting coefficient is \(\alpha_{k}=\frac{1}{K}\), where \(K\) is the total nodes participating in FL as shown in Equation (7). In this scenario, the author considered that each node would contribute equally to the aggregation function.