## 4 Discussion
### Data Distribution (Statistical Data Heterogeneity) Challenge
#### 4.3.1 Non-IID Characteristics  
The non-IID characteristics among healthcare nodes in the FL environment can take on four different forms such as (1) quantity distribution skew, (2) label distribution skew, (3) feature distribution skew, and (4) concept shift skew [24; 25]. The non-IID characteristic summarized from 24 published FL studies applied for medical application is listed in Table 2.  
**Quantity skew (imbalance data) characteristic**. Quantity skew characteristic in non-IID occurs when the class distribution of data instances \(\mathcal{I}\) is not equal or far from equal across nodes in the FL scheme. An illustration of quantity skew is shown in Figure 9. In the IID scenario, the amount ratio of positive and negative instances is almost equal. For instance, in node two, the negative and positive amount ratios are 45% and 55%, respectively. In the non-IID case, the ratio of positive and negative instances is far from equal. For example, in node one, positive instances are around 5%, while negative ones are 95%. [PERSON] et al. [57] divided imbalance data categories into slight imbalance and severe imbalance. A slight imbalance is when the majority class is uneven by a small amount in the training dataset, and the ratio ranges from 1:4 up to 1:100. Severe imbalance data distribution is when the data distribution of the majority class is uneven by a vast amount in the training dataset, the ratio is more than 1:100. For example, the ratio of imbalance data in fraud detection tasks is up to 1:1000. Figure 9: Non-IID from quantity skew (i.e., imbalanced dataset) characteristic. (**a**) IID: the amount ratio of positive and negative instances is equal or slightly equal; (**b**) non-IID: the ratio of positive and negative instances is far from equal. For example, the positive and negative instances ratio is 5% and 95% in node one, respectively. Quantity skew characteristic exists in FL for medical application experiment datasets such as [18; 19; 46; 52; 53]. Quantity skew (i.e., imbalanced dataset) is common in the medical dataset since it is acquired from multiple healthcare institutions, and the number of instances in a class is not equally distributed for each institution. For instance, larger hospitals have more patient records than small clinics in rural areas. [PERSON] et al. [19] tried to resolve this challenge by developing an imbalanced eICU dataset to predict patient mortality where the ratio is 5% and 95% for death and alive categories, respectively. **Label distribution skew characteristic**. For label distribution skew, the distribution of labels \(P(y_{i})\) varies between different nodes. In the medical case, larger hospitals generally have more disease-related records than small clinics in rural areas. An illustration for label distribution skew characteristic is shown in Figure 10. In the IID setting, the distribution of labels \(\mathcal{Y}\) is the same across all nodes. However, in the non-IID setting, the distribution of labels \(\mathcal{Y}\) varies between each node. Specifically, there is a label \(y_{i}\) that only exists in one or several nodes in the FL environment. This label distribution skew characteristic was initially demonstrated in [PERSON]'s experiment [16]. Data samples with the same label are divided into subsets, and each client is assigned to no more than two subsets with distinct labels. Following FedAvg, this configuration is employed in published FL studies for medical applications [38]. **Feature distribution skew characteristic**. In the feature distribution skew characteristic, the distribution of features \(P(x_{i})\) varies between different nodes. An illustration of features distribution skew is shown in Figure 11. In the case of IID, the distribution of features \(\mathcal{X}\) is the same across all nodes, while in the non-IID case, the distribution of features \(\mathcal{X}\) varies between each node. Specifically, there is a feature \(x_{i}\) that only exists in one or several nodes in the FL environment.