## 6 Discussion  
The results of the first experiment show that our proposed method has stronger adaptation for extreme non-IID cases where a group of clients have less influence on the global model compared to other clients. FedWAvg expresses exceptional strength when groups are mildly differed in number but hierarchical, nonetheless. By the results of the second experiment, we can tell that our method is running as intended by its design. Combined, we suggest that having less forgetting events has contributed to enhancing the performance of the global model. A vital issue to note is that the performance is highly sensitive to the update ratio hyperparameter \(\alpha\). Currently, \(\alpha\) is a hyperparameter, but could be developed to automatized. That is a topic of future research. The parity between top performance and average performance is too large to ignore. Therefore, we executed a 1000 round training with the same setup as the second experiment. In the beginning, the global model fluctuates greatly every round. Over time, the amplitude decreases. Moreover, the number of forgettable example track records show that, beyond 400 rounds, the numbers stay within 450 to 500 range. It is converging to a certain degree. Presumably, this pattern occurs due to the fact that weights change every round since the event ratio \(t\) is 1. The drop of forgettable examples in minor clients after a successful weighted aggregation afflicts the averaging weights. The averaging weights then reduce the influence of weaker--minority cluster--clients causing the global model to drastically lose knowledge previously obtained from weaker clients. In the next round, this will be amended only to be broken again in the upcoming round. In other words, recalculating averaging weights every round creates disincentive to weaker clients. When the global model performs adequately, it causes the global model to lose their knowledge in the next round. Then, the loss of knowledge creates incentive to weaker clients again. Before verifying our presumption, we sought to inspect possible sources of the pattern. It could have been affected by the non-IID environment. The result is visualized in Figure 2.  
Figure 2 shows each result of FedWAvg (orange), FedAvg (green), and FedProx (blue) in the same setup. Compared to FedWAvg, although the early trends are more stable, \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline
**Round** & **Minor1** & **Minor2** & **Major1** & **Major2** & **Major3** & **Major4** & **Major5** & **Major6** & **Major7** & **Major8** \\ \hline
100 & 1197 & 1129 & 0 & 1 & 0 & 1 & 2 & 0 & 3 & 5 \\
200 & 1169 & 1164 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\
300 & 1008 & 1012 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\
400 & 1030 & 1023 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\
500 & 1065 & 1063 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: FedAvg forgettable example counts in each client when non-IID (2-to-8). \begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline
**Round** & **Minor1** & **Minor2** & **Major1** & **Major2** & **Major3** & **Major4** & **Major5** & **Major6** & **Major7** & **Major8** \\ \hline
100 & 1200 & 1193 & 3 & 6 & 6 & 7 & 13 & 13 & 8 & 11 \\
200 & 1170 & 1172 & 1 & 0 & 1 & 0 & 2 & 0 & 0 & 0 \\
300 & 1185 & 1184 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 \\
400 & 1190 & 1177 & 0 & 0 & 0 & 0 & 0 & 2 & 0 & 0 \\
500 & 1157 & 1147 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: FedProx forgettable example counts in each client when non-IID (2-to-8).