## 4 Proposed Method  
As mentioned in the previous Section 2, we also make choices to define the federated learning problem to solve. In our case, we propose a method to build a federated averaged model in a label-skewed non-IID [25]. The primary focus of our proposed method is to modify the aggregation method to consider relativity among clients based on the difficulty of learning local data samples. Meanwhile, data samples are not leaked and only utilized within the possessing client. Our definition of difficulty to learn derives from the fact that some data samples tend to be forgotten during aggregation. Our proposed method applies a hyperparameter to control the change of global model parameters to avoid extreme divergence. As mentioned in the previous Section 3, an example forgetting problem is more concerning in federated learning than in centralized learning environment. We also observed when there is an unbalanced distribution of data among clients, the averaging aggregation would always favor the majority. We propose federated weighted averaging (FedWAvg), which rebalances the global model aggregation based on forgettable examples in clients. FedWAvg ensures the global model is more influenced by knowledge that would normally be neglected in averaging aggregation in a non-IID environment. It manipulates the weight of each client strengthening those with more forgettable examples. The goal of this method is to reduce information loss by model aggregation. The observations in Section 3 prove that the averaging algorithm is skewed towards the majority cluster. The resulting global model will forget most of the information from the minority cluster. This is shown in Tables 2 and 3. Forgetting nearly 90% of the minority (cluster) data allows us to assume that the global model will fail on the intended task if the inferred input is from the minority classes. Thus, FedWAvg aims to reduce the number of forgettable examples--training data samples that are learned then forgotten--after aggregation, which proves that information loss induced by model aggregation is mitigated. \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline
**Round** & **Minor1** & **Minor2** & **Major1** & **Major2** & **Major3** & **Major4** & **Major5** & **Major6** & **Major7** & **Major8** \\ \hline
10 & 1137 & 1148 & 55 & 73 & 90 & 84 & 103 & 93 & 60 & 79 \\
20 & 1180 & 1185 & 63 & 38 & 38 & 57 & 66 & 57 & 47 & 60 \\
30 & 1192 & 1197 & 34 & 22 & 39 & 32 & 45 & 36 & 28 & 39 \\
40 & 1195 & 1189 & 22 & 12 & 24 & 25 & 29 & 27 & 23 & 22 \\
50 & 1194 & 1196 & 21 & 8 & 22 & 17 & 27 & 24 & 19 & 22 \\
100 & 1197 & 1129 & 0 & 1 & 0 & 1 & 2 & 0 & 3 & 5 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Forgettable example counts in each client when non-IID (2-to-8). \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline
**Round** & **Minor1** & **Minor2** & **Minor3** & **Minor4** & **Major1** & **Major2** & **Major3** & **Major4** & **Major5** & **Major6** \\ \hline
10 & 1188 & 1151 & 1180 & 1178 & 285 & 295 & 308 & 312 & 286 & 292 \\
20 & 1191 & 1182 & 1182 & 1184 & 86 & 87 & 108 & 103 & 77 & 93 \\
30 & 1194 & 1181 & 1191 & 1194 & 21 & 23 & 25 & 27 & 22 & 22 \\
40 & 1186 & 1193 & 1181 & 1194 & 4 & 5 & 9 & 10 & 3 & 9 \\
50 & 1189 & 1173 & 1183 & 1192 & 0 & 2 & 3 & 2 & 0 & 3 \\
100 & 1121 & 1114 & 1130 & 1120 & 0 & 0 & 0 & 0 & 0 & 1 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Forgettable example counts in each client when non-IID (4-to-6). To achieve the above goal, FedWAvg counts forgettable examples every \(t\) rounds. The clients send the number of forgettable examples to the server alongside with the local model parameters. Since the number is a single integer, it would barely add to the size of payload. In addition, because the number does not reveal anything about the raw training data or its distribution, it is safe to be transmitted. The number of forgettable examples in each client means how much information is lost in each client.