# FLaMAS: Federated Learning Based on a SPADE [PERSON]
## 4 Experiments
### Experiment 4: Adding New Agents  
In the fourth experiment, using our \(FLaMAS\) tool, we sought to analyze the effect of adding new agents to the learning process during the execution at different times.  
As can be seen in Figure 12, this experiment used three Client agents that entered the system at different moments (initially, there was only agent 1 active, then after a while, agent 2 entered the system, and lastly, agent 3 incorporated itself into the learning process.  
Figure 11: Experiment 3.3: Local learning effect with three agents, five local epochs, and four global epochs.  
Figure 10: Experiment 3.2: Local learning effect with three agents, two local epochs, and four global epochs.  
According to the results obtained, the dynamic entry of new agents into the system was performed without affecting the system's performance, and they were immediately adjusted to the appropriate weights. In this way, higher flexibility was achieved, since the failure of any client could be easily solved without affecting the global performance. In addition, the scalability of the system was facilitated, if necessary.  
To conclude, a record was taken of the different parameters in each experiment to be compared (accuracy, training time and the number of epochs); the results obtained are summarized in the following Table 1. The distributed learning performed with \(FLaMAS\) achieved the goal proposed (with even more accuracy than the centralized learning) in lower epochs than the centralized experiment, and in a meaningfully lower amount of time. These numbers could be improved if we increased the number of Client agents. Moreover, as commented above, the usage of \(FLaMAS\) not only facilitated an easy communication mechanism, but also enabled adaptation to agents failing and to new agents entering the system.  
A statistical significance test was used. Ten experiments were conducted to determine whether the results obtained were non-random between the centralized training and one of the distributed training agents. A total of 211 accuracy values were collected. Before testing, we considered whether the two accuracy populations had equal variances. We can assume that the populations have equal variances if the ratio of the variance of the larger sample to the variance of the smaller sample is less than 4:1. The ratio of the variance of the larger sample to the variance of the smaller sample was 12.46/8.48 = 1.46, which is less than 4. This means that we can assume that the variances of the populations were equal. The t-test statistic was 2.29 and the corresponding two-sided p-value was 0.022. The p-value of the test was 0.022, which is below the alpha significance level (e.g., 0.05). This means that  
\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & **Experiment 1** & **Experiment 2** & \multicolumn{3}{c}{**Experiment 3**} & **Experiment 4** \\ \hline  & & & 3.1 & 3.2 & 3.3 & \\ \hline Accuracy (\%) & 95 & 96.5 & 93.56 & 93.58 & 96.49 & 97.1 \\ \hline Training Time (minutes) & \(\approx\)5 & \(\approx\)3 & \(\approx\)1 & \(\approx\)3 & \(\approx\)7 & \(\approx\)10 \\ \hline N-Global Epochs & 20 & 5 & 4 & 4 & 4 & 4 \\ \hline Mean (\%) & 94.69 & 95.10 & 94.68 & 94.93 & 94.95 & 95.05 \\ \hline Variance & 1.17 & 0.52 & 1.16 & 0.83 & 0.78 & 0.58 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Comparison table of experiment results.  
Figure 12: Experiment 4: Effect of adding new agents.  
we can conclude that the accuracy of the distributed model was statistically different from that of the centralized training.