# FLIP: A New Approach for Easing the Use of Federated Learning
## 4 V&V: Validation and Verification of the Platform
### Objectives Verification
To help interpret the results, it is important to first note the reference to each of the parameters considered. First, the SUS study score represents an average perceived usability score between 0 and 100, which is calculated based on the formula from [27], and considers 68 as the average rate. The percentile value [24] shows the SUS study scores on a cumulative percentile curve for the relation between the SUS score and the percentage of systems that exhibit a lower perceived usability level. The quartile scale, as the authors explain in [28], can be used to contextualise and compare the SUS scores with the scores in their dataset, which contains 3500 SUS scores divided into four quartiles. The acceptability scale contextualises the SUS study scores in descriptions ranging from \"Not acceptable\" to \"Acceptable\", based on data from [29]. The Net Promoter Score (NPS) [30] describes \begin{table}
\begin{tabular}{c l l l} \hline \hline
**Step** & **Action** & **Flower Alone** & **FLIP** \\ \hline \multirow{3}{*}{1} & Creation of server & 1. ssh connection in server node & Transparent to the user \\  & working environment & 2. Python virtual environment creation & (Automated by the platform) \\  & & 3. Dependencies installation & \\ \hline \multirow{3}{*}{2} & Creation of client & 1. ssh connection in client node & Transparent to the user \\  & working environment & 2. Python virtual environment creation & (Automated by the platform) \\  & working environment & 3. Dependencies installation & \\ \hline \multirow{3}{*}{3} & Flower server deployment & 1. Copy of server script to the server node via scp & 1. Login on the platform \\  & & 2. Execution of the server script on the server node & \\ \hline \multirow{3}{*}{4} & Flower clients deployment & 1. Copy of client scripts to all client nodes via scp & 1. Task creation on platform \\  & & 2. Execution of the client script on all client nodes & \\ \hline \multirow{3}{*}{5} & \multirow{3}{*}{Results gathering} & 1. Copy all the result files from server node via scp & 1. Download all the result files \\  & & & from the platform \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of model training steps between Flower and FLIP. Figure 5: Results of the initial set of questions. (**a**) Educational level; (**b**) users with a data scientist profile; (**c**) knowledge about federated learning; and (**d**) knowledge about computer system administration. how likely users of a product are to recommend the analysed system to others. Finally, the industry benchmark scale is a non-empirical scale derived from [31] where the authors claim that scores above 80 are an \"industrial goal\". Figure 6 and Table 5 show the results obtained for both systems. Taking into account the results obtained, we can claim that our platform presents a high level of perceived usability, as expected. The SUS score rates our solution at 84.64 (above the average of 68), which means that it has a perceived usability level higher than 95.15% of all products considered to establish the percentile scale. Moreover, with regard to the comparison with Flower, the SUS score of our platform is 40% higher and users' opinions are less dispersed (they offer a lower standard deviation). Users consider our platform as an acceptable system and, according to their responses, they can be considered as promoters of it, contrary to their opinion about Flower, which obtains only a marginal rate in the acceptability scale, and users may act as detractors of this option. Finally, the score obtained for the industry benchmark analysis suggests that our platform is above industry standards.