# Towards Blockchain-Based Federated Machine Learning: Smart Contract for Model Inference
## 4 Benchmarking Experiments
### EEG Eye State Dataset Results  
The median runtimes for all the tested network and data sizes are presented in Figure 11, median values presented similar results to the first experiment performed with the synthetic 2D dataset. Similar to the synthetic 2D dataset results, the biggest increase in the model inference time occurred when the data size reached more than 8192 records. The bigger number of features and dataset size increased the amount of time required to validate the model, thus displaying higher median values for the EEG eye state dataset. Identically to the results of the previous experiment, a linear increase in runtime with respect to the network size can only be noticed when testing on large datasets containing 16,348 or 32,768 rows. Figure 10: Distribution of runtime results using a network of 13 peers with the dataset of 32,768 records in the model validation experiment. The median runtime for model validation was 1 min 19 s for the chaincode (**left**) and 1 min 19 s for the oracle service (**right**). \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multicolumn{1}{c}{**Data**} & \multicolumn{5}{c}{**Number of Peers in the Blockchain Network**} \\ \cline{2-7}
**Records** & **3** & **5** & **7** & **9** & **11** & **13** \\ \hline
1024 & 2.66 & 2.56 & 6.28 & \(-\)4.31 & \(-\)1.51 & 6.59 \\
2048 & 2.18 & 4.48 & \(-\)0.63 & 3.72 & 4.99 & 5.31 \\
4096 & 1.17 & 5.61 & 3.93 & 5.37 & 4.27 & 4.89 \\
8192 & 3.16 & 5.01 & 2.85 & 3.79 & 2.96 & 3.59 \\
16,348 & 4.28 & 1.20 & 0.32 & 1.36 & 1.02 & 0.77 \\
32,768 & 2.79 & 0.73 & 0.63 & 0.40 & 0.05 & 0.03 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Model inference overhead (in%) for the synthetic dataset, calculated as a median of 100 runtimes time ratios. The overall median from 3600 runs (6 \(\times\) 6 \(\times\) 100 runtime ratios) was 1.99% and the overhead was above zero for 69.15% of all runs. For the largest dataset scenario, we can observe a linear relationship between the network size and model inference runtime (see Figures 11 and 12) except for outliers consistently resulting from the first run of the experiment. Similarly, to the first experiment, outliers for the chaincode were fixed at almost a constant low value, while the oracle service outlier values started fluctuating, especially when the network exceeded eight peers. The largest runtimes at the extreme scenario of large network and data settings are shown in Figure 13. The initial runs for both validation types provided the least amount of validation time clearly describing the overhead introduced by the oracle service. All sequential validation run results clustered around the validation time of 1 min and 24 s for the chaincode validation and around 1 min 25 s for the oracle service validation. Figure 11: Model inference runtime (in seconds) depending on the network size for different amounts of EEG (electroencephalogram) eye state dataset. Figure 12: Model inference runtime (in seconds) in relation to the network size for the largest variant of EEG eye state dataset: Chaincode (**a**) and oracle service (**b**) results. The linear relationship is evident, where an additional peer in the network increases runtime T by 6.55 s for the chaincode and 6.63 s for the oracle service. In Table 4, oracle service overheads in percentages are reported, revealing the suitability of the oracle service for the larger dataset and network sizes. Results indicate that the EEG eye state dataset had only positive overheads with the overall median overhead of 4.06%.