# Towards Blockchain-Based Federated Machine Learning: Smart Contract for Model Inference
## 4 Benchmarking Experiments
### Experimental Setup  
The experiments were performed using the architecture described in Section 3.1 on a server powered by an 8 Intel Xeon Silver 4114 CPU running at 2.20 GHz and featuring 16 GB of RAM with data stored on the SSD storage. The server was running an Ubuntu 18.04  
Figure 6: Running example of the _calculatePrediction_ function featuring the logistic regression inference. The shown data instance corresponds to the ground-truth class label, in this case, the target class instance (Data.class = 1), which is not used in the calculations. operating system and using Docker 19.03.6 containers as an environment for Hyperledger Fabric 1.4.9 with CouchDB 2.3.1 as the state database. All peers existing in the blockchain network were required to execute the chaincode to reach a consensus. Experiments were conducted with peers configured as a single organization. Chaincode was developed using Go, and the oracle was implemented as a RESTful service API endpoint using the Go _net/http_ library [35], encapsulating the benchmarked function, with the code content identical to the chaincode implementation. The deployment architecture of the model inference prototype is presented in Figure 7. The main components were hosted on a Linux Ubuntu server using Docker [36] containers. Hyperledger Fabric CLI (command-line interface) was deployed into a node, which was used to experiment by calling chaincode functions. The CLI was linked to the Orderer service and Certificate authority, which issues certificates. The Orderer regulates the transaction, commits among peers, and controls the chaincode execution order. Each of the peer nodes had a Hyperledger solution deployed, which consisted of a model inference chaincode and local ledger copy. Alongside the Hyperledger Fabric solution, a model inference web service was deployed to each node to accurately judge the performance of the oracle web service against the entirely blockchain-based solution. To evaluate the impact on the model inference runtime performance, we conducted model validation experiments on two datasets:  
1. The synthetic dataset contained points in the 2D feature space of two non-linearly separated banana-like shaped classes and were generated using the _make_moons_ function from the _scikit-learn_ Python library [18]. 2. The EEG eye state dataset [19] contained 14 features for detecting if the participant's eyes were open or closed during a 117 s recording of signal through 14 channels of EMOTIV EEG Neuroheadset. The number of rows for both datasets was determined according to the following grid: {1024; 2048; 4096; 8192; 16,348; 32,768}. Since the EEG eye state dataset contained only 14,980 instances it had to be expanded to contain 16,348 or 32,768 rows through bootstrapping of the original dataset. Data were stored using indexing, to speed up data  
Figure 7: Physical view of the utilized blockchain architecture: UML deployment diagram. All components were hosted on the same physical server, using Docker [36] containers. reading during the experiments. The sizes of synthetic and EEG eye state datasets in kB are presented in Table 2. The number of peers participating in the network was set according to the following grid: {3; 5; 7; 9; 11; 13}. These specific network sizes were chosen to represent a gradual increase and the maximum network size was capped according to the known limitations of the Hyperledger Fabric platform [37]. For each combination of data and network size, the blockchain infrastructure was initialized from scratch. For both datasets, we measured model inference runtimes and summarized a comparison between different implementations using the overhead derived by the following ratio:  
\[Overhead=100\times\left(median\left(\frac{T_{OracleAPI}}{T_{Chaincode}}\right)- 1\right) \tag{1}\]  
where \(T\) is a set of runtimes, obtained while repeating the experiment over 100 logistic regression models, which were prepared ahead of the experiments using the _GoML_ library [38] on a random subsample of the dataset.