# A Coordination Technique for Improving Scalability of Byzantine Fault-Tolerant Consensus
## 4 Evaluation
### RQ1: How Much Can the Scalability of PBFT Be Increased through Our Proposed Approach?  
This first research question is intended to figure out how much our suggested approach achieves our research aim which is to improve the scalability of the BFT-based consensus algorithm. We selected and implemented the PBFT consensus algorithm for this research question, which is the most popular BFT-based consensus algorithm. Then, we measured the performance of the PBFT equipped with and without our approach. Furthermore, we increased the number of nodes to figure out the scalability of our approach.  
Experimental setting for _RQ1_. To respond to RQ1, we built the PBFT network (Our source code for implementing the PBFT network is available at [[https://github.com/jungwonrs/JwRalph_Seo/tree/master/lab/Agent_Consensus](https://github.com/jungwonrs/JwRalph_Seo/tree/master/lab/Agent_Consensus)]([https://github.com/jungwonrs/[PERSON]_Seo/tree/master/lab/Agent_Consensus](https://github.com/jungwonrs/JwRalph_Seo/tree/master/lab/Agent_Consensus))) based on the [PERSON] and [PERSON]'s research [13; 14]. Initially, we structured four nodes and issued transactions every 10 ms for 10,000,000 (ms), so that we transmitted one million transactions for 2 h 40 min. In addition, we set a block generation time to be 10 ms, which implies that each node generates a new block every 10 s with transactions in their transaction pools (by using the sensitivity analysis, we obtained 10 s as the best block generation time for the best performance.). Then, we measured the total elapsed time until the consensus process of transactions is complete and computed an average elapsed time. We prepared 81 physical computers and deployed 80 PBFT nodes and one consensus coordinator into each computer. The hardware specification of each computer was Intel i5-3570 3.4 GHz CPU with 4 GB RAM, and Windows 10 OS installed.  
Experimental Result for _RQ1_. Figure 4 shows the result of the experiment. In the initial experiment with four nodes, the PBFT with the consensus coordinator expressed in _CC + PBFT_ achieved 0.0328 s for each transaction on average, while PBFT without our approach denoted as _PBFT_ showed 0.1237 s. The gap of the elapsed time of two approaches grew as the number of nodes increased. When the number of nodes reached 80, the elapsed time of PBFT and CC + PBFT became 1.1191 and 6.2212 s, respectively. Thus, the PBFT equipped with our approach obtained 3.77 times (=0.1237/0.0328) higher performance than PBFT with the initial four nodes. The performance gain was increased so that the PBFT with our approach achieved 5.56 (=6.2212/1.1191) times higher performance with 80 nodes. In all experiments throughout node changes, the performance of the PBFT with our approach increased an average of 4.75 times compared to the use of the PBFT alone. Therefore, we can recognize that our approach contributed to improving the performance of the PBFT consensus algorithm.  
In addition to this, we observed that the elapsed time increase rate of the PBFT is bigger than that of CC + PBFT. While the increase rate of the PBFT from 4 to 80 nodes is 50.29 times (=6.2212/0.1237), that of the CC + PBFT is 34.12 (=1.1191/0.0328). It implies that our approach contributes to improving PBFT consensus algorithm's scalability depending on the increase of nodes compared to only using the PBFT. The reason for the increase of the elapsed time of PBFT is because all nodes in the PBFT algorithm should participate in consensus process and the number of communications. In addition, the consensus process should always be executed for all transactions (i.e., one million transactions). However, our approach checks the equality of transactions and executes the consensus process only for trouble transactions. Thus, depending on the proportion of trouble transactions associated with the number of the nodes, the elapsed time of CC + PBFT is increased but not as steeply as that of PBFT.  
Figure 4: Elapsed time comparison between PBFT and PBFT equipped with the Consensus Coordinator (CC).