# A Coordination Technique for Improving Scalability of Byzantine Fault-Tolerant Consensus
## 4 Evaluation
### Threats to Validity  
Construct ValidityThe results of _RQ1_, _RQ2_, and _RQ3_ may be influenced by hardware specification and the version of Hyperledger Besu. Although there are diverse factors related to performance, the experimental results, such as elapsed times and the number of transactions, can differ. However, we tried to carry out our experiments on the same hardware specification for control and experimental groups, so that the relative comparison of the result is considered desirable for our experiment. In addition, the result of the experiment for RQ3 can be different due to the version of Hyperledger Besu. We selected the 1.5.1 version of Hyperledger Besu in the experiment for RQ3, which was the most recent version at the time. However, the version upgrade is frequent so that the use of different versions would show different results. Content ValidityIn this paper, the concept of scalability is measured by the extent of the decrease in performance, depending on the increase in the number of nodes. Based on this definition, we keep measuring the performance gap, depending on node changes. Besides, we also defined that the term _transaction issue_ indicates that a client issues one transaction, and the transaction is contained in a new block through block generation. However, the block generation interval was set to be every 10 s in \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**10 ms**} & \multicolumn{3}{c}{**25 ms**} \\ _N(Node)_ & **Equal Txs** & **Unequal Txs** & **Equality** & **Inequality** \\ \hline
**4** & 28 & 93.33\% & 2 & 6.67\% & 29 & 96.67\% & 1 & 3.33\% \\
**8** & 27 & 90\% & 3 & 10\% & 30 & 100\% & 0 & 0\% \\
**12** & 25 & 83.33\% & 5 & 16.67\% & 29 & 96.67\% & 1 & 3.33\% \\
**16** & 20 & 66.67\% & 10 & 33.33\% & 26 & 86.67\% & 4 & 13.33\% \\
**20** & 19 & 63.33\% & 11 & 36.67\% & 23 & 76.67\% & 7 & 23.33\% \\
**24** & 16 & 53.33\% & 14 & 46.67\% & 20 & 66.67\% & 10 & 33.33\% \\
**28** & 14 & 46.67\% & 16 & 53.33\% & 16 & 53.33\% & 14 & 46.67\% \\
**32** & 10 & 33.33\% & 20 & 66.67\% & 17 & 96.67\% & 13 & 43.33\% \\
**36** & 4 & 13.33\% & 26 & 86.67\% & 14 & 46.67\% & 16 & 53.33\% \\
**40** & 3 & 10.00\% & 27 & 90.00\% & 12 & 40\% & 18 & 60\% \\ \hline Average & 166 & 55.33\% & 134 & 44.67\% & 216 & 72\% & 84 & 28\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Result of monitoring equal and unequal transactions of consensus coordinator. Figure 6: Comparing the amount of transactions between IBFT and CC + IBFT when the transaction generation interval is 25 ms.  
the experiments for RQ1 and RQ3, while the unit of the measure of the elapsed time must be 10 s, which are not the exact time. Due to this issue, we performed our experiment for 2 h 40 min, which is a long time enough to ignore the 10 s time gap for measuring the average elapsed time for processing a transaction. In the experiment for RQ3, we fixed the experiment time into 30 block generations (i.e., 5 min) to resolve the issue. _Internal Validity_. Experiment results may be affected by different sets of the interval execution of the consensus coordinator (i.e., \(time_{p}\sim time_{c}\)) and the block generation time for PBFT and IBFT in Hyperledger Besu. To handle this issue, we performed the sensitivity analysis and observed that setting the execution time interval of the coordinator and block generation time of PBFT and IBFT to 10 ms showed the best performance. However, the performance may be different depending on the interval setting. _External Validity_. In this paper, we claim that our approach is efficient for BFT-based consensus algorithms, and we applied our approach to two consensus algorithms: PBFT and IBFT. It is hard to claim that our approach applies to all BFT-based consensus algorithms. However, we selected the most popular BFT-based consensus algorithm, PBFT. Many consensus algorithms such as IBFT, Zyzzyva [18], SBFT [19], Hotstuff [33; 34], and Tendermint [26] are derived from PBFT.