## 3 Challenges and the State of the Art Progresses
### Performance Metrics  
Because blockchain is implemented without a central authority, the system integrity and availability are enhanced by keeping redundant data across multiple nodes at some expense to performance. Thus, it is important to evaluate the performance metrics with the increasing node number and workload in blockchain. The Hyperledger performance and scale working group published a white paper that defines the platform-agnostic terms and key metrics in evaluating the performance of blockchain [30]. [PERSON] et al. presented a benchmarking framework for evaluating performance of private blockchain, and conducted a comprehensive evaluation of three major blockchain platforms including Ethereum, Parity, and Hyperledger Fabric [46]. In this section, we summarize the important performance metrics and present the performance comparison conclusion drawn by the research work [46] in Table 4. We only list Ethereum, Hyperledger and Parity in Table 4 to compare the performance. The reason is that the key performance metrics such as throughput and latency are only relevant when compared against a transaction of a similar nature on an identical set of constraints. The various blockchain technologies differ in how they define a transaction. For Ethereum, Hyperledger and Parity, the transaction is measured as the duration between the submission of a transaction to the network and the confirmation of the transaction to a block. For other blockchain platforms, the definition of a transaction might be very different. For instance, unlike the traditional blockchain, IOTA is a technology that is based on Tangle. IOTA uses a whole network, i.e., the tangle, instead of a chain to map transactions. Theoretically, IOTA will enjoy good throughput, latency and scalability since it trades the strict data consistency for performance improvement. Thus, it is difficult if not impossible to directly compare IOTA with other chain-based blockchain platforms in an identical environment. We also list IOTA in the performance comparison table, where the performance is evaluated with different methods or analyzed theoretically compared with the other three platforms. * Throughput refers to the number of successful transactions that a blockchain platform can process per second. * Latency is the response time per transaction of the blockchain platform. * Scalability evaluates the changes in throughput and latency when the blockchain experiences the increase of nodes and concurrent workloads. * Fault tolerance evaluates the changes in throughput and latency during node failure including system crashes, network delays and random message corruptions. * Security metrics are defined as the ratio between the total number of blocks of the main branch and the total number of confirmed blocks. When the ratios is high, the blockchain system is vulnerable to the security attack resulted from double spending or selfish mining. [PERSON] et al. conducted a comprehensive empirical study to characterize the performance of Hyperledger Fabric [47]. The study was conducted in an environment of eight peers and one orderer node with a kafka-zookeeper. All peers and kafka-zookeeper ran on the x86_64 virtual machines with 32 vCPUs and 32 GB of memory, and were connected to the 3 Gbps Datacenter network. The experiment increased the transaction arrival rate until it reached the saturation, and repeated the tests with varied block size. The work concludes that, with an increase in transaction arrival rate, the throughput increases linearly as expected until it reaches the maximum throughput around 140 tps. When the transaction arrival rate reaches the maximum throughput, the latency increases dramatically, more specifically from dozens or several hundred of milliseconds to hundreds of seconds. Furthermore, for an arrival rate less than the maximum throughput, the latency increases significantly as the block size increases; when the arrival rate reaches the maximum throughput, the latency decreases slightly as the block size increases.