GPU accelerated blockchain over key‐value database transactions

IET BlockchainVolume 2, Issue 1 p. 1-12 ORIGINAL RESEARCH PAPEROpen Access GPU accelerated blockchain over key-value database transactions Konstantinos Iliakis, Corresponding Author Konstantinos Iliakis kiliakis@microlab.ntua.gr orcid.org/0000-0002-1403-6851 Microprocessors and Digital Systems Laboratory, Department of Electrical and Computer Engineering, National Technical University of Athens, Zografou Campous, 9, Iroon Polytechniou str, Athens, Greece Correspondence Konstantinos Iliakis, Microprocessors and Digital Systems Laboratory, Department of Electrical and Computer Engineering, National Technical University of Athens, Zografou Campous, 9, Iroon Polytechniou str, Athens 15780, Greece. Email: kiliakis@microlab.ntua.grSearch for more papers by this authorKonstantina Koliogeorgi, Konstantina Koliogeorgi orcid.org/0000-0003-0064-7616 Microprocessors and Digital Systems Laboratory, Department of Electrical and Computer Engineering, National Technical University of Athens, Zografou Campous, 9, Iroon Polytechniou str, Athens, GreeceSearch for more papers by this authorAntonios Litke, Antonios Litke orcid.org/0000-0002-7658-2559 Distributed Knowledge and Media Systems group, Department of Electrical and Computer Engineering, National Technical University of Athens, Zografou Campous, 9, Iroon Polytechniou str, Athens, GreeceSearch for more papers by this authorTheodora Varvarigou, Theodora Varvarigou Distributed Knowledge and Media Systems group, Department of Electrical and Computer Engineering, National Technical University of Athens, Zografou Campous, 9, Iroon Polytechniou str, Athens, GreeceSearch for more papers by this authorDimitrios Soudris, Dimitrios Soudris orcid.org/0000-0002-6930-6847 Microprocessors and Digital Systems Laboratory, Department of Electrical and Computer Engineering, National Technical University of Athens, Zografou Campous, 9, Iroon Polytechniou str, Athens, GreeceSearch for more papers by this author Konstantinos Iliakis, Corresponding Author Konstantinos Iliakis kiliakis@microlab.ntua.gr orcid.org/0000-0002-1403-6851 Microprocessors and Digital Systems Laboratory, Department of Electrical and Computer Engineering, National Technical University of Athens, Zografou Campous, 9, Iroon Polytechniou str, Athens, Greece Correspondence Konstantinos Iliakis, Microprocessors and Digital Systems Laboratory, Department of Electrical and Computer Engineering, National Technical University of Athens, Zografou Campous, 9, Iroon Polytechniou str, Athens 15780, Greece. Email: kiliakis@microlab.ntua.grSearch for more papers by this authorKonstantina Koliogeorgi, Konstantina Koliogeorgi orcid.org/0000-0003-0064-7616 Microprocessors and Digital Systems Laboratory, Department of Electrical and Computer Engineering, National Technical University of Athens, Zografou Campous, 9, Iroon Polytechniou str, Athens, GreeceSearch for more papers by this authorAntonios Litke, Antonios Litke orcid.org/0000-0002-7658-2559 Distributed Knowledge and Media Systems group, Department of Electrical and Computer Engineering, National Technical University of Athens, Zografou Campous, 9, Iroon Polytechniou str, Athens, GreeceSearch for more papers by this authorTheodora Varvarigou, Theodora Varvarigou Distributed Knowledge and Media Systems group, Department of Electrical and Computer Engineering, National Technical University of Athens, Zografou Campous, 9, Iroon Polytechniou str, Athens, GreeceSearch for more papers by this authorDimitrios Soudris, Dimitrios Soudris orcid.org/0000-0002-6930-6847 Microprocessors and Digital Systems Laboratory, Department of Electrical and Computer Engineering, National Technical University of Athens, Zografou Campous, 9, Iroon Polytechniou str, Athens, GreeceSearch for more papers by this author First published: 04 March 2022 https://doi.org/10.1049/blc2.12011AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat Abstract Blockchain is a distributed ledger based on peer-to-peer networks, originally used for crypto-currency systems. Blockchains are being used as an enabling technology for decentralised applications in the areas of Internet-of-Things, finance, supply-chain and others. Consistency, data privacy, performance, and energy efficiency are of paramount importance in such applications. The full nodes of public, permissionless blockchains undertake the task of verifying the transactions generated by the network. Full nodes perform operations such as confirming balances, transactions, and history, i.e. mostly database search queries. Consequently, their throughput is crucial for the performance of blockchain systems. In this work, the benefits of accelerating the blockchain search and insert queries by leveraging GPU platforms are studied. An extensive comparison between the most dominant utilized database is provided, i.e. LevelDB, and MegaKV, a high-performance GPU-accelerated database. Realistic operations that take place in blockchain systems are emulated and evaluated over representative scenarios, showing three orders of magnitude gains in throughput and energy efficiency without compromising the security aspect. The extensive comparison between LevelDB and MegaKV indicates that GPU acceleration is an effective solution for runtime and energy efficiency enhancement of blockchain systems, and the integration of the two technologies is a promising field of research. 1 INTRODUCTION In the finance industry blockchain is broadly used in crypto-currency applications (e.g. bitcoin [1], litecoin [2], ethereum [3]) and other financial practices such as cross-border transactions, trade platforms, clearing and settlements, digital identity verification, and credit reporting. However the use of blockchain is more broad and the technology is adopted by various domains and applications, that are characterised by similar inherent prerequisites for their transactions. Blockchain technology fulfils these requirements and provides decentralisation, persistency, anonymity and auditability, to name a few of its advantages. Thanks to these attributes, it is further adopted in fields such as distributed cloud storage, smart property, Internet of Things, supply chain management, healthcare, ownership and royalty distribution, and decentralised autonomous organisations. A blockchain is a decentralised, distributed, and digital ledger, consisting of records called blocks, that is used to record transactions across many computers so that any involved block cannot be altered retroactively, without altering all subsequent blocks. A blockchain database is managed autonomously using a peer-to-peer network and a distributed timestamping server. Peer-to-peer blockchain networks eliminate central vulnerability points. Every node in a decentralised system has a copy of the blockchain and transactions are broadcast to all the nodes of the network. Nodes are typically classified into two categories: full nodes and light nodes. Full nodes include a single copy of an entire blockchain history including transactions, timestamps and all created blocks. Light nodes are much smaller in size and only hold information about partial blockchain histories, usually associated with a specific blockchain user. In typical blockchain applications, the network is accessed by a vast number of users that issue requests, verified by a relatively small number of full nodes. Therefore, blockchain search queries, such as confirming balance, transaction contents, and transaction histories, are handled by full nodes. As a result, the full node search throughput forms a bottleneck in blockchain systems. Hardware acceleration techniques in the blockchain research area focuses, among others, on topics such i) blockchain searching, ii) hardware-assisted trusted execution environments, iii) cryptographic acceleration, iv) cryptocurrency mining and other topics, something which reveals the importance of this research area as an enabling factor for the mass deployment of blockchains in widespread applications [4]. There are several types of hardware accelerators available today implemented mainly through field-programmable gate arrays (FPGAs), graphic processing units (GPUs) and application specific integrated circuits (ASICs). Today, the major proof of work, public blockchains are massively relying on GPU and ASIC based hardware acceleration techniques [4]. A GPU is a single-chip processor primarily used to manage and boost the performance of video and graphics, as their highly parallel structure makes them very efficient for algorithms that process large blocks of data in parallel. Modern applications in the areas of bioinformatics and artificial intelligence are also gaining significant benefits from the usage of this technology. ASICs are highly optimised circuits dedicated to a specific domain of applications ranging from communications and networking to computing and storage, making them particularly optimal for special purpose computing as it is the case for consensus algorithms such as the SHA-256. Most blockchain systems store blocks and metadata of transactions utilising NoSQL databases that meet the specific requirements of the application (consensus of nodes, fault tolerance). A common choice for many cryptocurrency applications is LevelDB [5]. Such databases however, are not necessarily the best in terms of performance or energy efficiency. In fact, there is a lack of literature review and studies that examine the use of alternative databases and technologies to accelerate the search part of the application. The database research community has identified GPUs as effective co-processors for data processing and many approaches have proposed to make use of GPUs at different levels of a database system. In this work, we examine the prospect of enhancing the performance of blockchain systems by leveraging GPU acceleration capabilities. Specifically, we target the key-value store operations performed at the heart of the search-insert transactions of the most widely used LevelDB database. We compare the performance of LevelDB operations with the respective ones performed by a GPU-based key-value store system, called MegaKV [6]. The value of this paper lies on implementing, benchmarking, and demonstrating the potential of a hybrid blockchain system that combines CPUs and GPUs to provide higher transaction processing throughput. For that purpose, we perform an extensive literature review and gather representative types of key-value store operations in terms of size and frequency. Then, we evaluate the two databases through a set of realistic blockchain transactions, that cover a wide range of real-world scenarios and stress the system's limits. The results show that a GPU-accelerated key-value store like MegaKV can offer two to three orders of magnitude greater performance compared to conventional CPU-based databases. This gain in performance is accompanied with significant reduction in terms of energy consumption, leading to greener blockchain applications. The proposed solution is fully compatible with existing blockchain systems and can be optionally leveraged by GPU-enable nodes for performance boost without interfering with normal operation. We believe that this work can motivate researchers to adopt GPU-accelerated databases in blockchain technologies, that can widen the applicability and popularity of blockchain systems even further. In the remainder of the paper, we elaborate on the blockchain application, the leveraged technologies and the benchmarking methodology followed. In Section 2 we present a literature overview of performance optimisations applied on blockchain systems with a special focus on performance optimised key-value stores and heterogeneous accelerating technologies. Followingly, in Section 3 we provide the theoretical background of blockchain applications. An extensive presentation of the key-value store implemented in LevelDB, the GPU execution model and the MegaKV architecture facilitate a better understanding of the acceleration achieved by the latter GPU-based key-value store system. In Section 4, we describe the benchmarking methodology followed to evaluate the two databases in respect to the chosen key-value sizes and transaction scenarios, and in Section 4.2 we compare the performance of the two aforementioned database systems. Finally, Section 5 concludes this paper. 2 RELATED WORK 2.1 Performance optimised blockchain systems To the best of our knowledge, there is no other prior work studying the feasibility and performance of a GPU-accelerated blockchain database. In this section we overview alternative techniques that aim to optimise the performance of blockchain platforms. Optimisation of blockchain systems is a challenging task that has been the objective of active research. The requirements of security, decentralisation and scalability often contradict each other and raise challenges for most optimisation techniques [7]. Currently, the most popular methods to improve the performance and scalability of blockchain system are sharding, directed acyclic structure (DAG), scalable consensus, side-chain, on-chain and system optimisation. Sharding is a traditional database technology applied to the blockchain architecture. According to this method, each block is fragmented into smaller pieces called shards and each node only needs to verify the transactions of its own shard. This reduces the verification operations performed per node, allows parallel processing of a single block by multiple nodes and improves the processing speed of the blockchain [8, 9]. Another popular method is designing the blockchain as a directed acyclic graph (DAG). The DAG topology is based on a tree structure and improves scalability by allowing asynchronous and concurrent write transactions [10]. Changing the consensus protocol can also improve performance and scalability. The various consensus protocols [11-13] exhibit both advantages and disadvantages. Several blockchain systems use the Proof-of-Work (POW) consensus algorithm [1, 3, 14]. This algorithm requires the nodes which participate in the consortium to calculate hash signatures to ensure the system's correctness. As a result a lot of computing power is required, and the throughput of POW-based systems is usually low. For instance, bitcoin can confirm on average three to four transactions per second (TPS) and ethereum about 14 TPS. To increase the system's transaction throughput, alternative distributed consensus algorithms have been proposed such as the proof-of-stake [15] (POS), delegated POS [16] (DPOS), proof-of-vote [17] (POV) and practical Byzantine fault tolerance (PBFT) [18]. Those approaches usually manage to speedup the block generation procedure and reduce the energy footprint, at the cost of increased transaction confirmation latency or weakened security. Side-chain techniques [19] have also proven to be efficient. These techniques improve scalability by locking a part of the blockchain on a main chain first, and perform currency operations on the side chain. After the operations are complete, the main chain is updated and verified. On-chain solutions [20-22] on the other hand, directly modify the blockchain design to support high performance. A key-value store, is a data storage technique aiming to store, retrieve, and manage associative arrays implemented as hash table data structures. There are various research efforts that deal with accelerating key-value stores (KVS) which store transmission IDs (TXID) as a key and Merkle root as a value. For instance, the authors in [23] present a technique which takes advantage of the reduced latency dynamic random access memory (RLDRAM) of the FPGA to gain latency reduction showing an improvement of blockchain queries throughput by almost a twofold factor. An alternative to FPGAs for searching is presented in ref. [24] where the authors propose an acceleration method of blockchain search using GPUs. They introduce an array-based Patricia tree structure suitable for GPU processing which makes more effective use of the blockchain features of having no update and delete queries. Despite the significant efforts and contributions made by the aforementioned systems, we observe that they manage to accomplish at best two of the following three desired properties: high performance, decentralisation, and security. Our approach, however, is orthogonal to the blockchain's decentralisation and security assurances, and optimises the database used to store and retrieve transaction and block records. Therefore, by optimising the blockchain database we manage to optimise the scalability and throughput of the system without compromising the systems security or decentralised nature. 2.2 Performance optimised key-value stores Database systems can be divided in SQL or relational, and NoSQL or non-relational databases. SQL databases are table based and are better suited for storing structured datasets. On the other hand, NoSQL databases are better suited for handling unstructured data and depending on their internal organisation and data representation, can be categorised as document, key-value, graph, or wide-column stores. Such databases can benefit from GPU-acceleration and an extensive overview of such systems is reported in [25]. The majority of this works concern SQL and relational databases which are evaluated with fixed datasets. In the scope of this paper, we focus on NoSQL, key-value databases, since this is the databases of choice of most blockchain systems [1-3]. To the best of our knowledge, this is the first attempt to study how GPU acceleration copes with the unique transaction patterns of a blockchain system and improves their performance. Key-value databases have mainly leveraged CPU-based optimisation solutions so far. A plethora of such solutions can be found in the literature [26-29]. Recent advances in DRAM technologies have increased the DRAM densities and decreased the prices, making CPU servers with DRAM capacities in the order of terabytes affordable [30]. As a consequence, key-value stores that keep the entire database in-memory have emerged [31-33]. To achieve maximum performance, CPU-based in-memory key-value stores optimise the index data structure, and the network packet processing phase [34-37]. Taking advantage of cutting-edge interconnect fabrics such as RAM-Cloud [33], Infiniband [38], and Pilaf [37] demonstrate query processing latencies in the order of 5–10 μ $\mu$ s. Utilising GPUs to accelerate key-value database accesses is a more recent trend. To profit from the massive throughput capacity of GPU platforms, GPU-based key-value stores batch incoming requests, trading-off higher latency for higher throughput. Consequently, the response time of GPU-based databases can be up to two orders of magnitude higher than CPU implementations [39]. Memcached, one of the most popular in-memory key-value stores, has been ported to GPU platforms using OpenCL [40]. However, the OpenCL implementation of Memcached fails to take advantage of the GPU-specific hardware characteristics. MegaKV [6] manages to outperform other GPU-based key-value stores by decoupling the index data structure from the key-value data, and offloading the index data structure on the GPU. The most expensive operations such as search, insert and delete take place on the GPU, returning only the location of the stored item to the CPU to alleviate the CPU–GPU memory transfer bottleneck. MegaKV uses a GPU-optimised Cuckoo hash table [41] at its core. Finally, the query processing kernels are run periodically in pre-defined intervals, placing an upper bound to the response latency while taking advantage of the SIMT [42] GPU model for batch processing. 3 BACKGROUND This section introduces and provides basic information about the main components of this paper, i.e. blockchain systems, databases, and GPU accelerators. This paper targets both blockchain developers and GPU experts, which are two diverse fields. Therefore familiarisation with these concepts is essential to more easily understand the work presented in this paper. 3.1 Blockchain functionality At its core, a typical blockchain system is composed of a linked-list (chain) of data-sets (blocks). An simple example of a block-chain can be seen in Figure 1. Each data block contains multiple transactions, and the complete blockchain contains the entire transaction history of the system. The transaction details are not the only data stored in a block. Metadata such as a creation time timestamp, and a number verifying the hash of the block (nonce) are also stored inside the block, but most importantly, every block contains a cryptographic hash value of its direct ancestor. This is crucial for the integrity of the blockchain, since to modify a single block, one would have to modify the entire blockchain from this point to the end of the chain. The first block of the chain is the only with no ancestor and is often referred to as the genesis block. New blocks can be added and extend the blockchain. Every time a new block is generated, by a so-called miner, a consensus algorithm is run among the nodes participating in the blockchain's network, and only if the majority of the nodes agree for the validity of the transactions listed in the block, the newly generated block can be appended. Upon validating and adding the block in the chain, the block is considered part of the blockchain's history, and can never be altered. Based on these principles, blockchain systems need can be used to process money transactions (i.e. crypto-currencies), as well as all sorts of asset transfers among peers (i.e. smart contracts). FIGURE 1Open in figure viewer Example of a blockchain. The initial block is also called genesis block. Every block contains a pointer to each predecessor, together with transaction related data and other meta-data Compared to conventional centralised services, decentralised blockchain systems offer many benefits. A third-party authority is not required to validate and keep record of the transactions, minimising the book-keeping fees. Furthermore, an asset ownership transfer usually involves handling delays, up-to several days or weeks. With blockchain systems, transfers take effect instantaneously and require a short interval (i.e. 10 min for bitcoin) to be validated and committed. Users of the system do not need to trust their peers or an intermediary, but the trust is embedded into the system via the crypto-graphic procedures and consensus protocols. Finally, contrary to centralised systems, blockchain networks are fault-tolerant and can withstand node multiple hardware failures and network disconnects. Although crypto-currencies were the first blockchain applications, a plethora of alternative applications have been suggested [3]. Smart-contracts [43] essentially rely on a blockchain system to keep track and ensure the validity of an arbitrary contract among peers. These contracts might deal with the ownership of properties such as houses, stock market shares or even artwork copyrights. The ethereum [3] network for instance, supports smart contracts. Blockchain models can be potentially applied in fields that traditionally require a central authority to establish trust, such as government services and elections [44]. Some more visionary and forward-looking supporters of the blockchain technology [45-47] believe that decentralised platforms will disruptively innovate and lead to major refactoring of present business models, industry sectors or even societies' organisation. 3.2 In-memory key-value stores and level DB Key-value store databases are widely adopted by single servers, across data centres and in the cloud thanks to their resilience and fault tolerance. Their distributed architecture lacks single point of failure and is characterised by built-in redundancy of both function and data. If one or more database nodes collapse, the rest of the nodes in the system continue their operations without compromising data. In a key-value store system, a database is a schema-less collection of key-value pairs, with data access operations on either individual key-value pairs or groups of related pairs (e.g. sharing part of the key) [48]. Each key includes a pointer to a particular value in the database. The values can only be accessed by their key. The keys are often organised in a hash table, that controls the distribution of data in the database. Some key-value stores hold the whole data in-memory, also called in-memory key value stores (IMKVS), which is essential to their high throughput. A typical IMKVS system can be accessed through an application programmable interface (API) that supports functions like get(key), set(key, value), and delete(key). The get(key) function retrieves the value stored that is associated with the key of the user query. The system searches for the key value in the index data structure, e.g. hash table, and if it is located, the respective value is fetched from the memory and returned to the user. In case of a set(key, value) query, the system stores a new key-value item in the database. For that purpose, memory is allocated for both the key and the value of the inserted item. The key is stored in the index structure, whereas the value in memory. Key-value store systems also support delete(key) operations. According to these operations both the key and value are removed from their respective locations and the memory is released. Therefore IMKVS systems perform network processing operations, memory operations such as allocation, deletion, read and write, and search index operations. Until recently, the bottleneck in such systems was the network speed. At the same time typical IMKVS systems cannot benefit from multi-threading due to atomic operations and the use of locks. Several optimisation techniques have resolved these issues and the bottleneck has shifted to be the performance gap between the CPU and the main memory. Recent research works [49, 50] focus on accelerating the memory access pattern of index operations in order to improve the overall performance of an IMKVS system. LevelDB [5] is a fast key-value store library developed by Google, that is widely used in blockchain applications, e.g. bitcoin[1], litecoin[2], ethereum[3]. The key characteristic of LevelDB is that it is based on log structure merge (LSM) trees and therefore provides an ordered mapping from string keys to string values. LevelDB enhances the simple operations supported by most key-value stores by functions to increase performance and fault tolerance[51]. The data is stored in sorted by key order. LevelDB also introduces a new operation of GetSnapshot(), which automatically saves previous states of the index table and therefore previous versions of values for different keys. The write() operation is also optimised by enabling to batch multiple writes together. Search operations are improved through the use of iterators that allows more efficient scanning and jumping to specific keys. Lastly, data is automatically compressed using the Snappy compression library. The LevelDB architecture comprises of an on-disk log file, two in-memory sorted skiplists and seven levels of on-disk sorted string table (SST) files. The SST file is the main unit for storing data in key order. These data include the type of compression, data blocks, index blocks, meta-index blocks, and filter blocks. The structure of the main blocks is illustrated in Figure 2. Data blocks contain a set of key-value pairs which is prefix-compressed in order to facilitate quick key lookups. The index blocks contain the first key to each data block. The meta-index block consists of a pointer that points to the filter block. This block indicates missing keys and saves considerable time from expensive block searches. FIGURE 2Open in figure viewer Structure of data and index blocks in LevelDB The overall architecture of LevelDB is depicted in Figure 3. LevelDB serves the write queries following a strategy that inserts new items in the hierarchy of memory starting from top to bottom and propagates them into larger memory when there is no more space in the first levels. Initially, LevelDB stores a new key-value pair in a log file and then in the in-memory memtable. Once these are full, LevelDB moves on to other memtables and log files for new write queries. In the meantime, the previous memtable becomes an immutable memtable and is being propagated to disk in the form of a new SST file. These SST fill the available levels from smaller to larger ones. Similar to write operations, LevelDB handles read queries by searching the available levels in the same order. The search begins from the memtable and continues to the immutable table before expanding to the several levels of SST starting from level 0. The search is further optimised by avoiding overlapping of keys within files of the same level. FIGURE 3Open in figure viewer LevelDB architecture and read/write protocol 3.3 GPU execution model and megaKV CPUs are traditionally tailored for high single-core performance utilising low-latency large memories and superscalar, speculative execution pipelines. On the contrary, GPUs are throughput-oriented machines. To provide maximum performance, GPUs employ massive multi-threading combined with single instruction, multiple data (SIMD