Benchmarking and evaluation of blockchain systems and applications

Abstract

In the past ten years, we have witnessed the evolution of cryptocurrencies. With market capitalizations of $189bn and $19bn respectively in September 2019, Bitcoin and Ethereum are the world’s most successful cryptocurrencies. Blockchain, which is a public ledger that is immutable, is the main innovation behind these cryptocurrencies. In Bitcoin, the blockchain is introduced to exchange and trade a single asset, whereas in Ethereum it is used to store and execute a smart contract with a Turing Complete Machine. Ethereum’s Gas mechanism, which charges the execution of each operation code, ensures the termination of smart contracts that run in the EVM (Ethereum Virtual Machine) and to compensate the computational usage. Thus, the gas awarded should be proportional to the required computational, to ensure aligned incentives and to avoid denial of services attacks. Currently, in Ethereum, gas awarded is set statically for each opcode in the smart contract, but it is unknown whether these values are correct for various computer architectures. Therefore, in this thesis, firstly, we propose a benchmark approach to measure the CPU times required to deploy and execute real smart contracts obtained from the Ethereum blockchain and compare it with the gas award in the PyEthApp client running over a single machine. The result of our benchmark study shows the collected Gas is not always proportional to the invested CPU for both deploying and executing smart contracts. Secondly, we focus more in-depth on the operational codes (opcodes) and conduct a benchmark study to investigate whether the Gas cost set by Ethereum for each opcode is aligned with the CPU usage. The experiments are conducted on three Ethereum clients running over different hardware platforms and operating systems. The results show that the Gas cost is not always proportional to CPU usage. Finally, we implement and analyze the performance of blockchain and smart contract technologies in different domains, in particular cloud computing and distributed database management systems. In cloud computing, we create the first smart contract implementation that achieves both verifiability and cost-efficiency using any two client providers. For distributed database management systems, we implement the first smart contract-based two-phase commit protocol in Ethereum’s blockchain. For both systems, we investigate the cost, the performance and trade-offs in the blockchain. We tested the implementations of these systems on Ethereum’s official, test and private networks. We also provide a financial and computational analysis of their costs.