# NewSQL Databases Assessment: CockroachDB, MariaDB Xpand, and VoltDB
## 6 Discussion  
The main purpose of this paper is to explore the topic of NewSQL databases and to provide an analysis of the best databases, using OSSpal methodology and a standard benchmark (SSB) to evaluate their performance and scalability. The study conducted shows that using OSSpal methodology, MariaDB Xpand achieves 4.16 points, VoltDB achieves 4.10 points, and CockroachDB achieves 3.99 points, as shown in Figure 15. These values are only determined by analyzing and selecting key functionalities and qualities that each database presents, which can bias the results. Despite having close results, each database has been designed to work with large volumes of data, as well as the capability of scaling. Therefore, having assessed each database, we conclude that MariaDB Xpand is highlighted as the tool with the best characteristics, according to OSSpal categories. Figure 14: Sum of queries between SF = 1 and SF = 10. In the experimental evaluation, the Star Schema Benchmark was applied to analyze the performance and scalability of each database, given the scale factor of 1 and 10, corresponding to 1 GB, and \(\sim\)10 GB of data, respectively. Figure 16 illustrates the total amount of time needed to upload 1 GB and 10 GB of data. We presume that each database would take 10 times to upload the data, but the results proved to be better. This is due to the upload system used by each database. CockroachDB uses an import mechanism that allows data to be processed and stored in key-value stores, passing through several layers of its architecture directly to the storage layer, improving the efficiency of data import, ranking first with a total of 139 s. VoltDB uses an integrated Java mechanism called csvloader, which imports CSV files into the database. This utility allows data to be imported efficiently into memory in 397 s, decreasing performance penalties of disk accesses. However, we expected that being an in-memory database, VoltDB would outperform the other two databases. Lastly, MariaDB Xpand uses a mechanism in Python called clustrix_import, which scans the data to know how much space it needs to allocate in storage before importing data. It also uses a rebalancing system that distributes data throughout the existing nodes in the cluster bringing optimization results. This database, despite having similarities to the MySQL storage mechanism, lagged behind CockroachDB and VoltDB. Figure 16: SSB total loading times with SF = 1 and SF = 10. Figure 15: OSSpal evaluation results. Additionally, to evaluate the performance of CockroachDB, MariaDB Xpand, and VoltDB, we used 13 suitable queries provided by SSB benchmark. These queries diverge into four groups with three or four queries each. As the number of groups increases, so does the complexity of queries. These 13 queries comprise several instructions designed to assess each database's capabilities. With SF = 1, VoltDB, being an in-memory database, displayed better results than CockroachDB and MariaDB Xpand. This type of database is generally faster and more efficient than disk databases because data can be accessed and processed directly from memory. Among the four query groups, VoltDB's best performance was with query group 1, which involves less complexity and table scans. This database works better with queries that do not include a large scan of data and complex queries. With this circumstance, the results of the execution time of other queries' performance were noticeable. Even increasing the scalability to SF = 10, VoltDB once again displayed fast execution time. In this part of the assessment, we experienced a decrease in query execution time from group 2 to group 4. CockroachDB was also a database that showed interesting results. We did not expect that there would be an almost linear execution time between all queries, either with SF = 1 or SF = 10. We analyzed each query with the help of the 'EXPLAIN' command to inspect and tune the query. This command is employed to analyze queries with a complex structure that returns large amounts of data.