# NewSQL Databases Assessment: CockroachDB, MariaDB Xpand, and VoltDB
## 5 Experimental Evaluation
### Performance Evaluation with Star Schema Benchmark
#### 5.2.2 SSB Performance Evaluation with SF = 1  
In this subsection, we evaluate the performance of CockroachDB, MariaDB Xpand, and VoltDB by analyzing query execution time and the average resource usage of RAM and CPU for each SSB query flight group with a scale factor of 1. It should be noted that the values presented are the average of five runs for each query. The results in Figure 8 show that in general, VoltDB has the best query execution time with SF = 1. Regarding the sum of all SSB queries' execution time, we have 32 ms, 47 ms, and 67 ms for VoltDB, CockroachDB, and MariaDB Xpand, respectively. This is due to the fact that VoltDB is an in-memory database, so the data are easily accessed, while the other databases use disk access. While analyzing VoltDB queries, we observed that from query flight Q2 to Q4, there was a substantial increase in execution time. These queries had a higher execution time than the first query flight Q1 because they searched for information in a larger number of tables and needed to perform more operations to present the results. The remaining group, Q1, searches for information in smaller tables, which explains why there is a shorter execution time. On the other hand, CockroachDB has an almost linear execution time compared to the other two databases. This demonstrates that no matter how complex the query is, the time to process and return a result would be almost the same as the other queries. Lastly, MariaDB Xpand takes more time to execute the queries. In the first flight group, we were surprised that the execution time is almost the same as VoltDB. We were expecting to see close results to VoltDB, but the remaining queries took more time to process,  
Figure 8: Results of query execution time with SF = 1. Figure 7: Results for loading data files with SF = 1 and SF = 10.  
especially Q2.1, Q3.1, Q3.2, Q4.2, and Q4.3, which are the queries that access more tables and return the most data. Figure 9 shows the results of RAM usage in the queries execution with SF = 1. VoltDB presents a better RAM consumption, with an average total of 34 MB, while CockroachDB and MariaDB Xpand results are higher, with 78 MB and 198 MB, respectively. VoltDB has a higher memory consumption in queries Q4.1 and Q4.2, with 85 MB and 98 MB usage, respectively. This last group of queries has more aggregations and data to be retrieved from tables with large data volumes. In the remaining queries, the memory consumption was below 100 MB. On the other hand, CockroachDB is the second database with the highest memory used, with 78 MB. This database displayed better control over the usage of RAM, with small peaks in queries Q1.2, Q3.1, and Q4.1. Moreover, given the complexity of queries executed in this experiment, CockroachDB has a controlled memory usage. MariaDB Xpand is the database with the highest use of memory. This database has an increase of 2.54 times the usage of memory comparing to CockroachDB and 5.83 times the VoltDB usage of memory. MariaDB Xpand requires more RAM when queries with more aggregated operations are performed, as we can see with groups Q2, Q3, and Q4. Group Q1 has one aggregator operation so, the use of memory was relatively low. Figure 10 illustrates the results of CPU used in the queries execution with a SF = 1. MariaDB Xpand is the database with the lowest percentage of CPU usage, with 36.3%, while VoltDB has an average of 40.3%, and CockroachDB with 41.1% of CPU usage. MariaDB Xpand results show an increase in CPU usage from Q1.1 to Q2.2, since more tables were accessed, and from query Q3.2 to Q4.3, we notice a decrease in CPU usage. Even though this group of queries has a higher number of tables with aggregations, MariaDB Xpand displays adequate results with the last group of queries. VoltDB is the second database with the highest use of CPU, with 40.3%. By analyzing the queries during VoltDB operation, we notice a substantial increase in queries Q2.1, Q2.2, Q2.3, and Q3.2.