# Fast Performance Modeling across Different Database Versions Using Partitioned Co-Kriging
## 5 Experimental Evaluation
### Experimental Settings  
**Subject databases, versions, and benchmarks**. We carried out an investigation into the subject systems of researches on configuration tuning for database systems [2; 6; 7; 8; 9; 10; 14; 16; 28; 29; 31; 39; 40; 41], and chose five widely used database systems to evaluate PCK approach: MySQL ([[https://www.mysql.com/](https://www.mysql.com/)]([https://www.mysql.com/](https://www.mysql.com/)) (accessed on 25 September 2020)), PostgreSQL ([[https://www.postgresql.org/](https://www.postgresql.org/)]([https://www.postgresql.org/](https://www.postgresql.org/)) (accessed on 26 September 2021)), SQLite ([[https://www.sqlite.o](https://www.sqlite.o)]([https://www.sqlite.o](https://www.sqlite.o)) rg/ (accessed on 26 September 2021)), Redis ([[https://redis.io/](https://redis.io/)]([https://redis.io/](https://redis.io/)) (accessed on 11 October 2020)), and Cassandra ([[http://cassandra.apache.org/](http://cassandra.apache.org/)]([http://cassandra.apache.org/](http://cassandra.apache.org/)) (accessed on 13 October 2020)). MySQL is an open-source relational database management system (RDBMS). PostgreSQL is an open-source object-relational database management system (ORDBMS). SQLite is an open-source embedded RDBMS. Redis is an open-source in-memory data structure store. Cassandra is an open-source column-oriented NoSQL database management system. In this experiment, we choose two (PostgreSQL, SQLite, and Cassandra) or three (MySQL,Redis) representative versions for convenience. The time gaps among these releases of three database are about five years for PostgreSQL, nearly a half year for SQLite, about five and a half years for Cassandra, and approximate three years for MySQL and Redis. In addition, we use sysbench ([[https://github.com/akopytov/sysbench](https://github.com/akopytov/sysbench)]([https://github.com/akopytov/sysbench](https://github.com/akopytov/sysbench)) (accessed on 25 September 2020)) for MySQL, pgbench ([[https://www.postgresql.org/docs/11/pgbench.html](https://www.postgresql.org/docs/11/pgbench.html)]([https://www.postgresql.org/docs/11/pgbench.html](https://www.postgresql.org/docs/11/pgbench.html)) (accessed on 26 September 2021)) for PostgreSQL, a customized workload for SQLite, YCSB [45] for Cassandra, and Redis-Bench ([[https://redis.io/topics/benchmarks](https://redis.io/topics/benchmarks)]([https://redis.io/topics/benchmarks](https://redis.io/topics/benchmarks)) (accessed on 11 October 2020)) for Redis. **Parameters**. For each database system, we use domain expertise to identify a subset of parameters that are considered critical to the performance, as in [2,9,10]. Reducing the number of considered parameters can reduce the search space exponentially, and numerous existing approaches [9,12] also adopt this manual feature selection strategy. Note that even with only these parameters, the search space is still enormous, and exhaustive search is infeasible. Table 1 summarizes the database systems and versions, along with the benchmarks, the numbers of selected parameters, and performance metrics, respectively. **Running environment**. In order to avoid interference in collecting samples from different subject database systems, we conduct experiments on different servers and computer. In addition, we ensure a consistent running environment for different versions of the same subject system. The running environments for different subject databases systems are listed as follows. MySQL, PostgreSQL: The physical server is equipped with two 2-core Intel(R) Core(TM) i5-4590 CPU @3.30 GHZ processors, 4 GB RAM, 64 GB disk, and running CentOS 6.5 and Java 1.8.0. SQLite: The computer is equipped with a Intel(R) Core(TM) i5-4460 CPU @3.20 GHZ processors, 8 GB RAM, 1 TB disk, and running Windows 10 and Java 1.8.0_291. Redis: The cloud server is equipped with two 2-core Intel(R) Xeon(R) Platinum 8163 CPU @2.50 GHz processors, 4 GB RAM, 53.7 GB disk, and running CentOS 7.6 and Java 1.8.0_261.