Package: amoeba-db
AMOEBA-6422Environment SetupInstall dependenciesInstall postgresql-12 (https://www.postgresql.org/download/)Download demo data fromdropboxand unzip it todatafolderStart postgresql server with data folder:/usr/lib/postgresql/12/bin/pg_ctl -D ./data startIf you install postgresql-12 from apt, stop it viasudo systemctl stop postgresql.serviceand give permission to the lock file viasudo chmod o+w /var/run/postgresqlInstall dependenciessudo apt-get install python3-dev libmysqlclient-dev(if not Debian/Ubuntu, seehttps://github.com/PyMySQL/mysqlclient#installfor more details)pip install -r requirements.txtfor python dependenciesSetup environmentInstallation ofdirenvis needed. And runeval "$(direnv hook bash)"to setup the environment (replacebashwithzshor any shell you're using).Build theMUTATORofAMOEBA(only necessary if you make changes to theMUTATOR)cd /workspace/calcite-fuzzing && ./gradlew build -x core:checkstyleMain -x test -x core:checkstyleTest -x core:forbiddenApisTest -x core:autostyleJavaCheckQuick StartAMOEBAis configurable, a launch command template looks like the following:$timeout{total_timeout}./test_driver.py--workers{num_workers}--output{outputfolder}--queries{num_queries_per_worker}--rewriter./calcite-fuzzing--dbms={dbms_undertest}--validate--num_loops={num_feedbackloops}--feedback={conf_feedback}--dbconf=db_conf_demo.json--query_timeout{per_query_timeout}You can customize the value of the following options:{total_timeout}: timeout for the entire run ofAMOEBA(unit is seconds){workers}: number of parallel workers to invokeGENERATORandMUTATOR{output}: location to store the intermediate results and bug reports{queries}: number of base queries that are generated by eachGENERATORinstance{dbms}: DBMS thatAMOEBAwill evaluate on (i.e.,postgresqlorcockroachdb){num_loops}: number of feedback loops{validate}: a boolean argument that decides whether to invoke theVALIDATORafter generating the equivalent query pairs{feedback}: what types of feedbacks to utilize (i.e.,both,none,mutator, orvalidator){query_timeout}: timeout for executing each query (unit is seconds)For example, you can launchAMOEBAwith the following command:timeout3600./test_driver.py--workers1--output/home/postgres/exp/1--queries200--rewriter./calcite-fuzzing--dbms=postgresql--validate--num_loops=100--feedback=none--dbconf=db_conf_demo.json--query_timeout30IfAMOEBAis working correctly, you should expect to see the following progress information is printed:startquerygenerator['mutator.py --prob_table=/home/postgres/test/190156/prob_table_190156.json --db_info=/workspace/amoeba_conf/db_conf_demo.json -s seq --queries 100 1>/home/postgres/test/190156/0/log_sa0 2>/home/postgres/test/190156/0/input.sql']finishquerygenerator
startqueryrewriter['java -cp calcite-core-1.22.0-SNAPSHOT-tests.jar:./*:. org.apache.calcite.test.Transformer /home/postgres/test/190156/0']finishqueryrewriter
startvalidator
begincompareplancostofequivalentqueries
compareplancost/home/postgres/test/190156/0/out/q20.sql
findplandiff/home/postgres/test/190156/0/out/q20.sql
compareplancost/home/postgres/test/190156/0/out/q13.sql
findplandiff/home/postgres/test/190156/0/out/q13.sql
compareplancost/home/postgres/test/190156/0/out/q18.sql
compareplancost/home/postgres/test/190156/0/out/q23.sqlThis example command should complete within 20 minutes. You can check the generated intermediate results in/home/postgres/exp/1. IfAMOEBAdiscovers potential performance bugs, the generated bug report will live at/home/postgres/exp/1/bugs.md.The shortcut CTRL+C can be used to terminateAMOEBAmanually. Otherwise,AMOEBAwill terminate either after a specified experiment timeout is reached or after a specified number of base queries have been examined. The optiontotal_timeoutcontrols the experiment timeout. The optionsworkers,queries, andnum_loopsaltogether determine the number of base queries thatAMOEBAis going to examine.Notecalcite part is based on Apache Calcite 1.22 and the only new class is added oncore/src/test/java/org/apache/calcite/test/Transformer.javaneed to remove all dependencies in this repo and ask users to download it as needed.grant appropriate permission to your user to access the database. ref:https://stackoverflow.com/a/23934693/10180666