Project: TDengine (App Definition and Development / Database)
File: README.md

<center><h1>User's Guide to the Crash_Gen Tool</h1></center> To effectively test and debug our TDengine product, we have developed a simple tool to exercise various functions of the system in a randomized fashion, hoping to expose maximum number of problems, hopefully without a pre-determined scenario. This tool can run as a test client with the following features: Any number of concurrent threads Any number of test steps/loops Auto-create and writing to multiple databases Ignore specific error codes Write small or large data blocks Auto-generate out-of-sequence data, if needed Verify the result of write operations Concurrent writing to a shadow database for later data verification User specified number of replicas to use, against clusters This tool can also use to start a TDengine service, either in stand-alone mode or cluster mode. The features include: User specified number of D-Nodes to create/use. To run this tool, please ensure the followed preparation work is done first. Fetch a copy of the TDengine source code, and build it successfully in the `build/` directory Ensure that the system has Python3.8 or above properly installed. We use Ubuntu 20.04LTS as our own development environment, and suggest you also use such an environment if possible. To run the tool with the simplest method, follow the steps below: Open a terminal window, start the `taosd` service in the `build/` directory (or however you prefer to start the `taosd` service) Open another terminal window, go into the `tests/pytest/` directory, and run `./crash_gen.sh -p -t 3 -s 10` (change the two parameters here as you wish) Watch the output to the end and see if you get a `SUCCESS` or `FAILURE` That's it! This tool also makes it easy to test/verify the clustering capabilities of TDengine. You can start a cluster quite easily with the following command: ``` $ cd tests/pytest/ $ rm -rf ../../build/clusterdnode?; ./crash_gen.sh -e -o 3 # first part optional ``` The `-e` option above tells the tool to start the service, and do not run any tests, while the `-o 3` option tells the tool to start 3 DNodes and join them together in a cluster. Obviously you can adjust the the number here. The `rm -rf` command line is optional to clean up previous cluster data, so that we can start from a clean state with no data at

When the tool runs a cluster, it users a number of directories, each holding the information for a single DNode, see: ``` $ ls build/cluster* build/clusterdnode0: cfg data log build/clusterdnode1: cfg data log build/clusterdnode2: cfg data log ``` Therefore, when something goes wrong and you want to reset everything with the cluster, simple erase all the files: ``` $ rm -rf build/clusterdnode* ``` The DNodes in the cluster all binds the the `127.0.0.1` IP address (for now anyway), and uses port 6030 for the first DNode, and 6130 for the 2nd one, and so on. In a separate terminal window, you can invoke the tool in client mode and test against a cluster, such as: ``` $ ./crash_gen.sh -p -t 10 -s 100 -i 3 ``` Here the `-i` option tells the tool to always create tables with 3 replicas, and run all tests against such tables. The exhaustive features of the tool is available through the `-h` option: ``` $ ./crash_gen.sh -h usage: crashgenbootstrap.py [-h] [-a] [-b MAXDBS] [-c CONNECTORTYPE] [-d] [-e] [-g IGNORE_ERRORS] [-i NUM_REPLICAS] [-k] [-l] [-m] [-n] [-o NUMDNODES] [-p] [-r] [-s MAXSTEPS] [-t NUM_THREADS] [-v] [-w] [-x] TDengine Auto Crash Generator (PLEASE NOTICE the Prerequisites Below) You build TDengine in the top level ./build directory, as described in offical docs You run the server there before this script: ./build/bin/taosd -c test/cfg optional arguments: -h, --help show this help message and exit -a, --auto-start-service Automatically start/stop the TDengine service (default: false) -b MAXDBS, --max-dbs MAXDBS Maximum number of DBs to keep, set to disable dropping DB. (default: 0) -c CONNECTORTYPE, --connector-type CONNECTORTYPE Connector type to use: native, rest, or mixed (default: 10) -d, --debug Turn on DEBUG mode for more logging (default: false) -e, --run-tdengine Run TDengine service in foreground (default: false) -g IGNOREERRORS, --ignore-errors IGNOREERRORS Ignore error codes, comma separated, 0x supported (default: None) -i NUMREPLICAS, --num-replicas NUMREPLICAS Number (fixed) of replicas to use, when testing against clusters. (default: 1) -k, --track-memory-leaks Use Valgrind tool to track memory leaks (default: false) -l, --larger-data Write larger amount of data during write operations (default: false) -m, --mix-oos-data Mix out-of-sequence data into the test data stream (default: true) -n, --dynamic-db-table-names Use non-fixed names for dbs/tables, for -b, useful for multi-instance executions (default: false) -o NUMDNODES, --num-dnodes NUMDNODES Number of Dnodes to initialize, used with -e option. (default: 1) -p, --per-thread-db-connection Use a single shared db connection (default: false) -r, --record-ops Use a pair of always-fsynced fils to record operations performing + performed, for power-off tests (default: false) -s MAXSTEPS, --max-steps MAXSTEPS Maximum number of steps to run (default: 100) -t NUMTHREADS, --num-threads NUMTHREADS Number of threads to run (default: 10) -v, --verify-data Verify data written in a number of places by reading back (default: false) -w, --use-shadow-db Use a shaddow database to verify data integrity (default: false) -x, --continue-on-exception Continue execution after encountering unexpected/disallowed errors/exceptions (default: false) ```