Project: MongoDB (App Definition and Development / Database)
File: how_to_use_cppsuite.md

This README demonstrates the process to create and run a cppsuite test from scratch. As an example, we will create a new test `multisizeinserts` that performs transactions that are either within or greater than the database cache size. We will cover the steps to , , and finally . New tests are created using the script in the root directory. This script takes the test name as its only argument, so we will call it with: `./createtest.sh multisize_inserts` This will create two new files: the `.cpp` file `tests/multisizeinserts.cpp` and the configuration file `configs/multisizeinserts_default.txt`. Working with these files is covered below. Note: `s_all` is called as part of `create_test.sh` which automatically updates some boilerplate sections of the test framework based on the contents of `test_data.py`. If you want to change the name of an existing test you will need to also update it in `test_data.py` and then call `s_all` manually. Having generated these two files we now need to modify them to define our test. The `.cpp` file allows users to define the behaviour of database operations for their test. This file is cloned from on creation and by default will do nothing as all functions are stubbed. In our example test `multisizeinserts` we will first update the functions that define `insert` and `populate`. The default behavior of populate (implemented in ) will create the table in the database, so we can delete the stubbed implementation of `populate` and make sure that it is defined in the later. When cppsuite executes the test it will create threads that run the `insert_operation` function. Initially, the insert function is stubbed as follows: ```cpp void insertoperation(threadworker *) override final { logger::logmsg(LOGWARN, "insert_operation: nothing done"); } ``` The user can either define their own insertion logic by filling out the function body, or remove the `insert_operation` stub to fall back on the frameworks default implementation. For our `multisizeinserts` example, we add code to to write transactions with either 10MB or 1KB updates. We will configure the `cache_size` to be 10MB later. ```cpp void insertoperation(threadworker *tw) override final { const int64t tenmb = 10 1024 1024; const int64t onekb = 1024; // Retrieve the collection created by the populate operation. collection &coll = tw->db.get_collection(0); scopedcursor cursor = tw->session.openscoped_cursor(coll.name); int valuesizebytes = one_kb; while (tw->running()) { // Swap value size between 1KB and 10MB. valuesizebytes = valuesizebytes == onekb ? tenmb : one_kb; tw->txn.begin(); bool success = true; // Each transaction should have targetopcount insert operations. This value is set in the configuration file for(int i = 0; i < tw->txn.gettargetop_count(); ++i) { tw->sleep(); // Populate the key and value with random strings. We only use the size of the value in this test const std::string key = randomgenerator::instance().generatepseudorandomstring(tw->key_size); const std::string value = randomgenerator::instance().generatepseudorandomstring(valuesizebytes); if(!tw->insert(cursor, coll.id, key, value)) { success = false; break; } } if(!success){ tw->txn.rollback(); } else { testutil_assert(tw->txn.commit()); } } } ``` The `settrackingcursor` function defines what data is saved by the operation tracker. In our example test we will save the timestamp, transaction ID, key and value size. ```cpp void settrackingcursor(WTSESSION *session, const trackingoperation &operation, const uint64t &collectionid, const std::string &key, const std::string &value, wttimestampt ts, scopedcursor &optrack_cursor) override final { uint64t txnid = reinterpretcast<WTSESSION_IMPL *>(session)->txn->id; optrackcursor->setkey(optrackcursor.get(), ts, txnid); optrackcursor->setvalue(optrackcursor.get(), key.cstr(),

} ``` Note: We also need to specify the key-value format of the tracking table where we save test data. This is covered . When the test finishes the framework calls the `validate` function in which users can perform any checks they want. In our test `multisizeinserts` the 10MB updates are too large for cache and not be written ( for how to configure cache size). As such we will check that any update made with a value size of 1KB is present in the database, and any update with a value size of 10MB is not present: ```cpp void validate(bool trackingenabled, const std::string &operationtablename, const std::string &schematable_name, database &db) override final { const int64t tenmb = 10 1024 1024; const int64t onekb = 1024; /* This validate logic uses data from operation tracking, it must have been enabled during the test. */ testutilassert(trackingenabled); scopedsession session = connectionmanager::instance().create_session(); // For this test we only need to check the insert operations data saved by the operation tracker. scopedcursor trackedoperationscursor = session.openscopedcursor(operationtable_name); testutilassert(!db.getcollection_ids().empty()); scopedcursor cursor = session.openscopedcursor(db.getcollection(0).name); while (trackedoperationscursor->next(trackedoperationscursor.get()) == 0) { int ts = 0, txnid = 0, valuesize = 0; char *key = nullptr; testutilcheck(trackedoperationscursor->getkey(trackedoperationscursor.get(), &ts, &txn_id)); testutilcheck(trackedoperationscursor->getvalue(trackedoperationscursor.get(), &key, &value_size)); testutilassert(valuesize == onekb || valuesize == ten_mb); cursor->set_key(cursor.get(), key); if(valuesize == onekb) { testutil_assert(cursor->search(cursor.get()) == 0); } else if(valuesize == tenmb) { testutilassert(cursor->search(cursor.get()) == WTNOTFOUND); } } } ``` The configuration file manages cppsuite level configuration for the test such as the total runtime of the test and the behaviour of the different components in the test. To see what fields can be set please refer to . In our `multisizeinsertsdefault.txt` file we define our test to run for 15 seconds, have a cache of 10MB, and create 10 insert threads. Each thread runs `insertoperation` where each transaction contains between 5 and 10 inserts and performs one insert every 10 milliseconds. We define the operation tracker key and value formats to match the types of the data we save in `settrackingcursor` . The types that can be used are found . ```txt duration_seconds=15, cachesizemb=10, workload_manager= ( populate_config= ( collection_count=1, ), insert_config= ( thread_count=1, op_rate=10ms, opspertransaction=(min=5,max=10), ), ), operation_tracker= ( trackingkeyformat=QQ, trackingvalueformat=SQ ) ``` Sometimes users may want to use the same test workload but run it at different intensities to stress different aspects of WiredTiger. To do so users may define multiple configuration files. As a simple example we can create a second configuration file `multisizeinsertsstress.txt` that is identical to the original configuration above, but changes `oprate` to 1ms. Having written our test we can now run it. Compile WiredTiger and then navigate to the `<build_dir>/test/cppsuite` folder. The `./run` binary executes the test. Further details are provided via the `-h` help flag, but the most common usage is `./run -t multisizeinserts -f configs/multisizeinserts_default.txt -l 2`. In this command the `-t` flag tells cppsuite to use the `test/multisizeinserts.cpp` test configuration, the `-f` flag to use the `configs/multisizeinserts_default.txt` cppsuite configuration, and the `-l` flag to run the test with level 2 verbosity. Further details on log levels can be . Warning: If you change your configuration file you will need to recompile WiredTiger to have the changes propagate into the build folder. Make sure you update the file in the `test/cppsuite` folder and not the `<build_dir>/test/cppsuite` folder as the latter will be overwritten when WiredTiger is recompiled.