Project: RethinkDB (App Definition and Development / Database)
File: README.md

Description ========== Run basic tests for all terms to look for regressions between releases. Requirements ========== Uses the Python driver Requires a release build in ../../build/release/ Use ========== Run tests with: ``` python test.py ``` Make more comparisons ``` python compare <file1> <file2> ``` Where file1 must contain the more recent results. Ex: ``` python compare result14.03.20-15:10:18.txt result14.03.19-00:35:18.txt ``` Add queries ========= Add queries in `queries.py` with a simple string or an object with two fields (`query` and `tag`). Note: `tag` must be unique. ix `make` output with the rule count (such as `[3/301]`). `make VERBOSE=1`: Display each command being executed and their output. `make -j <n>`: Build in parallel. A good default for `<n>` is the amount of cores plus one. Server The server may depend on some of the fetched dependencies. Accessing its web UI also requires the web assets. `make rethinkdb`: Only build the server executable. `make tags`, `make etags`: Generate tag files used by vim and emacs. `make ALLOW_WARNINGS=0`: Fail if the C++ compiler generates a warning, by enabling `-Werror`. `./configure CXX=... CXXFLAGS=...`: Build with a different compiler or flags. `./configure --ccache`: Speed up repeated builds using `ccache`. RethinkDB employees can use `./configure CXX=recc` to speed up the build. The executable, in `newton:/usr/local/bin/recc`, wraps both `ccache` and `distcc` to distribute the build across our server infrastructure and cache the result. There is a `check-syntax` target that can be used with emacs' flymake-mode. `make SYMBOLS=0`: Exclude debugging symbols. `make SPLIT_SYMBOLS=1`: Strip the executable and generate a symbols file. `make DEBUG=1`: Build in debug mode. When building on older distributions or porting to different platforms, these `make` options can also be useful: `THREADEDCOROUTINES=1` `NOEVENTFD=1` `NO_EPOLL=1` `BUILDPORTABLE=1` or `LEGACYLINUX=1` Web assets These are generated in a separate repository, named "rethinkdb-old-admin", and the generated file is placed in src/gen/web_assets.cc and committed. Instructions are in that repository. The web assets are compiled into the executable. To speed up development time and avoid rebuilding the executable, the `--web-static-directory` option can be used. For example: `build/release/rethinkdb --web-static-directory ~/rethinkdb-old-admin/build/web_assets` Drivers Drivers have been removed from this repository. Test These instructions are probably non-working -- you need to supply a driver. `make unit`: Build and run the unit tests. `make test`: Run the unit tests, reql tests and integration tests. The `TEST` variables determines which tests to run. See `test/run -h` for more documentation. Dependencies `make support` or `make support-<dep>`: Build the dependencies that have not been built

`make fetch` or `make fetch-<dep>`: Fetch the dependencies that have not yet been fetched. `make build-<dep>`: Rebuild a given dependency. `make clean-<dep>`: Clean the build directory of a given dependency. `install-include-<dep>`: Copy the include files from the given dependency. `VERIFYFETCHHASH=0`: Don't verify the hash of fetched archives. Installation `./configure --prefix`: Set the prefix folder used by `make install`. `make install`: Install RethinkDB to the prefix. If the `DESTDIR` variable is passed, install the files there. If `STRIPONINSTALL=1` is passed, also strip the executable. `make install-binaries`: Only install the server executable. Packaging Packaging is usually automated by the BuildBot instance running on `dr-doom:8010` in our intranet. `make build-deb-src UBUNTURELEASE=<name> PACKAGEBUILD_NUMBER=<n>`: Build a package for the given Ubuntu version. `make build-osx`: Build a `dmg` installer for OS X. Jenkins always builds the `dmg` on OS X 10.7.5. `make dist`: Build a source distribution. The CentOS RPM are built with `scripts/build-rpm.sh`. The build system Some of the flags and targets can be used to debug the build system itself. make `var-<VAR>`: Print the value of the given variable. `make -B`: Force rebuilding all files. `make -d`: Tell `make` to dump a lot of debug information. `make IGNOREMAKEFILECHANGES=1`: Do not rebuild files if the makefiles have changes. `make SHOWBUILDREASON=1`: Show which file triggered each rule. `make dump-db` or `make -p`: Dump the `make` database. `make command-line`: Print the real `make` command line used to build RethinkDB. On some old, unpatched versions of GNU make this is required to use `-j`. `make debug-count`: Debug the count feature. To be able to invoke make from a subdirectory, a Makefile must exist. It usually looks like this: ``` OVERRIDE_GOALS ?= default-goal=<default target for this directory> TOP := <relative path to the top of the rethinkdb source tree> include $(TOP)/Makefile ``` FreeBSD To build on FreeBSD (these instructions tested on FreeBSD 13), run ``` pkg install git gcc gmake python ./configure --allow-fetch CXX=g++ gmake ``` Note that despite the CXX=g++ parameter, the QuickJS dependency is hard-coded to build with Clang, out of laziness. If you try CXX=clang++, the executable is improperly linked. Some fixes to improve this are welcome. on to the client driver that initiated the request. Changefeeds are handled differently from normal reads. For normal reads, the machine handling the query will send a `readt` to the shards for every chunk of data it gets back. The exception to this rule is changefeeds, where the shards will instead push data to the server handling the

The server handling the query subscribes to a changefeed by sending the mailbox address for its `changefeed::clientt` to the shards inside of a `readt`. The shards will then subscribe that client to their `changefeed::servert` (which is a member variable of `storet`). The relevant classes can be found in `rdbprotocol/changefeed.hpp`. The above description has mentioned several types that are involved in the process of executing queries against a table: the `storet`, the `primaryqueryservert`, the various types defined in `clustering/immediateconsistency/`, and so on. The table's configuration (as exposed to the user in the `rethinkdb.tableconfig` system table) describes which servers should be performing which role for which part of the table. When the user changes the table's configuration, objects on various servers must be created and destroyed to implement the user's new configuration, while making sure that the tables' contents are correctly copied from server to server. There is one `multitablemanagert` (defined in `clustering/tablemanager/`) per server, created by `doserve()`. It keeps a record for each table that is known to exist, whether that table has a replica on that server or not. If the server is supposed to be a replica for a given table, the `multitablemanagert` will construct a `tablemanagert` for that table. RethinkDB uses the Raft consensus algorithm to manage table metadata, but not to manage user queries. RethinkDB uses a custom implementation of Raft that's designed to work with the RethinkDB event loop and networking primitives; it's implemented by the type `raftmembert`, defined in `clustering/generic/raftcore.hpp`. There is one Raft cluster per RethinkDB table. Each `tablemanagert` instantiates a `raftmembert`. The state managed by the Raft instance is defined as `tableraftstatet`, defined in `clustering/tablecontract/contractmetadata.hpp`. Among other things, the `tableraftstatet` contains the table's current configuration and a collection of `contractt` objects. The `contractt`s describe what each server is supposed to be doing with respect to each region of the table. The difference between the contracts and the config is that the user can change the config freely, but the contracts can only change in specific well-defined ways that ensure that the data in the tables is preserved and remains consistent. The `tablemanagert` constructs a `contractcoordinatort` (defined in `clustering/tablecontract/coordinator/`) on whichever server is the leader of the Raft cluster; it's responsible for updating the contracts to match the config. The `tablemanagert` also constructs a `contractexecutort` (defined in `clustering/tablecontract/executor/`) on every server; the `contractexecutort` is responsible for carrying out the instructions in the contracts by constructing the `storet`, the `primaryqueryservert`, the various types in `clustering/immediateconsistency/`, etc. The details of how this works are quite complicated; the comments in `clustering/tablecontract/contractmetadata.hpp` are a good introduction.