Project: MongoDB (App Definition and Development / Database)
File: multiversion.md

- - - - - Some tests test specific upgrade/downgrade behavior expected between different versions of MongoDB. Several versions of MongoDB are spun up during those test runs. Multiversion suites - resmoke suites that are running tests with several versions of MongoDB. Multiversion tasks - Evergreen tasks that are running multiversion suites. Multiversion tasks in most cases include `multiversion` or `downgrade` in their names. For some of the versions we are using such generic names as `latest`, `last-lts` and `last-continuous`. `latest` - the current version. In Evergreen, the version that was compiled in the current build. `last-lts` - the latest LTS (Long Term Support) Major release version. In Evergreen, the version that was downloaded from the last LTS release branch project. `last-continuous` - the latest Rapid release version. In Evergreen, the version that was downloaded from the Rapid release branch project. Many multiversion tasks are running tests against `latest`/`last-lts` or `latest`/`last-continuous` versions. In such context we refer to `last-lts` and `last-continuous` versions as the `old` version and to `latest` as a `new` version. A `new` version is compiled in the same way as for non-multiversion tasks. The `old` versions of compiled binaries are downloaded from the old branch projects with . `db-contrib-tool` searches for the latest available compiled binaries on the old branch projects in Evergreen. Multiversion suites can be explicit and implicit. Explicit - JS tests are aware of the binary versions they are running, e.g. . The version of binaries is explicitly set in JS tests, e.g. : ```js const versions = [ { binVersion: "4.4", featureCompatibilityVersion: "4.4", testCollection: "four_four", }, { binVersion: "5.0", featureCompatibilityVersion: "5.0", testCollection: "five_zero", }, { binVersion: "6.0", featureCompatibilityVersion: "6.0", testCollection: "six_zero", }, { binVersion: "last-lts", featureCompatibilityVersion: lastLTSFCV, testCollection: "last_lts", }, { binVersion: "last-continuous", featureCompatibilityVersion: lastContinuousFCV, testCollection: "last_continuous", }, { binVersion: "latest", featureCompatibilityVersion: latestFCV, testCollection: "latest", }, ]; ``` Implicit - JS tests know nothing about the binary versions they are running, e.g. . Most of the implicit multiversion suites are using matrix suites, e.g. `replicasetslast_lts`: ```bash $ python buildscripts/resmoke.py suiteconfig --suite=replicasetslast_lts executor: config: shell_options: global_vars: TestData: useRandomBinVersionsWithinReplicaSet: last-lts nodb: '' selector: exclude_files: jstests/replsets/initialsyncrename_collection.js jstests/replsets/initialsyncdrop_collection.js jstests/replsets/applypreparetxnwriteconflict_robustness.js jstests/replsets/invalidatesessionson_stepdown.js jstests/replsets/initialsyncfailsuncleanrestart.js excludewithany_tags: multiversion_incompatible backportrequiredmultiversion replicasetsmultiversionbackportrequired_multiversion disabledforfcv61_upgrade roots: jstests/replsets/*.js testkind: jstest ``` In implicit multiversion suites the version of binaries is defined on the resmoke fixture level. The of replica set fixture configuration override: ```yaml fixture: num_nodes: 3 oldbinversion: last_lts mixedbinversions: newnewold ``` The of sharded cluster fixture configuration override: ```yaml fixture: num_shards: 2 numrsnodespershard: 2 oldbinversion: last_lts mixedbinversions: newoldold_new ``` The of shell fixture configuration override: ```yaml value: executor: config: shell_options: global_vars: TestData: useRandomBinVersionsWithinReplicaSet: "last-lts" ``` In implicit multiversion suites the same set of tests may run in similar suites that are using various mixed version combinations. Those version combinations depend on the type of resmoke fixture the suite is running

These are the recommended version combinations to test against based on the suite fixtures: Replica set fixture combinations: `last-lts new-new-old` (i.e. suite runs the replica set fixture that spins up the `latest` and the `last-lts` versions in a 3-node replica set where the 1st node is the `latest`, 2nd - `latest`, 3rd - `last-lts`, etc.) `last-lts new-old-new` `last-lts old-new-new` `last-continuous new-new-old` `last-continuous new-old-new` `last-continuous old-new-new` Ex: uses a so the corresponding multiversion suites are - - - Sharded cluster fixture combinations: `last-lts new-old-old-new` (i.e. suite runs the sharded cluster fixture that spins up the `latest` and the `last-lts` versions in a sharded cluster that consists of 2 shards with 2-node replica sets per shard where the 1st node of the 1st shard is the `latest`, 2nd node of 1st shard - `last-lts`, 1st node of 2nd shard - `last-lts`, 2nd node of 2nd shard - `latest`, etc.) `last-continuous new-old-old-new` Ex: uses a so the corresponding multiversion suites are - Shell fixture combinations: `last-lts` (i.e. suite runs the shell fixture that spins up `last-lts` as the `old` versions, etc.) `last-continuous` Ex: uses a Shell Fixture, so the corresponding multiversion suites are - If `last-lts` and `last-continuous` versions happen to be the same, or last-continuous is EOL, we skip `last-continuous` and run multiversion suites with only `last-lts` combinations in Evergreen. Please refer to mongo-task-generator for generating multiversion tasks in Evergreen. Sometimes tests are not designed to run in multiversion suites. To avoid implicit multiversion suites to pick up the test `multiversion_incompatible` tag can be added to the test, e.g. . ```js // @tags: [multiversion_incompatible] ``` There is additional `requiresfcvXX` set of tags that can be used to disable tests from running in multiversion where `XX` is the version number, e.g. `requiresfcv61` stands for version `6.1` in . ```js // @tags: [requiresfcv61] ``` Tests with `requiresfcvXX` tags are excluded from multiversion tasks that may run the versions below the specified FCV version, e.g. when the `latest` version is `6.2`, `last-continuous` is `6.1` and `last-lts` is `6.0`, tests tagged with `requiresfcv61` will NOT run in multiversion tasks that run `latest` with `last-lts`, but will run in multiversion tasks that run `lastest` with `last-continuous`. In addition to disabling multiversion tests based on FCV, there is no need to run in-development `featureFlagXYZ` tests (featureFlags that have `default: false`) because these tests will most likely fail on older versions that have not implemented this feature. For multiversion tasks, we pass the `--runNoFeatureFlagTests` flag to avoid these failures on `all feature flag` variants. For more info on FCV, take a look at . Another common case could be that the changes on master branch are breaking multiversion tests, but with those changes backported to the older branches the multiversion tests should work. In order to temporarily disable the test from running in multiversion it can be added to the . Please follow the instructions described in the file.