Project: Sonobuoy (Provisioning / Security & Compliance)
File: e2eplugin.md

The Kubernetes end-to-end testing plugin (the e2e plugin) is used to run tests which are maintained by the upstream Kubernetes community in the repo. There are numerous ways to run this plugin in order to meet your testing needs. The most common point of customization is changing the set of tests to run. This is controlled by two environment variables the test image recognizes: E2E_FOCUS E2E_SKIP Each of these is a regular expression describing which tests to run or skip. The "E2EFOCUS" value is applied first and the "E2ESKIP" value then further restricts that list. These can be set using Sonobuoy flags: ``` sonobuoy run \ --e2e-focus=<run tests matching this regexp> \ --e2e-skip=<skip tests matching this regexp> ``` Note: These flags are just special cases of the more general flag `--plugin-env`. For instance, you could set the env vars by using the flag `--plugin-env e2e.E2E_SKIP=<value>` There are a few commonly run configurations which Sonobuoy hard-codes for convenience. Keep in mind that since these modes are shorthand for focus/skip values, you should not provide flags for both (mode and focus/skip): Note: You can see a list of these by running `sonobuoy modes` non-disruptive-conformance This is the default mode and will run all the tests in the `e2e` plugin which are marked `Conformance` which are known to not be disruptive to other workloads in your cluster. This mode is ideal for checking that an existing cluster continues to behave is conformant manner. NOTE: The length of time it takes to run conformance can vary based on the size of your clusterthe timeout can be adjusted in the Server.timeoutseconds field of the Sonobuoy `config.json` or on the CLI via the `--timeout` flag. quick This mode will run a single test from the `e2e` test suite which is known to be simple and fast. Use this mode as a quick check that the cluster is responding and reachable. certified-conformance This mode runs all of the `Conformance` tests and is the mode used when applying for the . Some of these tests may be disruptive to other workloads so it is not recommended that you run this mode on production clusters. In those situations, use the default "non-disruptive-conformance" mode. NOTE: The length of time it takes to run conformance can vary based on the size of your clusterthe timeout can be adjusted in the Server.timeoutseconds field of the Sonobuoy `config.json` or on the CLI via the `--timeout`

When specifying your own focus/skip values, it may be useful to figure out which tests will be run before actually spending the time/resources to run them. Sonobuoy has gathered the list of tests for different Kubernetes versions and allows you to quickly check the result of focus/skip values: ```gotemplate sonobuoy e2e --focus foo --skip bar ``` It can be particularly useful to see which "tags" (groups of tests) are being run and how many tests. The "e2e" command provides different output modes to facilitate that: ```gotemplate // Lists all tests by name. Precise but may be lots of data. sonobuoy e2e --focus foo --skip bar // Lists all the test tags (e.g. [sig-storage] or [Feature: Foo]) sonobuoy e2e --focus foo --skip bar --mode=tags // Lists all the test tags and how many tests had it (e.g. [sig-storage]: 13 or [Feature: Foo]: 2) sonobuoy e2e --focus foo --skip bar --mode=tagCounts ``` Instead of the "e2e" command, you can also run the tests in dry run mode: ``` sonobuoy run \ --plugin-env e2e.E2E_FOCUS=pods \ --plugin-env e2e.E2E_DRYRUN=true ``` By setting `E2EDRYRUN`, the run will execute and produce results like normal except that the actual test code won't execute, just the test selection. Each test that would have been run_ will be reported as passing. This can help you fine-tune your focus/skip values to target just the tests you want without wasting hours on test runs which target unnecessary tests. With such a of Kubernetes distributions available, conformance tests help ensure that a Kubernetes cluster meets the minimal set of features. They are a subset of end-to-end (e2e) tests that should pass on any Kubernetes cluster. A conformance-passing cluster provides the following guarantees: Best practices*: Your Kubernetes is properly configured. This is useful to know whether you are running a distribution out of the box or handling your own custom setup. Predictability*: All your cluster behavior is well-documented. Available features in the official Kubernetes documentation can be taken as a given. Unexpected bugs should be rare, because distribution-specific issues are weeded out during the conformance tests. Interoperability*: Workloads from other conforming clusters can be ported into your cluster, or vice versa. This standardization of Kubernetes is a key advantage of open source software, and allows you to avoid vendor lock-in. Individual Kubernetes distributions may offer additional features beyond conformance testing, but if you change distributions, these features can't be expected to be provided. See the for Kubernetes's existing conformance tests.