Project: Operator Framework (App Definition and Development / Application Definition & Image Build)
File: _index.md

title: Upgrade SDK Version weight: 4 description: Guide to upgrading sdk version for your operator When upgrading your version of Operator-sdk, it is intended that post-1.0.0 minor versions (i.e. 1.y) are backwards compatible and strictly additive. Therefore, you only need to re-scaffold your operator with a newer version of Operator-SDK if you wish to take advantage of new features. If you do not wish to use new features, all that should be required is bumping the operator image dependency (if a Helm or Ansible operator) and rebuilding your operator image. it also provides a means to execute custom test definitions. The scorecard tests make no assumptions as to the state of the operator being tested. Creating operators and custom resources for an operator are left outside the scope of the scorecard itself. Scorecard tests can however create whatever resources they require if the tests are designed for resource creation. A default set of kustomize files should have been scaffolded by `operator-sdk init`. If that is not the case, run `operator-sdk init` as you would have to initialize your project and copy scaffolded files: ```sh $ TMP_PROJECT="$(mktemp -d)/<current-project-name>" $ mkdir "$TMP_PROJECT" $ pushd "$TMP_PROJECT" $ operator-sdk init $ popd $ cp -r "$TMP_PROJECT"/config/scorecard ./config/ ``` The default config generated by this kustomization can be immediately run against your operator. See the for an explanation of the configuration file format. (Re)generate manifests and metadata for your Operator. `make bundle` will automatically add scorecard annotations to your bundle's metadata, which is used by the `scorecard` command to run tests. Execute the for an overview of command invocation. The scorecard test execution is driven by a configuration file named `config.yaml`, generated by `make bundle`. Note that if run `make bundle` that any changes you have made to `config.yaml` will be overwritten. To persist any changes to `config.yaml` you can update the kustomize templates found in the `config/scorecard` directory. The configuration file is located at the following location within your bundle directory (`bundle/` by default): ```sh $ tree ./bundle ./bundle ... tests scorecard config.yaml ``` The following YAML spec is an example of the scorecard configuration file: ```yaml kind: Configuration apiversion: scorecard.operatorframework.io/v1alpha3 metadata: name: config stages: parallel: true tests: image: quay.io/operator-framework/scorecard-test:latest entrypoint: scorecard-test basic-check-spec labels: suite: basic test: basic-check-spec-test image: quay.io/operator-framework/scorecard-test:latest entrypoint: scorecard-test olm-bundle-validation labels: suite: olm test: olm-bundle-validation-test ``` The configuration file defines the tests that scorecard executes. Tests are grouped into stages for fine-grained control of

The following fields of the scorecard configuration file define the test as follows: | Config Field | Description | | -- | image | the test container image name that implements a test | entrypoint | the command and arguments that are invoked in the test image to execute a test | labels | scorecard-defined or custom labels that which tests to run The scorecard command has the following syntax: ```sh $ operator-sdk scorecard <bundlediror_image> [flags] ``` The scorecard requires a positional argument that holds either the on-disk path to your operator bundle or the name of a bundle image. Note that the scorecard does not run your operator but merely uses the scorecard configuration within the bundle contents to know which tests to execute. For further information about the flags see the . The configuration file allows operator developers to define separate stages for their tests. Stages run sequentially in the order they are defined in the configuration file. A stage contains a list of tests and a configurable `parallel` setting. By default (or when a stage explicitly sets `parallel` to `false`), tests in a stage are run sequentially in the order they are defined in the configuration file. Running tests one at a time is helpful to guarantee that no two tests interact and conflict with each other. However, if tests are designed to be fully isolated, they can be parallelized. To run a set of isolated tests in parallel, include them in the same stage and set `parallel` to `true`. All tests in a parallel stage are executed simultaneously, and scorecard waits for all of them to finish before proceeding to the next stage. This can make your tests run much faster. Tests are selected by setting the `--selector` CLI flag to a set of label strings. If a selector flag is not supplied, then all the tests within the scorecard configuration file are executed. Tests are executed serially, one after the other, with test results being aggregated by scorecard and written to stdout. To select a single test (`basic-check-spec-test`) you would enter the following: ```sh $ operator-sdk scorecard <bundlediror_image> -o text --selector=test=basic-check-spec-test ``` To select a suite of tests, olm in this case, you would specify a label that is used by all the OLM tests: ```sh $ operator-sdk scorecard <bundlediror_image> -o text --selector=suite=olm ``` To select multiple tests, you could specify them as follows: ```sh $ operator-sdk scorecard <bundlediror_image> -o text --selector='test in (basic-check-spec-test,olm-bundle-validation-test)' ``` The scorecard ships with pre-defined tests that are arranged into suites. | Test | Description | Test Name | | -- | -- | -- | | Spec Block Exists | This test checks the Custom Resource (CRs) created in the cluster to make sure that all CRs have a spec block. | basic-check-spec-test | | Test | Description | Short Name | | -- | -- | -- | | Bundle Validation | This test validates the bundle manifests found in the bundle that is passed into scorecard. If the bundle contents contain errors, then the test result output will include the validator log as well as error messages from the validation library. See this for details on

olm-bundle-validation-test | | Provided APIs have validation |This test verifies that the CRDs for the provided CRs contain a validation section and that there is validation for each spec and status field detected in the CR. | olm-crds-have-validation-test | | Owned CRDs Have Resources Listed | This test makes sure that the CRDs for each CR provided via the `cr-manifest` option have a `resources` subsection in the to add resources. | olm-crds-have-resources-test | | Spec Fields With Descriptors | This test verifies that every field in the Custom Resources' spec sections have a corresponding descriptor listed in the CSV.| olm-spec-descriptors-test | | Status Fields With Descriptors | This test verifies that every field in the Custom Resources' status sections have a corresponding descriptor listed in the CSV.| olm-status-descriptors-test | The `--output` flag specifies the scorecard results output format. See an example of the JSON format produced by a scorecard test: ```json { "apiVersion": "scorecard.operatorframework.io/v1alpha3", "kind": "TestList", "items": [ { "kind": "Test", "apiVersion": "scorecard.operatorframework.io/v1alpha3", "spec": { "image": "quay.io/operator-framework/scorecard-test:latest", "entrypoint": [ "scorecard-test", "olm-bundle-validation" ], "labels": { "suite": "olm", "test": "olm-bundle-validation-test" } }, "status": { "results": [ { "name": "olm-bundle-validation", "log": "time=\"2020-06-10T19:02:49Z\" level=debug msg=\"Found manifests directory\" name=bundle-test\ntime=\"2020-06-10T19:02:49Z\" level=debug msg=\"Found metadata directory\" name=bundle-test\ntime=\"2020-06-10T19:02:49Z\" level=debug msg=\"Getting mediaType info from manifests directory\" name=bundle-test\ntime=\"2020-06-10T19:02:49Z\" level=info msg=\"Found annotations file\" name=bundle-test\ntime=\"2020-06-10T19:02:49Z\" level=info msg=\"Could not find optional dependencies file\" name=bundle-test\n", "state": "pass" } ] } } ] } ``` See the example below for the results of a scorecard test formatted in XML. The scorecard tool formats the XML output for XUnit schema compatibility. This format makes it easier for post-processing the test results. ```xml <testsuites name="scorecard"> <testsuite name="olm-bundle-validation-test" tests="1" skipped="0" failures="0" errors="0"> <properties> <property name="spec.image" value="quay.io/operator-framework/scorecard-test:v1.19.0"></property> <property name="spec.entrypoint" value="scorecard-test olm-bundle-validation"></property> <property name="labels.test" value="olm-bundle-validation-test"></property> </properties> <testcase name="olm-bundle-validation" time="0001-01-01T00:00:00Z"> <system-out>time=&#34;2022-04-12T19:21:52Z&#34; level=debug msg=&#34;Found manifests directory&#34; name=bundle-test&#xA;time=&#34;2022-04-12T19:21:52Z&#34; level=debug msg=&#34;Found metadata directory&#34; name=bundle-test&#xA;time=&#34;2022-04-12T19:21:52Z&#34; level=debug msg=&#34;Getting mediaType info from manifests directory&#34; name=bundle-test&#xA;time=&#34;2022-04-12T19:21:52Z&#34; level=debug msg=&#34;Found annotations file&#34; name=bundle-test&#xA;time=&#34;2022-04-12T19:21:52Z&#34; level=debug msg=&#34;Could not find optional dependencies file&#34; name=bundle-test&#xA;</system-out> </testcase> </testsuite> <!-- Some suites omitted for readability --> </testsuites> ``` See an example of the text format produced by a scorecard test: ``` -- Image: quay.io/operator-framework/scorecard-test:latest Entrypoint: [scorecard-test olm-bundle-validation] Labels: "suite":"olm" "test":"olm-bundle-validation-test" Results: Name: olm-bundle-validation State: pass Log: time="2020-07-15T03:19:02Z" level=debug msg="Found manifests directory" name=bundle-test time="2020-07-15T03:19:02Z" level=debug msg="Found metadata directory" name=bundle-test time="2020-07-15T03:19:02Z" level=debug msg="Getting mediaType info from manifests directory" name=bundle-test time="2020-07-15T03:19:02Z" level=info msg="Found annotations file" name=bundle-test time="2020-07-15T03:19:02Z" level=info msg="Could not find optional dependencies file" name=bundle-test ``` NOTE The output format spec for each test matches the type layout. The scorecard return code is 1 if any of the tests executed did not pass and 0 if all selected tests pass. Scorecard will execute custom tests if they follow these mandated conventions: tests are implemented within a container image tests accept an entrypoint which include a command and arguments tests produce v1alpha3 scorecard output in JSON format with no extraneous logging in the test output tests can obtain the bundle contents at a shared mount point of /bundle tests can access the Kubernetes API using an in-cluster client connection See the written in Go. Writing custom tests in other programming languages is possible if the test image follows the above guidelines.