Chapter 6. Running the certification test suite Complete the following steps on the test server to run certification tests on the OpenStack deployment under test or test client. Run the tests on OpenStack plugins and drivers for the following services - Block Storage and Shared File System Storage. Prerequisites You have prepared the test host system as per the instructions provided in Setting up the test environment . You have a working directory on the test host system with write privileges. Procedure To initialize the certification test environment, enter the OpenShift namespace for the RHOSO control plane (default is openstack ), and then select a certification test suite. The current release supports a cinder test suite and four manila suites. For more information on debugging test failures, see debug suite . The certification test environment is automatically configured with most of the tempest settings required by each test suite. Running the test suites requires additional settings that you can specify in a tempest-config-overrides file. Create the tempest-conf-overrides file and enter the necessary configuration settings. Note In the tempest-conf-overrides file, enter the necessary configuration settings as separate lines and do not enter comments. For example, you can configure the following options by providing an appropriate value in the angular braces (<>). Sample configuration settings Optional: Create tempest accounts. It is beneficial to use a dedicated set of tempest test accounts for some test suites, for example, the manila suites with DHSS=True . Using dedicated test accounts prevents a tempest test from deleting a dynamically created resource, while another test is using the resource. For more information about tempest test accounts, see Using tempest test accounts . Run the certification test suite. Each test suite will run a series of subtests, and when it completes it reports a Pass or Fail status. Note In the current RHOSO version, all the tests are executed, including both mandatory and optional tests, such as consistency groups and multi-attach volumes. The optional tests may fail if your driver does not support those features. This is acceptable and does not impact the overall certification process. You can view the detailed test results, including the debug data after the certification logs are generated. Extract the log files. The system prompts you to include a must-gather report. To speed up the extraction process of the tempest test results, skip the must-gather report by typing no . However, if you are submitting the test results for certification, you must include the must-gather report. The logs and debug data are generated in a subdirectory and the name of the subdirectory is displayed at the end of the command. Sample output Here is an example of the sample output received while extracting the logs for the cinder test suite without a must-gather report. In this example the extracted logs are stored in the directory rhoso-cert-cinder-2024-Aug-01_11-48-08 . Save the generated log files. A filtered subset of the generated test result files are stored in the default location /var/log/redhat-certification/ . You can upload the generated test result files for Red Hat's review. After uploading the test result files, clean up the host system. You can execute this command after completing the certification process, or before selecting a different test suite. 6.1. Optional: Running the debug test suite The debug test suite executes a single group of tests specified in the rhoso-cert-debug.yaml file. The debug test suite helps you to perform the following tasks: Debug the tempest test failures that you encounter when running the entire cinder, manila or neutron test suite. Rerun specific set of tests that failed but are required for result submission. Procedure Copy the YAML file related to the debug test suite to your working directory. Open the local rhoso-cert-debug.yaml file in an editor, and update the list of tests. The list can include just a single test, or a group of tests that you want to execute. Run the command rhoso-cert-init . Select the debug test suite. Rerun the commands rhoso-cert-run and rhoso-cert-logs to execute the selected list of tempest tests. Collect the logs for Red Hat's analysis and verification. 6.2. Optional: Using tempest test accounts Tempest provides a mechanism for specifying a set of test resources by using an accounts.yaml file. It is optional to use test accounts, but it is helpful if the tempest tests fail when a test resource gets prematurely deleted. You can pre-create the test accounts by providing an accounts.yaml file in the certification working directory. Refer to tempest's sample accounts.yaml for knowing the structure of this file. You must have three times the number of test accounts as there are concurrent test threads. Since the default test concurrency is 4, the accounts.yaml file used must list 12 test accounts. These accounts must have 'administrator' users in addition to 'member' users and must also include pre-created networks. Alternatively, the RHOSO certification tools can create the test accounts and the accounts.yaml file when you perform the following steps: Note You must manually clean up the test accounts and their resources after completing the certification process. Procedure Run the debug suite and ignore any failures. Generate the tempest test accounts and accounts.yaml file. Clean up after running the debug suite. Re-initialize the certification test environment and select the desired test suite.

Commands:
rhoso-cert-init
share.share_network_id <network_uuid> share.alt_share_network_id <network_uuid> share.admin_share_network_id <network_uuid> share.security_service <security_service_mapping> share.backend_replication_type <replication_style> share.username_for_user_rules <Username> share.override_ip_for_nfs_access <IP/CIDR> volume.build_interval 10 volume.build_timeout 300 volume.storage_protocol <iSCSI, FC, NVMe-TCP, NFS, etc.> volume.vendor_name <Driver's vendor name>
rhoso-cert-run
rhoso-cert-logs
rhoso-cert-logs Spawning a pod to access the logs in PVC rhoso-cert-cinder-6f855 pod/rhoso-cert-cinder-logs created Waiting for the rhoso-cert-cinder-logs pod to be ready Saving tempest logs Saving logs from the individual Tempest pods: Saving rhoso-cert-cinder-volumes-workflow-step-0-q9btc.log Saving rhoso-cert-cinder-backups-workflow-step-1-5p9xf.log Saving rhoso-cert-cinder-multi-attach-volume-workflow-step-2-zpwh2.log Saving rhoso-cert-cinder-consistency-groups-workflow-step-3-tmb24.log Collect a must-gather report? [y/N] : n Done. Logs are stored in rhoso-cert-cinder-2024-Aug-01_11-48-08
rhoso-cert-save
rhoso-cert-cleanup
cp /usr/share/redhat-certification-rhoso/rhoso-cert-debug.yaml <current test directory>
rhoso-cert-init rhoso-cert-run
rhoso-cert-test-accounts
rhoso-cert-cleanup
rhoso-cert-init