Package: drs-compliance-suite
drs-compliance-suiteTests to verify the compliance of a DRS implementation with GA4GH Data Repository Service (DRS) specification.
This compliance suite currently supports the following DRS versions and will aim to support more versions of DRS in the future.DRS 1.2.0InstallationsPython 3.xis required to run DRS Compliance Suite natively or using PyPI package.Docker Desktopis required to run DRS Compliance Suite using a docker image.Running DRS Compliance Suite1. NativelyInstall the packages from requirements.txtcd drs-compliance-suite
pip3 install -r requirements.txtAdd PYTHONPATH to env variablesexport PYTHONPATH=<absolute path to drs-compliance-suite>Run the compliance suitepython3 compliance_suite/report_runner.py --server_base_url "http://localhost:8085/ga4gh/drs/v1" --platform_name "ga4gh starter kit drs" --platform_description "GA4GH reference implementation of DRS specification" --drs_version "1.2.0" --config_file "compliance_suite/config/config_samples/config_basic.json" --serve --serve_port 56565Note: This specific command is an example of running the compliance suite on a local deployment of DRS that is running on port 8085.When running the compliance suite, it's important to configure the command line arguments according to the specific DRS implementation you're testing.
Please refer to theCommand Line Argumentssection for details on each of these arguments.2. Using PyPI PackageInstall the latest version of thedrs-compliance-suitePyPI package using pip3pip3 install drs-compliance-suite --upgradeRun the compliance suitedrs-compliance-suite --server_base_url "http://localhost:8085/ga4gh/drs/v1" --platform_name "ga4gh starter kit drs" --platform_description "GA4GH reference implementation of DRS specification" --drs_version "1.2.0" --config_file "compliance_suite/config/config_samples/config_basic.json" --serve --serve_port 56565Note: This specific command is an example of running the compliance suite on a local deployment of DRS that is running on port 8085.When running the compliance suite, it's important to configure the command line arguments according to the specific DRS implementation you're testing.
Please refer to theCommand Line Argumentssection for details on each of these arguments.3. Using DockerPull the latest docker image from dockerhub.docker pull ga4gh/drs-compliance-suite:1.0.3Run the compliance suite using the docker imagedocker run -d --name drs-compliance-suite -v $(PWD)/output/:/usr/src/app/output/ -p 57568:57568 ga4gh/drs-compliance-suite:1.0.3: --server_base_url "http://host.docker.internal:8085/ga4gh/drs/v1" --platform_name "ga4gh starter kit drs" --platform_description "GA4GH reference implementation of DRS specification" --report_path "./output/test-report.json" --drs_version "1.2.0" --config_file "compliance_suite/config/config_samples/config_none.json" --serve --serve_port 57568Note: This specific command is an example of running the compliance suite on a local deployment of DRS that is running on port 8085.When running the compliance suite, it's important to configure the command line arguments according to the specific DRS implementation you're testing.
Please refer to theCommand Line Argumentssection for details on each of these arguments.Command Line ArgumentsCommand Line ArgumentDescriptionOptional/RequiredDefault Value--server_base_urlThe base URL of the DRS implementation that is being tested by the compliance suite.RequiredN/A--platform_nameThe name of the platform hosting the DRS server.RequiredN/A--platform_descriptionThe description of the platform hosting the DRS server.RequiredN/A--drs_versionThe version of DRS implemented by the DRS server taht is being tested for compliance. It can be one of the following: "1.2.0"RequiredN/A--config_fileThe file path of the JSON config file. The config file must contain auth information for service-info endpoint and different DRS objects. Refer to theconfig-filesection for more details.RequiredN/A--report_pathThe path of the output JSON report file.Optional"./output/drs_compliance_report.json"--serveIf this flag is set to True, the output report is served as an HTML webpage at the port specified by--serve_port.OptionalFalse--serve-portThe port where the output report HTML is deployed.Optional57568Config File:The compliance suite is provided with information for testing the DRS server through a user-provided JSON config file. This file includes the following details:Authorization information for the service-info endpointA few DRS Object IDs that are present in the DRS serverAuthorization information for each of these DRS objectsIndication of whether the DRS object is a bundle or a single blobHere's a template for a config file that can be used to configure these details:{
  "service_info": {
      "auth_type": "basic",
      "auth_token": "dXNlcm5hbWU6cGFzc3dvcmQ="
  },
  "drs_objects" : [
      {
          "drs_id": "697907bf-d5bd-433e-aac2-1747f1faf366",
          "auth_type": "none",
          "auth_token": "",
          "is_bundle": false
      },
      {
          "drs_id": "0bb9d297-2710-48f6-ab4d-80d5eb0c9eaa",
          "auth_type": "basic",
          "auth_token": "dXNlcm5hbWU6cGFzc3dvcmQ=",
          "is_bundle": false
      },
      {
          "drs_id" : "41898242-62a9-4129-9a2c-5a4e8f5f0afb",
          "auth_type": "bearer",
          "auth_token": "secret-bearer-token-1",
          "is_bundle": true
      },
      {
          "drs_id" : "a1dd4ae2-8d26-43b0-a199-342b64c7dff6",
          "auth_type": "passport",
          "auth_token": "43b-passport-a1d",
          "is_bundle": true
      }
  ]
}The "auth_type" specifies the type of authorization, which can be one of the following: ["basic", "bearer", "passport", "none"]The "auth_token" field contains the corresponding authorization token value.If "auth_type" is set to "basic", the "auth_token" can be created by using base64 encoding.If "auth_type" is set to "none", the "auth_token" field should be left blank with a value of "".The "is_bundle" flag indicates whether the DRS object is a bundle or a single blob. If set to True, the object is a bundle, and if set to False, the object is a single blob.You can find some sample config fileshereUnittestingRun the unittests with coveragepytest --cov=compliance_suite unittests/Changelogv1.0.3provide flexibility in providing different auth information for drs object and drs access endpointsremove incorrect skip status settingv1.0.2Reduce the docker image size by using python:3.11-slim-bullseye instead of python:3v1.0.1Fixed a bug in the docker deployment of DRS Compliance SuiteUpdate README documentationv1.0.0DRS Compliance Suite forData Repository Service v1.2.0