Package: detoxed
DetoxedAboutDetoxedis an integration test harness that makes it easy to run a suite of integration tests in Python. It is easy to integrate into a CD pipeline and is currently being used to verify the correctness of Azure IoT Deployments inDevandQAstages. It is easy to host in any CD pipelines that allow execution of user-defined scripts as part of a deployment.Detoxedcan be used in a CD pipeline to:Run integration test suite after a deployment to target stagesRun sanity (is it up?) tests after deploying to production stagesTrigger rollback of failed deploymentInstallationDetoxedcurrently supports Python versions3.4,3.5,3.6and3.7. It can be installed by runningpip install detoxed.SamplesDetoxedships with a dockerized sample showing both passing and failing tests. The sample shows the following use cases:InvokingDetoxedas a CLI throughbashInvokingDetoxedas a Python library.To run the samples, follow these steps:# clone the repogitclonehttps://github.com/Microsoft/Detoxed.git# enter the Detoxed directorycdDetoxed/# ensure Docker is runningdocker--version# build the projectdockerbuild.-tdetoxed# run the samplesdockerrundetoxedBasic UsageA common case for usingDetoxedis to execute integration tests following the deployment of deployment either locally or to a cloud provisioned resource. Here is a simple integration test that will test if an application's HTTP endpoint is up and running:# assume module name is 'tests.integration.suite'importrequestsfromosimportgetenvfromdetoxed.integimportIntegTestBase,Fail,Pass,TestResultclassSimpleOutboundConnectionTestCase(IntegTestBase):def__init__(self):"""Initialize test with a name of your choice."""super().__init__('Application responds to HTTP GET')# optional method to execute test setup...# def setup(self):#     ...# optional method to execute test teardown...# def teardown(self):#     ...deftest(self)->TestResult:"""Test logic here. Should return Pass() or Fail('<failure message>')"""try:status_code=requests.get(getenv('APP_HTTP_ENDPOINT')).status_codeifstatus_code==200:returnPass()returnFail('status code expected to be 200 but was{}'.format(status_code))exceptExceptionasex:returnFail('failed to execute HTTP GET:{}'.format(ex))# More tests can live in this module or other module.Running Integ Tests LocallyHere is how you would invoke this integration test locally usingDetoxed:exportAPP_HTTP_ENDPOINT="..."exportTEST_MODULES="tests.integration.suite"# add any number of modules here...python3-mdetoxed$TEST_MODULESThis will produce the following logs:importingmodule:tests.integration.suite
STARTING:'Application responds to HTTP GET'✓PASSED:'Application responds to HTTP GET'# logs for any additional tests will be here...✓PASSED:'Integration Test Suite'Running Integ Tests through deployment pipeline in Azure Dev OpsHere is how you would invoke this integration test from an Azure Dev Ops Pipeline (steps for other CD technologies would be similar):Create a variable group linked to your release stage. It should include the following variables:TEST_MODULES: modules where your integration test classes liveAny test-specific environment variables. In the example test above, this would beAPP_HTTP_ENDPOINTCreate a script step that runs the test harness:python3-mpipinstalldetoxed# install other modules as needed, perhaps from requirements.txtpython3-mdetoxed$TEST_MODULESAdvanced UsageConfiguring Deployment ConditionsThe test harness will accept an optional parameter that specifies a deployment condition. This condition defines a criteria that must be met in order for the tests to run along with a timeout that specifies how long to wait for the condition to pass. The default condition will immediately be met and will not block test execution. This behavior can be overriden by creating the test harness in python using a deployment condition that matches this interface:classDeploymentCondition(ABC):"""Represents a deployment condition that should block the tests from running until met."""@abstractmethoddefis_met(self)->bool:"""True if the condition is met, False otherwise."""@abstractmethoddeftimeout(self)->int:"""Timeout for contition to be met."""Azure IoT Edge Deployment ConditionA common case for usingDetoxedis to execute integration tests following the deployment of an IoT application. Integration tests need to be runafterthe IoT Deployment is applied to the target devices. However the IoT Deployment is considered complete after it is applied to the IoT Hub itself. It may be quite some time until it is applied to the IoT devices. A built in deployment condition to handle this case ships withDetoxed.fromdetoxed.conditionsimportIoTDeploymentFinishedConditionfromdetoxed.integimportIntegTestRunnerTEST_MODULES=[# your test modules here...]condition=IoTDeploymentFinishedCondition(timeout=300,# seconds to wait for deployment to finishiot_hub='my-iot-hub-qa',# iot hub namedeployment_id=102,# iot deployment IDdevice_query="tags.environment='qa'")# iot device query to enumerate targeted devicestest_harness=IntegTestRunner(TEST_MODULES,condition)passed=test_harness.run()Notes aboutIoTDeploymentFinishedCondition:IoTDeploymentFinishedConditioncurrently has a hard dependency on theAzure CLIand theAzure IoT CLI Extension.az loginmust be executed forIoTDeploymentFinishedConditionto work properly. This can be accomplished in Azure Dev Ops by running the integration test using anAzure CLI Taskwith theUse Global Configurationoption set toTrue.Pending development workEnable parallel test executionContributingThis project welcomes contributions and suggestions.  Most contributions require you to agree to a
Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us
the rights to use your contribution. For details, visithttps://cla.microsoft.com.When you submit a pull request, a CLA-bot will automatically determine whether you need to provide
a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions
provided by the bot. You will only need to do this once across all repos using our CLA.This project has adopted theMicrosoft Open Source Code of Conduct.
For more information see theCode of Conduct FAQor
contactopencode@microsoft.comwith any additional questions or comments.For additional guidance, refer to the project-specificcontributing guidelinesas well as thepull request template