Package: pytest-terraform
Introductionpytest_terraform is a pytest plugin that enables executing terraform
to provision infrastructure in a unit/functional test as a fixture.This plugin features uses a fixture factory pattern to enable paramterized
construction of fixtures via decorators.Usagepytest_terraform provides aterraformdecorator with the following parameters:ArgumentRequired?TypeDefaultDescriptionterraform_diryesStringTerraform module (directory) to execute.scopenoString"function"Pytest scope- should be one of:function, orsession. Other scopes likeclass,module, andpackageshould work but have not been fully tested.replaynoBooleanTrueUse recorded resources instead of invoking terraform. SeeReplay Supportfor more details.namenoStringNoneName used for the fixture. This defaults to theterraform_dirwhenNoneis supplied.teardownnoString"default"Configure which teardown mode is used for terraform resources. SeeTeardown Optionsfor more details.Examplefromboto3importSessionfrompytest_terraformimportterraform# We use the terraform decorator to create a fixture with the name of# the terraform module.## The test function will be invoked after the terraform module is provisioned# with the results of the provisioning.## The module `aws_sqs` will be searched for in several directories, the test# file directory, a sub directory `terraform`.## This fixture specifies a session scope and will be run once per test run.#@terraform('aws_sqs',scope='session')deftest_sqs(aws_sqs):# A test is passed a terraform resources class containing content from# the terraform state file.## Note the state file contents may vary across terraform versions.## We can access nested datastructures with a jmespath expression.assertaws_sqs["aws_sqs_queue.test_queue.tags"]=={"Environment":"production"}queue_url=aws_sqs['test_queue.queue_url']print(queue_url)deftest_sqs_deliver(aws_sqs):# Once a fixture has been defined with a decorator# it can be reused in the same module by name, with provisioning# respecting scopes.#sqs=Session().client('sqs')sqs.send_message(QueueUrl=aws_sqs['test_queue.queue_url'],MessageBody=b"123")@terraform('aws_sqs')deftest_sqs_dlq(aws_sqs):# the fixture can also referenced again via decorator, if redefined# with decorator the fixture parameters much match (ie same session scope).# Module outputs are available as a separate mapping.aws_sqs.outputs['QueueUrl']Notethe fixture name should match the terraform module nameNoteThe terraform state file is considered an internal
implementation detail of terraform, not per se a stable public interface
across versions.MarksAll tests using terraform fixtures have aterraformmark applied so
they can be run/selected via the command line ie.pytest-kterraformtests/to run all terraform tests only. See pytest mark documentation for
additional details,https://docs.pytest.org/en/stable/example/markers.html#mark-examplesOptionsYou can provide the path to the terraform binary else its auto discovered--tf-binary=$HOME/bin/terraformTerraform modules referenced by fixtures are looked up in a few different
locations, directly in the same directory as the test module, in a subdir
named terraform, and in a sibling directory named terraform. An explicit
directory can be given which will be looked at first for all modules.--tf-mod-dir=terraformThis plugin also supports flight recording (see next section)--tf-replay=[record|replay|disable]Teardown Optionspytest_terraformsupports three different teardown modes for the terraform decorator.
The default,pytest_terraform.teardown.ONwill always attempt to teardown any and all modules viaterraform destory.
If for any reason destroy fails it will raise an exception to alert the test runner.
The next mode,pytest_terraform.teardown.IGNORE, will invoketerraform destroyas withteardown.ONbut will ignore any failures.
This mode is particularly help if your test function performs destructive actions against any objects created by the terraform module.
The final option ispytest_terraform.teardown.OFFwhich will remove the teardown method register all together.
This should generally only be used in very specific situations and is considered an edge case.There is a specialpytest_terraform.teardown.DEFAULTwhich is what theteardownparameter actually defaults to.Teardown options are available, for convenience, on the terraform decorator.
For example, set teardown to ignore:frompytest_terraformimportterraform@terraform('aws_sqs',teardown=terraform.TEARDOWN_IGNORE)deftest_sqs(aws_sqs):assertaws_sqs["aws_sqs_queue.test_queue.tags"]=={"Environment":"production"}queue_url=aws_sqs['test_queue.queue_url']print(queue_url)Hookspytest_terraform provides hooks via the pytest hook implementation.
Hooks should be added in theconftest.pyfile.pytest_terraform_modify_stateThis hook is executed after state has been captured from terraform apply and before writing to disk.
This hook does not modify state that's passed to the function under test.
The state is passed as the kwargtfstatewhich is aTerraformStateJsonUserString class with the following methods and properties:TerraformStateJson.dict- The deserialized state as a dictTerraformStateJson.update(state: str)- Replace the serialized state with a new state stringTerraformStateJson.update_dict(state: dict)- Replace the serialized state from a dictionaryExampledefpytest_terraform_modify_state(tfstate):print(str(tfstate))Example AWS Account scrubimportredefpytest_terraform_modify_state(tfstate):""" Replace potential AWS account numbers with 'REDACTED' """tfstate.update(re.sub(r'([0-9]+){12}','REDACTED',str(tfstate)))Flight RecordingThe usage/philosophy of this plugin is based on using flight recording
for unit tests against cloud infrastructure. In flight recording rather
than mocking or stubbing infrastructure, actual resources are created
and interacted with with responses recorded, with those responses
subsequently replayed for fast test execution. Beyond the fidelity
offered, this also enables these tests to be executed/re-recorded against
live infrastructure for additional functional/release testing.https://cloudcustodian.io/docs/developer/tests.html#creating-cloud-resources-with-terraformReplay SupportBy default fixtures will save atf_resources.jsonback to the module
directory, that will be used when in replay mode.Replay can be configured by passing --tf-replay on the cli or via pytest config file.RecordingPassing the fixture parameterreplaycan control the replay behavior on an individual
test. The default is to operate in recording mode.@terraform('file_example', replay=False)
def test_file_example(file_example):
assert file_example['local_file.bar.content'] == 'bar!'XDist Compatibilitypytest_terraform supports pytest-xdist in multi-process (not distributed)
mode.When run with python-xdist, pytest_terraform treats all non functional
scopes as per test run fixtures across all workers, honoring their
original scope lifecycle but with global semantics, instead of once
per worker (xdist default).To enable this the plugin does multi-process coodination using lock
files, a test execution log, and a dependency mapping of fixtures
to tests. Any worker can execute a module teardown when its done executing
the last test that depends on a given fixture. All provisioning and
teardown are guarded by atomic file locks in the pytest execution's temp
directory.Root module referencesterraform_remote_statecan be used to introduce a dependency between
a scoped root modules on an individual test, note we are not
attempting to support same scope inter fixture dependencies as that
imposes additional scheduling constraints outside of pytest native
capabilities. The higher scoped root module (ie session or module scoped)
will need to have output variables to enable this consumption.