Package: pytest-variant
Pytest Variantpytest-variant is a plugin for pytest that allows handling variants in test cases.
The plugin can deselect test cases that are only meant for certain variants or pass down the
variant settings to test cases.InstallationYou can install this library from PyPI using pip.pip install pytest-variantUsageDefine variant attributesParameters can be loaded from INI files from thevariantsection. To load from INI file usepytest --variant INI:config.ini.An example of such.inifile.[variant]os=win32arch=x86Parameters can also be loaded from JSON files, they are loaded as a dictionary usingjson.load.
To load from JSON file usepytest --variant JSON:config.json.An example of a.jsonfile.{"os":"win32","arch":"x86"}Variants can also be inferred from environment variables usingpytest --variant ENV:<prefix>.
When the prefix is empty all environment variables will be loaded, otherwise it will only load the
variables that start with the prefix and the attribute names will have the prefix stripped.For examplepytest --variant ENV:VARIANT_in a context whereVARIANT_OS=win32andVARIANT_ARCH=x86exist then the loaded variant will be:{"OS":"win32","ARCH":"x86"}Note that on Windows systems environment variable names are case insensitive and they always have to
be referred to using capital letters.Lastly the internal variant variable can be overwritten and therefore it's possible to implement
custom configuration and loading mechanisms. This can be done by using a custompytest hook.defpytest_configure(config):config.variant={'os':'win32','arch':'x86'}Accessing variant attributesThe plugin automatically registers a fixture calledvariant, which is a dictionary containing the
attributes that were loaded. You can use these to make a test behave a certain way given a variant
or you could conditionally skip the test case.deftest_case1(variant):ifvariant['os']=='win32':# windows specific behaviorpasselse:passMarking testsThere are two styles of marking tests for variant selection.When a dictionary is passed as the marker's argument the value of each key must match the currently
active variant's. Missing keys are ignored, you can think of each supplied key as anandoperation, and providing a list as a key as anoroperation.defpytest_configure(config):config.variant={'os':'win32','arch':'x86'}# Will be selected@pytest.mark.variant({'os':'win32'})deftest_case1():pass# Will be deselected@pytest.mark.variant({'os':'ubuntu'})deftest_case2():pass# Will be selected@pytest.mark.variant({'os':'win32','arch':'x86'})deftest_case3():pass# Will be deselected@pytest.mark.variant({'os':'win32','arch':'x64'})deftest_case4():pass# Will be selected@pytest.mark.variant({'os':'win32','arch':['x86','x64']})deftest_case5():passWhen a string is passed it's assumed to be a Python expression, which when evaluates toTrueindicates that the test should be selected. In the context of the expression all attributes are
available.defpytest_configure(config):config.variant={'os':'win32','arch':'x86'}# Will be selected@pytest.mark.variant("os == 'win32'")deftest_case1():pass# Will be deselected@pytest.mark.variant("os == 'win32' and arch == 'x64'")deftest_case2():passWhy not just use custom marks or skips?Pytest marks allow you to filter the tests based on the active marks, this is good when you have a
few test sets and you can control what marks are being used. For example you have tests specific to
Windows, you could use@pytest.mark.win32but this is hard to scale up, if you want to use integer
types as variant attributes then it becomes awkward to use.Theskipifmark can be a decent option however when used with a reporting tool like Jenkins they
will show up as skipped tests and looking at the report it won't be evident that a test shouldn't be
performed at all for the specific variant.Truly the only difference between custom marks and this plugin is that pytest marks are in a list,
while variant attributes are in a dictionary.License