Package: pytest-interface-tester
Interface-tester-pytestThis repository contains a library meant to facilitate compliance testing of charm relation interfaces.
The problem is best stated as follows:there is a repository containing relation interface specifications and tests (henceforth 'thespecrepo'), such ascharm-relation-interfaces.there is a charm repository such astraefik-k8sthat implements the provider side of theingressrelation interface (henceforth 'thecharmrepo').The maintainers of the spec repo want to be able to automatically verify if all the registered charms that claim to implement a relation interface do in fact comply with its specification.The maintainers of the charm repo want to be able to automatically verify that their implementation of the standardized relation interfaces are in fact compliant.The interface tester package facilitates both these verification flows.How to use the interface_tester in the spec repoFollow the instructionshere.How to use the interface_tester in the charm repoEnsure thatcharm-relation-interfaceshas one or more interface tests and a schema for the interface you want to test. If that is not the case, ask the maintainers of the interface (or its 'official' implementation) to add some.Install this package.Add a...charm-root/tests/interface_tests/conftest.pyfile containing at least:importpytestfromcharmimportMyCharm# your charm classfrominterface_tester.pluginimportInterfaceTester@pytest.fixturedefinterface_tester(interface_tester:InterfaceTester):interface_tester.configure(charm_type=MyCharm)yieldinterface_testerinterface_testeris a pytest fixture exposed by this plugin. You are expected to override it with your own. The idea is that in doing this you:configure this fixture so that the spec repo will be able to find your charm location and name (it will default to whatever CharmBase subclass it can find in...charm-root/src/charm.py, but it will give up soon if that is nonstandard or there are multiple charm classes).the test runner can be made aware of any special configuration that your charm needs in order to function. For example if your charm will do nothing unless it is configured in a certain way, a given file is present, a network is active, a container can connect, etc... then this is your chance to set your charm up so that it is ready for handling the necessary relation events.the runtime can be patched with anything necessary for your charm to function. For example, you can mock out anyPopencalls, HTTP requests,At this stage, if you commit this to yourmainbranch, the spec repo will already be able to find your charm and run the interface tests against it.The flow is (from the POV of the spec repo):gather all tests, schemas and charms for an interfacefor each charm:clone the charm repofetch theinterface_testerfixture from the charm's own tests, if any, otherwise assume no config is needed and try to grab the charm type off ofsrc/charm.pyuse the interface tester (configured or not) to run each test case:check that the scenario completes without uncaught charm errors, get the output statecheck whether the test case's own validator determines that the output state is validcheck whether the relations in the output state are valid according to the schemaWhat if you want to run the tests yourself, charm-side, in CI, to catch potential issues before they trigger failures in the spec repo?How to run the tests in the charm repoA minimal example of a test is:frominterface_testerimportInterfaceTesterfromcharmimportMyCharmdeftest_ingress_interface(interface_tester:InterfaceTester):interface_tester.configure(# you can skip this if your interface_tester fixture is already configured with the charm_type in conftest.pycharm_type=MyCharm,# put here the interface that you wish to test. Omitting it will test for all interfaces that your charm supports.interface_name='ingress')interface_tester.run()If you have a conftest.py where you configured aninterface_testerfixture and did all necessary mocking/patching already, then in principle you are good to go.The flow is (from the POV of the spec repo):clone the spec repo (or another)gather all tests, schemas and charms for the interface you want to testuse the interface_tester fixture from the plugin (or one you override) to run the test casessame 3 steps as aboveCustomizing the fixture's addressYou can customize name and location of the fixture, but you will need to include that data when registering your charm with the interface. Incharms.yaml, you can then specify:-name:my-charm-name# requiredurl:https://github.com/foo/my-charm-name# requiredtest_setup:# optionallocation:path/to/file.py# optional; default = tests/interface/conftest.pyidentifier:my_fixture_name# optional; default = interface_testerUpgrading from v1pytest-interface-testersupports both pydantic v1 and v2, but using v2 is recommended.
You might need to adjust your tested charm to also support v2. Seemigration guidefor more information.