Package: aiida_eos
aiida_eosA demonstration of creating a Python package for AiiDA plugins.
The goal is to create a plugin package for the Equation of State workflow demonstrated in:https://aiida-qe-demo.readthedocs.io/en/latest/6_write_your_own_workflow.html.Each commit in this repository corresponds to a step in the tutorial.Note,https://github.com/aiidateam/aiida-plugin-cuttercan be used to automate most of these steps,
but here we shall do it manually explain each aspect of the package.Initial creationThe first step is to create a new repository on GitHub.
We will call itaiida_eos.
The repository should be created with aREADME.mdand a.gitignorefile.Interacting with the repositoryWe shall useVisual Studio Codeto interact with the repository.
This is a free, open-source, cross-platform IDE, with nice integration with GitHub, and many useful extensions.Creating the package metadataThe first step is to create the package metadata.
This is done by creating apyproject.tomlfile in the root of the repository.
This can be used by pip to install the package, and by other tools to build the package:https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/We shall useflitto build the package.
This is a simple tool that is designed to build Python packages from apyproject.tomlfile.We can initialise thepyproject.tomlfile by running:flitinitThis also generates a license file, which is crucial for allowing others to use your package.Create the package and install itWe create the initial package with a single file:src/aiida_eos/__init__.py.
This file should have a docstring that describes the package, and a__version__variable.We now want to install the package in editable mode.
This means that we can make changes to the package, and they will be immediately available to Python.First we create avirtual environment, and activate it:python-mvenv.venvsource.venv/bin/activateVirtual environments are a way of isolating Python environments.We can now install the package in editable mode:python-mpipinstall--upgradepip
pipinstall-e.We can now import the package in Python:>>>importaiida_eos>>>aiida_eos.__version__'0.0.1'Adding formatting and linting with pre-commitWe shall usepre-committo automatically format and lint the code.
This will ensure that the code is formatted consistently, and that it conforms to the style guide.We can initialise a pre-commit configuration file with:pre-commitsample-config>.pre-commit-config.yaml
pre-commitautoupdate
pre-commitinstallWe shall add a few additional hooks to the configuration file:black: a Python code formatterflake8: a Python linterisort: a tool to sort the Python importsAdding testingWe shall usepytestto run tests on our package.
To install pytest, we shall add it to anoptional-dependenciessection in thepyproject.tomlfile.
This is because we only need pytest to run the tests, and not to use the package.We can now install the package, with the optional dependencies:pipinstall-e".[test]"We can now add a test to the package.
We shall add a test that checks that the package can be imported.
This is done by adding atestsdirectory, and atest_import.pyfile in it.Now we can run the tests:pytestTo check the coverage of the tests, we can run:pytest--cov=aiida_eosUsing toxThetoxCLI tool is an optional way to automate both setting up the virtual environment, then running the tests within it.
See thepyproject.tomlsection for the configuration.
You can then simply runtoxto run the tests, ortox -e py39to run with a certain python version.
Seetox-condafor an example of how to use tox with conda.Adding GitHub ActionsWe can useGitHub Actionsto automatically run the tests on each commit.
This is done by adding a.github/workflows/test.ymlfile.Adding the rescale calcfunctionWe shall add arescalecalcfunction to the package.
This is a simple function that takes a structure, and rescales it by a given factor.Note that up until now, we have not added any AiiDA specific code.
Now that we want to add an AiiDA specific calcfunction, we need to add theaiida-coredependency to thepyproject.tomlfile, and alsoasefor the structure manipulation.If usingtox, we can regenerate our virtual environment with:tox-rType annotationsWe will also notice that we added type annotations to the function.
This is a good practice, to provide static type inference, and also allows us to use tools likemypyto check the type annotations.Testing the calcfunctionWe can now add a test for therescalecalcfunction.
In fact, if we were to use test-driven development, we would first write the test, and then write the code to make the test pass!Since the calcfunction needs to store data to an AiiDa profile, we need to create a profile for the tests.
We can do this by utilising thepgtest packageto create a temporary, local PostgreSQL database cluster, which allows us to create a temporary AiiDA profile, which we add to thepyproject.tomltest dependencies.For this we will first need anPostgreSQLserver running, with which to connect.
There are numerous ways to do this, such as using theDocker image, or installing viaHomebrewon macOS, or via thePostgreSQL apt repositoryon Ubuntu.To create the profile for the tests, we can add aconftest.pyfile to thetestsdirectory.
Then we register the AiiDApytest fixturesin it.
You can see all the available fixtures by runningpytest --fixtures(ortox -- --fixtures).
The initial one we need is theaiida_profile_cleanfixture, which creates a temporary profile, and tears it down at the end of the test.We can now add a passing test for therescalecalcfunction.Setting up PostgreSQL on GitHub ActionsWe also need to start a PostgreSQL server for our GitHub Actions, by specifying a service in thetest.ymlfile.
This actually uses thePostgreSQL Docker image, which is started before the tests are run.Adding the EquationOfState workchainWe shall now add the workflow itself.We add anproject.entry-points."aiida.workflows"section to thepyproject.tomlfile, to register theEquationOfStateworkchain as an entry point, for AiiDA to access.
After re-installing the package (pip install --no-deps -e ".[test]"), we can now use theverdi plugin list aiida.workflows eos.basecommand and see the registered workflow.For the testing we need to set up some more resources, before running the workflow.
We can do this by adding pytest fixtures to theconftest.pyfile.We also need to run against the actualpw.xexecutable.
One way to do this is to use conda, to install thequantum-espressopackage, which will install thepw.xexecutable in thebindirectory of the conda environment.
Another way we are developing isaiida-testing.Publishing the packageOnce you have a working package, it is good to create a version tag, and aCHANGELOG.mdto keep track of the changes.We can now publish the package toPyPI.
First create an account on PyPI, and then create an API token.You can either use locallyflit publishto publish the package, or use the GitHub Actions:
Add the token to thePYPI_KEYsecret in the GitHub repository settings.
Then we add a publishing workflow that runs on a new release.Once released to PyPI (or before) you can make a PR to theAiiDA plugin registryfor others to find your plugin.