Package: pyrealm
ThepyrealmpackageThese are development notes for the package, user documentation can be found at:OverviewThis a Python 3 package intended to provide a common framework for a number of
related models of plant productivity, growth and demography.Code developmentPackage managmentThe package usespoetryto manage dependencies, generate virtual environments for code
development and then for package building and publication. You will need to installpoetryto develop the code.Source code managementThe codebase is developed ingitwith a repository at:It uses thegit flowmodel for development and release. Briefly:All code development should happen specificfeature/feature_namebranches.Pull requests (PR) should be made to merge feature branches into thedevelopbranch.Candidate release versions should be made on specificrelease/x.y.zbranches
and these are then committed to themasterbranch only after final checking.Themasterbranch should only ever contain commits representing new release
versions - do not work on themasterbranch.Both thedevelopandmasterbranches are protected on GitHub to avoid accidents!Code qualityThe project uses thepre-committool to enforce code quality. The configuration file.pre-commit-config.yamlshows the details of the tool chain, butisort,black,flake8andmarkdownlintare used to maintain code quality. You will need to installpre-committo develop package code, and each PR must pass the same set of checks.Code testingUsingdoctestThe package docstrings containdoctestexamples of code use. These are intended to
demonstrate use and to validate a reference set of inputs against expected outputs. They
do not provide extensive unit testing! To run the docstring tests, use:python-mdoctestpyrealm/pmodel.py
python-mdoctestpyrealm/*.pyFordocteston warnings, see the example forpyrealm.utilities.convert_rh_to_vpdwhich redirects the stderr to stdout to allow for the warning text to be included in the
doctest.UsingpytestThetestdirectory containspytestmodules to provide greater testing of different
input combinations and to check errors are raised correctly. These are the main tests
that the package is behaving as expected.pytestContinuous integrationThe project uses continuous integration via GitHub Actions to check that the package is
building correctly and that bothdoctestandpytesttests are passing. The status of
builds can be seen at:DocumentationThepyrealmpackage is documented usingsphinx, with source material in thedocs/sourcedirectory.The documentation insourceusesMyst
Markdownrather than the standardsphinxreStructuredText (.rst) format. This is because the documentation uses themyst_nbextension tosphinxthat supports running documentation as a Jupyter
notebook: the built documentation includes examples of running code and output plots to
demonstrate the use and behaviour of the package.Thesphinxconfiguration includes thesphinx.ext.mathjaxextension to support
mathematical notation. This has been configured to also load themhchemextension,
supporting the rendering of chemical notation.DocstringsThe module codes uses docstrings written in theGoogle
style.
Unlike the main documentation pages, the docstrings in code are written using
reStructuredText because theautodocfunctions insphinxcurrently rely onrstinputs. This allows the function documentation to be stored alongside the code and
included simply into the documentation.Building the documentationAdditional python packages given indocs/source/requirements.txtare needed to build
the documentation. To actually build the documentation, usemakein the package root,
which will use theMakefilecreated bysphinx-quickstart.cddocs
makehtmlOnline documentationThe documentation for the package is hosted at:This has been configured to build commits to themasterbranch, which should
generate version specific sets of documentation.ReferencingThe documentation uses thesphinxcontrib-bibtexpackage to support citations.
This uses Latex like citation keys in the documentation to insert references and
build a bibliography. The reference library insource/refs.bibneeds to be
kept up to date with the literature for the project. Thesphinx_astrorefspackage is
used to provide an Author Date style citation format.Release processReleasing a new version of the package follows the flow below:Create areleasebranch fromdevelopcontaining the new release code.Check that this branch builds correctly, that the documentation builds correctly and
that the package publishes to thetest-pypirepository.When all is well, create pull requests on GitHub to merge thereleasebranch into
bothdevelopandmaster, along with a version tag for the release.Once you have updated your local repository, then the tag can be used to build and
publish the final version to the main PyPi site.It is easier ifgitis configured to push new tags along with commits. This
essentially just means that new releases can be sent with a single commit. This only
needs to be set once.setgitconfig--globalpush.followTagstrueCreate the release branchUsinggit flowcommands as an example to create a new release:gitflowreleasestartnew_releaseObviously, use something specific, notnew_release! Ideally, you would do a dry run of
the next step and use the version - but it should be fairly obvious what this will be!Thepoetry versioncommand can then be used to bump the version number. Note that the
command needs a 'bump rule', which sets which part of the semantic version number to
increment (major,minororpatch). For example:poetryversionpatchThis updatespyproject.toml. At present, the package is set up so that you alsohave
to update the version number inpyrealm/version.pyto match manually.Publish and test the release branchWith those changes committed, publish the release branch:gitcommit-m"Version bump"pyrealm/version.py
gitflowreleasepublishnew_releaseThe GitHub Actions will then ensure that the code passes quality assurance and then runs
the test suites on a range of Python versions and operating systems.Check package publicationThesdistandwheelbuilds for the package can then be built locally usingpoetrypoetrybuildThe first time this is run,poetryneeds to be configured to add the Test PyPi
repository and an API token from that site. Note that accounts are not shared between
the Test and main PyPi sites: the API token fortest-pypiis different frompypiand you have to log in to each system separately and generate a token on each.poetryconfigrepositories.test-pypihttps://test.pypi.org/legacy/
poetryconfigpypi-token.test-pypi<your-token>The built packages can then be published totest-pypiusing:poetrypublish-rtest-pypiCheck the documentation buildsLog in to:which is the admin site controlling the build process. From the Versions tab, activate
therelease/new_releasebranch and wait for it to build. Check the Builds tab to see
that it has built successfully and maybe check updates! If it has built succesfully, do
check pages to make sure that page code has executed successfully, and then go back to
the Versions tab and deactivate and hide the branch.Create pull requests intomasteranddevelopIf all is well, then two PRs need to be made on GitHub:Thereleasebranch intomaster, to bring all commits since the last release and
any fixes on release intomaster.Thereleasebranch intodevelop, to bring anyreleasefixes back intodevelop.Once both of those have been merged, thefeaturebranch can be deleted.Tag, build and publishOnce theoriginrepository is merged, then usegit pullto bringdevelopandmasterup to date on a local repo. Then, create a tag using the release version.gitcheckoutmaster
gittag<version>
gitpush--tagsThe final commit onmasteris now tagged with the release version. You can add tags on
the GitHub website, but only by using the GitHub release system and we are using PyPi to
distribute package releases.Before publishing a package to the main PyPi site for the first time, you need to set an
API token for PyPi.poetryconfigpypi-token.pypi<your-token>And now you can build the packages frommasterand publish.poetrybuild
poetrypublish