Package: gird
GirdGird is a lightweight & general-purposeMake-like build tool & task
runner for Python.FeaturesA simple, expressive, and intuitive rule definition and execution scheme very
close to that of Make.Configuration in Python, allowing straightforward and familiar usage, without
the need for a dedicated rule definition syntax.Ability to take advantage of Python's flexibility and possibility to easily
integrate with Python libraries and tools.Emphasis on API simplicity & ease of use.Example use casesData science & data analytics workflows.Portable CI tasks.Less rule-heavy application build setups. (Build time overhead may become
noticeable with thousands of rules.)Any project with tasks that need to be executed automatically when some
dependencies are updated.InstallationInstall Gird from PyPI withpip install gird, or from sources withpip install ..Gird requires Python version 3.9 or newer, and is supported on Linux & macOS.UsageDefine rules ingirdfile.py. Depending on the composition of a rule
definition, a rule can, for example,define a recipe to run a task, e.g., to update a target file,define prerequisites for the target, such as dependency files or other rules,
anduse Python functions for more complex dependency & recipe functionality.A rule is invoked via the CLI bygird {target}. To list rules, rungird list.When invoked, a rule will be run if its target is considered outdated. This
is the case if the rulehas aPhonytarget,has aPathtarget that does not exist,has aPathtarget and aPathdependency that is more recent than the target,has an outdatedRule/target as a dependency, orhas a function dependency that returnsTrue.Rules with outdated targets are run in topological order within the
dependency graph, i.e., all outdated dependencies are updated before the
respective targets. By default, rules are run in parallel when possible.Example rulesA rule with files as its target & dependencyA girdfile.py with the following contents defines a single rule that, whengird package.whlis invoked, buildspackage.whlwhenevermodule.pyis modified. If module.py hasn't been modified, the packaging recipe will not be executed.importpathlibimportgirdRULE_BUILD=gird.rule(target=pathlib.Path("package.whl"),deps=pathlib.Path("module.py"),recipe="python -m build --wheel",)A rule with a phony targetPhony targets can be used when there's not any actual target to update. The recipe of a rule with a phony target is always executed.RULE_TEST=gird.rule(target=gird.Phony("test"),recipe="pytest",)A rule with other rules as dependenciesThis is equivalent to using the targets of the rules as dependencies. Here, a phony target is used to give an alias to a group of other rules.gird.rule(target=gird.Phony("all"),deps=[RULE_TEST,RULE_BUILD,],)A compound recipe for mixing Python functions with shell commandsFILE1=pathlib.Path("file1")gird.rule(target=FILE1,recipe=[FILE1.touch,f"echo text >>{FILE1.resolve()}",],)A Python function as a recipe with parametersUse, e.g.,functools.partialto turn a function and its arguments into a callable with no arguments.importfunctoolsimportshutilFILE2=pathlib.Path("file2")gird.rule(target=FILE2,deps=FILE1,recipe=functools.partial(shutil.copy,FILE1,FILE2),)A Python function as a dependency to arbitrarily trigger rulesBelow, have a remote file re-fetched if it has changed.defhas_remote_changed():returnget_checksum_local()!=get_checksum_remote()gird.rule(target=FILE1,deps=has_remote_changed,recipe=fetch_remote,)Implement theTimeTrackedprotocol for custom targets & dependenciesSuch types are treated identically toPathobjects, which respectively have a time of modification that is tracked for resolving outdatedness.For example, define platform-specific logic to apply dependency tracking on
a remote file.classRemoteFile(gird.TimeTracked):def__init__(self,url:str):self._url=url@propertydefid(self):returnself._url@propertydeftimestamp(self):returnget_remote_file_timestamp(self._url)gird.rule(target=FILE1,deps=RemoteFile(URL),recipe=fetch_remote,)Define rules flexiblyAll that matter are therulefunction calls that are executed when the girdfile.py is imported. The structure of the file and other implementation details are completely up to the user.RULES=[gird.rule(target=source.with_suffix(".gz"),deps=gird.rule(target=source,recipe=functools.partial(fetch_remote,source),),recipe=f"gzip -k{source.resolve()}",)forsourcein[FILE1,FILE2]]Example girdfile.pyThis is the girdfile.py of the project itself.fromitertoolsimportchainfrompathlibimportPathfromgirdimportPhony,rulefromscriptsimportassert_readme_updated,get_wheel_path,render_readmeWHEEL_PATH=get_wheel_path()RULE_PYTEST=rule(target=Phony("pytest"),recipe="pytest -n auto --cov=gird --cov-report=xml",help="Run pytest & get code coverage report.",)RULE_MYPY=rule(target=Phony("mypy"),recipe="mypy --check-untyped-defs -p gird",help="Run mypy.",)RULE_CHECK_FORMATTING=rule(target=Phony("check_formatting"),recipe=["black --check gird scripts test girdfile.py","isort --check gird scripts test girdfile.py",],help="Check formatting with Black & isort.",)RULE_CHECK_README_UPDATED=rule(target=Phony("check_readme_updated"),recipe=assert_readme_updated,help="Check that README.md is updated based on README_template.md.",)RULES_TEST=[RULE_PYTEST,RULE_MYPY,RULE_CHECK_FORMATTING,RULE_CHECK_README_UPDATED,]rule(target=Phony("test"),deps=RULES_TEST,help="\n".join(f"-{rule.help}"forruleinRULES_TEST),)rule(target=Path("README.md"),deps=chain(*(Path(path).iterdir()forpathin("scripts","gird")),[Path("girdfile.py"),Path("pyproject.toml")],),recipe=render_readme,help="Render README.md based on README_template.md.",)# Wrap the rule to build WHEEL_PATH in a phony rule for simpler invocation.# Don't include the inner rule in `gird list`.rule(target=Phony("build"),deps=rule(target=WHEEL_PATH,recipe="poetry build --format wheel",listed=False,),help="Build distribution packages for the current version.",)Respective output fromgird list:pytest
    Run pytest & get code coverage report.
mypy
    Run mypy.
check_formatting
    Check formatting with Black & isort.
check_readme_updated
    Check that README.md is updated based on README_template.md.
test
    - Run pytest & get code coverage report.
    - Run mypy.
    - Check formatting with Black & isort.
    - Check that README.md is updated based on README_template.md.
README.md
    Render README.md based on README_template.md.
build
    Build distribution packages for the current version.