Package: bandsaw
BandsawBandsaw is a python library that allows to split a python workflow into individual tasks
that can be run separately with different python interpreters and even on different
machines.What it doesBandsaw can be used to create distributed python scripts, that define workflows in
heterogeneous environments and with conflicting dependencies. It is especially meant
for  building complex machine learning processes, that use different machine learning
frameworks like tensorflow or pytorch in a single workflow, or need to run on multiple
different computation platforms, e.g. in different regions due to data restrictions.How it worksThis works by decorating python functions with the bandsaw @task decorator:importbandsaw...@bandsaw.taskdefmy_function(x):returnxThis decorator gives bandsaw the opportunity to run additional code before and after the
code withinmy_function(x)is executed. This additional code is defined in
classes that fulfill theAdviceprotocol.When this decorated function is called, bandsaw intercepts the call first and runs thebefore()methods of the configuredAdvices. Then the wrapped function is computed
and afterwards each advice'safter()method is called.All callback functions receive aSessionobject as single
argument, which is used for continuing the execution by calling itsproceed()method.Advices have the possibility toconclude()this early by providing aResulton
their own. This shortcuts the computation and can be used for caching results.
Additionally, the session can be serialized and transferred to other python
processes even on other machines using itssave(stream)andrestore(stream)methods.A full explanation of this can be found in the latestuser guide.DevelopBandsaw usestoxto build and test the library.
Tox runs all tests on different python versions, can generate the documentation and run
linters and style checks to improve the code quality.
In order to install all the necessary python modules, please run:pipinstalltoxAfterwards the tests can be run by just callingtoxfrom the project directory. For this to work, you need to have multiple python
interpreters installed. If you don't want to run the tests on all supported platforms
just edit the tox.ini file and setenvlist = py36,py37,py38to contain only the python version you want to use. Another option is to run tox with
the additional command line argument'--skip_missing_interpreters'which skips python versions that aren't installed.DocumentationThe latest version of the documentation can always be found underhttps://docs.kant.ai/bandsaw/latest.
The documentation is written inMarkdownand is located in thedocsdirectory of the project.
It can be built into static HTML by usingMkDocs.
In order to manually generate the documentation we can use tox to build the HTML pages from our markdown.tox-edocsReleaseReleasing a new package versionReleasing new versions of bandsaw is done usingflit.pipinstallflitIn order to be able to publish a new release, you need an account with PyPI or their
respective test environment.Add those accounts into your~.pypirc:[distutils]
index-servers =
  pypi
  pypitest

[pypi]
username: <my-user> 

[pypitest]
repository: https://test.pypi.org/legacy/
username: <my-test-user>Publishing a new release to testflitpublish--repositorypypitestReleasing a new version of the documentationThe package usesmiketo manage multiple versions of the documentation. The already generated documentation is kept
in thedocs-deploymentbranch and will be automatically deployed, if the branch is pushed to
the repository.In order to build a new version of the documentation, we need to use the corresponding tox environment:VERSION_TAG='<my-version>'tox-edocs-releaseTheVERSION_TAGenvironment variable should be set to the new version in format '.'.
This will build the documentation and add it as new commits to thedocs-deploymentbranch.By pushing the updated branch to the gitlab repository, the documentation will be automatically
deployed tothe official documentation website.