Package: datastreamservicelib
AsyncIO eventloop helpers and Abstract Base Classes for making services that use ZMQ nice, easy and DRYUsageUse the CookieCutter template athttps://gitlab.com/advian-oss/python-datastreamserviceapp_templateYou can also take a look at src/datastreamservicelib/console.py for some very simple test examples.DockerFor more controlled deployments and to get rid of “works on my computer” -syndrome, we always
make sure our software works under docker.It’s also a quick way to get started with a standard development environment.SSH agent forwardingWe needbuildkit:export DOCKER_BUILDKIT=1And also the exact way for forwarding agent to running instance is different on OSX:export DOCKER_SSHAGENT="-v /run/host-services/ssh-auth.sock:/run/host-services/ssh-auth.sock -e SSH_AUTH_SOCK=/run/host-services/ssh-auth.sock"and Linux:export DOCKER_SSHAGENT="-v $SSH_AUTH_SOCK:$SSH_AUTH_SOCK -e SSH_AUTH_SOCK"Creating a development containerBuild image, create container and start it:docker build --ssh default --target devel_shell -t datastreamservicelib:devel_shell .
docker create --name datastreamservicelib_devel -v `pwd`":/app" -it -v /tmp:/tmp `echo $DOCKER_SSHAGENT` datastreamservicelib:devel_shell
docker start -i datastreamservicelib_develpre-commit considerationsIf working in Docker instead of native env you need to run the pre-commit checks in docker too:docker exec -i datastreamservicelib_devel /bin/bash -c "pre-commit install"
docker exec -i datastreamservicelib_devel /bin/bash -c "pre-commit run --all-files"You need to have the container running, see above. Or alternatively use the docker run syntax but using
the running container is faster:docker run -it --rm -v `pwd`":/app" datastreamservicelib:devel_shell -c "pre-commit run --all-files"Test suiteYou can use the devel shell to run py.test when doing development, for CI use
the “tox” target in the Dockerfile:docker build --ssh default --target tox -t datastreamservicelib:tox .
docker run -it --rm -v `pwd`":/app" `echo $DOCKER_SSHAGENT` datastreamservicelib:toxProduction dockerThere’s a “production” target as well for quick running of testsubscriber and testpublisherdocker build –ssh default –target production -t datastreamservicelib:latest .
docker run –rm -it -v /tmp:/tmp datastreamservicelib:latest testpublisher -s ipc:///tmp/test_pub.sock -t foo
docker run –rm -it -v /tmp:/tmp datastreamservicelib:latest testsubscriber -s ipc:///tmp/test_pub.sock -t fooNote that on non-linux platforms the IPC sockets may not as expected between host and container over volume mounts.Local DevelopmentTLDR:Create and activate a Python 3.6 virtualenv (assuming virtualenvwrapper):mkvirtualenv -p `which python3.6` my_virtualenvchange to a branch:git checkout -b my_branchinstall Poetry:https://python-poetry.org/docs/#installationInstall project deps and pre-commit hooks:poetry install
pre-commit install
pre-commit run --all-filesReady to go, try the following:testpublisher --help
testsubscriber --helpUse Python 3.6 for development since it’s the lowest supported version so you don’t accidentally
use features only available in higher versions and then have to re-do everything when CI tests fail
on 3.6.Remember to activate your virtualenv whenever working on the repo, this is needed
because pylint and mypy pre-commit hooks use the “system” python for now (because reasons).Running “pre-commit run –all-files” and “py.test -v” regularly during development and
especially before committing will save you some headache.