Package: miniboss
minibossminiboss is a Python application for locally running a collection of
interdependent docker services, individually rebuilding and restarting them, and
managing application state with lifecycle hooks. Services definitions can be
written in Python, allowing the use of programming logic instead of markup.Why not docker-compose?First and foremost, good old Python instead of YAML.docker-composeis in the
school of yaml-as-service-description, which means that going beyond a static
description of a service set necessitates templates, or some kind of scripting.
One could just as well use a full-blown programming language, while trying to
keep simple things simple. Another thing sorely missing indocker-composeis
lifecycle hooks, i.e. a mechanism whereby scripts can be executed when the state
of a container changes. Lifecycle hooks have beenrequestedmultipletimes, but were not
deemed to be in the domain ofdocker-compose.Installationminiboss ison PyPi; you can install it
with the following:pip install minibossUsageHere is a very simple service specification:#! /usr/bin/env python3importminibossminiboss.group_name('readme-demo')classDatabase(miniboss.Service):name="appdb"image="postgres:10.6"env={"POSTGRES_PASSWORD":"dbpwd","POSTGRES_USER":"dbuser","POSTGRES_DB":"appdb"}ports={5432:5433}classApplication(miniboss.Service):name="python-todo"image="afroisalreadyin/python-todo:0.0.1"env={"DB_URI":"postgresql://dbuser:dbpwd@appdb:5432/appdb"}dependencies=["appdb"]ports={8080:8080}stop_signal="SIGINT"if__name__=="__main__":miniboss.cli()The first use of miniboss is in the call tominiboss.group_name, which
specifies a name for this group of services. If you don't set it, sluggified
form of the directory name will be used. Group name is used to identify the
services and the network defined in a miniboss file. Setting it manually to a
non-default value will allow miniboss to manage multiple collections in the same
directory.Aserviceis defined by subclassingminiboss.Serviceand overriding, in
the minimal case, the fieldsimageandname. Theenvfield specifies the
environment variables. As in the case of theappdbservice, you can use
ordinary variables anywhere Python accepts them. The other available fields are
explained in the sectionService definition
fields. In theabove example, we are
creating two services: The application servicepython-todo(a simple Flask
todo application defined in thesample-appsdirectory) depends onappdb(a
Postgresql container), specified through thedependenciesfield. As indocker-compose, this means thatpython-todowill get started afterappdbreaches running status.Theminiboss.clifunction is the main entry point; you need to call it in the
main section of your script. Let's run the script above without arguments, which
leads to the following output:Usage: miniboss-main.py [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  start
  stopWe can start our small collection of services by running./miniboss-main.py start. After spitting out some logging text, you will see that starting the
containers failed, with thepython-todoservice throwing an error that it
cannot reach the database. The reason for this error is that the Postgresql
process has started, but is still initializing, and does not accept connections
yet. The standard way of dealing with this issue is to include backoff code in
your application that checks on the database port regularly, until the
connection is accepted.minibossoffers an alternative withlifecycle
events. For the time being, you can simply rerun./miniboss-main.py start, which will restart only thepython-todoservice,
as the other one is already running. You should be able to navigate tohttp://localhost:8080and view the todo app page.You can also exclude services from the list of services to be started with the--excludeargument;./miniboss-main.py start --exclude python-todowill
start onlyappdb. If you exclude a service that is depended on by another, you
will get an error. If a service fails to start (i.e. container cannot be started
or the lifecycle events fail), it and all the other services that depend on it
are registered as failed.Stopping servicesOnce you are done working with a collection, you can stop the running services
withminiboss-main.py stop. This will stop the services in the reverse order
of dependency, i.e. firstpython-todoand thenappdb. Exclusion is possible
also when stopping services with the same--excludeargument. Running./miniboss-main.py stop --exclude appdbwill stop only thepython-todoservice. If you exclude a service whose dependency will be stopped, you will get
an error. If, in addition to stopping the service containers, you want to remove
them, include the option--remove. If you don't remove the containers,
miniboss will restart the existing containers (modulo changes in service
definition) instead of creating new ones the next time it's called withstart.
This behavior can be modified with thealways_start_newfield; see the details
inService definition fields.Reloading a serviceminiboss also allows you to reload a specific service by building a new
container image from a directory. You need to provide the path to the directory
in which the Dockerfile and build context of a service resides in order to use
this feature. You can also provide an alternative Dockerfile name. Here is an
example:classApplication(miniboss.Service):name="python-todo"image="afroisalreadyin/python-todo:0.0.1"env={"DB_URI":"postgresql://dbuser:dbpwd@appdb:5432/appdb"}dependencies=["appdb"]ports={8080:8080}build_from="python-todo/"dockerfile="Dockerfile"Thebuild_fromoption has to be a path relative to the main miniboss file.
With such a service configuration, you can run./miniboss-main.py reload python-todo, which will cause miniboss to build the container image, stop the
running service container, and restart the new image. Sincethe
contextgenerated at start is saved in a file, any context
values used in the service definition are available to the new container.Lifecycle eventsOne of the differentiating feature of miniboss is lifecycle events, which are
hooks that can be customized to execute code at certain points in a service's or
the whole collection's lifecycle.Per-service eventsFor per-service events,miniboss.Servicehas three methods that can be
overriden in order to correctly change states and execute actions on the
container:Service.pre_start(): Executed before the service is started. Can be used
for things like initializing mount directory contents or downloading online
content.Service.ping(): Executed repeatedly right after the service starts with
a 0.1 second delay between executions. If this method does not returnTruewithin a given timeout value (can be set with the--timeoutargument,
default is 300 seconds), the service is registered as failed. Any exceptions
in this method will be propagated, and also cause the service to fail. If
there is already a service instance running, it is not pinged.Service.post_start(): This method is executed after a successfulping.
It can be used to prime a service by e.g. creating data on it, or bringing it
to a certain state. You can also use the global context in this method; seeThe global contextfor details. If there is already a
service running, or an existing container image is started insted of creating
a new one, this method is not called.These methods arenoopby default. A
service is not registered as properly started before lifecycle methods are
executed successfully; only then are the dependant services started.Thepingmethod is particularly useful if you want to avoid the situation
described above, where a container starts, but the main process has not
completed initializing before any dependent services start. Here is an example
for how one would ping theappdbservice to make sure the Postgresql database
is accepting connections:importpsycopg2classDatabase(miniboss.Service):# fields same as abovedefping(self):try:connection=psycopg2.connect("postgresql://dbuser:dbpwd@localhost:5433/appdb")cur=connection.cursor()cur.execute('SELECT 1')exceptpsycopg2.OperationalError:returnFalseelse:returnTrueOne thing to pay attention to is that, in the call topsycopg2.connect, we are
usinglocalhost:5433as host and port, whereas thepython-todoenvironment
variableDBURIhasappdb:5433instead. This is because thepingmethod is
executed on the host computer. The next section explains the details.Collection eventsIt is possible to hook into collection change commands using the following
hooks. You can call them on the baseminibossmodule and set a hook by passing
it in as the sole argument, e.g. as follows:importminibossdefprint_services(service_list):print("Started ",' '.join(service_list))miniboss.on_start_services(print_services)on_start_serviceshook is called after theminiboss.startcommand is
executed. The single argument is a list of the names of the services that were
successfully started.on_stop_serviceshook is called after theminiboss.stopcommand is
executed. The single argument is a list of the services that were stopped.on_reload_servicehook is called after theminiboss.reloadcommand is
executed. The single argument is the name of the service that was reloaded.Ports and hostsminiboss starts services on an isolated bridge network, mapping no ports by
default. The name of this service can be specified with the--network-nameargument when starting a group. If it's not specified, the name will be
generated from the group name by prefixing it withminiboss-. On the
collection network, services can be contacted under the service name as
hostname, on the ports they are listening on. TheappdbPostgresql serviceabove, for example, can be contacted on the port 5432, the default
port on which Postgresql listens. This is the reason the host part of theDB_URIenvironment variable on thepython-todoservice isappdb:5432. If
you want to reachappdbon the port5433from the host system, which would
be necessary to implement thepingmethod as above, you need to make this
mapping explicit with theportsfield of the service definition. This field
accepts a dictionary of integer keys and values. The key is the service
container port, and the value is the host port. In the case ofappdb, the
Postgresql port of the container is mapped to port 5433 on the local machine, in
order not to collide with any local Postgresql instances. With this
configuration, theappdbdatabase can be accessed atlocalhost:5433.The global contextThe objectminiboss.Context, derived from the standard dict class, can be used
to store values that are accessible to other service definitions, especially in
theenvfield. For example, if you create a user in thepost_startmethod of
a service, and would like to make the ID of this user available to a dependant
service, you can set it on the context withContext['user_id'] = user.id. In
the definition of the second service, you can refer to this value in a field
with the standard Python keyword formatting syntax, as in the following:classDependantService(miniboss.Service):# other fieldsenv={'USER_ID':'{user_id}'}You can of course also programmatically access it asContext['user_id']once a
value has been set.When a service collection is started, the generated context is saved in the file.miniboss-context, in order to be used when the same containers are restarted
or a specific service isreloaded.Service definition fieldsname: The name of the service. Must be non-empty and unique for one
miniboss definition module. The container can be contacted on the network
under this name; it must therefore be a valid hostname.image: Container image of the service. Must be non-empty. You can use a
repository URL here; if the image is not locally available, it will be pulled.
You are highly advised to specify a tag, even if it'slatest, because
otherwise miniboss will not be able to identify which container image was used
for a service, and start a new container each time. If the tag of theimageislatest, and thebuild_fromdirectory option is specified, the container
image will be built each time the service is started.entrypoint: Container entrypoint, the executable that is run when the
container starts. SeeDocker
documentationfor
details.cmd:CMDoption for a container. SeeDocker
documentationfor
details.user:USERoption for a container SeeDocker
documentationfor
details.dependencies: A list of the dependencies of a service by name. If there
are any invalid or circular dependencies, an exception will be raised.env: Environment variables to be injected into the service container, as
a dict. The values of this dict can contain extrapolations from the global
context; these extrapolations are executed when the service starts.ports: A mapping of the ports that must be exposed on the running host.
Keys are ports local to the container, values are the ports of the running
host. SeePorts and hostsfor more details on networking.volumes: Directories to be mounted inside the services as a volume, on
which mount points. The value ofvolumescan be either a list of strings, in
the format"directory:mount_point:mode", or in the dictionary format{directory: {"bind": mount_point, "mode": mode}}. In both cases,modeis
optional. See theUsing
volumessection of Docker Python SDK documentation for details.always_start_new: Whether to create a new container each time a service
is started or restart an existing but stopped container. Default value isFalse, meaning that by default existing container will be restarted.stop_signal: Which stop signal Docker should use to stop the container,
by name (not by integer value, so don't use values from thesignalstandard
library module here). Default isSIGTERM. Accepted values areSIGINT,SIGTERM,SIGKILLandSIGQUIT.build_from: The directory from which a service can be reloaded. It
should be either absolute, or relative to the main script. Required if you
want to be able to reload a service. If this option is specified, and the tag
of theimageoption islatest, the container image will be built each time
the service is started.dockerfile: Dockerfile to use when building a service from thebuild_fromdirectory. Default isDockerfile.Release notes0.3.0LintingPull container image if it doesn't existIntegration testsMounting volumesPre-start lifetime event0.4.0Don't fail on start if excluded services depend on each otherDestroy service if it cannot be startedLog when custom post_start is doneDon't start new if int-string env keys don't differDon't run pre-start if container foundMultiple clusters on single host with group idBuild container if tag doesn't exist and it hasbuild_fromBetter pypi readme with release notes0.4.1Tests for CLI commandsCollection lifecycle hooks0.4.2Removed group name requirementLogging fixesSample app fixes0.4.3Entrypoint, cmd and user fields on serviceType hintsUse tbump for version bumping0.4.3Corrected docker lcient library version in dependenciesTodosUser attrs properly with typesAdd stop-only commandAdd start-only commandMaking easier to test on the cloud??Run tests in container (how?)Exporting environment values for use in shellRunning one-off containersConfiguration object extrapolationRunning tests once system startedUsing context values in testsDependent test suites and setups