dbmisvc-stack 1.0.1
A command line program to facilitate development on a stack of containerized services
License: BSD 2-Clause
Keywords: cli

# Stack

#### A program for running a stack of Docker services

## Purpose

This is a program to provide some convenience wrappers around common tasks
when developing with a containerized stack of micro-services. The program aims
to simplify some of the management of a local stack and make it easier to
develop on an entire stack whilst running and testing it. The current version
only supports Docker.

The application depends on configurations defined in the base `stack.yml`
file for the applications that will be built.

## App Properties

- `repository`: This should specify the URL to the app's git repository.
- `branch`: The particular branch to checkout when cloning the repo.

## Setup

0. Create your Python virtualenv and install requirements:
`pip install -r requirements.txt`

1. First step is to place any needed overrides in the `overrides/{APP}`
directory. These files are what will be used to build the image that
will be run in the stack. Typically they closely mirror the files
the app uses in its production environment, but with small tweaks to
run locally and to interface with other local apps. This directory
must contain a `Dockerfile` if the app is to run off a built image.

2. The next step is to clone all needed repositories into the `apps` directory.
`dbmisvc-stack` includes some convenience commands for managing the subtree
repositories. Do this with one command by running `dbmisvc-stack init`.
See below for further details on repository management.

3. Next, configure secrets. Ensure the secrets configuration is accurate in `stack.yml`. Specifically, ensure your
AWS profile is set to the account that contains this Stack's secrets. Secrets must be fetched and persisted locally
as `.env`, do this by running `dbmisvc-stack secrets [--force]`.

4. The next step is to make sure the `docker-compose.yml` file is fully
and accurately filled out to include the apps and their configurations.
This includes any volumes needed to mount the app's source within the
container. This allows for file changes to update the services
automatically so changes can be tested immediately. Verify current setup
by running `dbmisvc-stack check`.

5. Lastly, update the hook scripts with any extra bits of code
needed to run the stack. Whether you need to collect npm
dependencies after cloning a repo or you need a database to
be cleared when cleaning an app, this is where custom functionality
should live.

## Stack Commands

To check stack configurations and to ensure volume paths are correct,
required images exist, etc:

> `dbmisvc-stack check [<app>]`

Not passing an app will iterate through all services specified in the
`docker-compose.yml` file and check all configurations.

Run the initialize command to clone all needed repositories to their
respective branches:

> `dbmisvc-stack init`

Most applications will require sensitive secrets to function and stack assumes those will be saved in AWS Secrets Manager. Be sure
the correct configuration is set in `stack.yml` and remote secrets will be fetched and persisted to `.env` which is automatically
used by docker-compose as a source of environment variables for services.

To pull those secrets down from AWS by running (pass `-f` to overwrite existing `.env` file):

> `dbmisvc-stack secrets [-f]`

To get the stack going, run the following command (pass `-d` to daemonize
the process):

> `dbmisvc-stack up [-d]`

If a container needs to be rebuilt for some reason (updated requirements, etc),
run the following command (app is the key of the service in your `docker-compose.yml`):

> `dbmisvc-stack reup <app> [--clean]`

You could instead shell into the needed container and run the requirements
update command there:

> `dbmisvc-stack shell <app> [-sh]`

Stack defaults to trying to open a bash shell, but you can default to
sh if bash is not available.

You can also check logs on a container with a couple constraints to more
easily find the relevant logs:

> `dbmisvc-stack logs <app> [--minutes=n] [--lines=n] [-f]`

You can specify how many minutes in the past to start the log retrieval
or the number of lines to get. You can also pass the `-f` flag to follow
the logs as the container runs.

This will stop and remove the container, and then start it up again. The clean
flag will purge the existing container image and rebuild before running again.

To bring the stack down, run the following:

> `dbmisvc-stack down`

This merely wraps `docker-compose down --volumes` and brings the stack down
and removes any left-over data volumes.

> `dbmisvc-stack packages [package]`

This command will attempt to build and upload the package to the PyPi
mirror for use by the apps in the Stack. Any apps that were marked
as dependent on this package will trigger a reinstall of that
package automatically when a new build is successfully registered with
the local PyPi mirror.


## Git Subtree Helper Commands

Stack apps are included as git subtrees. Commands were added to Stack to
wrap and simplify the commands needed to work with these repositories.

Before cloning, make sure the app exists in the `docker-compose.yml` file
and has the required configurations, namely the repository URL. The current
`docker-compose.yml` file illustrates how to do this with the current set
of apps.

**Git subtree commands will not run with pending changes in the working
directory. Commit all changes before running subtree commands.**

To clone an existing repo into the `apps` directory:

> `dbmisvc-stack clone <app> <branch>`

This will clone the repo as a subtree in the `apps` directory
(default: {PROJECT_ROOT}/apps). If the app is already present in the
`apps` directory, that copy will be removed and the specified branch
will be cloned in its place.

To pull remote changes into the local branch, use the `pull` command:

> `dbmisvc-stack pull <app> <branch> [--squash]`

All commits pulled are added to the stack repository so squashing
the incoming commits keeps history tidy.

To create a new branch for a specified app:

> `dbmisvc-stack checkout <app> -b <branch>`

This splits the subtree into the new specified branch. Commit changes
as usual for the entire stack repository. Once an update is ready to push,
run the push command as usual:

> `dbmisvc-stack push <app> <branch> [--squash]`

This will collect commits relevant to the particular subtree and push those
to origin for the new branch. The `--squash` command will collapse those
commits into a single commit.

To get back to the base branch, checkout the branch as usual:

> `dbmisvc-stack checkout <app> <branch>`

This removes the subtree entirely, and clones the specified branch in
its place.