caktus/django-project-template — pull request #67 by mlavin (2014-01-03 18:36:02+00:00)
Title: Salt Deployment Refactor

# Summary
This refactor incorporates changes which we've made in recent projects which required more complex setups. There is some more work to do, mostly in the documentation and testing, but I wanted to get some eyes on this early. The major changes are:

- Deployments are now handled by Salt using Github deploy keys. Less mixing of Fabric and Salt. Salt does pretty much everything other than install the master and minion.
- States are targeted by the server roles instead of assuming there is a single server deployment

This relies on a couple changes which have yet to land in margarita namely: https://github.com/caktus/margarita/pull/12, https://github.com/caktus/margarita/pull/11, and https://github.com/caktus/margarita/pull/9 It also essentially moves the `postgresql.project`, `users.devs`, and `rabbitmq.project` states into the repo. The general trend is moving states which require pillar information out of margarita.

# Backwards Incompatible Changes
This breaks the ability to run multiple projects/environments on the same box. Since each server is a Salt Minion and can only have one master this just isn't feasible. In reality that was rarely if ever used by our deployments and added an absurd amount of complexity to some of the states/file names because of it.

There is also no easy upgrade path for existing projects.

# New Features

## Multi-Server Deployment

These changes make it much easier to run a multi-server deployment out of the box. Each server has its role declared when it is
provisioned (which can be updated later). A single server deployment would simply take on all of the relevant roles. For multi-server deployments the firewall rules are added to allow servers to communicate while blocking other connections. Since the deployment is handled by the master, servers are updated in parallel by default.

## Python 3
Along with the margarita changes it is easier to use Python 3. This needs a little more work add testing but the foundation is there.

## Better Defaults
Postgres and RabbitMQ are configured by default. Supervisor configs are only readable by root. `ALLOWED_HOSTS` is configured by default.

# Remaining Work

A few more things I'd like to do before I call this completely finished. Since we've done all of these things recently it shouldn't be hard to build from those experiences.

- [ ] Documentation
- [ ] Clean up Python 3 support
- [ ] Celery 3.1 support
- [ ] Functional testing with a new project
- [ ] Support dedicated/non-local cache/queue server
- [ ] Support for multiple webservers behind a single load balancer
- [ ] Support for DB slaves