Proposal - Network plugins

*This proposal documents a collective effort from @erikh (lead, network driver), @LK4D4 (sandbox), @aanand (UX, API), @tibor (state), @shykes (arch), and myself (daemon). It is only a draft: there is much more documentation to write (only basic UX is documented in `doc/` as part of this proposal), and some questions left answered. However, it should give a good overview on the ongoing work regarding network plugins.*

## Issue statement

Networking aspects in Docker suffer from two major flaws:
* It is naive, and most notably not multi-host aware.
* The networking related code is entangled with the rest of the daemon, and cannot be easily substituted with an alternative implementation.

The goal of this proposal is to introduce a new ways to customize Docker internal behavior with a first focus on networking. Following our "batteries included but removable" motto we want to keep providing a functional networking support out of the box, but we also want to allow anyone to come up with an awesome implementation of a networking stack for Docker and make it easy for users to take advantage of it.

A proof of concept was developed on [shykes/docker](https://github.com/shykes/docker) [`extensions` branch](https://github.com/shykes/docker/tree/extensions). Network is managed by a `simplebridge` driver which implements the already existing bridge + veth approach.

## Subsystems

The goal is to provide new "hook points" to Docker internals with a focus on networking. You might recognize in this approach was has been achieved in the past with `execdriver` or `graphdriver`. However, whereas these refer to low-level concepts, what we try to identify here are higher level responsibilities, and design them to be extensible from day one. We call these **subsystems**.

In the end, we expect the existing `docker/daemon` package to shrink as responsibilities are moved to subsystems, and he becomes more of a subsystems orchestrator.

### State

The **state** subsystem is the first introduced subsystem and provides persistency for the Docker engine. It was introduced early to provide a unified way for all subsystems to manage their internal state. The Docker engine has the responsibility of managing the state and providing each subsystem with their own persistent subsection of the global state.

The state subsystem lives inside the `/state` directory at the project root. Relevant links:
- [`State` interface](https://github.com/shykes/docker/blob/666fae8ee598c4aefd9fc4b65250e8ce837f4320/state/state.go#L6)
- [`GitState` implementation](https://github.com/shykes/docker/blob/666fae8ee598c4aefd9fc4b65250e8ce837f4320/state/gitstate.go) based on [docker/libpack](https://github.com/docker/libpack)

### Sandbox

The **sandbox** is a second introduced subsystem. It provides the mechanism to manipulate a sandboxed execution environment, and relies on the existing `execdriver` mechanism as an implementation detail. The reason sandbox was introduced in the context of network plugins is to abstract namespace manipulation: network drivers will rely on `sandbox` abstraction as a mean of inserting network interfaces inside a network namespace.

The sandbox subsystem lives inside the `/sandbox` directory at the project root. Relevant links:
- [`Sandbox` interface](https://github.com/shykes/docker/blob/666fae8ee598c4aefd9fc4b65250e8ce837f4320/sandbox/controller.go#L54) (implemented by the existing [`execriver.Driver`](https://github.com/shykes/docker/blob/666fae8ee598c4aefd9fc4b65250e8ce837f4320/daemon/execdriver/driver.go#L65))

### Network

Finally, the **network** subsystem provides mechanisms to create networks and have sandboxes join them. The subsystem manages *n* networks, themselves being managed by *m* drivers. Networks can be created through new API endpoint and are assigned a user-provided name. Having a sandbox join a network creates an **endpoint** with a user-provided name. This is very important change: containers don't have names, endpoint do: a container in no network has no name, while a container in *n* networks may have up to *n* distinct names. To provide a proper "out-of-the-box" experience, newly created sandboxes will be [joining a default network](https://github.com/shykes/docker/blob/666fae8ee598c4aefd9fc4b65250e8ce837f4320/daemon/create.go#L70) under either a randomly generated or user-provided name.

The network subsystem lives inside the `/network` directory at the project root. Relevant links:
- [`Driver` interface](https://github.com/shykes/docker/blob/666fae8ee598c4aefd9fc4b65250e8ce837f4320/network/driver.go#L8)
- The default [`simplebridge` implementation](https://github.com/shykes/docker/tree/666fae8ee598c4aefd9fc4b65250e8ce837f4320/extensions/simplebridge)
- New API endpoints implementation: [`daemon/net.go`](https://github.com/shykes/docker/blob/666fae8ee598c4aefd9fc4b65250e8ce837f4320/daemon/net.go)

## Extensions

Subsystems are conceived with flexibility in mind, which is why we introduce the concept of **extensions**. An extension is a collection of drivers that can be registered upon extension load. The idea behind having a *collection* of drivers rather than a single driver is to account for the possibility of a higher level extension which would required customizing different aspects of the Docker engine to provide their functionality. In a first step, extensions are compiled in with the Docker binary: this is not our end goal, as we intend to support out-of-process extensions.

The extension mechanism lives inside the `/extensions` directory at the project root. This is also where compiled-in extensions live for the moment.

## Remaining work

UX questions:
- How to pick which driver should manage a given network?
- How to specify a container should join a specific network in `docker run`?

Design questions:
- How to handle out-of-process plugins in the future?