Fleshed out layering and general structure of cockroach. The devilish details remain; I tried to capture many of the TODOs in comments.

Summary:
server/server.go is the entry point to the system. A server now creates
the following objects:

 - An HTTP server for handling RESTful APIs

 - An RPC server for handling Raft, Gossip, & Roach Key-Value RPCs

 - A Gossip instance which connects P2P to the larger gossip network
   by spawning clients in order to minimize total hops to all known
   data while still remaining under a maximum fan out limit. Each
   gossip instance also registers a gossip service on the RPC
   server.

 - A local storage Node which multiplexes incoming requests to the
   appropriate storage engine (one storage engine per physical storage
   device). Nodes register a Node service on the RPC server, which
   accepts incoming Roach Key-Value RPCs. The storage node is passed
   the gossip instance so the node can advertise its availability. It
   also uses the gossip instance to access information about other
   available nodes when allocating new replicas to build consensus
   groups.

 - A Roach Key-Value database (kvDB). This is a client which provides
   virtual access to the monolithic sorted map by looking up ranges to
   satisfy requests to read or write the map. For range scans, for
   example, it may decompose the request into multiple RPCs. This layer
   provides an async API for higher-level client APIs.

 - A Key-Value RESTful API server. Provides CRUD functionality via HTTP
   POST, GET, PUT & DELETE methods. The KV REST server is instantiated
   with the roach node's HTTP serve mux.

 - A structured schema database (structuredDB). This is a client built
   on top of the key-value database that understands how to construct
   schemas comprised of familiar RDBMS concepts such as tables, columns,
   keys, indexes, foreign keys and a query language. A structuredDB is
   intantiated with a reference to the underlying kvDB.

 - A structured schema RESTful API server. Provides CRUD functionality
   via HTTP methods. Unlike the simple KV RESTful server, the structured
   one generates endpoints for each table in the schema and depending
   on the indexes and foreign keys defined. It automatically exports a
   complete interface as well as documentation.

---

Extracted RPC server and client code from gossip/ into a more general
location in rpc/. This still has a ways to go.

Also, moved contents of former db/ package into storage/ and for the
moment decommissioned the Raft code. It's currently commented out in
storage/raft.go. I removed the delete and put commands with the
expectation that the commands being serialized into the Raft log match
those of the Cockroach "Node" RPC service. That API is defined by
storage/node.go and kv/messages.go.

As mentioned in a previous comment, a single instance of cockroach
exports a single Node server. That server multiplexes between one or
more "stores". Each store corresponds to one physical device. E.g.
you might have a store for ssd01, ssd02, sda1, sda2, etc. Each store
is backed by a storage engine. The interface is in storage/engine.go;
there are two implementations: storage/in_mem.go & storage/rocksdb.go.

Although each store has a single engine, there can be many ranges
(as high as 2^18). A store's job is to maintain a map of range objects.
Each range hosts a non-overlapping, probably non-contiguous region of
the overall keyspace. Each range maintains its own Raft instance to
mediate writes between the replicas in the range. Whichever replica
is the established leader makes decisions about new allocations. Each
range has a reference to the underlying engine (it can only write to
its key range), and to an allocator for maintaining range replicas.

I removed the protobuf API for the key value store. We won't need it
in the form originally envisaged. It's contents have been reworked into
kv/messages.go.

Also added an LRU cache, taken from another go package. Not in use,
but will need to press it into service soon.

Test Plan:
This is a partial review submission. It doesn't even compile.
I just wanted to get it looked over now rather than in a few more days.

Reviewers: petermattis, andybons

Reviewed By: andybons

Subscribers: levonlloyd

Differential Revision: http://phabricator.andybons.com/D15