tchandy/octopus — pull request #239 by mumoshu (2014-04-18 08:56:29+00:00)
Title: Introducing slave groups, can be used concurrently with sharding

Hi, @tchandy and commiters

First of all, I would like to thank you for sharing this great library :smile:
At CrowdWorks, we are using Octopus in production and it is working like a charm :+1:

In this pull request, I'm introducing "slave groups" to make Octopus distribute queries to a specific group of slaves.
They can be used concurrently with shards, e.g. merging this pull request also adds ["Replication + Sharding" in your TODO list](https://github.com/tchandy/octopus#replication--sharding).
This change is backward-compatible.

-----

A slave group is a group of slaves to which queries are distributed.
In other words, all the slaves in the same slave group are load-balanced.
Octopus does not load-balance between slaves across multiple slave groups.

An usecase of slave groups is to distribute/completely seperate loads of `SELECT` queries from each part of an application to seperate databases.
For example, consider that you have a Web application consists of the A part which serves Web pages for users, and the B part which serves Web pages for admins.
If you just send heavy `SELECT` queries while serving for admins, they will degrade performance serving Web pages for other users.
This is where slave groups come in.
You can configure seperate slave groups for A and B to make them not affect each other's performance.

You can configure any number of slave groups in `shards.yml`.
To configure two slave groups `slave_group_a` and `slave_group_b` for the `development` environment, you need `shards.yml` like:

```
octopus:
replicated: true
environments:
- development
development:
slave_group_a:
slave1:
# ...
slave2:
# ...
slave_group_b:
slave3:
# ...
slave4:
# ...
```

Then:

* In `Octopus.using(slave_group: :slave_group_a)` blocks, all the `SELECT` queries are distributed between `slave1` and `slave2`
* In `Octopus.using(slave_group: :slave_group_b)` blocks, all the `SELECT` queries are distribuetd between `slave3` and `slave4`
* Without `using`, all the `SELECT` queries go to the development database configured in `database.yml`

To use with sharding, you need `shards.yml` like:

```
octopus:
replicated: true
environments:
- development
development:
shard_1:
slave_group_a:
slave1:
# ...
slave2:
# ...
slave_group_b:
slave3:
# ...
slave4:
# ...
shard_2:
slave_group_a:
slave5:
# ...
```

Then:

* In `Octopus.using(shard: :shard_1, slave_group: :slave_group_a)` blocks, all the `SELECT` queries are distributed between `slave1` and `slave2`
* In `Octopus.using(shard: :shard_1, slave_group: :slave_group_b)` blocks, all the `SELECT` queries are distribuetd between `slave3` and `slave4`
* Without `using`, all the `SELECT` queries go to the development database configured in `database.yml`