basho/riak_kv — pull request #258 by rustyio (2011-12-09 19:53:50+00:00)
Title: Update multi_backend to work with 2i (AZ943)

Related to https://github.com/basho/riak_kv/pull/227

## Objective

Currently, 2i doesn't play well with `riak_kv_multi_backend`. 2i requires that the backend support an `indexes` capability. Multi-backend doesn't correctly thread capabilities of its child backends through to `riak_kv_vnode`, so it never exposes the `indexes` capability.

Some history: when capabilities were first created, they were erroneously scoped at the backend level. Multi-backend (and perhaps other backends in the future?) need to scope capabilities at a bucket level.

This pull request re-scopes the `api_version/N` function to have BOTH a backend-level AND a bucket-level version, giving `riak_kv_vnode` the ability to query (and backends the ability to expose) bucket level capabilities.

Currently, backends expose something like this:

```erlang
api_version() ->
{?API_VERSION, ?CAPABILITIES}.
```

With this pull request, they now expose something like this:

```erlang
api_version() ->
{ok, ?API_VERSION}.

%% Backend Capabilities
capabilities(_ModState) ->
{ok, ?CAPABILITIES}.

%% Bucket-Level Capabilities
capabilities(_Bucket, _ModState) ->
{ok, ?CAPABILITIES}.
```

This pull request also updates `riak_kv_vnode` and `riak_kv_multi_backend` to enable 2i queries through the multi_backend on child backends that support it.

## Testing

### Test Setup

Configure the system with multi-backend:

Put the following into the `riak_kv` section of `app.config`:

```erlang
{storage_backend, riak_kv_multi_backend},
{multi_backend, [
{bitcask, riak_kv_bitcask_backend, [
]},
{level, riak_kv_eleveldb_backend, [
]},
{memory, riak_kv_memory_backend, [
]}
]},
{multi_backend_default, bitcask}
```

Then configure `bucket1`, `bucket2`, and `bucket3` to use the bitcask, level, and memory backends as follows:

```erlang
riak_core_bucket:set_bucket(<<"bucket1">>, [{backend, bitcask}]).
riak_core_bucket:set_bucket(<<"bucket2">>, [{backend, level}]).
riak_core_bucket:set_bucket(<<"bucket3">>, [{backend, memory}]).
```

Finally, start the system and add some objects:

```bash
# This script loads objects called key1, key2, key3, and key4
# into bucket1, bucket2, bucket3, and bucket4.

curl -v -X PUT -d 'data1' -H "x-riak-index-field1_bin: val1" \
http://127.0.0.1:8098/riak/bucket1/key1

curl -v -X PUT -d 'data2' -H "x-riak-index-field1_bin: val2" \
http://127.0.0.1:8098/riak/bucket1/key2

curl -v -X PUT -d 'data3' -H "x-riak-index-field1_bin: val3" \
http://127.0.0.1:8098/riak/bucket1/key3

curl -v -X PUT -d 'data4' -H "x-riak-index-field1_bin: val4" \
http://127.0.0.1:8098/riak/bucket1/key4

curl -v -X PUT -d 'data1' -H "x-riak-index-field1_bin: val1" \
http://127.0.0.1:8098/riak/bucket2/key1

curl -v -X PUT -d 'data2' -H "x-riak-index-field1_bin: val2" \
http://127.0.0.1:8098/riak/bucket2/key2

curl -v -X PUT -d 'data3' -H "x-riak-index-field1_bin: val3" \
http://127.0.0.1:8098/riak/bucket2/key3

curl -v -X PUT -d 'data4' -H "x-riak-index-field1_bin: val4" \
http://127.0.0.1:8098/riak/bucket2/key4

curl -v -X PUT -d 'data1' -H "x-riak-index-field1_bin: val1" \
http://127.0.0.1:8098/riak/bucket3/key1

curl -v -X PUT -d 'data2' -H "x-riak-index-field1_bin: val2" \
http://127.0.0.1:8098/riak/bucket3/key2

curl -v -X PUT -d 'data3' -H "x-riak-index-field1_bin: val3" \
http://127.0.0.1:8098/riak/bucket3/key3

curl -v -X PUT -d 'data4' -H "x-riak-index-field1_bin: val4" \
http://127.0.0.1:8098/riak/bucket3/key4

curl -v -X PUT -d 'data1' -H "x-riak-index-field1_bin: val1" \
http://127.0.0.1:8098/riak/bucket4/key1

curl -v -X PUT -d 'data2' -H "x-riak-index-field1_bin: val2" \
http://127.0.0.1:8098/riak/bucket4/key2

curl -v -X PUT -d 'data3' -H "x-riak-index-field1_bin: val3" \
http://127.0.0.1:8098/riak/bucket4/key3

curl -v -X PUT -d 'data4' -H "x-riak-index-field1_bin: val4" \
http://127.0.0.1:8098/riak/bucket4/key4
```

### Test: List Buckets

Run:

```bash
curl http://localhost:8098/riak\?buckets\=true
```

Verify that the command returns bucket1, bucket2, bucket3, and bucket4.

### Test: List Keys

Run:

```bash
curl http://localhost:8098/riak/bucket1?props=false&keys=true
curl http://localhost:8098/riak/bucket2?props=false&keys=true
curl http://localhost:8098/riak/bucket3?props=false&keys=true
curl http://localhost:8098/riak/bucket4?props=false&keys=true
```

Verify that each command returns key1, key2, key3, key4.

### Test: Streaming List Keys

Run:

```bash
curl http://localhost:8098/riak/bucket1?props=false&keys=stream
curl http://localhost:8098/riak/bucket2?props=false&keys=stream
curl http://localhost:8098/riak/bucket3?props=false&keys=stream
curl http://localhost:8098/riak/bucket4?props=false&keys=stream
```

Verify that each command returns key1, key2, key3, key4.

### Test: Legacy Keylisting

Run:

```erlang
{ok, C} = riak:local_client().
C:list_keys(<<"bucket1">>).
C:list_keys(<<"bucket2">>).
C:list_keys(<<"bucket3">>).
C:list_keys(<<"bucket4">>).
```

Verify that each command returns key1, key2, key3, key4.

### Test: Indexing

Run:

```bash
curl http://localhost:8098/buckets/bucket1/index/field1_bin/val2/val3
curl http://localhost:8098/buckets/bucket2/index/field1_bin/val2/val3
curl http://localhost:8098/buckets/bucket3/index/field1_bin/val2/val3
curl http://localhost:8098/buckets/bucket4/index/field1_bin/val2/val3
```

Verify that an index query on bucket2 returns key2 and key3.
Verify that an index query on bucket1, bucket3, and bucket4 throws an error.

### Test: Backups

Back up and restore data:

```bash
bin/riak-admin backup riak@127.0.0.1 riak riak_backup all
bin/riak stop
rm -rf data/eleveldb
rm -rf data/bitcask
bin/riak start
bin/riak-admin restore riak@127.0.0.1 riak riak_backup
```

Re-run all of the previous tests.

### Test: Synchronous Folds

By default, the system is set to use asynchronous folding.

Add (or change) the following to the `riak_kv` section of `app.config` to disable asynchronous folding, and re-run all the previous tests:

```erlang
{async_folds, false}
```

### Test: Last Write Wins

Add (or change) the following to the `riak_kv` section of `app.config`:

```erlang
{vnode_vclocks, false}
```

Then, run the following to set last write wins on all buckets.

```erlang
riak_core_bucket:set_bucket(<<"bucket1">>, [{last_write_wins, true}]).
riak_core_bucket:set_bucket(<<"bucket2">>, [{last_write_wins, true}]).
riak_core_bucket:set_bucket(<<"bucket3">>, [{last_write_wins, true}]).
```

Now run the following in the python client:

```python
import riak

# Create an object...
client = riak.RiakClient()
bucket = client.bucket('bucket1')
obj = bucket.get('key1')
obj.set_data('data1')
obj.store(None, None, False)
print bucket.get('key1').get_data()

# Fetch two copies of the object...
oldClient = riak.RiakClient()
oldBucket = oldClient.bucket('bucket1')
oldObj = oldBucket.get('key1')

newClient = riak.RiakClient()
newBucket = newClient.bucket('bucket1')
newObj = newBucket.get('key1')

# Write both objects, but only return the body on the
# new one. This causes oldObj to have an
# outdated vector clock.
oldObj.store(None, None, False)
newObj.store()

# Update the data, creating a new vector clock...
newObj.set_data("data1new")
newObj.store()
print bucket.get('key1').get_data()

# Now, try to write the old object.
# If lww = true, this should overwrite existing value.
# Otherwise, the write should be ignored.
oldObj.store(None, None, False)

# Should print 'data1' if lww = true,
# or 'data1new' if lww = false
print bucket.get('key1').get_data()
```