basho/yokozuna — pull request #230 by rzezeski (2013-11-12 01:14:33+00:00)
Title: Integrate with bucket types

Overall
-------

Bucket types are the future of namespacing and property creation in
Riak. They allow efficient storage of "bucket properties" outside of
the Ring and 2-level namespacing of `Type` and `Name`.

Essentially the bucket type can now be either a lone `binary()`
(legacy) or a 2-tuple of `{Type :: binary(), Name ::
binary()}`. Internally, when the legacy version is encountered it is
considered to live under the `default` bucket type. For example the
bucket `<<"my_bucket">>` would become `{<<"default">>, <<"my_bucket">>}`.

Up until this point Yokozuna has used the bucket property `yz_index`
to determine where to index data. This commit changes that in some
ways. Legacy users will have existing data in buckets. Those buckets,
in 2.0, will be considered to live under the default type as described
above. For legacy buckets (the default type) Yokozuna will NOT respect
the `yz_index` property. Rather it will act like Riak Search and use
an index named the same as the bucket AS LONG AS the `search` property
is set to true. Once users upgrade to 2.0 they should start opting
into non-default bucket types since it is more efficient and newer
features require the use of non-default type. For these types of
buckets Yokozuna will still use the `yz_index` property. This property
will typically be set at the type level but can also be overridden per
name under a type. Yokozuna doesn't care. If that `{Type, Name}` has a
`yz_index` property then it will be indexed.

In summary:

* Legacy buckets (default type) will act like Riak Search. The index
used must have same name as bucket and `search` property must be
true. This is used to users migrating from Riak Search.

* All new users MUST use new style buckets made of Type + Name. In
most cases the `yz_index` property will be set on the type and thus
inherited by all names under it (many buckets to one index). The
index DOES NOT have to have the same name.

Handoff
-------

Another important change revolves around handoff. Since Yokozuna
leeches off the KV vnode it doesn't have control over handoff like it
would if it were a true vnode. When a node joins KV can start shipping
data before the bucket type data has been shipped over. In that case
there will be no `yz_index` property and indexes will be missing. AAE
would eventually catch this but it is poor form that node join would
cause a degradation in harvest, especially in a quiescent cluster.

To fix this Yokozuna needs more control over the lifecycle of the KV
vnode. Yokozuna needs to hook into the `handoff_starting` stage and
verify that the bucket types data is shipped before data handoff
begins. This is accomplished by adding the `yz_kv:should_handoff` hook
which is hard-coded in the KV vnode for now.

This is important for removing the hack around index creation as
well. Currently Yokozuna has a pretty horrible busy-wait hack in its
index hook to make sure indexes are created on joining nodes before
doing the first write of a handoff. This busy-wait blocks the KV vnode
and is dangerous for vnode latency. In a future commit this busy-wait
will be replaced with a check in this new handoff hook.

Removal of Automatic AAE Tree Clearing
--------------------------------------

Remove all functionality around automatic clearing of trees when
adding or removing the `yz_index` property on a bucket with data. This
was referred to as `sync_data` in the `yz_events` module. Also called
"flags" harking back to when Yokozuna had a one-to-one bucket-to-index
mapping.

The original intention was that adding an index to a bucket with data
should clear the AAE trees so that exchanges would start repairing
missing indexes. If setting the index property to the tombstone value
(removal) then a) data for that bucket should be purged from the index
and b) AAE trees should be cleared. After much thought I think this
implicit behavior hurts more than helps.

Actions like clearing all AAE trees can be very expensive. It will not
be obvious to all users that adding or changing `yz_index` could cause
expensive operations to occur. For example, clearing the AAE trees for
a database with billions or trillions of objects will be expensive to
rebuild. Rather than relying on AAE a more direct operation could be
offered that allows the user to re-index a bucket or subset of
data. When removing an index it makes more sense to let the user
delete the index entirely rather than do an implicit delete-by-query
which is doing a bunch of extra work for a index that is going to be
deleted anyways.

Misc Changes
------------

* Update all tests to work with bucket types.

* Update Basho Bench driver to work with bucket types.

* Make map-reduce extraction more efficient. This is the ugly hack
found in `yokozuna:positions`.