basho/yokozuna — pull request #368 by rzezeski (2014-04-23 16:42:23+00:00)
Title: Add JSON Object Value Generator

Add `json_obj_valgen` to the Yokozuna BB driver. I used this to test
indexing "large" JSON objects while investigating #358. The config
line to use this value generator looks like so:

```
{value_generator, {function, yz_driver, json_obj_valgen,
[]}}.
```

This will generate a static JSON object (i.e. generated once at start
up) which contains `` integer fields. All workers will
index the identical object. Here is an example config from one of my
testing runs:

```
{mode, max}.
{concurrent, 32}.
{driver, yz_driver}.
{code_paths, ["/root/work/yz-bb/misc/bench"]}.
{secure, false}.
{bucket, {<<"data">>, <<"largeobj">>}}.
{index, <<"largeobj">>}.
{pb_conns, [{"10.0.1.201", 8087}]}.
{http_conns, []}.
{duration, infinity}.
{key_generator, {to_binstr, "~B", {partitioned_sequential_int, 0,
1000}}}.
{value_generator, {function, yz_driver, json_obj_valgen, [4000]}}.
{operations, [{load_pb, 1}]}.
```

This uses 32 workers to write the same JSON object containing 4000
integer fields over the key space 0 to 999.

NOTE: I purposely hard coded the timeout to 90s for the purposes of my
testing. Yes, ideally this would be configurable but this patch is
about minimal amount of work to achieve the task at hand. Please keep
this in mind while reviewing. This is a tool, not production code.