basho/basho_bench — pull request #53 by joedevivo (2012-11-02 20:24:30+00:00)
Title: 2i driver: Added enforce_keyrange setting and edge case verification

Ok, first of all, I fixed the configuration of the 2i driver to use a more palatable and consistent syntax for IP addresses.

It used to be this:
```erlang
{pb_ips, [{127,0,0,1}]}.
{http_hosts, ["127.0.0.1"]}.
```
Now, it uses the exact syntax and logic from the `basho_bench_driver_riakc_pb`.

```erlang
{pb_ips, ["127.0.0.1"]}.
{pb_replies, 1}.

{http_ips, ["127.0.0.1"]}.
{http_port, 8098}.

%%It can also be configured for devrel!
{pb_ips, [{"127.0.0.1", 8081},
{"127.0.0.1", 8082},
{"127.0.0.1", 8083},
{"127.0.0.1", 8084}
]}.
{pb_replies, 1}.

{http_ips, [{"127.0.0.1", 8091},
{"127.0.0.1", 8092},
{"127.0.0.1", 8093},
{"127.0.0.1", 8094}
]}.
```

Now that that's out of the way, say hello to my little friend: `enforce_keyrange`. The point of this setting is to throw errors when the keyrange is an unexpected value. This should be used with `sequential_int` key generator, and be set to the same value. This guy is going to come in handy for `riak_test`, allowing us to determine if 2i queries are returning correct values.

The existing 2i driver behavior will be the same if `enforce_keyrange` is not set. That behavior is to output lines whenever there's an unexpected amount of results. With a key distribution, that could be often, but with sequential_int, it's only on edge cases.

Here's an example. I seeded riak with `{key_generator, {sequential_int, 10000}}`. With `enforce_keyrange` undefined, you get the original output... look!
```
Not enough results for query_http: 9993/10002/[<<"9999">>,<<"9995">>,
<<"9998">>,<<"9996">>,
<<"9993">>,<<"9994">>,
<<"9997">>,<<"10000">>]
Not enough results for query_pb: 9995/10004/[[<<"mybucket">>,<<"9995">>],
[<<"mybucket">>,<<"9998">>],
[<<"mybucket">>,<<"9996">>],
[<<"mybucket">>,<<"9999">>],
[<<"mybucket">>,<<"10000">>],
[<<"mybucket">>,<<"9997">>]]
Not enough results for query_http: 9993/10002/[<<"9998">>,<<"9996">>,
<<"9995">>,<<"9999">>,
<<"10000">>,<<"9994">>,
<<"9997">>,<<"9993">>]
Not enough results for query_mr: 9994/10003/[[<<"mybucket">>,<<"9998">>],
[<<"mybucket">>,<<"9999">>],
[<<"mybucket">>,<<"9995">>],
[<<"mybucket">>,<<"9996">>],
[<<"mybucket">>,<<"10000">>],
[<<"mybucket">>,<<"9997">>],
[<<"mybucket">>,<<"9994">>]]
```

Now, I set it up so `{enforce_keyrange, 10000}` and those cease to be output. Yay!

```
I'd but the output here, but there isn't any!
```

You can intentionally break it by setting `{enforce_keyrange, 2*SequentialIntMax}`

```
11:57:30.134 [info] Errors:[{{{query_http,10},{query_http,10}},1},{{{{query_http,10},{query_http,10}},"Not enough results for query_http: 9993/10002/[<<\"9995\">>,<<\"9999\">>,\n <<\"9998\">>,<<\"9996\">>,\n <<\"10000\">>,<<\"9993\">>,\n <<\"9994\">>,<<\"9997\">>]\n"},1}]
11:57:36.649 [error] Worker <0.90.0> exited with {{case_clause,{ok,"Not enough results for query_pb: 9995/10004/[[<<\"mybucket\">>,<<\"10000\">>],\n [<<\"mybucket\">>,<<\"9999\">>],\n [<<\"mybucket\">>,<<\"9996\">>],\n [<<\"mybucket\">>,<<\"9995\">>],\n [<<\"mybucket\">>,<<\"9998\">>],\n [<<\"mybucket\">>,<<\"9997\">>]]\n",{state,<0.91.0>,"127.0.0.1",8091,<<"mybucket">>,20000}}},[{basho_bench_worker,worker_next_op,1,[{file,"src/basho_bench_worker.erl"},{line,231}]},{basho_bench_worker,max_worker_run_loop,1,[{file,"src/basho_bench_worker.erl"},{line,282}]}]}
11:57:36.650 [error] Error in process <0.90.0> with exit value: {{case_clause,{ok,"Not enough results for query_pb: 9995/10004/[[<<\"mybucket\">>,<<\"10000\">>],\n [<<\"mybucket\">>,<<\"9999\">>],\n ...
11:57:36.652 [error] Supervisor basho_bench_sup had child basho_bench_worker_3 started with basho_bench_worker:start_link(basho_bench_worker_3, 3) at <0.89.0> exit with reason normal in context child_terminated
```

Look at that. Bona fide errors. Riak_test will love them. And so will your mom. So make mother happy and +1 this pull request.