First draft (3rd edition), ready for some review

Vnode replies always go via reply(), and reply() always uses unreliable
messaging.  (As opposed to the usual (and more reliable) send-and-pray
messaging.)

During handoff, all forwarding requests use unreliable vnode master
commands to avoid net_kernel blocking interference.

My testing procedure:

1. Use "make stagedevrel" on box A for nodes 1-4.  Configure for
multi-box use (i.e. change all 127.0.0.1 appearances in vm.args and
app.config as appropriate).

2. Use "make stage" on box B for node 5.  Configure for multi-box
use.

3. Start all nodes, join them all together, and commit.

4. Run basho_bench on box A, using this config:

    {mode, {rate, 50}}.
    {report_interval, 1}.
    {duration, 30}.
    {concurrent, 50}.
    
    {driver, basho_bench_driver_riakc_pb}.
    {riakc_pb_ips, [
                    {{127,0,0,1}, 10017}, %% {Ip, Port}
                    {{127,0,0,1}, [10027, 10037, 10047]} %% {Ip, Ports}
                   ]}.
    {riakc_pb_replies, 1}.
    {operations, [{get, 1}, {update, 1}]}.
    {pb_timeout_general, 5000}.

    {key_generator, {int_to_bin, {uniform_int, 990000}}}.
    {value_generator, {fixed_bin, 10}}.
    
5. On box B:

    sh -c 'for i in 1 2 3 4 5; do date; sleep 1 ; ( date ; ifdown eth0 ; sleep 100 ; ifup eth0 ; date ) > /tmp/slf 2>& 1 ; cat /tmp/slf; sleep 70; done'

6. Check the results via:

    awk -F, '{if ($5 == 0) { print "ok"; } else { print "error" } }' tests/current/summary.csv | uniq -c | less

7. There's a pretty darn obvious qualitative way to view the results
of the pre-patch and post-patch behavior.  The sampling rate used by
the basho_bench config above, i.e., `{report_interval, 1}`, creates
some noise in the step 6 results.  So, the best advice that I can give
for looking at the quantative data is to look for ~100 seconds of `ok`
stability after node 1's network interface has been shut off.

7a. Good results from step 6:

      1 error
     19 ok
      3 error
      2 ok
      4 error
      1 ok
     57 error (definitely down ifdown iteration  1)
      2 ok
      1 error
    105 ok (about ~100 seconds of goodness after down ifdown iteration  1)
      4 error
      2 ok
      4 error
      1 ok
     61 error (definitely down ifdown iteration  2)
      1 ok
      2 error
    101 ok (about ~100 seconds of goodness after down ifdown iteration  1)
      3 error
      2 ok
     55 error (definitely down ifdown iteration  3)
      2 ok
      2 error
    112 ok (about ~100 seconds of goodness after down ifdown iteration  1)
      4 error
      1 ok
     58 error (definitely down ifdown iteration  4)
      1 ok
      3 error
    109 ok (about ~100 seconds of goodness after down ifdown iteration  1)
      3 error
      2 ok
      4 error
      1 ok
      4 error
      1 ok
     52 error (definitely down ifdown iteration  5)
      1 ok
      4 error
    188 ok  (about ~100 seconds of goodness after down ifdown iteration  1)
            (end of interface flapping, ignore the quantity here)

7b. Bad results from step 6:

      1 error
     24 ok
      4 error (probably down ifdown iteration  1)
      1 ok
     10 error (definitely down ifdown iteration  1)
      1 ok
     46 error
      1 ok
      7 error
    106 ok (about ~100 seconds of goodness after down ifdown iteration  1)
      3 error
      2 ok
      4 error
      1 ok
     61 error (definitely down ifdown iteration 2 ... but this is
               where it becomes much less obvious where ifdown and
               ifup events actually happened)
      2 ok
      5 error
      1 ok
     21 error
     75 ok
      4 error
      1 ok
      5 error
      1 ok
      4 error
      1 ok
     46 error (down ifdown iteration  3?)
      1 ok
     38 error
     75 ok
      3 error
      2 ok
      4 error
      1 ok
      5 error
      1 ok
     84 error (Huge problem ... Step 7a's results never had an error
               rate larger than 61 samples in a row.)
     75 ok
      3 error
      2 ok
      5 error
      1 ok
     57 error (probably down here ifdown iteration  5)
      1 ok
     32 error (if "57 error" was ifdown, then node 1's interface is
               now up, and this error rate is too high.)
    974 ok (end of interface flapping, ignore the quantity here)