Shopify/sarama — pull request #215 by eapache (2014-11-28 19:52:27+00:00)
Title: Consistent partitioning

@wvanbergen this is a fairly non-trivial PR. I would prefer to do the commits one-at-a-time but we're only back to sane default behaviour once all three land. It is still definitely worth reviewing one commit at a time, and I gave each one a substantial stand-alone commit message to help with that.

The goal here is related to better behaviour in cases documented like #210 and #213. Specifically:
- When using a consistent partitioner (aka the `HashPartitioner`), if some partitions go away (all replicas gone) then those messages should fail fast and messages to other partitions should keep flowing.
- When using an inconsistent partitioner (round-robin, random, etc), if some partitions go away then all messages should keep flowing to the available partitions and none should fail.

This PR gets us a lot closer, but still leaves at least one unsolved problem:
- with a consistent partitioner, we still don't fail "fast" (i.e. performance probably degrades quite a bit when a partition goes down entirely)