Support for consumer-aware OffsetFetchRequest and OffsetCommitRequest

The OffsetFetchRequest and OffsetCommitRequest requests to the broker, which are used for broker storage of consumer offsets (to replace Zookeeper storage of offsets) require a ConsumerMetadataRequest. Similar to the topic metadata request, this returns the coordinator for a given consumer group, which is the broker that you must send the offset fetch and commit requests to. If you do not do this, you will more often than not get an error code 16, which is NotCoordinatorForConsumer.

I've added support for the ConsumerMetadataRequest, as well as a _send_consumer_aware_request method to support getting the coordinator and then sending a request to that broker. I have updated the OffsetFetchRequest and OffsetCommitRequest to use this method instead of the  _send_broker_aware_request method. I've also added error handling for the metadata request (in the case that it fails).
Yeah, it’s really old and needs to be rebased. I haven’t had a chance to do it yet, however.

-Todd

From: Mark Roberts <notifications@github.com<mailto:notifications@github.com>>
Reply-To: mumrah/kafka-python <reply@reply.github.com<mailto:reply@reply.github.com>>
Date: Monday, December 8, 2014 at 9:45 AM
To: mumrah/kafka-python <kafka-python@noreply.github.com<mailto:kafka-python@noreply.github.com>>
Cc: Todd Palino <tpalino@linkedin.com<mailto:tpalino@linkedin.com>>
Subject: Re: [kafka-python] Support for consumer-aware OffsetFetchRequest and OffsetCommitRequest (#164)


In kafka/common.py:

> @@ -63,6 +63,9 @@
>      10 : 'MESSAGE_SIZE_TOO_LARGE',
>      11 : 'STALE_CONTROLLER_EPOCH',
>      12 : 'OFFSET_METADATA_TOO_LARGE',
> +    14 : 'OFFSETS_LOAD_IN_PROGRESS',


This pull request is pretty old, and was made against the dev branch. It might have changed names since then.

—
Reply to this email directly or view it on GitHub<https://github.com/mumrah/kafka-python/pull/164/files#r21470157>.