Repository: Parsely/pykafka

Issue #701: balancedconsumer will get a PartitionOwnedError
Created: 2017-07-15T09:04:56Z
start a kafka server and not include any data, create a topic with 3(any number greater than one)partition.
create a balancedconsumer, set consumer_timeout_ms = -1.
start the balancedconsumer client, and then start it again, here are two client named clientA, clientB.
after rebalance_max_retries*rebalance_backoff_ms ms, clientB will get a PartitionOwnedError.
i have debug the code, find the reason is consume() function will always acquire the _rebalancing_lock and not to release it, so _rebalance() will block at with self._rebalancing_lock.

it occur in both master and dev version.

Duplicate issue #619: PartitionOwnedError / ConsumerStoppedException upon Consumer startup
Created: 2016-12-01T06:24:14Z
PyKafka Version: Have had this issue with numerous versions. Presently, 2.5.0
Kafka Version: 0.9.0
Partitions: 15
Consumer
This happens in multiple projects on multiple data centers, all with different Kafka clusters (albeit they are the same version, and all were installed and managed by the Ambari tool).
When I start up a consumer, unless I slowly turn them on one by one, some or all will get caught in a perpetual restart loop where it is unable to start subscribing. It sees, and usually bounces back and forth between either PartitionOwnedError or ConsumerStoppedException. It shuts down and is then restarted by my Scheduler or docker-compose, where it errors again.
Example as of right now:

Use a base CentOS Docker image I created with PyKafka and PIP installed.
Using Nomad, attempt to spin up 14 containers. (Never gets above 12)

print 'Start file'
from pykafka import KafkaClient
print 'Create client'
client = KafkaClient(hosts="my.ip.here:6667")
print 'Get topic'
topic = client.topics['stored_logs_json']
print 'Create consumer'
balanced_consumer = topic.get_balanced_consumer(
    consumer_group='test_group',
    zookeeper_connect='my.ip.here:2181'
)
print 'Loop through messages.'
for message in balanced_consumer:
    print message
    balanced_consumer.commit_offsets()
Partition Owned ---
Start file
Create client
Get topic
Create consumer
Loop through messages.
Traceback (most recent call last):
  File "test.py", line 14, in <module>
    for message in balanced_consumer:
  File "/usr/lib/python2.7/site-packages/pykafka/balancedconsumer.py", line 745, in __iter__
    message = self.consume(block=True)
  File "/usr/lib/python2.7/site-packages/pykafka/balancedconsumer.py", line 726, in consume
    self._raise_worker_exceptions()
  File "/usr/lib/python2.7/site-packages/pykafka/balancedconsumer.py", line 271, in _raise_worker_exceptions
    raise ex
pykafka.exceptions.PartitionOwnedError

After looking at a few logs, I haven't seen the ConsumerStoppedException with this example. It's possible that is caused by some convolution in my code, and could simply be solved with a stop(), start().
But here it is anyway.

Begin message process loop.
Error subscribing to pubsub. (<class 'pykafka.exceptions.ConsumerStoppedException'>)
Critical exception!!! Exiting: (<class 'pykafka.exceptions.ConsumerStoppedException'>)
Traceback (most recent call last):
  File "daemon.py", line 25, in <module>
    main()
  File "daemon.py", line 21, in main
    run()
  File "/usr/local/my-app-path/bin/../lib/my-app/some_path/main.py", line 35, in run
    process_messages(handler)
  File "/usr/local/my-app-path/bin/../lib/my-app/some_path/application.py", line 14, in process_messages
    for message in handler.subscribe_to_logstream():
  File "/usr/local/my-app-path/bin/../lib/my-app/some_path/handler.py", line 26, in subscribe_to_logstream
    for message in self.pubsub_consumer.subscribe():
  File "/usr/local/my-app-path/bin/../lib/my-app/some_path/adapter.py", line 77, in subscribe
    for message in self.consumer:
  File "/usr/lib/python2.7/site-packages/pykafka/balancedconsumer.py", line 745, in __iter__
    message = self.consume(block=True)
  File "/usr/lib/python2.7/site-packages/pykafka/balancedconsumer.py", line 726, in consume
    self._raise_worker_exceptions()
  File "/usr/lib/python2.7/site-packages/pykafka/balancedconsumer.py", line 271, in _raise_worker_exceptions
    raise ex
pykafka.exceptions.ConsumerStoppedException

Closing comment (2017-09-19T12:06:44Z): @emmett9001 Could this also be related to #701?