Repository: dpkp/kafka-python

Issue #674: Single-thread SimpleProducer says "KafkaError: this should not happen - are you threading?"
Created: 2016-04-29T11:05:29Z
The stack trace is
.tox/py27/lib/python2.7/site-packages/frontera/tests/test_message_bus.py:62: in sw_activity
    self.sw_us_p.send(None, 'message' + str(0) + "," + str(c))
.tox/py27/lib/python2.7/site-packages/frontera/contrib/messagebus/kafkabus.py:83: in send
    self._producer.send_messages(self._topic, *messages)
.tox/py27/lib/python2.7/site-packages/kafka/producer/simple.py:50: in send_messages
    topic, partition, *msg
.tox/py27/lib/python2.7/site-packages/kafka/producer/base.py:379: in send_messages
    return self._send_messages(topic, partition, *msg)
.tox/py27/lib/python2.7/site-packages/kafka/producer/base.py:422: in _send_messages
    fail_on_error=self.sync_fail_on_error
.tox/py27/lib/python2.7/site-packages/kafka/client.py:641: in send_produce_request
    resps = self._send_broker_aware_request(payloads, encoder, decoder)
.tox/py27/lib/python2.7/site-packages/kafka/client.py:283: in _send_broker_aware_request
    conn.recv()
...
E               KafkaError: this should not happen - are you threading?

Kafka-python version 1.1.1, kafka version is 0.8.2.2. See the full log and source code here

Duplicate issue #707: SimpleConsumer throws error when committing offsets
Created: 2016-05-26T14:32:41Z
2016-05-26 16:29:04,777 DEBUG    kafka.consumer.simple calling _get_message block=True timeout=0.85804104805
2016-05-26 16:29:04,777 DEBUG    kafka.consumer.simple internal queue empty, fetching more messages
2016-05-26 16:29:04,777 DEBUG    kafka.conn      <BrokerConnection host=localhost/::1 port=9092> Request 50: FetchRequest_v0(replica_id=-1, max_wait_time=858, min_bytes=1, topics=[(topic='fro
ntier-done', partitions=[(partition=0, offset=12224, max_bytes=1048576)])])
2016-05-26 16:29:04,825 DEBUG    kafka.consumer  Committing new offsets for frontier-done, partitions [0]
2016-05-26 16:29:04,825 DEBUG    kafka.consumer  Commit offset 12221 in SimpleConsumer: group=general, topic=frontier-done, partition=0
2016-05-26 16:29:04,826 DEBUG    kafka.conn      <BrokerConnection host=localhost/::1 port=9092> Request 51: OffsetCommitRequest_v0(consumer_group='general', topics=[(topic='frontier-done', p
artitions=[(partition=0, offset=12221, metadata=None)])])
2016-05-26 16:29:04,857 ERROR    messagebus.kafka this should not happen - are you threading?
Traceback (most recent call last):
  File "/Users/sibiryakov/src/sh/crawl-frontier/frontera/contrib/messagebus/kafkabus.py", line 59, in get_messages
    timeout=timeout):
  File "/Users/sibiryakov/src/sh/kafka-python/kafka/consumer/simple.py", line 285, in get_messages
    update_offset=False)
  File "/Users/sibiryakov/src/sh/kafka-python/kafka/consumer/simple.py", line 320, in _get_message
    self._fetch()
  File "/Users/sibiryakov/src/sh/kafka-python/kafka/consumer/simple.py", line 379, in _fetch
    fail_on_error=False
  File "/Users/sibiryakov/src/sh/kafka-python/kafka/client.py", line 662, in send_fetch_request
    KafkaProtocol.decode_fetch_response)
  File "/Users/sibiryakov/src/sh/kafka-python/kafka/client.py", line 283, in _send_broker_aware_request
    conn.recv()
  File "/Users/sibiryakov/src/sh/kafka-python/kafka/conn.py", line 436, in recv
    raise Errors.KafkaError('this should not happen - are you threading?')
KafkaError: this should not happen - are you threading?
2016-05-26 16:29:04,857 ERROR    kafka.conn      <BrokerConnection host=localhost/::1 port=9092>: socket disconnected
2016-05-26 16:29:04,859 DEBUG    messagebus.kafka <BrokerConnection host=localhost/::1 port=9092>:4499316496
2016-05-26 16:29:04,859 ERROR    kafka.consumer  FailedPayloadsError saving offsets:

Closing comment (2016-05-26T16:42:41Z): looks like the same as / similar to #674 ?