Livefyre/kafka-python — pull request #6 by jimmyislive (2014-10-10 20:56:13+00:00)
Title: MemoryError check

On tests (after setting max size to 15 bytes):

WARNING:kafka:Fetch size too small, increase to 2 (2x) and retry
WARNING:kafka:Fetch size too small, increase to 4 (2x) and retry
WARNING:kafka:Fetch size too small, increase to 8 (2x) and retry
ERROR:kafka:Message size exceeded maximum allowed of 15
ERROR:kafka:Current buffer_size is: 16
ERROR:kafka:topic: activity.stream, partition: 0, offset:24
Exception in thread Thread-2:
Traceback (most recent call last):
File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "/Users/jjohn/dev/lfdemux/env/lib/python2.7/site-packages/kafka/consumer.py", line 436, in _fetch_loop
self._fetch()
File "/Users/jjohn/dev/lfdemux/env/lib/python2.7/site-packages/kafka/consumer.py", line 489, in _fetch
raise MessageTooLargeError
MessageTooLargeError