edenhill/librdkafka — pull request #103 by bartw72 (2014-04-20 10:15:58+00:00)
Title: Add support for infinite timeout on produced messages

I've created a patch to debian 0.8.3 tree to allow also infinite timeouts on produced messages. I like to be able to choose if messages expire at all, or not. Currently the expiration timing is set between 1ms and 15min. With setting the time to 0ms, they will not expire (any time soon).

On testing I've discovered also that the scan on expired session is not triggered even when the queue is full but the queue contains loads of expired messages. The messages will be rejected at once in passing them into rd_kafka_produce. I am not sure how this was designed, but I think if the scan is triggered before the max amounts are hit (in bytes or message count), you might not have to reject messages in rd_kafka_produce but you can free some space in advance by dropping already expired messages. After a while, this mechanism kicks in and starts working as I would expected (freeing and adding concurrently).