Medium/draccus — pull request #4 by x-ma (2014-09-06 07:14:54+00:00)
Title: Allow multiple pending sqs.receiveMessage() calls

Hello @dpup, @dudeche, @giannic,

Please review the following commits I made in branch 'xiao-usain-bolt'.

646f9372ba05a4dba7d3e06fecdce68c1bf7cd39 (2014-09-06 00:05:23 -0700)
```
Allow multiple pending sqs.receiveMessage() calls

The current code waits until one sqs.receiveMessage() finishes before
making another sqs.receiveMessage() call. This is inefficient, given
that Draccus is almost completely I/O bound.

This changes allows multiple sqs.receiveMessage() calls to be made
concurrently. However, each of them will act individually. The maximum
number of calls can be capped by a new cli option - max_concurrent_receivers.

Upon errors or empty messages, Draccus will still back off with longer
and longer timeout. When things go back normal, multiple receiving calls
will start active again. This respects the original behavior.

One pitfall of this change is that we are missing control on back
pressure. More specifically, if the store classes cannot write data
out fast enough, and there are a lot of messages in the queue, SqlSink
will quickly fill up memory. So be cautious about the concurrent
value.

Ideally, the sinking performance is proportional to the number of
concurrent receivers, until the network is saturated or SQS's output
throughput is saturated. A limited experiement agrees with it.

1 receiver to sink 300 messages - 14.012 sec
2 receivers to sink 300 messages - 6.301 sec
4 receivers to sink 300 messages - 3.94 sec
```

R=@dpup
R=@dudeche
R=@giannic