gleber/erlcloud — pull request #86 by ransomr (2014-05-21 17:06:13+00:00)
Title: switch to use lhttpc instead of httpc

I intend to merge this in a few days, but since it is a significant change I want to open it up for comments before I do.

This is a significant performance improvement when doing a large number of AWS requests. In the primary scenario I have been testing I see a modest (10%) improvement in the number of requests/second. I also see a significant (3x) improvement in the time it takes to complete each request.

Why lhttpc? I had originally intended to use hackney and implemented that first, but my performance testing showed it to be much slower than httpc. I wrote a benchmark https://github.com/talko/httpcbench which showed lhttpc having the best performance of the options that I considered.

As part of this change I have added the erlcloud_httpc module that is used for all requests. This will make it easy to change http clients in the future, or make the http client configurable, if that is necessary.

Unit tests pass, and we have deployed this change and it seems to be working. But we only use a small subset of the library (DDB, S3, EC2, SES).