tarcieri/http — pull request #39 by tarcieri (2013-09-15 00:09:20+00:00)
Title: HTTP::ResponseBody

This encapsulates HTTP response bodies in their own object which can be streamed incrementally. Seems useful for the Twitter gem there, @sferik ;)

**CURRENTLY A WIP**: This code is broken. I am working to get it working.

It includes a number of other changes for features I don't want to support any more, but can arguably be added back in:

- *Callback system*: No longer needed when we control the request lifecycle. We can just call methods at the appropriate times. Death to callbacks!
- *Automagical request parsing*: We need to figure out a better API for this. Perhaps `HTTP::ResponseBody#parse`? Seems simple enough, and could be completely automatic based on the response's Content-Type

## Inspiration

I would like to get the streaming support in the Http Gem on par with what just shipped in Reel. Reel now has support for HTTP Pipelining as well as streaming both request and response bodies, and I think The Http Gem should as well.

I think The Http Gem can be restructured to work more like Reel, and this branch contains some of that work in progress. Here are the relevant classes in the HTTP Gem, how they map onto the corresponding classes in Reel:

- `HTTP::Client` => [Reel::Connection][reel-connection]: Arguably there should be a HTTP::Connection object too. The HTTP Gem does a bad job of handling connection pooling and keep-alive :( In Reel this class manages the connection lifecycle. The HTTP Gem presently only supports a single request per connection. This class feeds data to the parser so it can be consumed by the other classes
- `HTTP::ResponseParser` => [Reel::RequestParser][reel-request-parser]: Parses the incoming data (a response in the case of the HTTP Gem, or a request in the case of Reel). The parser hands the data off to the active request/response object.
- `HTTP::Response` => [Reel::Request][reel-request]: Serves as the interface between `Reel::Connection` objects and `Reel::RequestBodies`. This is important with HTTP pipelining, as we may be serving a different response fron the one we're currently reading. `HTTP::Response` would serve as the indirection mechanism for the response we're *currently* reading
- `HTTP::ResponseBody` => [Reel::RequestBody][reel-request-body]

[reel-connection]: https://github.com/celluloid/reel/blob/master/lib/reel/connection.rb
[reel-request-parser]: https://github.com/celluloid/reel/blob/master/lib/reel/request_parser.rb
[reel-request]: https://github.com/celluloid/reel/blob/master/lib/reel/request.rb
[reel-request-body]: https://github.com/celluloid/reel/blob/master/lib/reel/request_body.rb

I feel like there's a lot of logical overlap between Reel and The HTTP Gem, even though one's a client and the other's a server ;) Perhaps there's a way to extract it into a single gem to manage the HTTP lifecycle (perhaps this gem!)