hook.py: Document reasons argument for accept_url().
Adds wrapper module documentation.
http.py: Adds RichClient.
robotstxt.py: Adds RobotsTxtRichClientSessionMixin.
http.py: Adds RobotsTxtRichClientSession.
engine.py: Fixes typo in docstring.
Moves http.RedirectTracker, http.RichClient to web module.

Removes RobotsTxtRichClientSessionMixin.
Adds docs for web module.
Refactors Engine and Processor interaction to be asynchronous.

Moves request client concern out of Engine and into Engine.
Engine handles stats start and stop.
Moves cookie, robots, redirect concerns out of WebProcessor.
conversation.py: Adds some docstring.
changelog: Adds entry about API Engine and Processor refactor.
http_test.py: Handles possible closed connection in test_buffer_overflow.
web.RichClientSession: Joins base URL for next_location. Fixes unit tests.
processor.py: Fixes items not processed if single redirect request.
http.Connection: Fixes url_info not set on response.
Merge branch 'topic/engine_refactor' into develop
doc/api.rst: Notes about usage of Tornado coroutines.
Tweaks Fusil fuzz tester to be more aggressive, random content-types.
fuzz_fusil/runner.py: Supports writing gzip encoding.
Implements --no-warc-digests.