Use logger instead of printing to stdout.
Construct hyperlink graph and control spider with traversal algorithms.

* networkx graph:
- in addition to dumping URLs in a queue, calls to queue_append() also
construct a directed graph
- every URL is represented as a node in that graph
- if a source URL is provided, a directed edge is inserted
- ExtractorActionDecorator passes graph data for a given URL on calls
to WARP.extract() for use by processors

* class DepthFirstTraversal: provides a depth first traversal of the
graph structure as it is being constructed

* append_url_extr_re(): convenience method for building and attaching
a URL extractor based on regex patterns on anchor's href attribute
Supply ALL processors with input_dict and extend it with 'url' item.

- now parsers ('extractors') receive input_dict as well
- fix bugs in auth and warp
Encapsulate replication-related functionality in distinct module.

All code related to reading/writing from/to the filesystem or DB
as well as the repository's organization logic, is now encapsulated
in the repo module which is exposed to WARP through the methods
has_get(), get(), store() and store_data().
Add repo config options PROC_AS_LEAF and UNIFIED_REPO.
Add data attribute to spider, refactor event bus.

* spider.data: dictionary that can be accessed by action objects during
  a crawl as a medium for data exchange and reporting.

* repo: clarified interface
- more explicit methods: has_response(), has_data(), get_response(),
  get_response_info(), get_data(), store_response(), store_data()
- static support functions: response_repl_loc(), data_repl_loc(),
  repl_loc()

* event bus: add cfg options ADD_RESPONSE_EXT, CREATE_RESPONSE_VERSION

* proc: new module for functionality specific to processing
- parse.py: helper functions for parsing