Basic cache coming into being. Still rough at this stage.
Midway through cache development
Merge pull request #1 from cgiffard/development

Development Merge
Updated filesystem backend - bug if content-type was not set
Updated filesystem backend - bug if content-type was not set
Merge branch 'master' into development
Merge branch 'master' of github.com:cgiffard/node-simplecrawler
Proxy support, and a cache bug fixed.
Enabled domain whitelisting. Fixed a bug with subdomain comparison.
Made performance improvements to queue.
Updated readme to reflect recent changes in queue and crawler architecture.
Updated package.json.
Updated package.json.
Update README.markdown
Fixed a couple of boneheaded mistakes in package. Cache FS backend now handles really long filenames.
Merge branch 'master' of github.com:cgiffard/node-simplecrawler
Renamed main file for better compatibility.
Added homepage and keywords to package.json.
Added in basic auth support
Merge pull request #2 from ncrohn/master

Added basic auth support