Add the ability to externalize the enforcement of the robots.txt Crawl-delay directive.
Remove Gemfile.lock.
Merge pull request #6 from NearbyFYI/ignore_crawl_delay

Enable externalization of Crawl-delay directive enforcement from allowed?() and disallowed?().
Merge back the previous change to the source file.
Change the :ignore_crawl_delay option to a handler option :crawl_delay.
'lib' should be in test.libs by default.
Add .travis.yml.
Add rake as development dependency.  Let's see if it satisfies Travis-CI.
Tweak Travis-CI settings.
Allow failures with rbx.
Bump VERSION to 0.1.0.