[gh-14] Add tests for RobotsTxt filter
Merge pull request #18 from mfairchild365/master

Update tests to reflect recent changes
[gh-14] Fix RobotsTxt test by using file_get_contents

trying to get http://wwww.basepage.com/robots.txt in the test doesn't work
[gh-14] Fix case of Spider_Downloader
Handle base tag regardless of xhtml namespace (without Tidy)
Merge branch 'robots' of https://github.com/ericras/Spider into ericras-robots

Conflicts:
	src/Spider/Filter/RobotsTxt.php
	tests/spider.getCrawlableUris.phpt
	tests/spider.spider.phpt
PHPCS fixes, whitespace, visibility, alignment