Merge pull request #3 from mfairchild365/master

html5 support
Merge pull request #4 from mfairchild365/master

Allow max_depth to be configurable and add a page_limit
Merge pull request #5 from mfairchild365/master

Refactor and add tests
Merge pull request #6 from mfairchild365/master

Base Tag Support and Tests
Merge pull request #7 from mfairchild365/master

Options, options and more options
Merge pull request #8 from mfairchild365/master

Correctly get the base path of uris with a query string
Merge pull request #9 from mfairchild365/master

improve the tidy parsing
Merge pull request #10 from mfairchild365/master

update and add tests to spider::absolutePath()
Whitespace
Add option to respect robots.txt disallow directives
Call robots_allowed with $this-> instead of self::
Merge pull request #12 from ericras/robots

Respect robots.txt
Merge remote-tracking branch 'upstream/master'

Conflicts:
	src/Spider.php