Add support for an external spidering skip list
Merge pull request #150 from pmuir/user_specified_skips

Add support for an external spidering skip list