spider crawler overrides existing spider txt to empty hash
Merge pull request #185 from afkmaster/spider-crawl-overriding-txt

spider crawler overrides existing spider txt to empty hash