Rewriting crawling rules -- it'll now process all subpages.
Merge pull request #1 from rdegges/master

Rewriting crawling rules -- it'll now process all subpages.