Can now quickly run the spider and see that something is in fact happening...
Working on readable implementation which uses form requests as start requests (rather than trying to figure out exact format of URL)