Refactoring scraper

Tidying up the scraper because the code there is pretty hideous and also
adding some unittests. TTD FTW.