Moved most of what was in the views module to a new Crawler class, mostly to make the code more available for testing.
Added a few tests for new Crawler class.