removed unnecessary code and tests and simplified things
renamed main file to crawler
removed CLI stuff for now
test is a little hacky but crawls good enough for now