Crawler sort of done, could be prettier/faster, probably needs timeout etc, but functions mostly