Fixed a few more bugs after code cleanup, and added a howto on running crawler