Bug changes, finalizing web crawler.  With web interface it's fully functional at this commit.