cache crawl results
change filenames to md5 and tag as crawlcache
detected many problems after implementing my own crawler (for establishing the cache)
compress cache files
new technique after talking with Dave.  generate very simple versions of the pages for the crawler directly from the services (where the nice urls still point to the full pages)
from code review: include entity annotations
body
microsoft site verification
update home link
Merge pull request #207 from jay-hodgson/develop

handle crawl requests