made more reliable
added crawler to help with SEO