added robots, sitemap and banners; some cleanups