added some spam checking, a robots.txt, and a tiny bit of cleanup