added docs. added robots.txt