Add robots.txt to disallow agents
Documentation dependencies added to test