Add `Disallow:` to `robots.txt`

Also, update a broken url in `robots.txt`