The robots.txt file must be placed at the root level of the domain name, accessible at e.g., https://example.com/robots.txt or ftp://ftp.example.com/robots.txt