Assertion: search engine | /r/CapableOf | index public parts of their websites
Facets:
value: with robots.txt
type: OTHER
Source sentences:
text: Apart from standard web application security recommendations website owners can reduce their exposure to opportunistic hacking by only allowing search engines to index the public parts of their websites (with robots.txt) and explicitly blocking them from indexing transactional parts (login pages, private pages, etc.).