I am on a shared web hosting linux server. I want search engine robots/spiders to be able to read the robots.txt but not any one typing www.mysite.com/robots.txt.

As per the following google group post, the user specifies that by setting file permission to 640, it's possible to deny access to robots.txt file by the world but still enable search engine robots to read them.

Is that true? If not how it's possible to deny general public access to robots.txt but still allow Search engine robots to read them.


Question: Where is Query String (URL) Filter?
Answer: Filter web parts require enabling SharePoint Enterprise site collection feature. That means this belongs to only enterprise SharePoint versions.