update nginx config for dcs instant
add new URLs to robots.txt to prevent crawling