Jc working

updates the gitignore and makes robots.txt and robots metadata dynamic based on environment