Added requirements and robots.txt