Deleted unnecessary files and added to robots.txt