add license file
add robots.txt generator
Merge pull request #7 from take-cheeze/robots-exclusion

robots.txt generator and license file