New ROBOTS_EXCLUSIONS option for sitemaps and generated robots.txt
Generate robots.txt
Warn and abort when files/robots.txt
flake8
Use FILES_FOLDERS option instead of default value
Debug eventualities
debug msg
Update robots.plugin
no len() now that there is a default value
Merge pull request #1250 from getnikola/robots-are-cool

New ROBOTS_EXCLUSIONS option for sitemap exclusions and new robots.txt generation
Private and draft posts debug mesg
Fix reference to missing real_scan_files method in check plugin.
merged master