Added robots.txt and .htaccess generation.