D6 corrected rules for the robots.txt, disable access to .pot files, correct some MIME types according to RFC