fixed: reduced_redundancy=True and cleanup/upload robots.txt scripts