various html page improvements like doctype and title
fix tests with robots.txt