Remove unused file; fix spelling

include robots.txt
fix tests that rely on structure of bundled html generated