added index,follow metatag and robots.txt

Signed-off-by: Luís Pedro Algarvio <lp.algarvio@gmail.com>