robots.txt added.

sticking with plain text sitemap, reducing the possibility of errors and allowing for easy parsing.  Closes #32
See webmasters question: http://webmasters.stackexchange.com/questions/43407