added robots.txt file for robot friendly crawling.  Updated unit tests to check for that file.

Made it so static files are to be referenced without /static/ prefix.