Respect robots.txt
Merge pull request #1 from flightstats/fs/robots

I am sorry for that, I have updated it.