Add option to disallow search engines.
Merge pull request #15 from obduk/disalow_robots

Add option to disallow search engines.