added try/except block for database.py import to avoid errors in dev
changes to add robots.txt