updated address.py to respect Robots.txt