setting download delay 3 second, and follow robots.txt.