Support custom user agents

This adds support for custom user agents for the default downloader class and robots.txt filter