change user agent default to mediawords bot
Merge branch 'master' of github.com:berkmancenter/mediacloud
add more prefixes to remove from normalized urls
use MediaWords::Util::URL::normalize_url instead of local version
use MediaWords::Util::URL to normalize url in generate_medium_url_and_name_from_url; only reload new media in lookup_medium_by_url
Merge branch 'master' of github.com:berkmancenter/mediacloud