catch more bot useragents and catch bad url decoding
catch more non cors supported browsers