support crawling urls without extension, get subtype from response
get title from browser only when subtype is html