Improved links priority algorithm

Added new priority level for "non domain" links. This improvement
speeds up crawling for spiders with "stay_in_domain": False directive.