Beginning to classify URLs if worth to follow
Merge branch 'master' of github.com:ChristophKnabe/akka-crawl
Implemented isWorthToFollow for links.
Merge branch 'master' of github.com:ChristophKnabe/akka-crawl