create script to run hyperlinks
stop testing on first fail
remove redundant test
simplified flow of collector

don't add already visited URLs to the queue
ignore logs
Refactored to introduce concurrent HTTP requests
mock request.head call
extract explicitly usage of requests.XXX
Concurrency options is added