refactored link checking
link crawler
updated requirements
add comment and removed useless condition
Merge pull request #61 from sashakruglov/issue_56

Issue 56