refactored link checking
link crawler
updated requirements
add comment and removed useless condition
Merge pull request #61 from sashakruglov/issue_56

Issue 56
fix except issue
Merge pull request #63 from bebef1987/except

fix except issue
Added test for non-latin chars. and arked it as xfail
Merge pull request #64 from AndreiH/non-ascii-characters

Added test for non-latin chars. and marked it as xfail
Merge remote-tracking branch 'upstream/master' into issue_57

merge with master
make case-sensitive and replace double quotes to single quotes