refactoring crawler class and thinking about effective shutdown
Merge branch 'dev' of github.com:SerjNarbut/webcrawler into dev