Fixed some bugs in WebCrawler client
- Added unit test for all external links