Sample crawl & web server tests

Added tests to sample crawl tests
Added tests to web server