A basic web crawler that crawls up to a specified depth and saves crawled urls into a file. Accepts command line arguments for filename, maximum depth and seed url. Obeys Robots.txt. Avoids crawling previously crawled pages.