concurrent_tree_crawler (pypi): A generic concurrent tree crawling algorithm with a sample implementation for website crawling.
Keywords: concurrent|tree|website|crawler