GitHub repository: marcostolosa/katana
Description: A next-generation crawling and spidering framework.