GitHub repository: sjdirect/abot
Primary language: C#
Description: Cross Platform C# web crawler framework built for speed and flexibility. Please star this project! +1.