Revert "Crawler herpderp"

This reverts commit c3cbf6923e1883752c00ae251262406c39a50f5b.
Revert "Limited the number of simultaneous crawler downloads"

This reverts commit 7fc791996e7ce14a3b6f522fd3caa88c71bed7fe.