async version of clojure crawler