Fixed a bug that made the crawler download hang