Distributed crawling with Redis