Adding RabbitMQ into the mix to break up scraping urls, and processing them with PhantomJS