CC-338: Ramp up publishing to redis; publish in batches; back off and ra...

...mp up again on failure