Major refactoring of queue code using the new CompositeEntityManager

This should fix various queue related bugs and make it possible to plug in more robust sharding which can reduce the impact of tombstones.