Use serial numbers as a way of measuring the bounded queue burn rate
This requires only incrementing the serial number on a publish, not when we get the current serial. This in turn needed more precision around which messages we want when working back for retroactive subscribes
Work towards using a vector to store the router shard horizons. Doesn't quite work
Comments and rearranging
Fix a bug/testfailure when one of the shards didn't have any topics and wiped out buffered messages from the other shards
Bulk out get horizon command and add test for it. Doesn't pass yet, since we don't give the horizon back in any way
Allow subscribing using a horizon from a previous call to get_horizon, the test now passes
Tidy up offline retro tests
Partial work on uncontended horizon - doesn't compile
More partial work on uncontended horizons
Finish the uncontended horizon behavior. Need to tidy still
Tidying up and comments
Change semantics of register, so if you register twice, you get the most recent one.
Asana sometimes forgets to subscribe after registering, and when that happens, you probably meant to subscribe to zero topics on the first one (i.e. forget it) and subscribe to the new topics on the second.
Also improve debug output around a horizon being too old
Update the cached horizon after publishes, so we don't get stuck with a mega-old horizon when a lot of publishes, but no registers, happen
Only sent one request to the horizon updater at a time, to avoid overfilling its queue
Also fix tests
Restart the system when the horizon updater goes down
Allow exactly two horizon updates to be in flight at once, to guarantee both eventually updating to the latest, and avoiding unnecessary updates
Redo retroactivity based on timestamps rather than serial number - based horizons
Restore sane default to messages_stored_per_router_shard