twitter/scala-zookeeper-client — Issue #8: Session expiration handling
Author: maddogg14524
Created: 2012-02-12 05:24:35+00:00
State: open

The ZooKeeper documentation discusses at length the fact that session expiration should be treated as a devastating event in most cases. I can foresee times where it is not for people (as seems to be the case with Twitter's use of this library) but I think completely ignoring the situation leads a lot of people first picking up ZooKeeper down a path towards a nasty surprise in production.

I would remind users of the following in the README:

If a session expires, the scala zookeeper client will close() the now 'dead' zk handle and create a new instance essentially establishing a new session. As a result, all of the ephemeral nodes that had been created with the original session are now gone. If the app using the scala zk client was depending on these nodes existing, this can cause some significant coordination issues between how the rest of the cluster views the node state vs what is reality. To use the client as is for maintaining ephemeral node state, there will need to be work done outside of the client to appropriately detect these nodes not existing and take actions.

I think it is a good idea to allow the client to be configured whether to execute the reconnect logic or to simply call a listener/watcher when session expires. Up to this point, our team (Boston Technologies) had been using Apache's ZooKeeper client directly and shutting down our system in response to the session expired event via the default watcher. This is in line with the behavior recommended by the ZooKeeper library writers.