Project: springxd
Issue XD-1580: Undeploy modules when container disconnected from ZK
Consider a module running in a container when it is disconnected from ZK: {noformat} 12:30:13,021 WARN inbound.t.0-redis:queue-inbound-channel-adapter1 logger.t:150 - 2014-04-15 12:30:13 12:30:14,025 WARN inbound.t.0-redis:queue-inbound-channel-adapter1 logger.t:150 - 2014-04-15 12:30:14 12:30:15,029 WARN inbound.t.0-redis:queue-inbound-channel-adapter1 logger.t:150 - 2014-04-15 12:30:15 12:30:16,031 WARN inbound.t.0-redis:queue-inbound-channel-adapter1 logger.t:150 - 2014-04-15 12:30:16 12:30:32,590 WARN inbound.t.0-redis:queue-inbound-channel-adapter1 logger.t:150 - 2014-04-15 12:30:32 12:37:42,985 WARN inbound.t.0-redis:queue-inbound-channel-adapter1 logger.t:150 - 2014-04-15 12:37:42 12:37:43,398 INFO main-SendThread(fe80:0:0:0:0:0:0:1%1:2181) zookeeper.ClientCnxn:1096 - Client session timed out, have not heard from server in 430809ms for sessionid 0x145662be03e0002, closing socket connection and attempting reconnect 12:37:43,985 INFO main-SendThread(localhost:2181) zookeeper.ClientCnxn:975 - Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error) 12:37:43,986 INFO main-SendThread(localhost:2181) zookeeper.ClientCnxn:852 - Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session 12:37:43,988 WARN inbound.t.0-redis:queue-inbound-channel-adapter1 logger.t:150 - 2014-04-15 12:37:43 12:37:43,989 INFO main-SendThread(localhost:2181) zookeeper.ClientCnxn:1094 - Unable to reconnect to ZooKeeper service, session 0x145662be03e0002 has expired, closing socket connection {noformat} Currently the module for the disconnected container continues to execute: {noformat} 12:37:45,994 WARN inbound.t.0-redis:queue-inbound-channel-adapter1 logger.t:150 - 2014-04-15 12:37:45 12:37:46,997 WARN inbound.t.0-redis:queue-inbound-channel-adapter1 logger.t:150 - 2014-04-15 12:37:46 12:37:48,000 WARN inbound.t.0-redis:queue-inbound-channel-adapter1 logger.t:150 - 2014-04-15 12:37:47 12:37:48,094 ERROR main-EventThread imps.CuratorFrameworkImpl:512 - Watcher exception org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /xd org.apache.zookeeper.KeeperException.create(KeeperException.java:127) org.apache.zookeeper.KeeperException.create(KeeperException.java:51) org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155) org.apache.curator.framework.imps.GetDataBuilderImpl$4.call(GetDataBuilderImpl.java:302) org.apache.curator.framework.imps.GetDataBuilderImpl$4.call(GetDataBuilderImpl.java:291) org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:287) org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279) org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41) org.springframework.xd.dirt.server.ContainerRegistrar$StreamModuleWatcher.process(ContainerRegistrar.java:609) org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:56) org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498) 12:37:48,097 INFO main-EventThread state.ConnectionStateManager:194 - State change: SUSPENDED 12:37:48,097 INFO ConnectionStateManager-0 zookeeper.ZooKeeperConnection:262 - >>> Curator disconnected event: SUSPENDED 12:37:48,097 WARN ConnectionStateManager-0 server.ContainerRegistrar:325 - >>> disconnected container: 88ba115b-6190-497a-a67c-df1e295bf158 12:37:49,001 WARN inbound.t.0-redis:queue-inbound-channel-adapter1 logger.t:150 - 2014-04-15 12:37:49 12:37:50,004 WARN inbound.t.0-redis:queue-inbound-channel-adapter1 logger.t:150 - 2014-04-15 12:37:50 12:37:51,008 WARN inbound.t.0-redis:queue-inbound-channel-adapter1 logger.t:150 - 2014-04-15 12:37:51 12:37:52,012 WARN inbound.t.0-redis:queue-inbound-channel-adapter1 logger.t:150 - 2014-04-15 12:37:52 12:37:53,016 WARN inbound.t.0-redis:queue-inbound-channel-adapter1 logger.t:150 - 2014-04-15 12:37:53 12:37:54,021 WARN inbound.t.0-redis:queue-inbound-channel-adapter1 logger.t:150 - 2014-04-15 12:37:54 12:37:55,023 WARN inbound.t.0-redis:queue-inbound-channel-adapter1 logger.t:150 - 2014-04-15 12:37:55 ... {noformat} This container should not continue executing the module because the leader admin will likely select another container to execute this module. If and when this container reconnects to ZK, it can be (re)assigned modules for deployment. This can be done via a simple undeployment; or we may even consider closing and reopening the application context.
Story points: 4