Project: springxd
Issue XD-2004: Containers stopped responding to Admin
SHA = a205d43f0b59e1984bf55c3368b031a373a03712 Environment: Rabbit Transport Test 1 admin 2 containers. [Initial Event] During the run of FileJdbcTest.testPartitionedFileJdbcJob the containers quit responding to the admin server. After the initial failure at 12:26:46 no other streams can be deployed. [Secondary Event] When shutting down one of the container 1 the following exception occurs on the admin server: 12:51:12,004 INFO DeploymentSupervisorCacheListener-0 server.DepartingContainerModuleRedeployer - Container departed: Container{name='5353dc4b-6068-49a0-8981-fa175869edf0', attributes={id=5353dc4b-6068-49a0-8981-fa175869edf0, host=domU-12-31-39-07-81-02, pid=1270, groups=, ip=10.209.130.240}} 12:51:12,004 ERROR DeploymentSupervisorCacheListener-0 cache.PathChildrenCache - org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /xd/deployments/modules/allocated/5353dc4b-6068-49a0-8981-fa175869edf0 org.apache.zookeeper.KeeperException.create(KeeperException.java:111) org.apache.zookeeper.KeeperException.create(KeeperException.java:51) org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1590) org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:214) org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:203) org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107) org.apache.curator.framework.imps.GetChildrenBuilderImpl.pathInForeground(GetChildrenBuilderImpl.java:199) org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:191) org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:38) org.springframework.xd.dirt.server.DepartingContainerModuleRedeployer.deployModules(DepartingContainerModuleRedeployer.java:101) org.springframework.xd.dirt.server.ContainerListener.childEvent(ContainerListener.java:104) org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:509) org.apache.curator.framework.recipes.cache.PathChildrenCache$5.apply(PathChildrenCache.java:503) org.apache.curator.framework.listen.ListenerContainer$1.run(ListenerContainer.java:92) com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297) org.apache.curator.framework.listen.ListenerContainer.forEach(ListenerContainer.java:83) org.apache.curator.framework.recipes.cache.PathChildrenCache.callListeners(PathChildrenCache.java:500) org.apache.curator.framework.recipes.cache.EventOperation.invoke(EventOperation.java:35) org.apache.curator.framework.recipes.cache.PathChildrenCache$10.run(PathChildrenCache.java:762) java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) java.util.concurrent.FutureTask.run(FutureTask.java:262) java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) java.util.concurrent.FutureTask.run(FutureTask.java:262) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) The attached container logs are only partial, because they have rolled over. The attached admin log is fairly complete.
Story points: 12