spring-projects/spring-xd — pull request #782 by pperalta (2014-04-25 15:38:03+00:00)
Title: XD-1590: Move ephemeral nodes to /xd/deployments

In order to have a clearer separation of definition vs runtime information, the ephemeral nodes written by containers were moved from `/xd/streams/stream-name` to `/xd/deployments/streams/stream-name`. Same for jobs.

The work of building these path streams falls under `StreamsPath` and `JobsPath`. These classes were renamed to `StreamsDeploymentsPath` and `JobsDeploymentsPath`, respectively.

Class `DeploymentVerifier` was introduced into the test package. This class is used to block execution until a stream or job is created, destroyed, deployed, or undeployed. Instead of extending this class, stream and job path information is provided via the `DeploymentPathProvider` interface which is passed along to the `DeploymentVerifier` constructor. A new exception type `RuntimeTimeoutException` was introduced in order to indicate a timeout. This is a runtime version of `TimeoutException` which is a checked exception in the JDK.

In `SingleNodeIntegrationTestSupport`, if a `PathChildrenCache` object is created in `addPathListener`, the listener is added before invoking start to ensure that no events are missed.

In `ContainerRegistar`, a subtle race condition was fixed in registerWithZooKeeper. The admin leader/supervisor will delete nodes from `/xd/deployments/modules/[container-id]` for containers that are not present under `/xd/containers`. Therefore if a new container creates a `.../deployments/modules` node before creating its `/xd/containers node`, the leader could delete its deployments node. This has the nasty side effect of rendering any `PathChildrenCache` instances for the deployments path useless since it won't receive events from a path that is deleted and recreated.

`StreamListener` now offloads module deployment and undeployment via a dedicated thread in order to prevent the Curator thread from being blocked. We might consider making the executor service multi threaded at some point if there is a benefit.

`ZooKeeperStreamRepository` will delete the data under `/xd/deployments/streams` in the order in which the stream modules were defined. This allows for an orderly undeployment of streams. It is obvious that the source should be undeployed first and the sink last. Other than loading the stream definition itself (and parsing it, etc) the only way to know what order to undeploy the processor modules is to examine the order in which they were deployed. This can be done by examining the ZooKeeper transaction id as all writes in ZooKeeper are ordered.

Some of the test asserts were enhanced with extra debugging information to assist in test failure analysis.

The default timeout in `DisposableFileSupport` was reduced from 10 seconds to 1 second. This was done because the asserts that use this class all use the `eventually` matcher which executes underlying matchers multiple times. For tests that fail due to a module deployment issue and thus preventing the expected file from being created, this combination would cause the tests to execute for a very long time before timing out. In fact this condition could be easily mistaken for a "hung" test.

On a related matter, the defaults for `EventuallyMatcher` were reduced from 50 tries to 20 tries.