GitHub PushEvent at 2014-04-01 19:45:27
Repository: spring-xd
Actor: markfisher
Payload: {"shas":[["91ce9a7c53468a689fae7360bc069b1cebc64863","ba670f9476975834a2e3e8fee18ae4e033b6ee52@gopivotal.com","New ZooKeeper Runtime\n\nsupporting simple module deployment\n\nthis is just a stepping stone commit, still requiring a lot of rework\n\ndoes not yet support options values, composed modules, jobs, or source/sink channels\n\nAdded ZooKeeperStreamDefinitionRepository\n\nThat is now instantiated within both redis-store.xml and memory-store.xml.\n\nIt will later be expanded to supplant those (the --store option will no longer be necessary).\n\nThe /streams endpoints in the REST API are now backed by this implementation.\n\nAdded support for Module undeployment\n\nImplemented ZooKeeperStreamDefinitionRepository findAll methods needed to support 'stream list'\n\nRemoved RedisStreamDefinitionRepository and InMemoryStreamDefinitionRepository\n\nTests (work in progress)\n\nStream delete functionality\n\nIntroduced the ability to undeploy and delete a stream.\nThe Stream object has a new property in its map to\ndetermine if the stream's modules should be deployed\nupon creation. This property is exposed through\na \"getter\" in the Stream object and is examined\nwhen a stream is created or updated in ZK.\n\nStream module graceful undeployment/destroy\n\nWhen a stream is undeployed (by marking the 'deployed' flag to\n'false' in the stream data in ZK) the admin listener will\nremove the source, processors, and sink paths in the order\nin which the stream is processed.\n\nFurthermore, when a stream is destroyed via the REST controller,\nthe stream will be undeployed before it is deleted from ZK.\nSee StreamDeployer.beforeDelete() for details.\n\ninitial support for jobs\n\nsome general simplifications (e.g. removed 'undeployChannel')\n\nadded support for stream module parameters\n\nadded support for stream module classloader\n\nnamed channel support in stream definitions\n\nhandling job undeployment\n\nWhen a stream is created (but not deployed) a znode will be created in the following format:\n\n/xd/streams/stream-name\n\nWhen the stream is deployed, /source, /sink/, and\n(optionally) /processors znodes will be created, with\ntheir children being the labels for the modules of those\ntypes that belong to the stream.\n\nWhen the stream is undeployed, the admin will delete\nthe /source, /sink, and /processors nodes to bring\nthe znode back to its original undeployed state:\n\n/xd/streams/stream-name\n\nsupport job redeployment when container departs cluster\n\nsupports composite modules\n\njob launching via MessageBus\n\ncleanup after control bus removal\n\nUniquely identify deployed modules\n\nContainerRegistrar maintains a map of deployed modules via\nModuleDescriptor.Key. The key uses module type and label\nto identify a module. This prevents the tracking of multiple\nmodules of the same type and label for different streams.\nOne of the problems this causes is the inability to undeploy\na module for a stream if a previous stream had the same module\nlabel and type undeployed before it.\n\nThis fix introduces the stream name to the key in order\nto correctly identify the module runtime instance.\n\nAlso, now using job label in key used for deploy and undeploy.\n\nremoved launch booleans from ModuleDeploymentRequests\n\nFixed defect in deleteAll\n\nThe ZooKeeper repository implementations were attempting to\ndelete children in a znode (kind of like a truncate option)\nbut this isn't supported by ZK. Therefore the deleteAll\nimplementation will iterate all of the children and delete\nthem one by one. This isn't efficient but this is the best\nthat can be done via the ZK API.\n\nsplitting on colon plus space for label\n\nFixed AbstractSingleNodeInitializationTests\n\nFixed invalid map references\n\nFixed AbstractSingleNodeInitializationTests\n\nAdded embeddedZooKeeper.start() in setup\n\nModified Curator path cache listener\n\nModified the Curator path cache implementation to\ncorrectly add and remove listeners for a path's\nchildren.\n\nFixed ZooKeeperClientConnectionTests. Make RetryPolicy Pluggable\n\nFixed ZooKeeperClientConnectionTests. Make RetryPolicy Pluggable(2)\n\nspring-xd-dirt/src/test/java/org/springframework/xd/dirt/stream/AbstractSingleNodeStreamDeploymentIntegrationTests.java\n\nfixed module indices and test's undeploy order\n\nreversing order of parsed modules for StreamFactory\n\nno longer reverse order in Stream build\n\navoiding test crosstalk\n\nsupport named channels on both ends of a stream\n\nZooKeeperStreamDefinitionRepositoryTests pass; minor cleanup\n\nparser needs StreamDefinitionRepository for taps\n\nrefactoring StreamTestSupport\n\nensuring the /xd/streams path exists\n\nincreased test sink polling timeouts\n\nAdded stream name to getModule in SingleNodeIntegrationTestSupport\n\ninitialize StreamDefinitionRepository before tests\n\ncreating /xd/streams with create instead of ensurePath\n\nJobsContollerIntegrationTests passing\n\nStreamsControllerIntegrationWithRepositoryTests passing\n\nfixed check for composite module\n\nremoved StreamPluginModuleDeploymentTests and created a similar test in AbstractSingleNodeStreamDeploymentIntegrationTests\n\nusing creatingParentsIfNeeded instead of ensurePath\n\nproperly deleting module dependencies for streams\n\ntap can be deleted after its stream\n\nImplemented paging in ZKSDR\n\nxd-shell-test work in progress\n\nZooKeeperConnection autoStartup=true\n\nFix shell tests - work in progress\n\n- Added DeploymentsListener in shell tests that would check deploy/undeploy event before any assertions\n\n Added DefinitionsListener (wip)\n\nFix hanging FileSink tests\n\n- Throw RuntimeException instead of returning false after waitFor times out\n\nImplemented ZooKeeperStreamRepository\n\nAdded listeners for Stream and Job commands\n\ncorrected path for JobCommandListener\n\nincreased test file sink timeout\n\nimplemented ZooKeeper backed RuntimeContainerInfoRepository\n\nFix ModuleCommandTests\n\nimproving reliability of command listeners\n\nusing JobsPath\n\nimproved reliability of JobCommandTests\n\nimplemented ZooKeeperJobRepository\n\nFix AggregatorProcessorModuleOptionsMetadata's release prop\n\nlaunch error is now async\n\nFix XdConfigLoggingInitializer\n\n- Handle Embedded zookeeper connect\n\nFix SingleNode non-embedded ZooKeeperServer\n\n- Also moved the properties to be referred from SharedServerContextConfiguration\n\nFixed ZooKeeperClientConnectionTests\n\nFixed compile warnings\n\nImproved stream-command-tests\n\nMisc cleanup, Javadoc, refactoring\n\nRemoved PathChildrenCache for jobs and streams as members\nof AdminServer and placed them as local variables in\ntakeLeadership since they are only used in that method.\n\nAdded JavaDoc to AdminServer.\n\nMinor formatting.\n\nXD-1427 upgrade zookeeper to 3.4.6\n\nMisc cleanup\n\nMostly JavaDoc, whitespace, and minor refactoring.","Mark Fisher",true]],"size":1,"ref":"refs/heads/zkpr-squashed","head":"91ce9a7c53468a689fae7360bc069b1cebc64863"}