XD-1730: Zookeeper NoNode exception when deploying stream

If the admin supervisor does not receive a response for
a module deployment request in the allotted time, it will
consider the deployment timed out and it will remove
the ZK path for the module deployment. In the meantime,
if the module is successfully deployed on the container
side, it will attempt to write the module metadata
to the node that was already deleted by the supervisor,
thus causing a NoNodeException.

This fix splits out the module deployment and the
updating of the module deployment ZooKeeper node into
two separate try blocks. If a NoNodeException is thrown
the container logs the fact that the supervisor decided
to undeploy the module while it was being deployed.

Instead of undeploying the module right away, we will
leave it alone and let the upcoming CHILD_REMOVED event
handler perform the undeployment.

Additionally, the default timeout for module deployment
was increased to 30 seconds.
XD-1710/XD-1734 Improve JMX checks for processors taking into account error channels

* ProcessorTest.testfailedSink needs to use http as its test source
* Cleaned up Javadoc
* Match method names for overloaded methods
XD-1731 Support for UUID suffix for hdfs file names in acceptance tests
XD-1671: Added NamedColumnJdbcItemReaderFactory to configure MySql and
SQLite to handle large datasets correctly in prepackaged jobs.
XD-1708 Improve module re-deployment fixes

Proposed changes for XD-1708:

I was confused by the method signature in
ModuleDeploymentWriter.writeDeployment:

Collection<Result> writeDeployment(
        Iterator<ModuleDescriptor> descriptors,
        ModuleDeploymentPropertiesProvider provider,
        ContainerMatcher containerMatcher,
        Iterator<Container> containersToMatch)

The third parameter is called `containerMatcher` and
the fourth parameter is `containersToMatch`. It seems
to me that an object called `containerMatcher` should
be able to match containers on its own.

Therefore I introduced a new ContainerMatcher implementation
that wraps another ContainerMatcher and also accepts
container ids that should be excluded from matches.
This puts the logic in ContainerMatcher (which is where
it should go since it's one job is to match containers)
and allows for easier reuse down the line if we need it.

Note that this uses the Google Collections library.
XD-1338 RabbitMessageBus Module Deployment Props

XD-1338 Partition Metadata Initial Commit

Make ModuleDeployment Properties available in
Messagebus.bind*().

Currently, properties are only used in the RabbitMessageBus.
Properies arrive in the bind*() methods and an internal
property accessor is used to extract the properties (or fallback
to the defaults configured in the application.yml if a
particular property does not exist).

XD-1338 prep part 2 (work in progress)

Ultimately the goal is to provide the ModuleDeploymentProperties within the
runtime Module instance (e.g. SimpleModule) so that it can be accessed by
plugins. For example, the StreamPlugin will then be able to use any "consumer"
and/or "producer" properties when calling bindConsumer() and bindProducer()
methods on the MessageBus for a given Module.

While initially the plan was simply to pass those deployment properties along
within the DeploymentMetadata, that class itself overlapped with the recently
added ModuleDescriptor. So, this commit actually removes the DeploymentMetadata
class altogether. The ModuleDescriptor replaced the ModuleDeploymentRequest
object, and it wraps a ModuleDefinition (the "essence" of a module that is
shared by all instances within any stream) while adding the context for a
particular module instance within a stream (the group, index,
sourceChannelName, and sinkChannelName). Those properties are the ones that
were also exposed via DeploymentMetadata.

The most significant part of this commit is the modification to the constructor
for SimpleModule as shown below.

BEFORE:

	public SimpleModule(ModuleDefinition definition, DeploymentMetadata metadata) {

AFTER:

    public SimpleModule(ModuleDescriptor descriptor, ModuleDeploymentProperties deploymentProperties)

David fixed failing tests

passing deployment properties along for StreamPlugin

Conflicts:

	spring-xd-dirt/src/main/java/org/springframework/xd/dirt/server/ContainerRegistrar.java

Resolved.

Pass Consumer/Producer Props to MessageBus

RabbitMessageBus Overrride Default Properties

Add test cases.

TODO:

- Producer properties for more exotic producers
(dynamic producers from the router sink, tap producers, etc).

- Add logic to detect/log/(or error) if an invalid property is supplied for a bus

- Add logic for "affinity" partitioning where the producer evaluates where a
particular message should go and the consumers listen to partitioned queues.

--

XD-1338 Polishing; PR Comments (JavaDocs)
XD-1735 FileJdbc & JdbcHdfs test now use http source

When running tests in single & on EC2 these tests were successful.  However when running admin/container on the same machine these tests would fail because of the trigger.
With this update the tests work on all 3 types of deployments.
XD-1740 Add admin IP address at ZK node

Set AdminServer application context id to use admin server's `<ip-address>:<adminport>`

Modified RuntimeUtils.getIpAddress to return the first non loopback
address it can find. This was modified because
InetAddress.getLocalHost().getHostAddress() on Linux may return
127.0.0.1.
Misc changes (see below for details)

Follow ups to review (by Mark F) of XD-1548 including:

- Renamed methods in ContainerListener
- Renamed configuredTimeout to timeout in ModuleDeploymentWriter

Also moved logCacheEvent from ZooKeeperConnection to ZooKeeperUtils.
XD-1739

Container reconnection to ZK can fail intermittently

If a container disconnects and reconnects to ZooKeeper, sometimes
the ephemeral node for that container isn't cleaned up before
the container attempts to create a new ephemeral node. This fix
will delete the node if it exists prior to creating a new one.

In theory this should never happen, but in cases where the ZK
server and the container are both suspended (i.e. a laptop goes
to sleep) the container could reconnect before ZK cleans up
the old ephemeral node.

Also added the @SuppressWarnings("serial") annotation to
ZooKeeperAccessException.

Using ZooKeeperUtils to handle exception per review comment
XD-1737 Added --verbose option
XD-1741

Added StringToByteArrayConverter in app config

Added unicode chars to test
Fix Test Rabbit Queue Deletion

foo.props.0, xdbus:queue, xdbus:bar

Test for custom prefix left foo.props.

doTest was using the underlying Rabbit bus, not the wrapper.
XD-1742 Support Binary in TCP and HTTP Sources

JIRA: https://jira.spring.io/browse/XD-1742

Previously, the TCP and HTTP sources unconditionally
converted byte[] to String.

Remove the transformer from the tcp source; now, if people
want String conversion, they would use

    tcp --outputType=text/plain

Tested with

    stream create --name foo --definition "tcp --decoder=RAW | file --binary=true " --deploy
    stream create --name foo --definition "tcp --decoder=RAW --outputType=text/plain;charset=UTF-8 | file --binary=true " --deploy

Update the inbound http channel adapter to detect a
content-type of `application/octet-stream` and emit
a byte[] instead of performing String conversion.

This allows POSTing binary data...

    curl --data-binary @foo.zip -H'content-type: application/octet-stream' http://localhost:9000

Add test case to verify. Also tested with

    stream create --name bar --definition "http | file --binary=true " --deploy
Fixed compile warnings
XD-1672 filepollhdfs Acceptance Test

* Hdfs based tests will use assertValidHdfs
* Remove overloaded SimpleFileSink.toDSL method
* Replaced FileSourceTest's triggers with http
XD-1747: Module deployment order is not guaranteed

Modified writeDeployment to deploy each module serially (but still
write out deployment requests for a module to multiple containers
at once). This is required to guarantee that modules for a stream
are deployed in the correct order.
XD-1520 Update ModuleOptions stream name defaults

- Fix Module options that use stream name as the default value
  - Add ${xd.stream.name} as the default value in appropriate
module options POJO and module options properties
  - This will allow `module info` command to display the default values
with ${xd.stream.name} wherever stream name is used

Following modules are fixed:
File Sink
Gemfire sinks
Aggregator
File source
Metrics
HDFS dataset
Hdfs + move into own dir
Post
Tail
JMS
SimpleFileSink uses StringUtils.hasText to check for null and empty strings for the fileName and directory when creating DSL string.
XD-1622: Updated to use latest Spring Batch version