XD-1338 RabbitMessageBus Module Deployment Props

XD-1338 Partition Metadata Initial Commit

Make ModuleDeployment Properties available in
Messagebus.bind*().

Currently, properties are only used in the RabbitMessageBus.
Properies arrive in the bind*() methods and an internal
property accessor is used to extract the properties (or fallback
to the defaults configured in the application.yml if a
particular property does not exist).

XD-1338 prep part 2 (work in progress)

Ultimately the goal is to provide the ModuleDeploymentProperties within the
runtime Module instance (e.g. SimpleModule) so that it can be accessed by
plugins. For example, the StreamPlugin will then be able to use any "consumer"
and/or "producer" properties when calling bindConsumer() and bindProducer()
methods on the MessageBus for a given Module.

While initially the plan was simply to pass those deployment properties along
within the DeploymentMetadata, that class itself overlapped with the recently
added ModuleDescriptor. So, this commit actually removes the DeploymentMetadata
class altogether. The ModuleDescriptor replaced the ModuleDeploymentRequest
object, and it wraps a ModuleDefinition (the "essence" of a module that is
shared by all instances within any stream) while adding the context for a
particular module instance within a stream (the group, index,
sourceChannelName, and sinkChannelName). Those properties are the ones that
were also exposed via DeploymentMetadata.

The most significant part of this commit is the modification to the constructor
for SimpleModule as shown below.

BEFORE:

	public SimpleModule(ModuleDefinition definition, DeploymentMetadata metadata) {

AFTER:

    public SimpleModule(ModuleDescriptor descriptor, ModuleDeploymentProperties deploymentProperties)

David fixed failing tests

passing deployment properties along for StreamPlugin

Conflicts:

	spring-xd-dirt/src/main/java/org/springframework/xd/dirt/server/ContainerRegistrar.java

Resolved.

Pass Consumer/Producer Props to MessageBus

RabbitMessageBus Overrride Default Properties

Add test cases.

TODO:

- Producer properties for more exotic producers
(dynamic producers from the router sink, tap producers, etc).

- Add logic to detect/log/(or error) if an invalid property is supplied for a bus

- Add logic for "affinity" partitioning where the producer evaluates where a
particular message should go and the consumers listen to partitioned queues.

--

XD-1338 Polishing; PR Comments (JavaDocs)
XD-1735 FileJdbc & JdbcHdfs test now use http source

When running tests in single & on EC2 these tests were successful.  However when running admin/container on the same machine these tests would fail because of the trigger.
With this update the tests work on all 3 types of deployments.
XD-1740 Add admin IP address at ZK node

Set AdminServer application context id to use admin server's `<ip-address>:<adminport>`

Modified RuntimeUtils.getIpAddress to return the first non loopback
address it can find. This was modified because
InetAddress.getLocalHost().getHostAddress() on Linux may return
127.0.0.1.
Misc changes (see below for details)

Follow ups to review (by Mark F) of XD-1548 including:

- Renamed methods in ContainerListener
- Renamed configuredTimeout to timeout in ModuleDeploymentWriter

Also moved logCacheEvent from ZooKeeperConnection to ZooKeeperUtils.
XD-1739

Container reconnection to ZK can fail intermittently

If a container disconnects and reconnects to ZooKeeper, sometimes
the ephemeral node for that container isn't cleaned up before
the container attempts to create a new ephemeral node. This fix
will delete the node if it exists prior to creating a new one.

In theory this should never happen, but in cases where the ZK
server and the container are both suspended (i.e. a laptop goes
to sleep) the container could reconnect before ZK cleans up
the old ephemeral node.

Also added the @SuppressWarnings("serial") annotation to
ZooKeeperAccessException.

Using ZooKeeperUtils to handle exception per review comment
XD-1737 Added --verbose option
XD-1741

Added StringToByteArrayConverter in app config

Added unicode chars to test
Fix Test Rabbit Queue Deletion

foo.props.0, xdbus:queue, xdbus:bar

Test for custom prefix left foo.props.

doTest was using the underlying Rabbit bus, not the wrapper.
XD-1742 Support Binary in TCP and HTTP Sources

JIRA: https://jira.spring.io/browse/XD-1742

Previously, the TCP and HTTP sources unconditionally
converted byte[] to String.

Remove the transformer from the tcp source; now, if people
want String conversion, they would use

    tcp --outputType=text/plain

Tested with

    stream create --name foo --definition "tcp --decoder=RAW | file --binary=true " --deploy
    stream create --name foo --definition "tcp --decoder=RAW --outputType=text/plain;charset=UTF-8 | file --binary=true " --deploy

Update the inbound http channel adapter to detect a
content-type of `application/octet-stream` and emit
a byte[] instead of performing String conversion.

This allows POSTing binary data...

    curl --data-binary @foo.zip -H'content-type: application/octet-stream' http://localhost:9000

Add test case to verify. Also tested with

    stream create --name bar --definition "http | file --binary=true " --deploy
Fixed compile warnings
XD-1672 filepollhdfs Acceptance Test

* Hdfs based tests will use assertValidHdfs
* Remove overloaded SimpleFileSink.toDSL method
* Replaced FileSourceTest's triggers with http
XD-1747: Module deployment order is not guaranteed

Modified writeDeployment to deploy each module serially (but still
write out deployment requests for a module to multiple containers
at once). This is required to guarantee that modules for a stream
are deployed in the correct order.
XD-1520 Update ModuleOptions stream name defaults

- Fix Module options that use stream name as the default value
  - Add ${xd.stream.name} as the default value in appropriate
module options POJO and module options properties
  - This will allow `module info` command to display the default values
with ${xd.stream.name} wherever stream name is used

Following modules are fixed:
File Sink
Gemfire sinks
Aggregator
File source
Metrics
HDFS dataset
Hdfs + move into own dir
Post
Tail
JMS
SimpleFileSink uses StringUtils.hasText to check for null and empty strings for the fileName and directory when creating DSL string.
JobPluginTests - Remove explicit datasource url

- Since this test uses hsqlDataSource which randomly gets configured
by `RandomConfigurationSupport` explicit datasource url is not needed.
XD-1757 Resolve runtime module option values

- Fix ZooKeeperModuleMetdataRepository's findOne(String metadataPath)
   - Check if the module option value has property placeholder syntax and retrieve
the placeholder value from module metadata from ZK metadata data
For example, if the `log` module's name option has the value ${xd.stream.name} then
look for `xd.stream.name` in module metadata to resolve the module option `name`
XD-1748 Spring Integration -> 4.0.1 + Spring IO

JIRA: https://jira.spring.io/browse/XD-1748

Spring Integration to 4.0.1 - performance improvement in
the aggregator for large groups (e.g. hdfs-dataset). Update
the sink to enable the improvement.

Also move other deps to Spring IO versions.

Hold back spring-data-redis, because 1.3.0 is not
compatible with boot 1.0.x.
XD-1758 Resets the amqUrl to amq.url for brokerUrl
XD-1756 Upgrading to SHDP 2.0 RC4, modifying configs to match new properties
WebUI: enable/disable schedule action

- Instead of show/hide the schedule button, use
`enable/disable` the button based on the validity
of the job trigger stream name and fixed delay number