XD-1760 Rabbit/Redis Bus Short Circuiting

For colocated modules, when instructed by the lead admin,
hard wire the modules together.

- When binding a short-circuited producer, save his properties in
the binding so they can be used to revert, when the consumer is
unbound.

- When the consumer is unbound, bind the producer to the bus
(concurrently with the direct binding), remove the direct binding,
then stop the consumer (so there is always at least one subscriber
to the producing module's output channel.

- If the consumer is re-deployed, switch back to short circuit.

- TODO: Currently requires a producer property `shortCircuitAllowed`

XD-1760 Polishing (Tests)

Add delay to redis check for empty queue.
XD-1856 Fix failing RemoteFileToHadoopTests after fsUri change
XD-1701: Remove need of Hadoop on admin classpath
XD-1856 Fix RemoteFileToHadoopTests

- Use `fsUri` instead of `spring.hadoop.fsUri`
XD-1442: Remove HadoopDistro enum

- Use ${xd.home} in the glob for hadoop distro
XD-1816: Generate module options doc in asciidoc

* Remove default from option description itself
* upgrade to grgit 0.8
* Generate shell reference in the git repo, and allow its push
XD-1360: Add deployment status for single instance

* Update to new REST API
XD-1853: Tap Fixture refactoring
XD-1822: Re-organize XD batch schema

- Move `IS_RESTARTABLE` and `IS_INCREMENTABLE` columns into new table
`XD_JOB_REGISTRY` along with `JOB_NAME`
- Rename `JOB_REGISTRY_STEP_NAMES` to `XD_JOB_REGISTRY_STEP_NAMES`
- Update SQL files for create/drop
- Update `DistributedJobLocator` to handle the insertion/updates
based on the schema changes.
- Fix Schema based tests
- Update regex for date check in jobcommandtests for up to 4 char timezone
JobCommandTests - revert to original regex
Fix test following https://github.com/spring-projects/spring-shell/commit/b19c8cecc9f517382ba5fca1bb10162e313d4a87
XD-1270: Add states to the deployment of stream

Summary of changes:

1. ModuleDeploymentWriter now returns ModuleDeploymentStatus
   which encapsulates everything related to individual
   module state.

2. The ModuleDeploymentStatus objects are aggregated and
   passed into an implementation of new interface
   DeploymentUnitStateCalculator which calculates the
   state of the job/stream based on the individual module
   states.

3. Interface DeploymentUnit was introduced to cover the
   common attributes for streams and jobs.

4. The status for streams/jobs is written into ZK under
   /xd/deployments/streams/<name>/status and
   /xd/deployments/jobs/<name>/status, respectively.

5. As a result, modules for streams and jobs are now under
   /xd/deployments/streams/<name>/modules and
   /xd/deployments/jobs/<name>/modules, respectively.

6. The stream and job repositories have a new method
   getDeploymentStatus which may be invoked to obtain
   the state of a stream/job.

7. Stream and job states are recalculated when a container
   joins or leaves the cluster.

Addressed PR comments
XD-1819 Runtime executables share jars in xd\lib

- remove filter in add to cp function
XD-1872 Close Taps pathcache upon ZK disconnect

- In AbstractMessageBusBinderPlugin, close the `taps` PathChildrenCache
upon ZK disconnect. This will avoid existence of multiple `TapsPathChildrenCache`
over period of time (during multiple ZK connect/disconnect)
XD-1831 Mask Database Passwords

* Add module option type `Password`
* In `JdbcConnectionMixin` - Change password property from `String` to type `org.springframework.xd.module.options.types.Password`
* In `DetailedModuleDefinitionResourceAssembler` mask default password module option
* In `JobsController#list()` return PagedResources<JobDefinitionResource> that contain masked passwords in the job definitions
* Create AngularJS directive `xdFormatStream`
* Create unit tests for AngularJS directive `xdFormatStream`
* In `module-create-definition.js` make sure the option type is correctly set
* In `definitions.html` make sure the `xd-format-stream` directive is called
* In `module-create-definition.html` set the form input field to type password, if the module option type is `Password'
* In `module-create-definition.html` set the form input field to be a checkbox, if the module option type is `boolean`
* Add E2E tests `create-new-definition.spec.js`
* In `module-create-definition.html` apply the xd-format-stream directive when outputting the `calculatedDefinition`
* Add `lodash` JS library
* Add Java utility class `org.springframework.xd.dirt.stream.util.StreamUtils`
* Add `StreamUtils#maskPasswordsInStreamDefinition`
* Add unit tests for `StreamUtils#maskPasswordsInStreamDefinition`

Jira: https://jira.spring.io/browse/XD-1831

XD-1831 Code review fixes + polishing

* Admin UI - support Unicode RegEx
* Add XRegExp library (For Unicode support)
* Add support to match both `password` and `passwd` parameters
* Fix `JobsController#list()`
* Fix JavaDoc in `StreamUtilsTests
* Add one more test to `StreamUtilsTests
* Add `toString()` to `org.springframework.xd.module.options.type.Password`
* Add tests for `org.springframework.xd.module.options.type.Password`
* Add more tests to `directives.js`

XD-1831 Fix failing test + rebase with MASTER

XD-1831 Code Review changes - Add JavaDoc

XD-1831 Code Review Changes

XD-1831 Remove `lodash`
ensure InterruptedException handling
Moved 2 tangled classes from core to server pkg.
Clarify logging for HSQL shutdown
XD-1846 Simplify DeploymentVerifier

- With the Stream/Job Deployment State changes, the `DeploymentVerifier`
has been simplified to use the DeploymentUnitStatus.State to verify if
the Stream/Job is successfully deployed/undeployed.
    - Add `getDeploymentStatus` in DeploymentPathProvider and remove other
deployment path specific method contracts.
    - Have `Stream/JobPathProvider`s implement this method and use Stream/JobRepository's
`getDeploymentStatus` (via DeploymentStatusRepository) to return DeploymentUnitStatus
    - Since the StateCalcualtor already determines the number of modules that need to be
deployed for a given deployment unit, the verification of `DeploymentUnitStatus.State`
can be sufficient to assert if the deployment unit is successfully deployed/undeployed
with the expected number of modules count etc.,
    - Update `DeploymentVerifier`'s deployment verifier methods waitForDeploy/waitForUndeploy
Commenting out tests in `jobcontrollerSpec.js`