XD-1139 FTP to HDFS Partitioned Batch Job

- Partitioner
- Tasklet
- Job Configuration (FTP)

XD-1139 Polishing; Add Test Case

* Update to SI 4.0.0.BUILD-SNAPSHOT - fix package for MessageHandlingException.
* Use a new get() method on RemoteFileTemplate to avoid having to build a Message.
* Add a test case to JobCommandTests (Ignored for now - needs hadoop).
* Make DAOs available in all contexts.

XD-1139 Polishing

- Use Hadoop rule in tests
- Add module options

XD-1139 Polishng; PR Comments

* Fix default port and option description violations
* Enforce trailing / on remoteDirectory
* Add 'stepConcurrency' parameter
* Fix RemoteFileToTmpTests
Fix commented out test
Merge remote-tracking branch 'gary/XD-1139'

- Merging gary's pr.
- Small conflict in build.gradle which was just caused by
  new modules.job.ftphdfs project.
Remove unintentional System.out
XD-874 Support for deleting job files after import

- filejdbc and filepollhdfs now support the 'deleteFiles' option. If set
  the job will delete the file(s) on successful completion.
- The class 'FileDeletionJobExecutionListener' is responsible for doing
  the work.
- script tests for both modules have been added which check files are
  actually deleted in practice.
Fix integration scripts test

- One of the tests in hdfs_import_export_tests used `xargs` to get
the input from `ls` and subsequently `wc` to count the number of files
in a directory. In case when this kind of check is used to assert when
there are no files, the assertion fails in *nix environment.

This is because in *nix the `xargs` runs even if there are no input and
generates a blank line and `wc` subsequently counts this incorrectly.
To avoid this from happening, `xargs` can be run with `--no-run-if-empty`
option but this is only available in GNU extensions and fails in Mac.

One possible fix is to remove xargs and use `ls -1` to output the results
in single column and use `wc` to count.
XD-1298: Fix System.in.read() following JLine upgrade
Fix tests and compiler warnings

- Added an additional check to make sure the testMessageBus doesn't refer to a different
messageBus set by previous test that also uses AbstractSinglenode test.
For instance, if RabbitSingle node tests run before
RedisSingle node tests and Rabbit wasn't available, the Redis tests still use
Rabbit message bus that was set by the AbstractSingleNode test class.
Since the TestSupportResource is evaluated after the testmessagebus
is set, adding this check is needed.

- Fixed compiler warnings
XD-1087 Create REST API for restarting a specific job instance

* Add tests
* Add rudimentary UI support

Jira: https://jira.springsource.org/browse/XD-1087
Upgrade to Spring HATEOAS 0.9
XD-1305 File Sink now supports configurable modes

Originally the file sink only supported the append mode.
Now supports APPEND, REPLACE, FAIL IGNORE
XD-1294 Update xd reactor dependency

- Move the reactor Environment bean definition out of module common context
- Remove the 'spring-xd-extension-reactor' runtime dependency from spring-xd-dirt
- Add back 'spring-xd-extension-reactor' runtime dependency to reactor-syslog module

Add reactorEnv bean to syslog module

- Add the reactor environment bean to the reactor-syslog source
module so that the reactor environment is only loaded when this
module is deployed.
XD-1316 UI:Fix E2E test warning
Update xd ui test task

 Rename ui test task to 'ui_test' so that gradle test won't
  call spring-xd-ui tests. To run the E2E tests, the ui_test
  task needs to be invoked explicitly.
 The 'ui_test' task will also build the UI and runs the unit tests
  as it depends on 'grunt_build'

 Once we have valid UI tests, we will rename the task ui_test to 'test'
  so that will run as a 'test' task.
XD-804 Added integration.support package to dirt and used in AbstractSingleNode...Tests

XD-804. Refactored SingleNodeTestSupport and SingleNodeApplication per review

XD-804 Renamed package integration.test and added module resource location support

XD-804 Updated javadocs and added moduleResourceLocation arguments to processing chain creation methods

Javadoc fixes + random ports
Refactor jdbc script tests.

Extract variables for driver, url etc.
adding missing compile dependency on hibernate-validator
XD-1318 Start Container before Admin

- Moved the DAO beans back to 'adminServer' profile in batch.xml
  This will make sure these DAOs are only available in AdminServer's
  parent configuration
- Added the necessary DAO beans into singlestep-partition-support
  This will make sure these beans will only be loaded when the job
  module that requires singlestep partition support is deployed into
  the container. And, by this time the admin should be up as that's
  where the ModuleDeploymentRequest is coming from.
- Fixed the tests context files