XD-1139 FTP to HDFS Partitioned Batch Job

- Partitioner
- Tasklet
- Job Configuration (FTP)

XD-1139 Polishing; Add Test Case

* Update to SI 4.0.0.BUILD-SNAPSHOT - fix package for MessageHandlingException.
* Use a new get() method on RemoteFileTemplate to avoid having to build a Message.
* Add a test case to JobCommandTests (Ignored for now - needs hadoop).
* Make DAOs available in all contexts.

XD-1139 Polishing

- Use Hadoop rule in tests
- Add module options

XD-1139 Polishng; PR Comments

* Fix default port and option description violations
* Enforce trailing / on remoteDirectory
* Add 'stepConcurrency' parameter
* Fix RemoteFileToTmpTests
Remove tests related to substreams
Move a composed module related test
XD-1293 Remove JobModule beanId constraint

- Inject JobRegistry and jobName into JobLaunchRequestTransformer
- Check for FlowJob in BatchJobRegistryBeanPostProcessor as this
  is the type created by JobParserJobFactoryBean that loads the
  job module
- Avoid creating GroupAwareJob by not setting the groupName in
  JobRegistryBeanPostProcessor
- Update BatchJobsController to use actual job name instead of
  job name with suffix ".job"
- Updated OOTB job modules job bean id with the name that identifies
  them
XD-1093 Add shell command to display info on step execution

Jira: https://jira.springsource.org/browse/XD-1093

XD-1093 Remove display of Job Name - Not delivered via Rest Call
Fix commented out test
Merge remote-tracking branch 'gary/XD-1139'

- Merging gary's pr.
- Small conflict in build.gradle which was just caused by
  new modules.job.ftphdfs project.
Remove unintentional System.out
XD-874 Support for deleting job files after import

- filejdbc and filepollhdfs now support the 'deleteFiles' option. If set
  the job will delete the file(s) on successful completion.
- The class 'FileDeletionJobExecutionListener' is responsible for doing
  the work.
- script tests for both modules have been added which check files are
  actually deleted in practice.
Fix integration scripts test

- One of the tests in hdfs_import_export_tests used `xargs` to get
the input from `ls` and subsequently `wc` to count the number of files
in a directory. In case when this kind of check is used to assert when
there are no files, the assertion fails in *nix environment.

This is because in *nix the `xargs` runs even if there are no input and
generates a blank line and `wc` subsequently counts this incorrectly.
To avoid this from happening, `xargs` can be run with `--no-run-if-empty`
option but this is only available in GNU extensions and fails in Mac.

One possible fix is to remove xargs and use `ls -1` to output the results
in single column and use `wc` to count.
XD-1298: Fix System.in.read() following JLine upgrade