XD-1139 FTP to HDFS Partitioned Batch Job

- Partitioner
- Tasklet
- Job Configuration (FTP)

XD-1139 Polishing; Add Test Case

* Update to SI 4.0.0.BUILD-SNAPSHOT - fix package for MessageHandlingException.
* Use a new get() method on RemoteFileTemplate to avoid having to build a Message.
* Add a test case to JobCommandTests (Ignored for now - needs hadoop).
* Make DAOs available in all contexts.

XD-1139 Polishing

- Use Hadoop rule in tests
- Add module options

XD-1139 Polishng; PR Comments

* Fix default port and option description violations
* Enforce trailing / on remoteDirectory
* Add 'stepConcurrency' parameter
* Fix RemoteFileToTmpTests
XD-1229 + XD-1230 Create build infrastructure for AngularJS based UI

The Grunt-based build infrastructure is based on Yeoman's AngularJS support and then further customized. As part of `XD-1230`, this build process is then further integrated into the larger Spring XD Gradle build. Please see `README.md` for further details.

* Add AngularUI Router dependency
* Add UI-Router routes
* Add LESS support + UI Admin Styles
* Instead of customizing Bootstrap's `variables.less`, we instead import its styles into `main.less` and customize only the variables that changed. That way we can easily upgrade Bootstrap in the future
* Fix a `grunt-usemin` issue with images in CSS files
  - Use fix as detailed here: https://github.com/yeoman/yeoman/issues/824#issuecomment-29137057
  - See also issue: https://github.com/yeoman/grunt-usemin/issues/242
* Exclude vendor CSS, we already import the Bootstrap CSS via `main.less`
* Don't use port 8080 for Karma tests in order to minimize conflicts with Tomcat
* Add "Loading Indicator" support using `angular-busy`
* Add Job launch support + Growl support + define XD Server URL globally
* Use Bootstrap fonts from the distribution
* Improve Job launch support
* Update `.gitignore`
* Update Nodejs dependencies, fix test dependencies
  - Running `npm install` should install all relevant dependencies from `package.json`
* Add Gradle build support
* Import CSS files as LESS resources in `main.less`
  - Otherwise you run into minification issues as Grunt's nsmin does not consider those CSS files
* Remove old `admin-ui`
* Improve resolving of the admin-ui URLs
  - http://localhost:9393/admin-ui --> http://localhost:9393/admin-ui/#/jobs/definitions
  - http://localhost:9393/admin-ui/ --> http://localhost:9393/admin-ui/#/jobs/definitions
* Add task `cleanDist`
* Improve Gradle build - Depend on `spring-xd-dirt:backgroundAdminServer`
* Fix `xd.ui.home` in `application.yml`
  - `xd.ui.home` in `application.yml` should contain a trailing slash
* Add `grunt-bower-task` to Grunt build
  - Allow Bower to be called via Grunt
* Test that the UI builds with `gradle npmInstall installGrunt build`
* Update Node dependency versions
* Fix "Required config property "bower-install.app.src" missing"
  - Bower Install - In `Gruntfile.js`, change config `html: '<%= yeoman.app %>/index.html',` to `src: '<%= yeoman.app %>/index.html',`
* Add Proxying to XD Admin Server via Grunt build
  - Add https://github.com/drewzboto/grunt-connect-proxy
* Better error messages when pulling Job Definitions
* Add `.bower_cache` directory local to project dir
* Add Karma shell scripts
  - I find these shell scripts quite useful when writing unit and e2e tests as you can configure Karma to keep running as a daemon (automatically re-running the tests of code-changes do occur)
* Add Grunt task `testfiles` - useful to prepare Spring XD test files
* Add Karma Safari launcher
* Add Unit and E2E tests
* Remove unnecessary files
  - .gitattributes
  - app/.buildignore
  - app/robots.txt
* Remove old loading Gif `loading.gif`
  - Remove unused related CSS
* Ensure we use only single quotes in JS files
* Improve Favicon support including mobile
* Remove main.css + main.css.map (autogenerated)
* Make sure LESS creates CSS files when executing `grunt serve`
* Update `.gitignore`
* Change Grunt Server Port to 8000
  - Port 9000 conflicts with Sonar
* Improve Gradle Build - `gradle build` will:
  - Install (if needed) node
  - Install (if needed) grunt
  - Download node dependencies for XD UI
  - Run the Grunt build
* `gradle clean` will remove all unnecessary spring-xd-ui folders (e.g. local node modules, bower files etc)
* Run Spring XD UI Grunt build on Eclipse STS import
  - Add task `afterEclipseImport`
  - See for details: https://issuetracker.springsource.com/browse/STS-2059
* Add `README.md`
* Fix live-reloading of Html files
* Fix copying of images in `Gruntfile.js`

XD-1229 Code Review changes

* Remove `.htaccess`
* Run E2E as part of the Gradle test task
* For distribution, copy Admin UI to `spring-xd-ui/dist`
Polishing the UI tasks

- Make spring-xd-ui project independent and not run the build
  as part of 'gradle build'
- Added tasks 'setupUI' and 'cleanUI' that takes care of building/cleaning
  the UI project
- Removed an invalid URL from index.html
XD-1084 Create REST API for getting information on a given step execution

Jira: https://jira.springsource.org/browse/XD-1084
Fix JVM startup error for build on Windows
XD-1285: Command completion for module names
Remove tests related to substreams
Move a composed module related test
XD-1293 Remove JobModule beanId constraint

- Inject JobRegistry and jobName into JobLaunchRequestTransformer
- Check for FlowJob in BatchJobRegistryBeanPostProcessor as this
  is the type created by JobParserJobFactoryBean that loads the
  job module
- Avoid creating GroupAwareJob by not setting the groupName in
  JobRegistryBeanPostProcessor
- Update BatchJobsController to use actual job name instead of
  job name with suffix ".job"
- Updated OOTB job modules job bean id with the name that identifies
  them
XD-1093 Add shell command to display info on step execution

Jira: https://jira.springsource.org/browse/XD-1093

XD-1093 Remove display of Job Name - Not delivered via Rest Call
Fix commented out test
Merge remote-tracking branch 'gary/XD-1139'

- Merging gary's pr.
- Small conflict in build.gradle which was just caused by
  new modules.job.ftphdfs project.
Remove unintentional System.out
XD-874 Support for deleting job files after import

- filejdbc and filepollhdfs now support the 'deleteFiles' option. If set
  the job will delete the file(s) on successful completion.
- The class 'FileDeletionJobExecutionListener' is responsible for doing
  the work.
- script tests for both modules have been added which check files are
  actually deleted in practice.
Fix integration scripts test

- One of the tests in hdfs_import_export_tests used `xargs` to get
the input from `ls` and subsequently `wc` to count the number of files
in a directory. In case when this kind of check is used to assert when
there are no files, the assertion fails in *nix environment.

This is because in *nix the `xargs` runs even if there are no input and
generates a blank line and `wc` subsequently counts this incorrectly.
To avoid this from happening, `xargs` can be run with `--no-run-if-empty`
option but this is only available in GNU extensions and fails in Mac.

One possible fix is to remove xargs and use `ls -1` to output the results
in single column and use `wc` to count.
XD-1298: Fix System.in.read() following JLine upgrade
Fix tests and compiler warnings

- Added an additional check to make sure the testMessageBus doesn't refer to a different
messageBus set by previous test that also uses AbstractSinglenode test.
For instance, if RabbitSingle node tests run before
RedisSingle node tests and Rabbit wasn't available, the Redis tests still use
Rabbit message bus that was set by the AbstractSingleNode test class.
Since the TestSupportResource is evaluated after the testmessagebus
is set, adding this check is needed.

- Fixed compiler warnings
XD-1087 Create REST API for restarting a specific job instance

* Add tests
* Add rudimentary UI support

Jira: https://jira.springsource.org/browse/XD-1087
Upgrade to Spring HATEOAS 0.9
XD-1305 File Sink now supports configurable modes

Originally the file sink only supported the append mode.
Now supports APPEND, REPLACE, FAIL IGNORE