Add a Spark job type for EDP

* Added support for JOB_TYPE_SPARK
* Modifed create_workflow_dir to not use job_workflow_postfix (only for hdfs)
* Rewrote unit tests in test_job.py to be oriented around
  validation behavior for classes of job types rather than
  features of validation, and included JOB_TYPE_SPARK.
* Moved test_job_executor_java.py into test_job_executor and added JOB_TYPE_SPARK
* Added tests for create_workflow_dir and upload_job_files

Partial-implements: blueprint edp-spark-job-type
Change-Id: Ifd91123afea9e921ac441751a37aa6afae0bbd66
Fixed a ValueError on provisioning cluster

If the machine is slow, the remote command may output an empty string.
Converting an empty string to an integer results a ValueError, which
caused a failure. This patch attempted to fix the error by adding a
check for the outputted string before converting to integer.

Closes-Bug: 1351624
Change-Id: I38aadaa8c6b3ae9c7f93f90d5144ac8de7aa54d4
Group tests by class

It guarantees that all tests inside the test class will be executed in
sigle thread. It makes us able to group some tests that uses same
external resource into one tests execution thread to avoid races.
Example: tests that are use external MySQL/PostgreSQL.

Change-Id: I67f10d20ea84caa9335f909fb5f30da4b4e7f1ca
Add test for storing data in DB for 007 migration

Change-Id: Iefd73e8ac1899b30555b6b2427720f1fbe62b5c8
Fix closing HTTP session in Ambari plugin

Change-Id: I4028ff177766c0d6e46d1308b774250375039b10
Closes-bug: #1350388
Fixed cluster rollback on scaling with heat engine

We can't allow stack to move into Error state if we want to use it futher.
disable_rollback parameter controlls such behavior. It is Ok to have errored
stack for cluster create and rollback since we will not be using it anyway. But
for scaling we need to keep it Active since we can just rollback in case of
error.

Change-Id: Iab1a58f4924ca6a7689060f71217956a72316517
Closes-Bug: #1337006
Added ability to specify security group for node group

Extended API to support list of security groups in node groups and
node group templates. Changed db schema and provisioning engines (both direct
and heat) accordingly.

Partially implements: blueprint cluster-secgroups

Change-Id: I49d3551f6987552ff6398e8c87e22388f914f287
Merge "Fixed a ValueError on provisioning cluster"
Removed a duplicate directive

Change-Id: I72d7c86cd04bbc4a66d8ddf3f108713d21a2fe51
Imported Translations from Transifex

Change-Id: I9027de9d387b4962fff674070b0749e60e5f098e
Merge "Removed a duplicate directive"
Merge "Add test for storing data in DB for 007 migration"
Move middleware package to api package

All middlewares could be used only for api stuff, so, moving them to the
api package.

Change-Id: I1303f4b3c3bd2a8b4a883d86ba4f91aba92496e0
Updated from global requirements

It's done manually to prevent upgrading to alpha versions that'll
require updating conf sample file.

Change-Id: I914b9d24702fec12856f90d3c6806f40691453d3
Update oslo.config to the alpha/juno version

Change-Id: I3849baae371c1661a5b5c42ed292f8096a411af6
Update oslo.messaging to alpha/juno version

It should be done manually to update sample conf.

Change-Id: I39935c710af1864892d1b6b288c0163da2f05166
Merge "Group tests by class"
Merge "Updated from global requirements"
Merge "Update oslo.config to the alpha/juno version"
Merge "Update oslo.messaging to alpha/juno version"