Open Juno dev

Bump preversion to 2014.2 to formally open Juno development.
The Icehouse release branch (milestone-proposed) will be cut
from the last commit before this one.

Change-Id: I24d72b9c4a7d746152150093a0e1d37be22986ad
Fixed tests failures when SKIP_ALL_TESTS_FOR_PLUGIN=True

Moved skipif statement to be called before SetUp function in tests for IDH2
and VANILLA_TWO plugins to avoid failures caused by call to
get_image_id_and_ssh_username function in SetUp function.
The fix makes test skipped instead of being failed.

Co-Authored-By: Sergey Reshetnyak <sreshetniak@mirantis.com>

Fixes bug: #1301273

Change-Id: I5e2f67b7dd48c268f62fe94374e96a61e85aff18
Added parameters to configure a list of node group processes

Added MASTER_NODE_PROCESSES and WORKER_NODE_PROCESSES parameters
to HDP plugin config. The default list of processes that will be launched
on a cluster remains as before, but an ability to configure it is added
in order to test different configurations.

Change-Id: Id100b1dbabd89f1a4bdae22f0c45639faff77077
Added rackawareness to Hadoop 2 in vanilla plugin

* Now Hadoop 2.3.0 in vanilla is able to push topology data for
  datanodes and nodemanagers
* Only 3-level topology (rackawareness) pushes to nodes. Why 4-level is
  not? Because this is not implemented in YARN yet (see YARN-18 and YARN-19):
  https://issues.apache.org/jira/browse/HADOOP-8468
* Even if admin configures 4-level topology configs Sahara will force
  down to 3-level.

Co-Authored-By: Sergey Reshetnyak <sreshetniak@mirantis.com>

Partially implements: blueprint vanilla-plugin-hadoop-2

Change-Id: If5b93dde1f6aaaf619def60634da8c8819a3c75a
Update EDP requirements for hadoop v2

Change-Id: Ib59a6fb200e0e47a3070c2509ae7e84c8ef1fd15
Partial-Implements: blueprint update-docs-icehouse
Updating the setup development environment docs for icehouse

Changes
* rewording title paragraph to be more descriptive of the document
* adding a link to devstack.org in the "... inside DevStack" section
* adding a sentence about developing on OSX to the "... inside DevStack"
section
* changing the toctree link to the OSX VM install doc to be inline
* minor grammar cleanup on section titles
* adding a header paragraph to the ui dev environ page
* rewording the "... as part of DevStack" section to be similar to dev
environ doc
* minor cleanups to ui dev environ page
* correcting sahara-dashboard location in /opt/stack

Partial-Implements: blueprint update-docs-icehouse

Change-Id: Ideed96512ee7f28280577f2520bacc620952e305
Add examples of upstream files that we should not change

Partial-Bug: #1277584
Partial-Implements: blueprint update-docs-icehouse
Change-Id: I37b25b4cec49dce9f3237953fe84f953c63eb849
Merge "Open Juno dev"
Check that all po/pot files are valid

Perform minimal checks on po and pot files so that no broken files can
get imported.

Change-Id: I24e8543006830d0b0b1781bc2746b8eac700a971
Co-Authored-By: Clark Boylan <clark.boylan@gmail.com>
Related-Bug: #1299349
Add a custom filter method to scan wrapped dict results

Basic filtering is available via Resource._filter_fields and
Resource._children but partial filtering of fields is not supported.

This change allows a Resource to define custom sanitization methods
for individual fields. The specified method is applied after the
field has been generated.

For nested structures, particularly when elements are not Resource
types, it is easier to scan the field after generation and remove
specific elements than it is to modify the recursive descent already
implemented in to_wrapped_dict().

JobExecutions are filtered with this mechanism to set swift credential
values to empty strings and to remove the 'conf' section reported
for running Oozie actions.

Closes-Bug: #1300291
Change-Id: Ifb0b99a3e13d40306139b0d0021aead69e870205
Merge "Added parameters to configure a list of node group processes"
Merge "Fixed tests failures when SKIP_ALL_TESTS_FOR_PLUGIN=True"
Fix up installation guide

 + added a new page dedicated to Sahara configuration

Partial-Implements: blueprint update-docs-icehouse
Change-Id: If9a65ab165e1ecf52f0e71d7f1dbedeea1661470
Removed impossible branch of 'if' statement

Scaling is called with the following code:

----------
    cluster = conductor.cluster_update(ctx, cluster,
                                       {"status": "Decommissioning"})
...

    cluster = conductor.cluster_update(ctx, cluster, {"status": "Scaling"})
...
    instances = INFRA.scale_cluster(cluster, node_group_id_map)
----------

Cluster can't be in decommissioning state inside scale_cluster.

Removed code looks like attempt to move cluster to Error state in case if some
nodes were removed. According to comment in Heat engine, our policy is to remove
added node, but do not add removed nodes back (need to update docs?).

So, just removing impossible branch of if statement.

Change-Id: Iafc35807c7b03b8e4a19f2b5775143a5830cee5f
Removed possibility to run job w/o Oozie

* Added a validation function which checks if cluster contains oozie
  service in its node groups
* Added unit-test covers both valid and invalid cases when cluster
  contains and doesn't oozie service in attempt of job execution
* Fixed small issue in unit tests in validation utils where pathced
  api.get_cluster function had never been unpatched.

Change-Id: I5c5de6cfb1812ff6795797df63b8ec0292395fd1
Closes-Bug: #1291649
Doc's update for integration tests

* Description of tests was changed and complemented;
* List of tests to each plugin and its version was added.

Change-Id: I8de087d745cb83dfc6da67d1bc5036cacd1b6b80
Fix big job binary objects in mysql

Fixes bug: #1302579

Change-Id: I3acc13864c41127c341875e350b4611ff7035e29
Minor fixes to Sahara UI Installation Guide

 * removed item #7 - it is not needed and is misleading
 * rephrased clarification for AUTO_ASSIGNMENT_ENABLED
 * renamed 'Nova Network' to 'Nova-Network' - seems like the later is
   the only name which is used

Partial-Implements: blueprint update-docs-icehouse
Change-Id: Ibb177820da8d44e1f70a8fd7273dadd57e81ad1e
Update links for vanilla images in doc

* Update links for vanilla 1 plugin images
* Add links to Hadoop 2 images

Change-Id: I71cf69d424c7b53082d0980479967a8f4128a167
Merge "Fix up installation guide"