settings.yml generation script

- run.sh: fix parsing args
- settings.py: fix tempest settings generation
- Adds test to test run.sh
- run.sh: fix collect_logs: failure when REMOTE_USER is not set
- test_run tests aio-rhel7
- run.sh: properly parse args, respect TAGS and SKIP_TAGS
- Remove Foreman from the add hosts task
- (cherry picked from commit 040baa227d9a00cd5ef498a8d5d9beeb0d57f8bf)
- Change pre to post in the foreman playbook
- (cherry picked from commit 5ed1d2c91fca1c3e81378b7620d748f476ac5f6f)
- settings: do not store tags and skip tags
- Ansible is broken is tags and skip tags are stored as
- variables. Probably reserved!
- run.sh: show settings files

- settings.py rewrite using CRUD functions

  Settings class to hold and handle the settings
  KeyPath class and keypaths (strings of keys separated by a dot)
  All configuration settings can be now addressed using a keypath.

  Order of the operations and defaults:
  job defaults are loaded from build/defaults.yml and tempest/defaults.yml
  The build parameters are loaded from
  - job name (is split into basic components and assigned to config
    variables)
  - site/<site>.yml
  - build/<variant>.yml
  - tempest/<testsuite>.yml

  A parameter can be overridden using four value sources, in order of
  ascending priority:
  - existent setting addressed via keypath
  - environment variable
  - cli command argument (when available)
  - generic CRUD function (--{update,set,delete,create}-variable)

  More than one generic CRUD function command line argument is allowed,

  CAVEATS
  - keys in keypaths cannot contain dots ".". When a dot is needed, a colon
  should be used instead (Ex "6:5" instead of "6.5")
  - numeric keys in keypaths during "create,set" are interpreted as list
  index and a list will be created

- improved documentation
- Adds commented sample settings variable source files
- fixes principal parameters from job-name
- fixes get_values functions
- fixes tempest branch
- improved logging
- new settings generation is optional and must be activated with
  --use-python-generator
- fixed generic variable handling
- prevent password from showing when printing settings file
- execute foreman jobs

Change-Id: I79f22a41c0d5462ad6fc8efff9193fa7a692d9a3
Add snapshot function and its conditional execution

Change-Id: Ib35b11d0ce26678a09b7b3c17360345c85ab763e
Change logic of ruby SCL repo enablement

If the product is RHOS and the release is Havana, do not enable the Ruby
SCL repo. If it is anything else, we need the repo. Also, only enable if
the distribution is RHEL.

Change-Id: I4387d8e3b6813f08af3212c9e7334fc4a2cd51ca
echo selinux denials out to the console after a run

* use tag provision, we should always check for selinux denials
* using tags is not ideal here

Change-Id: I20e81f78b3d87c3e11045647fa37dafb98c5b5b1
Add workaround for glance utf8 issue in foreman installs

Change-Id: I6597b07706edb41e0031f2ef78083570b7ba8c61
Remove python 2.6 code path as this is no longer supported

Change-Id: I144a774a83c779dcbc065085f03c31ff85862841
Funky workaround to ignore syslinux for yum update

Use a little trickery to exclude syslinux* from yum update, by injecting
into the yum update, then notifying the real yum update handler.

Change-Id: I644ade7e6b346996005577db84c8d58548e61e89
settings.py: wrong data fetched for productrelease

Change-Id: Ia7a89e0cee7b9037c41dbb97013727964d616c6c
Make default yum_excludes be empty

Yum excludes should only be done on RHEL 6 at the moment. This makes it
so.

Change-Id: I5a38be541ead6f20f34bfece52f08b1406c92f6d
Revert "Make default yum_excludes be empty"

This reverts commit cd0651f28bd06fe01f8784cdda394cf30a671b27.

Change-Id: I40f56d0cd1aea3cf7fc621e6a75e3dec78bbf759
Revert "Funky workaround to ignore syslinux for yum update"

This reverts commit b905d442eec5ac64e250eeb368b98524380419d4.

Change-Id: I20446edd989415428b349985a43c0cd44a2337f7
Tempest POC: add internal RedHat tempest setup

We need to compare the tempest results of our internal tool to the new tempest
configuration script, this commit makes it possible - but should be reverted
soon.

Added a separate 'tempest_rhosqe' role that duplicates, but simplifies a
lot of code that is in the 'tempest' role. This is because there were
problems merging it into the 'tempest' role - it conflicts with
setup-openstack-elements, the tempest configuration dictionary gets overwritten
in multiple configuration files (even in those outside of this repository) and
tempest_run is hard-coded to only use smoke tests. All of this could have been
fixed, but it would take time - this is a quick hack. It is just a POC and
will be removed soon anyway.

It uses the tag "tempest_rhosqe" instead of "tempest_setup,tempest_run" and the
variable "tempest_rhosqe" instead of "tempest", so that the values don't get
overwritten in configuration scripts.

It will run on the controller node and not on the extra tempest node, since
this is what the tools require.

Change-Id: I634f690c3e4a734a092b28cec1d405d6e86c4faf
Fix the new rhosqe tempest role so it does not interfere with Foreman

Change-Id: Ibbad4376fabf239f86d6eb5ca458137bb8762457
Tempest image: handling difference between icehouse and havana image

Change-Id: Iec917f091c503006c7659f705c2de452519ee36d
Setting.py: tempest remote user handling

Change-Id: I5894c0827d33e0c0a7a8f4ec34a5a14567df8026
Add selinux check role to foreman playbook

Change-Id: I599c23a8c2e138cdfd0e1497a41d9576f83d4755
Remove 'include passenger' line from Foreman Havana installer

Change-Id: I1b25b952ff362dc4aefcc2bb904fb9b851f02400
Simple playbook for Sahara and its dashboard

Right now there are no working puppet modules for Sahara and its
dashboard; the playbook follows the installation instructions
to configure the component.
It assumes that Sahara is installed on the controller node,
and the web server and mysql servers are running there too.

Change-Id: If85cafb6adfdf6572cca77afdbc7c62e10cb7a9e
tempest: setup tempest using package

To turn on this method, set
- tempest.setup_method to tempest/rpm
- tempest.use_virtual_env to false
- tempest.checkout_dir to the dir where tempest will be installed
   /usr/share/openstack-tempest

Change-Id: I37235e89b70265dc61b6d5acaf78ddb5e197272f
logs: set sudo to no for localhosts

Change-Id: Iba6995fa0d90c0a7cc6a72c7ff17594fc81bc93d
(cherry picked from commit 3721c92b8e94b1430662d9e7032b0b6ed318da9f)