ubuntu: simplify the kernel updates

At the moment the kernel version to be installed on Ubuntu nodes is hard
coded in config.mk (and several other files, such as requirements-deb.txt).
This version must coninside with the version of the kernel used by Debian
installer (otherwise Debian installer won't be able to load any kernel
modules which in general breaks the provisioning). As a result updating
the kernel (say, to incorporate the upstream security fixes) requires
a lot of manual intervention: downloading the netboot image, finding out
the exact kernel version it uses, updating config.mk, requirements-deb.txt,
and so on.

This patch makes the process much simpler: updating the kernel (to a version
provided by Ubuntu) boils down to changing the UBUNTU_KERNEL_FLAVOR
variable (and possilby the UBUNTU_NETBOOT_FLAVOR variable).

 * config.mk: introduce UBUNTU_KERNEL_FLAVOR and UBUNTU_NETBOOT_FLAVOR
   variables. The former specifies which kernel should be installed on
   nodes, and the latter tells which netboot variant should be used.

 * image/ubuntu/module.mk: install the kernel specified by
   the UBUNTU_KERNEL_FLAVOR variable. There's no need to use exactly
   the same version of the kernel as Debian installer does, as image
   based provisioning does not use Debian installer at all.

 * mirror/ubuntu/boot.mk: guess and write the installer kernel version
   into a file which can be include'd by other modules. While at it
   reduce the copy-paste noise.

 * mkrepo.sh: make sure that the kernel (and the correspodning headers)
   (meta-)packages specified by UBUNTU_KERNEL_FLAVOR are included into
   the mirror.

Change-Id: Ia8db615284644b0e2bcdb70c1c7bb1f4388d5e46
Related-bug: #1390044
Related-bug: #1383641
Fail Fuel Master deployment if puppet or check fails

Puppet apply for Fuel Master deployment runs with
--detailed-exitcodes in order to capture any failures
and fails if any resource fails to apply, including
docker containers.
Cleaned up some obsolete lines about hiera and astute.yaml.

Fixed automated environment deployment scripts to detect
failed deployments and fail as well.

Change-Id: I29238a27f1335d63a6c82274a6dab9b25e1ecf0a
Related-Bug: #1390129
mirror-ubuntu: use the OSCI repo for bootstrapping the chroot

If some of the base packages (like python-minimal) has been rebuilt
we should use them when bootstrapping the temporary chroot. Otherwise
the newly created apt repo will contain either only upstream version of
the package in question, or both the upstream and customized versions.

Assume that the base package X has been customized. The customized version
of X is located on the OSCI mirror. Multistrap does not use that mirror, so
the upstream version of X gets pulled into the apt cache (typically
/var/apt/cache/archives) during bootstrapping of the chroot. Packages
depending on X pull its customized version (since the OSCI repo is added
to the sources.list.d after bootstrapping the chroot). If X is an essential
package (so other packages are not obliged to declare dependence on X)
only the upstream version of X gets into the apt repo being created.

Use the OSCI mirror when bootstrapping the chroot to avoid the problem.

While at it reduce copy-paste noise in createchroot.mk and make it a little
bit less obscure.

Related-bug: #1390416
Related-bug: #1390044
Related-bug: #1383641

Change-Id: Ic418c760a1635d5c72cc108658d412228fb83a62
Upgrade uses virtualenv

Related-Bug: #1341564

Change-Id: Ie93034ee209234bf811ceaa622bc599aac3bc3bb
Fix output of unmet dependences, add support for 'udeb' packages.

- When debian package manager selected to install older version
  of a package, but newer version requested by other package,
  then output with error is slightly different then was expected:
  Depends: package-name (= v2.0) but (v1.0) is to be installed

- Add support for 'udeb' packages for debian-installer.
  Now *.udeb packages can be placed to the folder
  /var/www/nailgun/ubuntu/x86_64/pool/debian-installer , repository
  can be updated with the sctipt 'regenerate_ubuntu_repo' with the
  same way as for *.deb packages.
  Also 'package_testing_deb' now can be used for testing gerrit
  reviews that contain any of *.deb or *.udeb packages.

Change-Id: Ib11546f7aa75e23e7f879170416c7183ff519df7
Closes-Bug: #1391569
Merge "Fix output of unmet dependences, add support for 'udeb' packages."
Merge "Upgrade uses virtualenv"
Merge "mirror-ubuntu: use the OSCI repo for bootstrapping the chroot"
Add regular expression to search for
mysql in pacemaker config

- Format of mysql resource was changed in pacemaker
configuration file. Also it's going to be master/slave
in the nearest future. To cover this case regular
expression was added

Change-Id: I7c5eb28fbfdf364b56bc2135af6317533b3faff4
Closes-Bug: #1391093
Merge "Add regular expression to search for mysql in pacemaker config"
Merge "Fail Fuel Master deployment if puppet or check fails"
packages: make it possible to (re)build only Debian/RPM packages

- (re)building Debian packages does not create RPMs mirror any more.
- Building Debian packages can be suppressed by setting the
  BUILD_DEB_PACKAGES variable to zero.

* packages/rpm/module.mk:
  - make sure sources are prepared before building a deb package, i.e. add
    an explicit dependency.

* packages/deb/module.mk:
  - building Debian packages does not require RPMs mirror, adjust
    the dependencies accordingly.
  - don't build packages if BUILD_DEB_PACKAGES is zero.

Related-bug: #1390416
Change-Id: I049342e7619bd3be80bc2e67c89ad77618b67b59
Merge "ubuntu: simplify the kernel updates"
Merge "packages: make it possible to (re)build only Debian/RPM packages"
packages/deb: build packages using the correct mirror (LP #1390416)

As of now Fuel related deb packages are built using the upstream mirror.
This yeilds inconsistent dependencies, for instance, packages depend on
both the upstream and the customized python versions, and provisioning of
Ubuntu node fails as a result.

To resolve the problem setup the build chroot using the official Fuel apt
repo. While at it use the locally created apt mirror instead of downloading
packages once more, and fix a number of inconsistencies/bugs.

* packages/deb/sandbox-ubuntu.sh:
  - new file, creates a minimal buildd-alike chroot from the local apt repo.

* packages/deb/module.mk:
  - use the helper script to create build sandbox.

* requirements-deb.txt:
  - make sure tools required for building Debian packages are included
    into the local apt repo.

Change-Id: I9f7b06bba94a65b2d65168cc1122257f4bec78be
Related-bug: #1390416