libvirt: do not fail instance destroy, if mount_device is missing

In some cases, a volume does not have a mount_device field populated
in the device_mapping, especially for LXC instances. LXC instances
does not have a disk_bus mapping in get_dev_prefix_for_disk_bus()
and that results in mount_device being set to None.

Related to blueprint libvirt-start-lxc-from-block-devices

Change-Id: Idbd4ced0337c64003af08ac0497008878fb1c2e1
VMware: Remove unused variable in test_configdrive

TrivialFix

Change-Id: I3b4c55f22e515d9a66311e4c608febf617babc5a
Make strutils.mask_password more secure

Make some enhancements to strutils.mask_password to allow it to
catch more cases of passwords in strings.

The following is a listing of patterns that will be handled. The
keyword that mask_password uses (a list of four now) is represented
by <key> and the password is shown as <password>. Quotes (both single
and double) are represented as <quote>.

--<key> <password>
--<key> <quote><password><quote>
<key> = <password>
<key> = <quote><password><quote>

All existing tests and patterns are still handled.

Cherry-pick from review.openstack.org/113407
Partial-Bug: #1345233

Change-Id: I693baaf225c5b393401d499d14f271cadb63eb2c
Mask passwords in exceptions and error messages

When a ProcessExecutionError is thrown by processutils.execute(), the
exception may contain information such as password. Upstream
applications that just log the message (as several appear to do) could
inadvertently expose these passwords to a user with read access to the
log files. It is therefore considered prudent to invoke
strutils.mask_password() on the command, stdout and stderr in the
exception.

Cherry-pick from review.openstack.org/109417
Partial-Bug: #1343604

Change-Id: I9741dcdebdb4bed295ddc5ec4c4b117fffcfe88c
Make neutronapi use NetworkRequest for allocate_for_instance()

This makes the neutronapi code use the NetworkRequest object for
the allocate_for_instance() operation. It also prepares validate_networks()
by using the object internally, which will make it easier when we
actually start passing that from the caller.

Change-Id: I71c85ed4124e1e00b8c4c6d8dc02f2f280d69cfa
Change v3 quota-sets API to v2.1

This patch changes v3 quota-sets API to v2.1 and makes v2 unit tests
share between v2 and v2.1.

The differences between v2 and v3 are described on the wiki page
https://wiki.openstack.org/wiki/NovaAPIv2tov3 .

Partially implements blueprint v2-on-v3-api

Change-Id: I96377c2c5c02e00cb70eadde225f3e17d0e0025c
Add support for ipv6 nameservers

Update interfaces.template to support ipv6 based dns servers. Make
sure we validate the version of the ip addresses before we
fill them in.

Closes-Bug: #1355777

Change-Id: I7229656bfa5ff5a29c63befc3b9ce91c46e04723
Add user namespace support for libvirt-lxc

This change adds two config values that allow deployers
to enable user namespacing for libvirt-lxc containers.
User namespacing allows the mapping of uids and gids in
a container to separate uids and gids outside of the
container. The main advantage of this is to allow a
container's processes to appear to be running as root,
while actually running as a different user on the host.

Implements: bp libvirt-lxc-user-namespaces

DocImpact: This adds two config options to the libvirt
group, uid_maps and gid_maps. Default values are empty
strings and will leave user namespacing disabled for
libvirt-lxc containers.

Change-Id: I5f517f228b477f8238eccbb604b9ffd0ebaa5927
Improve Ironic driver performance: patch 6

As part of some large scale operational testing, some needed
performance improvements in the ironic driver have been
identified.  These changes have been reviewed and have been
merged into the ironic tree previously (reviews 108545 & 113036)

Change-Id: I635b974353a9dee542739857aa9778bf1e78cca8
Co-authored-by: Jim Rollenhagen <jim@jimrollenhagen.com>
Co-authored-by: Chris Behrens <cbehrens@codestud.com>
Ironic driver backports: patch 7

The patch to remove the dependency on python-ironicclient was
misplaced in the split of 103167 into the series of smaller
patches that we're now reviewing.  This patch reinstates this
change so ironicclient is loaded dynamically, and does not
become a requirement for Nova when not using Ironic.

Change-Id: I70a2fa8fc45e18524ea4d777d72da24d5d329c25
Co-authored-by: Devananda van der Veen <devananda.vdv@gmail.com>
Adds ephemeral storage encryption for LVM back-end images

This patch adds ephemeral storage encryption for LVM back-end instances.
Encryption is implemented by passing all data written to and read from
the logical volumes through a dm-crypt layer. Most instance operations
such as pause/continue, suspend/resume, reboot, etc. are supported.
Snapshots are also supported but are not encrypted at present. VM rescue
and migration are not supported at present.

The proposed code provides data-at-rest security for all ephemeral
storage disks, preventing access to data while an instance is
shut down, or in case the compute host is shut down while an instance is
running.

Options controlling the encryption state, cipher and key size are
specified in the "ephemeral_storage_encryption" options group. The boolean
"enabled" option turns encryption on and off and the "cipher" and "key_size"
options specify the cipher and key size, respectively.

Note: depends on cryptsetup being installed.

Implements: blueprint lvm-ephemeral-storage-encryption
Change-Id: I871af4018f99ddfcc8408708bdaaf480088ac477
DocImpact
SecurityImpact
maint: various spelling fixes

Fix typos, identified mainly with misspell-check

TrivialFix

Change-Id: Icb747f9982ce1551042b832e4823c480a5492928
VMware: Remove get_copy_virtual_disk_spec from vmops and vm_util

VMwareVMOps.get_copy_virtual_disk_spec() became a no-op with the
removal of VMwareVCVMOps. This change removes it, and consequently
removes the copy_spec argument to vm_util.copy_virtual_disk, as it was
only ever being passed None.

We also remove vm_util.get_copy_virtual_disk_spec(), which is no
longer called at all after the removal of VMwareVCVMOps.

Change-Id: Ibdc6127143c39b084ea956950503af842a72dc24
Sync oslo lockutils to nova

The commit to be merged:

942e1aa Use file locks by default again
ac995be Fix E126 pep8 errors
15b8352 Remove oslo.log from lockutils

942e1aa is hopefully the fix for the "n-cpu stop about 15 minutes 
before the rest of the test run" problem at the gate. We need to 
make sure this actually fixes the gate problem

Closes-Bug: #1349452

Change-Id: I31265d4e08245879ad72d3e91dcd8be2359ca811
Add amd64 to arch.canonicalize()

Allows mapping 'amd64' to 'x86_64'.

Change-Id: I0b8a527bacc2d3b4308a1eb2ce3b6d1efd71e487
Closes-Bug: 1362699
Merge "libvirt: do not fail instance destroy, if mount_device is missing"
Merge "Improve Ironic driver performance: patch 6"
Merge "Ironic driver backports: patch 7"
Merge "Change v3 quota-sets API to v2.1"
Merge "Make neutronapi use NetworkRequest for allocate_for_instance()"