Make BDM dict __init__ behave more like a dict

Currently we can't pass values as kwargs to BlockDeviceDict classes.
This is a common way to call a dict constructor so it breaks the dict
abstraction a bit.

This patch fixes it in a backward compatible way.

Change-Id: I31117eecba4f060cb9c7fc3d329f70fd369b168d
Merge "Avoid possible timing attack in metadata api"
Merge "Make sure that metadata handler uses constant_time_compare()"
VMware: Consolidate fake_session in test_(vm|ds)_util

Both test_vm_util and test_ds_util contained a fake_session class.
The implementation of test_ds_util.fake_session was irrelevant to
almost all of its users, which was confusing.

fake_session from these 2 modules is consolidated into fake. We now
have fake.FakeSession(), which expects _call_method and _wait_for_task
to be mocked, and fake.FakeObjectRetrievalSession(), which is a
utility object for faking the return of an object list.

The implementation details of test_ds_util.fake_session have been
pushed into test_file_exists and test_file_exists_fails, where they
belong.

Change-Id: I168af46d28629c8b2cf8adae856f0f66ae3ee922
VMware: consolidate datastore code

This is largely a straight-forward code movement.
Given that we have utility classes/code in ds_util.py, it makes sense
to move the remaining datastore-related code in vm_util.py to ds_util.py
as well.

partial blueprint vmware-spawn-refactor
Change-Id: I2ede44306acf91a13f2ff6f0d8f210642a1c1e69
VMware: DatastorePath join() and __eq__()

There are quite a number of use cases where we construct additional
datastore paths out of one representing some ancestor directory,
so implemented a join() method that allows for just that.
It is useful to define equality operator when comparing two DatstorePath
objects, so added that as well.

partial blueprint vmware-spawn-refactor
Change-Id: Iec44fdd0beba232b57bcd0a07c39e628208ca9b2
VMware: use datastore classes get_allowed_datastores/_sub_folder

Convert the above-mentioned functions to use the datastore utility
classes.

partial blueprint vmware-spawn-refactor
Change-Id: I20c37367c8dca1c8dc20a308ce2cea5229097701
VMware: use datastore classes in file_move/delete/exists, mkdir

Straightforward conversion of a few functions in ds_util to
accepts DatastorePath objects instead of literal strings.

partial blueprint vmware-spawn-refactor
Change-Id: I9dff390ce020dcc39a05184a7ee69902e3412c9f
VMware: Trivial indentation cleanups in vmops

This change has been split out of
https://review.openstack.org/#/c/87002/, which was written by Shawn
Hartsock.

TrivialFix

Co-authored-by: Shawn Hartsock <hartsocks@acm.org>
Change-Id: I6c3ee19b019668a7509ee9835a53271f98805f8f
VMware: Convert vmops to use instance as an object

This is mostly mechanical, but it does expose what appears to be a bug
in the interaction between object.Instance's config_drive field and
configdrive.required_by(). object.Instance defines config_drive to be
a String, but configdrive.required_by() expects it to be a Boolean. In
assigning a boolean value to config_drive, it is cast to a String,
which evaluates as True when cast back to a Boolean. We therefore
update the tests which use config_drive to set either a String value,
or None, which is what other code does.

Note that Nova does not pass an Instance object to all driver methods,
so not all uses can be updated.

Change-Id: Ifc9ce5d96da30abfa09b608a9c8615ede1485a50
VMware: Remove unnecessary deepcopy()s in test_configdrive

test_configdrive was doing a deepcopy of its test_instance object in
some tests. This was unnecessary, as the object is created fresh in
setUp().

Also, change a couple of dict lookups of test_instance to object
properties.

Change-Id: I3a5039fde9f8fba5285d90b28743b7af8cd1a168
Merge "libvirt: fix recent test changes to work on libvirt < 0.9.13"
Partial oslo-incubator sync

Generated with:
python update.py --base nova --dest-dir ~/Develop/tmp/nova/ --modules
timeutils,sslutils,importutils,memorycache,strutils,context,fileutils,versionutils,systemd

context:
 9b73877 Add a RequestContext.from_dict method
 85d1ce6 Python 3: enable tests/unit/middleware/test_request_id.py
fileutils:
  2b966f9 Fix deletion of cached file for policy enforcer
gettextutils:
 3d90045 Backport code for i18n to check lazy at runtime
jsonutils:
 ef37e03 Added missing jsonutils.dump() function
log:
 5cac11d Merge "Add default log level for websocket"
 433fa0b Make logging_context_format_string optional in log.set_defaults
 ac92c06 Add default log level for websocket
 0aa2bd4 Merge "Ability to customize default_log_levels for each project"
 5fd77eb Ability to customize default_log_levels for each project
 i4d9328c Python 3: enable tests/unit/test_log.py
 722f418 Merge "update new requests logger to default WARN"
 cb5a804 Move `mask_password` to strutils
memorycache:
 90ae24b Remove redundant default=None for config options
 297d772 Raise exception when importing memcache error
strutils:
 cb5a804 Move `mask_password` to strutils
systemd:
 17c4e21 Fix docstring indentation in systemd
 667d1ba Fixed spelling error - occured to occurred
versionutils:
 a2ad3a2 Allow deprecated decorator to specify no plan for removal
 05ae498 Add JUNO as a target to versionutils module

Change-Id: I9e8e8e9b15075d99cf394170c6f5300e7bd0d4cc
Merge "Defer raising an exception when deleting volumes"
libvirt: speed up _get_disk_over_committed_size_total method

Switch the _get_disk_over_committed_size_total method to
use the new list_instance_domains method.

Blueprint: libvirt-domain-listing-speedup
Change-Id: I4ef00ea5584eda14c4781d3fbf8bb737f3abdc90
libvirt: remove unused list_instance_ids method

All code has been converted to use list_instance_domains so the
list_instance_ids method can finally be removed.

Blueprint: libvirt-domain-listing-speedup
Change-Id: I9ce33cda5608796e0eec4ec0804ee179a4704d9d
libvirt: remove flawed get_num_instances method impl

The get_num_instances method impl simply asks libvirt
how many running domains there are. This is flawed for
a number of reasons.

 - It doesn't filter out 'Domain-0' on Xen
 - It doesn't count inactive instances

As a result the value is returns is inconsistent with
the number of instances shown by 'list_instances'.
This will cause the _sync_power_states method in the
compute manager to constantly try to re-sync the list
of instances.

Remove the flawed impl and just allow the base class
default impl to be used. This just does a len() call
over the return of 'list_instances'. Previously this
was quite a heavy API call, but with new enough libvirt
it can be completed in a single API call which should
be fine.

Blueprint: libvirt-domain-listing-speedup
Change-Id: I96242717afa5c928f1cfb5f307b71739f2147e69
Merge "Add missing image to instance booted from volume"
Fix Cells ImagePropertiesFilter can raise exceptions

In a misconfigured scenario the ImagePropertiesFilter can raise an
exception which causes the image to be unbuildable in the region. This
happens when the capabilities is set for prominent_hypervisor_version
to an empty list.

Change-Id: I17bd9d87fa69ad34b19591018543a34dfd050978
Closes-Bug: 1343462
Merge "Deallocate the network if rescheduling for Ironic"