Enforce metadata string type on key/value pairs

The compute API is currently enforcing quota limits for user metadata
and length restrictions on the dict entries, but is not enforcing that
they are string types. This patch adds the checks for string type on the
keys and values in the metadata dict.

Closes-Bug: #1240319

Change-Id: Ic4aacc342cf37d929871287e6e4917f4f48b63e8
Remove useless api sample template files for flavor-rxtx v3

flavor-rxtx template files flavor-get-resp.* and flavors-detail-resp.*
isn't used by any api sample test. So remove them.

Change-Id: I74760bcd5367487aca31cdf94fa5e5715f19f4c6
Add decorator expected_errors for flavors_extraspecs v3

This patch add decorator expected_errors and also add catching
for FlavorNotFound.

Partially implements bp nova-v3-api

DocImpact

Change-Id: I56556871f649ba56211d3f523a184e3171dd3b7e
Remove unused flag 'host_state_interval'

Patch I749204d7d6461993d0c42bf5877b65c5ec17c62f removed periodic task
`_report_driver_status()`, which fanout this host's capabilities to
nova scheduler. So the flag 'host_state_interval' is not used.
Just remove it in this patch.

DocImpact: the 'host_state_interval' flag has been removed. It was last used in
grizzly and was not used in havana.

Change-Id: Ieb047ccb33cdecc6f25b6ae128dcac4e1c00cd98
Periodic task _heal_instance_info_cache can now use slave db

This periodic task will now use a slave db for all reads if a slave
is configured. For deployments with no slave, behavior remains unchanged.

Change-Id: Iafda7ff7a895b21a75c6d3923269f45bc84c81b0
Partially-implements: blueprint periodic-tasks-to-db-slave
LOG.warn() and LOG.error() should support translation

Based on a search done in nova code,
some logs have to be changed in order to support translation

Change-Id: I371b714d004bcd0091ae569d4b82ae8d19fba708
Closes-Bug: #1235088
Extends V3 servers api for pci support

All pci passthrough code is merged into nova, but the API does not support.
this patch extends the servers to show PCI information for instance and compute
node. This is a part of PCI APIs, for more information see the  blueprint.

Partially implement blueprint pci-api-support

Change-Id: I99eeeca67983c49433ab0079f490a454d250f5e0
Add shutdown option to cleanup running periodic

Add a 'shutdown' option to the periodic task that cleans up instances
which are both deleted *and* running.

The option powers down instances and marks them as non-bootable.  This
is a flag for system administrators who would like to reclaim the
resources consumed by each instance, but leave the records around for
later examination/debugging.

DocImpact
Implements: blueprint cleanup-running-deleted-shutdown

Change-Id: Id95845eb995b62ed8858fe61c6fdb34045721674
libvirt: add a test to guard against set_host_enabled raising an error

Commit cc70e3a adds a finally block to _get_new_connection to
automatically disable the host if making a connection to libvirt fails.

The set_host_enabled method currently swallows all exceptions which is
good so it doesn't mask the original libvirt failure in
_get_new_connection, but we need to have a test that makes sure that
isn't unintentionally regressed. This patch adds the test.

Related-Bug: #1257644

Change-Id: I4beec053794edc75ef2d0060a05285f7ed0b99ac
Make obj_from_primitive() preserve version information

When we hydrate an object from a remote client, we need to know
the version and not pretend like it's just the current version.

Related-Bug: #1258256
Change-Id: I68bbfa18addfb7e51efc10a757135e338302a787
Hide injected_file related quotas for V3 API

The injected_files, injected_file_content_bytes and
injected_file_path_bytes are all related to the
os-personalities extension which has been removed. They
can't be removed from the core quotas code as that has
to still support the V2 API. So for the V3 API we just
hide them.

DocImpact

Partially implements blueprint v3-api-remove-extensions

Change-Id: Iedb6e1427ac22f1926e929f3391e56403c37d088
Remove the api_thread_pool option from libvirt driver

This gave us the option to select if calls to libvirtd are done
in a native thread or directly. This was added because the person
who wrote the thread pool code was afraid of regressions for existing
deployments. But now we have enough features in nova that require
that this is always set to True or else users can have issues with
them. To this point the firewall overrode this option to make
sure that its call was always in a native thread no matter the
value of this option was.

The reasons behind this were discussed during this review:
https://review.openstack.org/#/c/57000/

Closes-Bug: #1254811

DocImpact: removes api_thread_pool flag from the libvirt group

Change-Id: Ifad7f1bdf2af69e4fcd6f9a04bf2f7ff5251afd6
Libvirt: Making the video driver element configurable

hw_video_model can be provided via the image properties in Glance.
It may be set to one of the following values: "vga", "cirrus", "vmvga", "xen" or "qxl",
as these are currently being supported by libvirt.

    glance image-update \
       --property hw_video_model=qxl \
       [image]

DocImpact
Partially implements blueprint libvirt-video-driver-selection
Change-Id: Ie710eb9d8d094865d651373c6f9882f1ac12b49a
Ensure api_paste_conf is an absolute path

If api_paste_conf is specified as a relative path and the directory from
where the nova-api service is started contains a file with the same
name, the wsgi loader won't try to find the absolute path to that file
which is required when not passing the base directory to the
paste.deploy library.

This change ensures the config_path is an absolute path so it can be
found by the paste.deploy library.

Closes-Bug: #1259183
Change-Id: I0eb6ef1d69d302634b8c842449f997ec10c13933
Move calls to os.path.exists() in libvirt imagebackend

The checks for the path existence were removed from the base Image class
to avoid redundant checks (some of implementations also perform them and
behave differently depending on the results). Also special handling for
some backends were removed from the base Image class.

Change-Id: Ie6cdf0d6d9801f84fae7a58c1034547611712323
Closes-Bug: #1152748
Sync oslo lockutils for "fix lockutils.lock() to make it thread-safe"

Pull in oslo-incubator change Ia18e6e9f "fix lockutils.lock() to make it
thread-safe" along with dependencies.

Note: I've intentionally excluded log.py from the sync since it pulls in
Change Ic2cf3e52: Adding domain to context and log which is (1) unrelated
to the lockutils change and (2) also changes context.py, which lockutils
does not depend on.

Changes by module (from oldest to newest)
=========================================

excutils:

Change Ic36fa050: Enable H302 hacking check
Change Ibf3c56e4: BaseException.message is deprecated since Python 2.6
Change I87fd89ff: excutils: use six.reraise to re-raise
Change If640e551: excutils: replace unicode by six.u
Change Ic6f0c0ef: Remove vim header

fileutils:

Change Ia51d416b: Add utils for creating tempfile
Change Ic6f0c0ef: Remove vim header

gettextutils:

Change Ic6f0c0ef: Remove vim header

importutils:

Change Ic6f0c0ef: Remove vim header

jsonutils:

Change Ic6f0c0ef: Remove vim header
Change I90be8797: Use six.iteritems to make dict work on Python2/3

local:

Change Ic6f0c0ef: Remove vim header

lockutils:

Change I64fccddc: Allow lockutils to get lock_path conf from envvar
Change I9e1260e2: Add main() to lockutils that creates temp dir
Change Ia18e6e9f: fix lockutils.lock() to make it thread-safe
Change Ic6f0c0ef: Remove vim header

timeutils:

Change I397bae40: Add helper method total_seconds in timeutils.py
Change Ic6f0c0ef: Remove vim header

Related-Bug: #1065529

Change-Id: Ia3530c0d3e78d90fd6de8ac186b252e0fbbba85e
Merge "Sync oslo lockutils for "fix lockutils.lock() to make it thread-safe""
Rename instance_type to flavor in baremetal virt driver

Goes through the baremetal virt driver and associated classes and tests
and renames instance_type to flavor. This patch does not handle renaming
instance_type in nova.test.utils or the instance_type_id column in the
instance db model so those parts are left unchanged here.

DocImpact: Renames the 'instance_type_extra_specs' option in the
baremetal group to 'flavor_extra_specs'.

Part of blueprint flavor-instance-type-dedup

Change-Id: I4405464aea2b2e2142ed1207fb9c25ed9e863cb0
Updated from global requirements

(Also add the sphinx issue which wasn't in the automatic job and
blocked this from landing.)

Change-Id: I94bc14b0ce448ea7a103a6faab73c29b38fbdf3d
Merge "Updated from global requirements"