Convert host value from unicode to a string.

The memcached API expects string format, not unicode.
Convert the host value to a string before creating
the cache_key and passing it to memcached.

Fixes: bug #1158958
Change-Id: I26e86b7fb61b4b3380a14686bd2faa9f58017999
(cherry picked from commit a11c41b48dff83a91eaa65630cc6aeb0e51e6461)
Ensure only pickle-able objects live in metadata

Conductor api has been added to InstanceMetadata object, but it cannot
be serialised. This breaks storing metadata in memcache, because the
information needs to be passed through pickle.

Since the field was not used outside of the constructor, it can be
removed to fix the problem.
Also add a test that pickles a sample InstanceMetadata to catch similar
failures in the future.

Fixes bug 1160900

Change-Id: I7e8cef5a131290eaa1402d48295026bcac6baff6
(cherry picked from commit 2da5ea6e5685ae5c349fc1d175594aecbe0a0e23)
Fallback to conductor if types are not stashed.

If the instance types are not stashed, fallback to
the conductor API to retrieve them.

bug 1164110

Change-Id: I1f0ef48e072f8580980de5bd8d43cf2206cd7d27
(cherry picked from commit e9c88b7d3657d83ab84c081fbd56183061ed09c7)
Update latest oslo.setup.

Change-Id: If3d397a1a33e14912abb1f7c94790640fe255d3a
(cherry picked from commit 62d74809ed356ec2962a0ddcfdbfeeed6246f135)
Fix error message in pre_live_migration.

We are using locals() to reference a variable which doesn't exist
in this error message. Also rename the instance variable while I
am here to be more consistent with other uses.

Resolves bug 1164072.

Change-Id: I96f5e2a81ac2e97181c3e87df83e19b381f4bcd2
(cherry picked from commit 676b16bfe5ff41eb28d782c888c17dae6bdff76a)
Accepts aws-sdk-java timestamp format

The aws-sdk-java timestamp format is "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"
which contains millisecond representation. However, the current
implementation does not accept this kind of format.

References:
https://github.com/aws/aws-sdk-java/blob/master/src/main/java/com/amazonaws/auth/QueryStringSigner.java#L173

Adds ability to parse this format in ec2utils.is_ec2_timestamp_expired.

Fixes bug 1156445.

Change-Id: I389ff8b9c6c91b699538b889add264d66dbb8131
(cherry picked from commit 69e940c661731c8ac3cc0cf4e874929c86dcdeb2)
Fixes argument order of quantumv2.api.get_instance_nw_info

Fixes bug #1159042

Change-Id: I04136707cdf2ca716dabd92aa7a471c291a3b4eb
(cherry picked from commit be5ff02f7484a3594f3adeac56ed7aa1ddb475d3)
Remove unnecessary db call in scheduler driver live-migration code

When the scheduler selects a destination host for live-migration, a db
call is made to get instance_type info for the instance. To avoid the
db call, this should instead call instance_types.extract_instance_type
on the instance.

Resolves bug 1167811.

Change-Id: I2025fdf1d34e70158360ff71e8545c8e3bc6bc80
(cherry picked from commit 5f68160354d240b191bcbaabca7d3c0684758cc6)
Cleans up orphan compute_nodes not cleaned up by compute manager

Fixes bug 1161193

Orphan compute_node records can cause the scheduler to pick
compute nodes that are not handled by driver anymore. This
can happen if you rename your hypervisor_hostname or in a
multi-node support where driver does not support a node
anymore for whatever reason (hardare failure for example).

Also, removing resource tracker logic to delete nodes since
resource trackers built in compute manager will never accept
nodes not reported by driver

Change-Id: I742d2e81ec0592d952ee5736aa8dce1e5598ef80
Fix bug in os-availability-zone extension.

Fixes bug 1167956.
The call to get the list of availability zones (GET V2/os-availability-zone)
doesn't hide internal availability zone if all services in that zone are
disabled.

Change-Id: I2c49ad68c2ee5ddea0536dfa770f649fbb392b63
(cherry picked from commit 7172fdc2bf4061039fe34cdb97e442c073a7570a)
Don't swallow PolicyNotAuthorized for resize/reboot actions

Removes 'except Exception' lines in the api which were preventing
PolicyNotAuthorized exceptions from bubbling up and returning 403 to the
caller.  This addresses policy exceptions in confirmResize,
revertResize, and reboot.

The try block calls into the compute api which sends a cast to perform
to bulk of the work, so there is little chance of an exception bubbling
up which is not already being caught.

Removes a unit test for a condition that should not exist.

Bug 1170453

Change-Id: I516a19894ab3d183057c774e84c4faa7053a6463
(cherry picked from commit 7e006ca39fc57e35244f471c0f0bb6773fdbbc28)
Change DB API instance functions for selective metadata fetching

This makes the DB API for instance_get_all() variants able to take a
columns_to_join parameter like the base instance_get_all() function,
which provides control over whether we go to the trouble of pulling
the metadata out of the database before we return the instances.

Related to bug 1164737

(cherry-picked from ba9cd2a545a0c198990f649b6a2985a22102930f)
Change-Id: Ie7c012e2e12bb91ef4294cdc90d828af255b67c5
Make compute/manager use conductor for unrescue()

The compute_api.unrescue() call makes direct database queries, which means
it can't be called directly from compute manager. This patch adds support
in conductor for proxying this request and makes compute manager utilize
it.

Fixes bug 1167619

(cherry-picked from 188f306aa88a6a5b73959c74f1642720d7627789)
Change-Id: I158c64cabdfe436462f5669047fb930819a13c50
Raise correct exception for duplicate networks

When using Quantum, if duplicate network ids are passed in on a
nova boot, nova-api raises a NetworkNotFound exception, which is
not correct.  This change causes it to raise a NetworkDuplicated
exception instead.

Fixes bug 1165088

Change-Id: I4f1f9aa5e1fa0e740b78e67760d924e2dd1d4d98
(cherry picked from commit 8eadd2aee9086815f9fe600d1e2d26145d7b5c16)
set timeout for paramiko ssh connection

set powervm connection timeout to 60 seconds
to eliminate crashing if ssh server is slow in
responding.

(cherry-picked from a62e6233ea6efa1a596678ad302022a646234fcf)
Change-Id: Ic7f4d092de73e73c6edb909713bf3a2347a50334
Encode consoleauth token in utf-8 to make it a str

This is needed for memcached backend, because the key cannot be a
unicode object in any of the exposed functions.

Fixes bug 1164784

Co-authored-by: Stanislaw Pitucha <stanislaw.pitucha@hp.com>

Change-Id: Ieb1429116d96eefe97564fe92631063dc16117cb
(cherry picked from commit 6dadf73305c0a3d779dd625a8349b4a7341df18d)
get context from req rather than getting a new admin context

This patch removes the call to context.get_admin_context() and instead
obtains the context from the req object. Allowing the context.auth_token
to be reused.

Fixes bug 1161081

Conflicts:

	nova/api/openstack/compute/contrib/security_groups.py

Change-Id: I3b104edc2966fc3263fad824b1c4c99df832672e
(cherry picked from commit 1ea02dbcf5ae3e822378a580ac1e54467f664be7)
Query quantum once for instance's security groups

This patch adds the method get_instance_security_group_bindings()
which returns all the instance_id's and security groups for a given
tenant. This method is then used to avoid querying quantum for it's
security groups when using GET /v2/<project_id>/servers/detail.

Fixes bug 1161518

Conflicts:

    nova/tests/api/openstack/compute/contrib/test_security_groups.py

Change-Id: Ia28d5573c264f0316ab9fb257ca8e7ae35679883
(cherry picked from commit afeb95dfc924ed7e768d76bc5ae4bd4c55e9dcb3)
Correct disk's over committed size computing error

There's a error in the disk's over committed size computing for the
instance which using RAW type disk. This issue caused
disk_available_least resource counting increasing.

Fixes bug: 1167695

Change-Id: I4e6cf22a9cf66a757e3641e2a0be0b7699cae69d
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
(cherry picked from commit 32e455282250efa385984085f1ecd8847cc6be2c)
Merge "Fixes argument order of quantumv2.api.get_instance_nw_info" into stable/grizzly