Resolve conflicting mac address in resize

This patch fixes bug 1161226 and mac address conflicts when migrating or
resizing an instance using the PowerVM driver. This will ensure that if
a migration or resize is taking place on the same host, a temporary mac
address will be assigned to the original LPAR so there is no conflict on
the VIOS when a second LPAR is started with the same address.

Change-Id: I59db484defb789f3a7c0eddbd5ef9e92efc22419
(cherry picked from commit 2a43c4020927c936ed3b3f07416413f5171d62f8)
Catch NoValidHost exception during live-migration

To avoid leaving the instance state as MIGRATING if scheduling of
a target host fails, add this exception to the list of exceptions
for which the instance task state is reset to None.

Resolves bug 1166771.

Change-Id: I23343091725051624c0eb0b82e9f03a7f9d5f4f4
(cherry picked from commit b5732a454a8221bb98cb6a7241b4d4fc17840c6a)
Security groups may be unavailable

Currently all missing security group situations are reported as errors
and exceptions. This is not the case - any user requesting a wrong id
or name can cause it and it's not an issue in Nova. A busy system can
generate a large number of those messages causing unnecessarily high
error count.

This change stops reporting this situation as exception in the logs.

Change-Id: Ide856cbbde2db025b0e2def79652cc6c9b00566b
(cherry picked from commit 7d14d024f0d92cd6f9596596cdcaf3d870ba1839)
Correct network uuid field for os-network extension

"nova net-list" does not return network uuid even when the uuid is
available, and instead returns only the id. This is because the
"network_dict" function in the extension improperly uses "in" to check
for a Network model object. This also reduces compatibility in the API.

Fixes bug: 1162409

Change-Id: I30d4c8bfa37d89123f35ee8cc7c67f0e819262a7
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>(cherry picked from commit 9254403281fad200f871c5e6ff052dc79c739d6e)
Fix a typo in attach_interface error path

This prevents an attach failure from raising the proper exception to
the caller.

Change-Id: I51559cf395fc99738127b7a443437197d15f3d4c
(cherry picked from commit 0ce92fcb233837f4715a173601707c6d65cf177c)
Fix legacy_net_info guard

The existing code assumes that `legacy_net_info` is always in legacy
mode, meaning a list of tuples which causes it to break when passed a
new-style NetworkInfo object.

Fixes bug 1164152

Change-Id: I2131d9b24045cd7531454b65d97776b11ec3ab02
(cherry picked from commit 948d1fefe6cb1e17c5567d3cf3313e13c16a3d57)
After migrate, catch and remove deleted instances

On the host init, starting with a deleted instance which has been
previously evacuated from the host results in an InstanceNotFound
exception. Catch and log this, and then call driver.destroy() so
that the hypervisor driver can clean up the deleted instance.

If we don't do this during host init, it will cause problems during
periodic tasks.

Fixes bug 1155152

Co-authored-by: Devananda van der Veen <devananda.vdv@gmail.com>

Change-Id: I979a698b8e739b9335f37b81e789285f91977a8e
(cherry picked from commit 306046c7d5c20454035fcea22ce3efeac1d11cfc)
libvirt: Get driver type from base image type.

If we are using the raw backend (no cow), we should set the driver
type based on the type of the source image instead of always
using raw.

Calls to to_xml were moved after _create_image so that the image
type could be determined when creating the xml for libvirt.

Fixes bug 1163009

Change-Id: Ic8d5f0ab83d868a42f834d39c0afb64818d7e027
(cherry picked from commit 494a3cb5749d52aa90daeacd980362df5f971c0d)
Fix _error_out_instance exception handler

Since Eventlet clears exceptions between greenthread switching, we need
to save that exception so it can be re-raised.

The _error_out_instance event handler did not wrap the quotas call with
the save_and_reraise_exception, so the original exception was lost.

Fixes bug 1163552

Change-Id: I69d28917aae1d787351cd800f8996532e5485eb2
(cherry picked from commit 25afdbbda8d56c509a4b34e0d6feeaafb2e46e5b)
improve handling of an empty dnsmasq --domain

Currently all domains passed through the dnsmasq --domain option,
are single quoted.  It's not fully understood how dnsmasq handles
single quotes in this situation, so it's safer to just avoid the
--domain option in this case.

Fixes bug: 1161506
Fixes bug: 1167447
Change-Id: I8ecd996676189f53908abd48fee81132091ee820
Cherry-picked: 24b9731af34fbc2fb17c849cb8cdb5c7a10db77d
Set defaultbranch in .gitreview to stable/grizzly

This allows people run git-review without any arguments.

Change-Id: I7adde6608451268e8247944c297576a70280bf16
Optimize resource tracker queries for instances

instance_get_all_by_host_and_node() had 4 different joins being done.
The resource tracker is the only consumer of this method and does not
need the joins.  This optimization will save many unnecessary rows from
being fetched during the update_available_resource() audit task.

(cherry-picked from e8449460334de7632887c8685658a8dfc3d3a5b3)
Change-Id: I414c219e4563634e270007fe3d0fa980b273e795
Replace metadata joins with another query

Right now, we always join metadata and system_metadata when we query for
instances. This inflates the number of rows returned by a factor of ten
at a minimum, and over 30 in some situations. This patch avoids the
direct join of these tables in favor of a "manual join" which performs
additional queries to fill the needed data. Latency may be increased,
but with the gain of a potentially significant amount of reduction in
data transfer.

This also provides the ability for the queries which utilize it to
selectively decide whether either or both of these additional fills are
performed, paving the way for future high-level optimizations.

Related to bug 1164737

(cherry-picked from c3568f9a1a28b269d88d2dcc175e6b5a2f7abf37)
Change-Id: I0610fb16ccce2ee95c318589c8abcc30613a3fe9
Don't join metadata twice in instance_get_all()

(cherry-picked from 842a6ac97b508b01f4773c94e8eb9416f997472e)
Change-Id: I2952243b093cda3d397f239ebe0db19246443a00
Convert host value from unicode to a string.

The memcached API expects string format, not unicode.
Convert the host value to a string before creating
the cache_key and passing it to memcached.

Fixes: bug #1158958
Change-Id: I26e86b7fb61b4b3380a14686bd2faa9f58017999
(cherry picked from commit a11c41b48dff83a91eaa65630cc6aeb0e51e6461)
Ensure only pickle-able objects live in metadata

Conductor api has been added to InstanceMetadata object, but it cannot
be serialised. This breaks storing metadata in memcache, because the
information needs to be passed through pickle.

Since the field was not used outside of the constructor, it can be
removed to fix the problem.
Also add a test that pickles a sample InstanceMetadata to catch similar
failures in the future.

Fixes bug 1160900

Change-Id: I7e8cef5a131290eaa1402d48295026bcac6baff6
(cherry picked from commit 2da5ea6e5685ae5c349fc1d175594aecbe0a0e23)
Merge "Optimize resource tracker queries for instances" into stable/grizzly
Fallback to conductor if types are not stashed.

If the instance types are not stashed, fallback to
the conductor API to retrieve them.

bug 1164110

Change-Id: I1f0ef48e072f8580980de5bd8d43cf2206cd7d27
(cherry picked from commit e9c88b7d3657d83ab84c081fbd56183061ed09c7)
Update latest oslo.setup.

Change-Id: If3d397a1a33e14912abb1f7c94790640fe255d3a
(cherry picked from commit 62d74809ed356ec2962a0ddcfdbfeeed6246f135)
Merge "Replace metadata joins with another query" into stable/grizzly