Test for remote directory creation before shutting down instance

This patch moves the creation of the directory on the migration target
earlier in the code path so we can catch failures like missing or incorrect
keypairs on the source and destination node.

This fixes part of bug #1061024 but not all of it yet.

Change-Id: I097cf007cdd8de0c9c3a9390cbc6dc202eddb2b5
Don't update DB records for unchanged stats.

Change-Id: I6eb0adba12676cf057c92fb0b54431cfe6a76210
Added validation for networks parameter value

A validation check is added for networks parameter value.
In server create request body, if networks parameter value
is not stored in a list then 'Bad networks format' error
is thrown with bad request exception.A new test case is
added to verify whether it is returning bad request exception
when invalid syntax is used.

Fixes: bug #1175683

Change-Id: I457a1626c56281f89d8c4a606ccc0d0e3aea05fa
Fixes encoding issues for nova api req body.

* Ensure utf-8 encoding for POST/PUT body for api requests
* As per https://wiki.openstack.org/wiki/Encoding.

Fixes bug #1180344

Change-Id: Iea0f16b0c024795fb3aee1cd38bb54ee9da6d90d
Fix error in instance_get_all_by_filters() use of soft_deleted filter

Change-Id: I7c2fab48944e34765b3fff8ce10bc64a5cd826c8 introduced the
'soft_deleted' filter to the above method to tweek the behavior of the
existing 'deleted' filter. In doing so, an error was introduced that
changed the original behavior of the 'deleted' filter when used by
itself, in how it treated both soft- and hard-deleted instances the
same.

The original merged patch did not include test coverage for the changes
made to the db api. This change fix the error so that the original
behavior of the 'deleted' filter is restored while also adding full
test coverage for the modifications to the db api that has already
merged.

Finally, the support method create_instances_with_args() used by the
new tests was renamed create_instance_with_args() to reflect that
when called, it just creates a single instance.

Resolves bug 1181865.

Change-Id: Ibb82af09d3876904455ca7c93e14e9722ed31d35
Fix missing argument to logging warning call

Change-Id: I04ff613e785b6b39476b7766b94c831d7e22148a
Disallow resize if image not available

If a resize is attempted on an instance that was started from an image
that has since been deleted, a resize will fail. This change makes error
reporting a bit cleaner.

This change is needed since in order to actually properly support
resize/migrate when the image is deleted - it is necessary for nova to keep
a copy of the image metadata and re-use it in case of migration/resize.

Fixes bug: 1160773
Fixes bug: 1177001

Change-Id: Ifaea71f79c97046a4cde094e3a5e676772fcceb4
get_instance_security_groups() fails if no name on security group

The following patch fixes a bug in get_instance_security_groups() where
it tries to access security_groups['id'] but security_groups is a string.
This patch also removes some stubs that were not needed in a few unit test.

Fixes bug 1182738

Change-Id: Ia64db2e40d4a0e163344b4234112dbde71b1e132
Optimize SecurityGroupsOutputController by len(servers)

The following patch adds an optimization for _extend_servers() so that
it calls get_instance_security_groups() when len(servers) == 1, and
get_instances_security_groups_bindings() when > 1. In addition, this patch
adds a return statement if not len(servers) as there is no reason to
perform a query if there are no servers.

Fixes bug 1182730

Change-Id: I149bf32176d60e72d76576168d15995f7215b657
Add missing tests for nova.db.api.agent_build_* methods

blueprint db-api-tests

Change-Id: If8f8a3f02bc0ea73b9174caad60107c7d87f62c0
Raise AgentBuildNotFound on updating/destroying deleted object

Methods below should raise an exception when attempting to operate
with deleted AgentBuild objects:

 nova.db.sqlalchemy.api.agent_build_destroy()
 nova.db.sqlalchemy.api.agent_build_update()

Fixes: bug 1181967

Change-Id: If6df202afb7ea41634c35f7a52988e19677f4d65
Using unicode() to handle image's properties

Replace str() to unicode() to allow nova-api work well with the fix I
did in glanceclient (https://review.openstack.org/#/c/29245/6).

Fixes: Bug #1180377

Change-Id: I31e532bb50e12e355b9dd26c9ce1e129da858cbe
Signed-off-by: Zhi Yan Liu <lzy.dev@gmail.com>
Removed session from fixed_ip_*() functions.

Removed session objects from public fixed_ip_*() functions
in nova/db/sqlalchemy/api module. Minor code refactiring.

blueprint db-session-cleanup

Change-Id: Ic2a96270745799496e365aee0380ac079e1edeff
Remove path_exists from NFS/GlusterFS drivers

mkdir -p will not throw an error if the path already exists,
so this method is unneeded in these classes.

Change-Id: I65a67cb75f83f65a6f80c7a90f77c2c5133bd41b
Share checks between create and rebuild

DRYs up the checks between create and rebuild in preparation for additional
checks going in.

Change-Id: I04a8448f7a3a4e1176b40f943ea9c90c06ba8ece
Move ImageTooLarge check to Compute API

The existing ImageTooLarge check is in the compute manager, meaning that a
validation error puts the instance into an ERROR state without the ability to
communicate the fault back to the user.

This patch moves the check to the API where we can return the appropriate
error to the user.

Change-Id: I2183449351e3b70f98d0552ea247184a3641d85a
Remove ImageTooLarge exception

InstanceTypeDiskTooSmall and ImageTooLarge are really two sides of the same
coin; an image is larger than the root_gb of the instance_type. Rather than
have two exceptions for this, this patch consolidates down to just
InstanceTypeDiskTooSmall.

Change-Id: I401205072c111a960beb2932c6c1889141ae03c3
xenapi: Disable VDI size check when root_gb=0

Nova uses root_gb=0 to indicate that we should use the image as-is without
trying to resize the root disk. Currently the xenapi virt-driver doesn't
properly implement this behavior.

This patch gets us a step closer by disabling the VDI size check when
root_gb=0. We still need to ensure that the disk-resize code handle root_gb=0
as well.

References bug 1155113

Change-Id: If91963e079e197a1315c48b8643f3419b42b0b2c
xenapi: Moving tests under tests/virt/xenapi/

We have both a tests/xenapi/ and a tests/virt/xenapi/ directory which
is confusing. This patch standardizes on tests/virt/xenapi/ which
matches the nova/ directory layout.

Change-Id: I324a4730e2477aa1863b17c7ea404ce2932a681c
Reset task_state when resetting vm_state to ACTIVE

A virt driver can raise InstanceFaultRollback to indicate that the
instance should be reset to ACTIVE.  However, the code that resets to
ACTIVE does not reset task_state to None.

This patch fixes that.  Technically, there's currently no bug here,
because the compute manager methods that matter here all have the
reverts_task_state() decorator to always reset task_state to None on
exceptions.  However, a bug would be introduced later if we remove
those decorators.

Change-Id: Ibcf8ae3473f601781d8aae80f88df3da54bbaac8