Make nova/quota use keypair objects

Related to blueprint icehouse-objects.

Change-Id: Ic06d2337c7ef1b028d305d0a7ec354238c239380
Cells rebuild regression fix

The introduction of 'preserve_ephemeral' in rebuilds created a
regression issue with cells.

Closes-Bug: 1281858

Change-Id: Ic8446e8611a41691d12c5dd0c4e1285a586248da
Fix unhandled InvalidServerState exceptions in server start/stop

Catch InvalidServerState exceptions in the server start/stop api
and return a 409 conflict rather than causing 500 errors

Closes-Bug: 1266379

Change-Id: I9d2437319d2cef93b9b10184e257acbd8eb45359
Fixed a problem in iSCSI multipath

Multipathing for attach volume and detach volume operations
doesn't work properly if there are different targets
associated with different portals for a mulitpath device.

Change-Id: Iab316526cf64de60325d90043f73fa3e83fafb4e
Closes-Bug: #1266051
Calculate default security group into quota usage

Create the related quota usage when creating the default
security group for user.

Change-Id: Ide1de160814a5937f71e07f3ab10ce2bba8886f1
Closes-bug: 1271381
Ensure that exception raised in neutron are handled correctly

We are now logging exceptions so we need to use save_and_reraise.

The patch set has also added log messages for exceptions that were
raised. This will be very helpful for debugging and troubleshooting.
This is done in the following cases:
 - unable to access a port when validating networks
 - unable to find a requested floating ip

Change-Id: I946126c2fb5aa229f917f8cb36ecb23aab528e14
Remove admin auth when getting the list of Neutron API extensions.

Currently _refresh_neutron_extensions_cache() always authenticates
with Neutron using the configured Neutron admin credentials because
it doesn't have access to the calling context.  But it's actually
always called when a valid context exists, and the call to get the
list of extensions is not itself an admin operation.  By passing
the context down we can avoid this additional authentication step.

Change-Id: I9e7d9c5e424a6b39c8f8ee1c79f5fd8f48ad81a2
Closes-Bug: 1250850
Make exception message more friendly

Exceptions that get returned via REST api
should have more user friendly message,
users shouldn't have to know backend concept.
more detail,see bug.

Change-Id: Idc20680e46c36fa3441838177ce61b6f5e1d2979
Closes-Bug: #1263204
VMware: fix missing datastore regex with ESX driver

When using shared datastore and the ESX driver the administrator
is unable to indicate which datastore to use.

Note: there were extensive tests added for the VC driver for the
regex support. These are invoked by passing the flag for the regex
when the ESX driver is created.

Change-Id: Ifb0a4a8df2b26f3521b1301462d1f2c3eeba5c14
Closes-bug: #1246206
Fix cache lock for image not consistent

Currently 2 external (file) locks with different name
are created to protect image concurrent access for libvirt driver.
This may lead to image corruption in rare case. This patch change
the lock file name from nova-_var_lib_nova_instances_base_xxxx
to nova-xxxx to make the lock name consistent

Change-Id: I3e60c3bcf456d2392409aa56ba05f4ad8fdc0444
Closes-Bug: #1265452
Minor changes to make certificates test cases use HTTPRequestV3

The existing certificates test in v3 API version test correctly
but it use HTTPRequest keyword inside its test cases and has v2
keyword. Though it will not affect the test, change to HTTPRequestV3
will be helpful in the future.

Change-Id: I0992152e56a3f5b5972e48cb6cda16ef39cacdd2
Catch NotImplementedError in get_spice_console in v2/v3 API

Currently get_spice_console doesn't catch NotImpelementedError
and it will lead to inaccurate information returned back
to API caller. This patch adds the catch exception routine and
returns a HTTPNotImplemented excpetion.

Closes-Bug: #1279195
Change-Id: I16cb0a83c0e1aa9a041dd21af324a39388ca3bb8
Call get_pgsql_connection_info from _test_postgresql_opportunistically

The _test_postgresql_opportunistically method was calling the method to
parse the mysql connection URL which is wrong when there is one
specifically for postgresql, so call the right method.

This wasn't causing issues obviously but is something that needs to be
cleaned up regardless.

Closes-Bug: #1274718

Change-Id: Ic572e612c7ce31d854dc7258aa9c8b89a197260e
libvirt: making set_host_enabled to be a private methods

Currently _set_host_enabled() is being used only for internal libvirt
driver purposes, which doesn't correlate to set_host_enabled in the
compute API.

_set_host_enabled in the driver is disabling and re-enabling
the compute service automatically and shouldn't be exposed to the users.

Also, adding disable_reason as an optional variable and enabled variable
to be boolean only.

Change-Id: I5fd13ef352fcbb4a1c96764a9e473cdaf7b128da
Closes-Bug: #1251803
Add missing translation support

A log message was missing the translation support.

Change-Id: Ib4320a7fe1609c6cf33835f9873a2ddbda211c8c
Fixes Hyper-V VHDX snapshot bigger than instance

In Hyper-V, the deployed images will be extended having their
metadata size added to their base image size. This may lead to
snapshots bigger than the flavor size.

This patch adds support to the V2 utils module, so that the
maximum internal size of a requested image may be calculated
in a similar way it is done in the V1 module.

VHDX format specs:
http://www.microsoft.com/en-us/download/details.aspx?id=34750

Closes-bug:  #1231911

Change-Id: I0d6aacb275f8af24554d3e8b6f04fe8a454b913f
Adds host_ip to hypervisor show API

After no-compute-fanout-to-scheduler, host_ip was stored in the table
of compute_nodes. Host ip address should be considered as the hypervisor
attribute similar to the hypervisor_type, hypervisor_version etc, and
now those attributes such as hypervisor_type, hypervisor_version etc
are all listed as the hypervisor attribute when calling "nova
hypervisor-show host", so we can also set "host_ip" as a new attribute
output for this command.

DocImpact
1) Only administrators can view hypervisor detail in nova.
2) It can help improve debug capabilities for nova. For example, if
admin using SimpleCIDRAffinityFilter, then after VM is deployed, admin
can check if the VM was deployed successfully to the desired host by
checking ip address of the host via "nova hypervisor-show host".
3) Add host_ip to the output for "nova hypervisor-show"

Implement bp hypervisor-show-ip
Change-Id: I006a504d030be1f47beb68a844647026a6daf0ce
V2 Pause: treat case when driver does not implement the operation

Some virt drivers may not support this operation.

Same change will be added to V3 when https://review.openstack.org/#/c/58432
is merged.

Change-Id: I7763f462dd8f61c29c79038d881dc015482a9b5c
Make libvirt use Flavor object instead of using VirtAPI

This makes the libvirt driver use the Flavor object instead of
getting that information through the VirtAPI/conductor call.

Related to blueprint compute-manager-objects

Change-Id: I484c98068a46c9780d82c886779eb58e783f9b6f
Guard against oversize flavor rxtx_factor float

We now check that rxtx_factor provided is not greater than
the Float type to be stored in the db. While the check is not
100% accurate due to float approximation it will be very
close to the max allowable. A 400 error is returned if the
provided value is too big.

Closes-Bug: 1259213
Change-Id: I72c52fa46d75275c45e72b6f5a36106d5abf8220
Co-authored-by: Takashi Natsume <natsume.takashi@lab.ntt.co.jp>