Describe addresses in ec2 api broken with neutron

It attempts to retrieve the fixed ip directly,
but it's not supported by neutron.

After the modification,
the instance uuid will be pulled without using get_fixed_ip().

Change-Id: Ia1b95d4c18aeb5012919af89148947694d80e2e7
Closes-Bug: #1273837
Ignore InstanceNotFound while getting console output

During we cold-migrating or resizing an instance to another host,
or during deleting it, the get_console_output method may raise an
InstanceNotFound excetpion if the instance is not on the hypervisor,
this is an expected error, so we should add it to expected
exceptions list in compute manager.

Closes-bug: #1292339
Change-Id: I23a99438d73826f01ff5662130dcbfa72dc39a6e
XenAPI: Calculate disk_available_least

The disk_available_least item is now needed for the tempest test.
Calculate based on the virtual-allocation from the SR.

Change-Id: I1a08f229de7cd77d4798472c82304247fae290e5
Closes-bug: 1300242
Rename instance_actions v3 to server_actions

* Renames the os-instance_actions v3 plugin to os-server-actions
* Also modifies api specific tests and expected user input JSON
* DocImpact only for v3 Nova api documentation for this plugin.

Closes-Bug: #1206032

DocImpact

Change-Id: Id5049e10f5ef540d0d8e3871d800e9d9cb33da73
ec2: use SecurityGroup object

Convert one place in the ec2 API code that was using the db API
directly to get a security group.  Use the SecurityGroup object for
this instead.

Some test fixes were needed to account for fields that the
SecurityGroup object does not have set as nullable.

Change-Id: I4b15b6bcb35f95955911a48faa8d74f6167593a0
ec2: use BlockDeviceMappingList object

Convert a direct use of the db API with the nova object equivalent.
Some test fixes were needed to include fields the object expects to be
preset in the fake db objects.

Change-Id: I1d562aa2535af9e713c6486c5c50a2ffe625c198
GET details REST API next link missing 'details'

When executing a pagination query a "next" link is included in the
API reply when there are more items then the specified limit.

See pagination documentation for more information:
http://docs.openstack.org/api/openstack-compute/2/content/
Paginated_Collections-d1e664.html

The caller should be able to invoke the "next" link (without
having to re-format it) in order to get the next page of data.
The documentation states "Subsequent links will honor the
initial page size. Thus, a client may follow links to traverse
a paginated collection without having to input the marker parameter."

The problem is that the "next" link is always scoped to the
non-detailed query.

For example, if you execute "/v2/<tenant>/servers/detail?limit=1",
the "next" link does not have the URL for a detailed query and is
formatted as "/v2/<tenant>/servers?limit=1&marker=<marker>". In this
case the "next" link needs to be scoped to "/v2/<tenant>/servers/detail".
Common code is used for both the v2 and v3 APIs.

Existing test cases will be updated to verify that the next link is
correctly scoped for both the v2 and v3 APIs.

The user could work around this issue my manually inserting '/details'
into the "next" link URL.

Change-Id: Ib06a6cc6e2dd5e2c8c16986ee256a58752626eb9
Closes-bug: 1299247
remove unneeded call to network_api on rebuild_instance

When rebuilding an instance we call the network_api and which results in
calling neutron and updating the info cache. We do not actually need to
do this as we can get the nw_info directly from the instance's info cache.

Change-Id: I2bcc2f15b1e5bb52d972098c7bc3e12f160b6feb
Closes-bug: 1294914
Fix the explanations of HTTPNotFound for keypair's API

Gets the "explanation" from HTTPNotFound exception and returns
correct message. the commit Ia784e09cda986a421e83c2820dfd0a2267867cdd
has fixed this kind of bug already, but this bug is remained.

Closes-Bug: #1307338
Change-Id: I6811eecba86938e4253f437b7677cb7e4468fd67
VMware: Log additional details of suds faults

When a suds requests results in a fault response, the current
code loses details on that fault in the sequence of exceptions
propagated through the driver.  A NoPermission fault will contain
additional metadata on the privilegeId and object type which
needs to be logged.  The fault string will be propagated with this
fix, along with details of a NoPermission fault.

An example of the new exception:
NoPermissionException: Permission to perform this operation was denied.
{u'privilegeId': Resource.AssignVMToPool, u'object': domain-c7}

An example of new exception within retrieveproperties_fault_checker:
Error(s) NoPermission occurred in the call to RetrievePropertiesEx
{'privilegeId': System.Read, 'object': datacenter-16}

Change-Id: Iafbf052750c2835f304b2edf21d7300d1fbd7e5a
Closes-bug: #1289627
VMWare - Check for compute node before triggering destroy

While booting an instance, if Claim on that compute node is not successful
compute manager triggers a call to destroy instance. Destroy fails since the
compute node (Cluster/ESX host) is set only after claim is successful.
Hence the check for node is made before destroy, to ensure that exception
does not get thrown. This ensures that the instance gets rescheduled
Closes-Bug: #1307408

Change-Id: Iaaf931e9c1e6cf046497e2c64952f92e802ad4be
Don't translate debug level scheduler logs

Our translation policy
(https://wiki.openstack.org/wiki/LoggingStandards#Log_Translation) calls
for not translating debug level logs. This is to help prioritize log
translation. Furthermore translation has a performance overhead, even if
the log isn't used (since nova doesn't support lazy translation yet).

Remove unnecessary debug level log translation in the scheduler to
comply with our translation policy. This has the added benefit of making
the scheduler slightly faster too.

Add a local hacking rule to enforce this.

Change-Id: Ie1068d2d2c7c37f27c62d0e4e5a64a1a504af9f5
Add helper method to determine disk size from instance properties

Add a _disk_size_from_instance method which by using the disk name
determine the size of the disk refered in the instance properties.
Update the method finish_migration to use this new helper.

Change-Id: I45bd3804fdabff35189421d01a0503c886c24a4c
Related-Bug: #1298975
Xen: Retry plugin call after connection reset

When using the call_plugin_serialized_with_retry session method it will
retry if the plugin raises a RetryableError, but not if it loses the
connection to xenapi.  This adds a retry for the 'Connection reset by
peer' error that is occasionally seen.

Change-Id: Ibb36a0170e0e807ad7a4198df7b6e574c4279bed
Closes-Bug: #1200591
Make instance_exists() take an instance, not instance_name

Some virt drivers may not support/need to use instance.name and have
a different way to find out if an instance exists. NOTE: this still
uses dict access on instance, as the _run_instance() path is not
converted to Instance objects yet.

NOTE: This also removes a duplicate call to self._lookup_by_name()
in libvirt's resume_state_on_host_boot(). It was calling
self.instance_exists() and then self._lookup_by_name(), but instance_exists()
calls _lookup_by_name() itself.

Change-Id: I701cb5416ec47262268d1a088018b6be2774d1c5
Imported Translations from Transifex

Change-Id: I4da8e7e117a65221e1d3afeef3774e1e42698c51
Correct the log in conductor unshelve_instance

The log in conductor unshelve_instance is not correct, this patch
adjust it.

Change-Id: I9706862fa39d3c9cb5af0d412368546b504bc4a5
Use eventlet.tpool.Proxy for DB API calls

There is no eventlet in common db code anymore, so, if we want to use
thread pooling for DB API calls, we should use eventlet.tpool.Proxy()
instead of tpool_wrapper() from openstack.common.db.api.

Config option CONF.database.use_tpool was moved from module
nova.openstack.common.db.api to nova.db.api.
Config file `nova.conf.sample` was updated as well.

See also patch to Oslo - I6fa140f2b1f03c43ae7cab98ac8a12d71188af76

Co-Authored-By: Chris Behrens <cbehrens@codestud.com>
Change-Id: Id7de85dce11f5ea61dfccddf80de7800a2bc4e37
Fix migrate_instance_*() using DB for floating addresses

This fixes the network api's migrate_instance_start() and
migrate_instance_finish() methods that are still hitting the database
directly for an oddball call to get a list of floating addresses.

Change-Id: Ia310e31d31aaf5c979e41c64af8223202a18e03a
Closes-bug: #1310966
XenAPI: Use local rsync rather than remote if possible

Using ssh depends on the host being set up to be able to SSH into
itself which is not a common scenario.  While this is unavoidable for
the current implementation of resize across multiple hosts, if there
is a single host (i.e. a test scenario) or the resize is restricted
to the same host then we can succeed without SSH access

Dependency on Ia310e31d31aaf5c979e41c64af8223202a18e03a is because the
tests will always fail without Ia310 therefore this fix cannot be tested
without taking Ia310 as well.

Closes-bug: 1308064

Change-Id: I15802a1d97d380b1c5b74fc9f92ece2494fe789a