Fixes messages logged on Glance plugin retries

Retry logic is now being used for both upload_vhd and download_vhd,
but nova/virt/xenapi/client/session.py still had 'upload_vhd'
hardcoded in its logged messages. This change causes it to accept the
function being used as an argument and display that.

Fixes-Bug: #1265054
Change-Id: I6203aef81960078f92fa7839e47ee1c84df95745
Fix invalid facilities documented in rootwrap.conf

The values user0 and user1 do not map to valid facility values.
local1, etc.  Using user0 results in a  pri value that does not map
back to a facility of the same name in syslog.

RFC5424 suggest values values of local0 through local7.  Setting
syslog_log_facility to one of those values results in a message with a
priority that can be mapped back to the original string value.

This fix adjusts the comment in rootwrap.conf to suggest the local
prefix instead of the user prefix.

Closes-Bug #1280879

Change-Id: Ibefffb5619641b75b4f8ac8ce8112a34cf03f0ce
Catch InstanceNotFound exceptions for V2 API instance_actions

Use the common.get_instance method to get the instance object
in the V2 API instance_actions extension rather than call
the compute api directly. The utility function
handes InstanceNotFound exceptions and tranlsates them to HTTPNotFound
rather than letting them propagate up causing a traceback.

Co-Authored-By: Michael Still <mikal@stillhq.com>

Change-Id: I1e69fed0c7c0c611fcfd48abbd4b4e0b1c8b403f
Closes-Bug: 1271146
Describe addresses in ec2 api broken with neutron

It attempts to retrieve the fixed ip directly,
but it's not supported by neutron.

After the modification,
the instance uuid will be pulled without using get_fixed_ip().

Change-Id: Ia1b95d4c18aeb5012919af89148947694d80e2e7
Closes-Bug: #1273837
Fix the validation of flavor_extraspecs v2 API

"create flavor_extraspecs" v2 API does not validate the data type
of a request body. If invalid parameter is passed, an internal error
happens. If many invalid requests come, a log file would be occupied
with traceback.

In addition, it does not validate the lengths of both key and value of
extra_specs. extra_specs are stored into the instance_type_extra_specs
table, and the key and value are defined as String(255).

This patch fixes the validation code from the viewpoint of data type
and key/value length.

Closes-Bug: #1264220

Change-Id: I195bd5d45a896e9b26dd81dab1e49c9f939b4805
attach/detach interface should raise exception when instance is locked

when instance is locked, exception should be raised if attach and
detach an interface operation is performed. This patch add instance
lock check for the 2 APIs, catch and process the exception at API layer.

Change-Id: I6c0b05dafb9d480c8f0002bcc8306cb65a32aaf7
Closes-Bug: #1275741
Remove 3 unnecessary variables in scheduler

Remove 3 lines which is unnecessary because already set its value

Change-Id: I8ba107963b8798e15b6336dbe355f86d2ef505f8
Add return value to some network API methods

Add return value to api methods add_fixed_ip_to_instance() and
remove_fixed_ip_from_instance(). This allows the @refresh_cache
decorator to use returned nw_info model for updating instance
info cache instead of needing to make another rpc call to get
that same information.

Tests are added to make sure return values are used by the
refresh_cache decorator for methods where the rpc call returns
with the necessary nw_info model.

Change-Id: I748e4db9de62d82e4dbcb0b9c444b16dabf0ec5d
Partial-Bug: #1284345
Updated development.environment.rst

Enhanced the discussion of tools/install_venv.py to say that it
installs the items listed in test-requirements.txt as well as
those listed in requirements.txt.

Re-wrapped the relevant paragraph.

Since the change to delete run_tests.sh and tools/install_venv.py
(https://review.openstack.org/#/c/76627/)
has been abandoned, the doc of install_venv.py should still be
accurate.

Change-Id: I5cfd9f179f40b2418129067d0940bfdbf47512cb
Closes-Bug: #1266671
VMware: add missing translations

Commit 1c6898efecd7d78855efcb22d9c38d6127339000 added a number
of exceptions that did not have translation support.

Change-Id: I38715b8b14bdf9ccee08db44cf171f9c860b4fa8
Make test computations explicit

This test is full of magic numbers, making it very difficult to fix when
the underlying code changes. There are some obvious errors, such as an
instance having a different memory_mb value than its flavor. This patch
attempts to sort these issues out, making it easier to work on the
resource tracker with confidence.

Change-Id: I4549f0d9b10a5ac65f169199b98b8975ef2a8cd3
Handle IpAddressGenerationClient neutron

Handles IpAddressGenerationClient neutron exception which
can occur when allocating a floating ip fails because
the pool has been exhausted. Convert the exception
to a nova NoMoreFloatingIps exception which will be handled
by the api layer and converted to a 404.

Change-Id: Ib5320a13b5cc0c493ef0763d0e2b4575dc699bdf
Partial-Bug: 1210598
VMware: remove unused code (_delete method in vmops.py)

This code was added in commit 24d609447399e468937f8a278854690864ffabcd and
never used.

Change-Id: I5ad022113666e7be32b512c05165999de2d0c267
Remove action-args from nova-manage help

During help [action_args [action_args ...]] could be
confusing for users

$ nova-manage network list -h
usage: nova-manage network list [-h]
         [action_args [action_args ...]]

Because of that action_args is set to "SUPPRESS" from the help

Change-Id: I0fdc480724d3b59a8eebb2d294f324441c5e46b2
Closes-Bug: #1260538
Include next link when default limit is reached

The /servers and /servers/detail APIs support pagination and a
"next" link should be included when more data is available, see
http://docs.openstack.org/api/openstack-compute/2/content/
Paginated_Collections-d1e664.html. In the documentation it states
that 'collections are required to contain atom "next" links'.

When the default "osapi_max" limit is reached then the "next" link
is not included in the API reply. In this case, the caller cannot
determine if there are any more servers and has no marker value
such that they can retrieve the rest of the servers.

For example, assume that there are 1050 VMs and the osapi_max value
is the default of 1000. The /servers API will return the first 1000
VMs but it will not include the "next" link; the caller has no way
to determine if there are more then 1000 VMs and no way to retrieve
the other 50 VMs.

The fix for this is to include the "next" link when the number of
servers being returned is the maximum limit, even if the "limit"
parameter is not supplied. Note: the caller could workaround this
by specifying the exact "osapi_max" limit value (ie, limit=1000)
on the API call but this requires that the caller knows the
configured limit.

Change-Id: I9b3da852b0ab11dc980524e2317ee876a3ec28b8
Closes-bug: 1288466
Ignore InstanceNotFound while getting console output

During we cold-migrating or resizing an instance to another host,
or during deleting it, the get_console_output method may raise an
InstanceNotFound excetpion if the instance is not on the hypervisor,
this is an expected error, so we should add it to expected
exceptions list in compute manager.

Closes-bug: #1292339
Change-Id: I23a99438d73826f01ff5662130dcbfa72dc39a6e
Remove PciDeviceList usage in pci manager

Pci manager keeps a PciDeviceList and even try to append
object to the list. This is not correct, as the ObjectList
is designed as immutable, although the objects in the list is
mutable. We should simply use a list to keep devices.

Closes-Bug: #1273803

Change-Id: Ie80b6f5f47a1d2d2e20aadb4779be142c17b5b38
InvalidCPUInfo exception added to except block

When a live migration between two nodes with different
CPU info features is launched, an InvalidCPUInfo exception
is raised and not captured. That lets the vm in an
unusable state because the task_state rollback is not
executed.

Change-Id: Ib418fa324964cabd5f342d1b764f9ac7608b7ade
Closes-Bug: #1294210
Fix service API and cells

The move to the object model broke the service API when used together
with cells. The underlying problem is how cells will overload the id
field with a cell path to uniquely identify each service in each cell.

Changing the id to a string would defeat the purpose of the object model
so this patch uses a proxy to modify the id on access to make the
difference transparent to the API.

Change-Id: I165b5d64b0c1556511eb02d179ca9fc6f101c008
Closes-bug: 1251043
Handling unlimited values when updating quota

Changes:
- Adding a general approach to handle unlimited values when performing
calculations with quota values.
- New test that fails without the fix.
- Updated test cases to support the fix.

Change-Id: I762a8e67261c0e0286307832cd13ad12e6d96504
Closes-Bug: #1292494