Fixes messages logged on Glance plugin retries

Retry logic is now being used for both upload_vhd and download_vhd,
but nova/virt/xenapi/client/session.py still had 'upload_vhd'
hardcoded in its logged messages. This change causes it to accept the
function being used as an argument and display that.

Fixes-Bug: #1265054
Change-Id: I6203aef81960078f92fa7839e47ee1c84df95745
Fix invalid facilities documented in rootwrap.conf

The values user0 and user1 do not map to valid facility values.
local1, etc.  Using user0 results in a  pri value that does not map
back to a facility of the same name in syslog.

RFC5424 suggest values values of local0 through local7.  Setting
syslog_log_facility to one of those values results in a message with a
priority that can be mapped back to the original string value.

This fix adjusts the comment in rootwrap.conf to suggest the local
prefix instead of the user prefix.

Closes-Bug #1280879

Change-Id: Ibefffb5619641b75b4f8ac8ce8112a34cf03f0ce
Catch InstanceNotFound exceptions for V2 API instance_actions

Use the common.get_instance method to get the instance object
in the V2 API instance_actions extension rather than call
the compute api directly. The utility function
handes InstanceNotFound exceptions and tranlsates them to HTTPNotFound
rather than letting them propagate up causing a traceback.

Co-Authored-By: Michael Still <mikal@stillhq.com>

Change-Id: I1e69fed0c7c0c611fcfd48abbd4b4e0b1c8b403f
Closes-Bug: 1271146
Fix the validation of flavor_extraspecs v2 API

"create flavor_extraspecs" v2 API does not validate the data type
of a request body. If invalid parameter is passed, an internal error
happens. If many invalid requests come, a log file would be occupied
with traceback.

In addition, it does not validate the lengths of both key and value of
extra_specs. extra_specs are stored into the instance_type_extra_specs
table, and the key and value are defined as String(255).

This patch fixes the validation code from the viewpoint of data type
and key/value length.

Closes-Bug: #1264220

Change-Id: I195bd5d45a896e9b26dd81dab1e49c9f939b4805
Remove 3 unnecessary variables in scheduler

Remove 3 lines which is unnecessary because already set its value

Change-Id: I8ba107963b8798e15b6336dbe355f86d2ef505f8
Add return value to some network API methods

Add return value to api methods add_fixed_ip_to_instance() and
remove_fixed_ip_from_instance(). This allows the @refresh_cache
decorator to use returned nw_info model for updating instance
info cache instead of needing to make another rpc call to get
that same information.

Tests are added to make sure return values are used by the
refresh_cache decorator for methods where the rpc call returns
with the necessary nw_info model.

Change-Id: I748e4db9de62d82e4dbcb0b9c444b16dabf0ec5d
Partial-Bug: #1284345
Updated development.environment.rst

Enhanced the discussion of tools/install_venv.py to say that it
installs the items listed in test-requirements.txt as well as
those listed in requirements.txt.

Re-wrapped the relevant paragraph.

Since the change to delete run_tests.sh and tools/install_venv.py
(https://review.openstack.org/#/c/76627/)
has been abandoned, the doc of install_venv.py should still be
accurate.

Change-Id: I5cfd9f179f40b2418129067d0940bfdbf47512cb
Closes-Bug: #1266671
VMware: remove unused code (_delete method in vmops.py)

This code was added in commit 24d609447399e468937f8a278854690864ffabcd and
never used.

Change-Id: I5ad022113666e7be32b512c05165999de2d0c267
Remove action-args from nova-manage help

During help [action_args [action_args ...]] could be
confusing for users

$ nova-manage network list -h
usage: nova-manage network list [-h]
         [action_args [action_args ...]]

Because of that action_args is set to "SUPPRESS" from the help

Change-Id: I0fdc480724d3b59a8eebb2d294f324441c5e46b2
Closes-Bug: #1260538
Include next link when default limit is reached

The /servers and /servers/detail APIs support pagination and a
"next" link should be included when more data is available, see
http://docs.openstack.org/api/openstack-compute/2/content/
Paginated_Collections-d1e664.html. In the documentation it states
that 'collections are required to contain atom "next" links'.

When the default "osapi_max" limit is reached then the "next" link
is not included in the API reply. In this case, the caller cannot
determine if there are any more servers and has no marker value
such that they can retrieve the rest of the servers.

For example, assume that there are 1050 VMs and the osapi_max value
is the default of 1000. The /servers API will return the first 1000
VMs but it will not include the "next" link; the caller has no way
to determine if there are more then 1000 VMs and no way to retrieve
the other 50 VMs.

The fix for this is to include the "next" link when the number of
servers being returned is the maximum limit, even if the "limit"
parameter is not supplied. Note: the caller could workaround this
by specifying the exact "osapi_max" limit value (ie, limit=1000)
on the API call but this requires that the caller knows the
configured limit.

Change-Id: I9b3da852b0ab11dc980524e2317ee876a3ec28b8
Closes-bug: 1288466
Remove PciDeviceList usage in pci manager

Pci manager keeps a PciDeviceList and even try to append
object to the list. This is not correct, as the ObjectList
is designed as immutable, although the objects in the list is
mutable. We should simply use a list to keep devices.

Closes-Bug: #1273803

Change-Id: Ie80b6f5f47a1d2d2e20aadb4779be142c17b5b38
InvalidCPUInfo exception added to except block

When a live migration between two nodes with different
CPU info features is launched, an InvalidCPUInfo exception
is raised and not captured. That lets the vm in an
unusable state because the task_state rollback is not
executed.

Change-Id: Ib418fa324964cabd5f342d1b764f9ac7608b7ade
Closes-Bug: #1294210
Handling unlimited values when updating quota

Changes:
- Adding a general approach to handle unlimited values when performing
calculations with quota values.
- New test that fails without the fix.
- Updated test cases to support the fix.

Change-Id: I762a8e67261c0e0286307832cd13ad12e6d96504
Closes-Bug: #1292494
Include proper Content-Type in the HTTP Headers

Nova Metadata Service was not setting the Content-Type
in the HTTP Response headers. Because of this, clients
were receiving the default Content-Type which is "text/html".
The changes done in this patch will include proper Content-Type
in the HTTP response headers.

Closes-Bug: #1243878
Change-Id: Ia65cdf8099c85f63d7f44f93835c606d679a7ab3
Use correct project/user for quotas

This fixes the compute manager to ensure the proper project_id/user_id
is used when making quota calls by converting most of the compute
manager code to use the new objects. This does not change the RPC
messages to the manager, yet.

Change-Id: I50601d12edef735f3c01ad73445e0826d05efc51
Closes-bug: 1271541
Remove unused quota_* calls from conductor

Compute manager's use of quota calls has been converted to using
objects internally and conductor's quota_commit and quota_rollback
methods are no longer used.

Change-Id: I6f43cb7f94775632e97d40e65ca08a3ba5dc195f
Use the list when get information from libvirt

If the libvirt adds new elements to the domain info struct or others,
when use the new libvirt, there will be a valueError (too many values
to unpack). When we get information from libvirt, we should use the
original variable format. This change is for compatibility with upcoming
versions of libvirt which change this signature.

Change-Id: I0271a260a53fe8f5fc17b2934ebe3a3c9ee0c130
Closes-Bug: #1291805
Change HTTPUnprocessableEntity to HTTPBadRequest

Change HTTPUnprocessableEntity to HTTPBadRequest, and use the
parameter 'explanation' in place of'detail' to show the correct
error message.
This patch fixes this bug.

Change-Id: I6dc2cbe29f24634138ba4c5255b0edd7b8ad6b0e
Closes-Bug: #1283876
Use project/user from instance for quotas

If an admin confirms a resize, a periodic task auto-confirms a resize,
or if an admin reverts a resize, these should all use the instance
project/user for quota updates.

Closes-bug: 1271429
Closes-bug: 1278686
Closes-bug: 1278695

Change-Id: I1e128cd4035d0140e35582de795eb4d6fb8603f2
_translate_from_glance() can cause an unnecessary HTTP request

After returning from a get() call to python-glanceclient, nova runs a
translation function on the returned Image to get the data it wants. Part of
this process is checking for an expected set of attributes, one of which is
the deletion time ('deleted_at'). However, if the image has not been deleted,
deleted_at key will not exist. This forces another call to glance to occur for
the same image. A similar problem exists for the checksum attribute, which does
not exist before an image is active. The fix here is to only consider
deleted_at and checksum if they are expected to be present.

Change-Id: I67b7dd16a94fe60d873c012f6bd246ab24500d5a
Closes-Bug: #1275173