Fix the validation of flavor_extraspecs v2 API

"create flavor_extraspecs" v2 API does not validate the data type
of a request body. If invalid parameter is passed, an internal error
happens. If many invalid requests come, a log file would be occupied
with traceback.

In addition, it does not validate the lengths of both key and value of
extra_specs. extra_specs are stored into the instance_type_extra_specs
table, and the key and value are defined as String(255).

This patch fixes the validation code from the viewpoint of data type
and key/value length.

Closes-Bug: #1264220

Change-Id: I195bd5d45a896e9b26dd81dab1e49c9f939b4805
Remove 3 unnecessary variables in scheduler

Remove 3 lines which is unnecessary because already set its value

Change-Id: I8ba107963b8798e15b6336dbe355f86d2ef505f8
Updated development.environment.rst

Enhanced the discussion of tools/install_venv.py to say that it
installs the items listed in test-requirements.txt as well as
those listed in requirements.txt.

Re-wrapped the relevant paragraph.

Since the change to delete run_tests.sh and tools/install_venv.py
(https://review.openstack.org/#/c/76627/)
has been abandoned, the doc of install_venv.py should still be
accurate.

Change-Id: I5cfd9f179f40b2418129067d0940bfdbf47512cb
Closes-Bug: #1266671
Remove action-args from nova-manage help

During help [action_args [action_args ...]] could be
confusing for users

$ nova-manage network list -h
usage: nova-manage network list [-h]
         [action_args [action_args ...]]

Because of that action_args is set to "SUPPRESS" from the help

Change-Id: I0fdc480724d3b59a8eebb2d294f324441c5e46b2
Closes-Bug: #1260538
Include next link when default limit is reached

The /servers and /servers/detail APIs support pagination and a
"next" link should be included when more data is available, see
http://docs.openstack.org/api/openstack-compute/2/content/
Paginated_Collections-d1e664.html. In the documentation it states
that 'collections are required to contain atom "next" links'.

When the default "osapi_max" limit is reached then the "next" link
is not included in the API reply. In this case, the caller cannot
determine if there are any more servers and has no marker value
such that they can retrieve the rest of the servers.

For example, assume that there are 1050 VMs and the osapi_max value
is the default of 1000. The /servers API will return the first 1000
VMs but it will not include the "next" link; the caller has no way
to determine if there are more then 1000 VMs and no way to retrieve
the other 50 VMs.

The fix for this is to include the "next" link when the number of
servers being returned is the maximum limit, even if the "limit"
parameter is not supplied. Note: the caller could workaround this
by specifying the exact "osapi_max" limit value (ie, limit=1000)
on the API call but this requires that the caller knows the
configured limit.

Change-Id: I9b3da852b0ab11dc980524e2317ee876a3ec28b8
Closes-bug: 1288466
Handling unlimited values when updating quota

Changes:
- Adding a general approach to handle unlimited values when performing
calculations with quota values.
- New test that fails without the fix.
- Updated test cases to support the fix.

Change-Id: I762a8e67261c0e0286307832cd13ad12e6d96504
Closes-Bug: #1292494
Use correct project/user for quotas

This fixes the compute manager to ensure the proper project_id/user_id
is used when making quota calls by converting most of the compute
manager code to use the new objects. This does not change the RPC
messages to the manager, yet.

Change-Id: I50601d12edef735f3c01ad73445e0826d05efc51
Closes-bug: 1271541
Remove unused quota_* calls from conductor

Compute manager's use of quota calls has been converted to using
objects internally and conductor's quota_commit and quota_rollback
methods are no longer used.

Change-Id: I6f43cb7f94775632e97d40e65ca08a3ba5dc195f
Use the list when get information from libvirt

If the libvirt adds new elements to the domain info struct or others,
when use the new libvirt, there will be a valueError (too many values
to unpack). When we get information from libvirt, we should use the
original variable format. This change is for compatibility with upcoming
versions of libvirt which change this signature.

Change-Id: I0271a260a53fe8f5fc17b2934ebe3a3c9ee0c130
Closes-Bug: #1291805
Change HTTPUnprocessableEntity to HTTPBadRequest

Change HTTPUnprocessableEntity to HTTPBadRequest, and use the
parameter 'explanation' in place of'detail' to show the correct
error message.
This patch fixes this bug.

Change-Id: I6dc2cbe29f24634138ba4c5255b0edd7b8ad6b0e
Closes-Bug: #1283876
Use project/user from instance for quotas

If an admin confirms a resize, a periodic task auto-confirms a resize,
or if an admin reverts a resize, these should all use the instance
project/user for quota updates.

Closes-bug: 1271429
Closes-bug: 1278686
Closes-bug: 1278695

Change-Id: I1e128cd4035d0140e35582de795eb4d6fb8603f2
_translate_from_glance() can cause an unnecessary HTTP request

After returning from a get() call to python-glanceclient, nova runs a
translation function on the returned Image to get the data it wants. Part of
this process is checking for an expected set of attributes, one of which is
the deletion time ('deleted_at'). However, if the image has not been deleted,
deleted_at key will not exist. This forces another call to glance to occur for
the same image. A similar problem exists for the checksum attribute, which does
not exist before an image is active. The fix here is to only consider
deleted_at and checksum if they are expected to be present.

Change-Id: I67b7dd16a94fe60d873c012f6bd246ab24500d5a
Closes-Bug: #1275173
Make 'VDI too big' more verbose

Added sizes to error logging for easier debugging. Changed log level
from error to debug.

Change-Id: I397b79df8cae01d736733b7b141f3b4a8513f708
VMware: clean up VNC console handling

Commit 61bfac8881dd6a71a572a54b2ea1680248fc4bc4 introduced
the VMwareVCVMOps class. This class specifcally treats VC methods.
The patch set moves the treatment for get_vnc_console to this
class.

The log message for the VNC console now has the instance as a
parameter.

Please note that there are already test cases for this.

Change-Id: I2b943558d96328580f5c8744945e5f5f1790c06f
Reserve 10 migrations for backports

Reserve 10 migrations so that we have the option of backporting fixes
that include a db migration to the stable/icehouse branch.  We did this
same thing to allow backports to stable/grizzly in
b52a2157bedba693c5da7dcb783b7a151769d6b1 and stable/havana in
ab2c467da951071a8aac4eb6ca032371c69053ab.

The choice of 10 migrations is just a magic number.  We haven't used any
of the 10 reserved in previous stable branches, so 10 seems to be enough.

Implements blueprint backportable-db-migrations-juno

Change-Id: Ifdeba76809a705413bacee571d6d0d43ce1f062c
Ensure network interfaces are in requested order

_build_network_info_model was iterating current_neutron_ports
instead of port_ids which contains ports in their correctly requested
order. Because of this the requested nic order was no longer being
perserved. This patch fixes this and also changes the order of ports
in test_build_network_info_model() so this case is tested.

Change-Id: Ia9e71364bca6cbc24ebc1c234e6a5af14f51cd62
Closes-bug: #1300325
Correct the keypairs-get-resp.json API sample file

Current keypair sample files 'keypairs-get-resp.json/xml contain
'keypairs' as the first key and that means its sample response
of "list keypairs" API not "get keypair" API.

The tests don't pass a keypair id, so current tests also are for
"list keypairs" API.

Details-
Below API sample files- from their name it looks like these are for
get keypair API. But in actual content of these files are written as
List keypair API response. So it create the confusion that for which
API these API sample file are. Name suggest for GET and content
suggest for LIST Keypair.

    /nova/tests/integrated/api_samples/os-keypairs/
    keypairs-get-resp.json.tpl
    /nova/tests/integrated/api_samples/os-keypairs/
    keypairs-get-resp.xml.tpl

Their API sample test cases are written corrosponding to list keypair
APIs.

This patch correct the above API sample file name from get to list
Keypair APIs.
API Sample test cases names are also modified accordingly.

Partial-Bug: 1298769

Change-Id: I88d894ff9b0f6236221fa922601b641f26a87301
Drop nova-rpc-zmq-receiver man-page

Doesn't make sense to install and build docs after it was removed.

Change-Id: Ia6193808cca625e83cb3497b5033db5c08dcdd1e
Change NotFound to InstanceNotFound in server_diagnostics.py

According to the get() method in nova/compute/api.py,
the index() method in nova/api/openstack/compute/
contrib/server_diagnostics.py should catch the
InstanceNotFound, rather than the NotFound.
This patch just applies to V2 API, V3 API is already
catching InstanceNotFound.

Change-Id: I9e021837b9050dade9af4725a0712f260eac6f7a
Closes-bug: #1296662
Moved the registration of lifecycle event handler in init_host()

The event handler has to be registered before the libvirt connection
is used for the first time.  Otherwise it doesn't register and events
such as shut down initiated from within the instance are not registered
in nova until a period task compares the content of hypervisor with
content of database.

Closes bug: #1297485

Change-Id: I70887bea343734a977f621fedc634b6fab0e4ad5