Add instance to debug log at compute api

Add instance info to 2 debug logs at compute api layer.

Change-Id: Icbe8e45ace74b8bf0282cf6d6dd1ead792532961
Rollback quota in os_tenant_network

when we call API with 'os_tenant_networks' to delete an existing network
it might fail due to unexpected reason, we should rollback the
quota we reserved otherwise we will face no quota later.

Change-Id: Ic74a7a8df337070f17b1ea6916809fa93880a287
Closes-Bug: 1296489
handle AutoDiskConfigDisabledByImage at API layer

nova compute api layer might raise AutoDiskConfigDisabledByImage,
so API layer need to catch them and report correct info.
Also, create api handle lots of exceptions which inherit from Invalid,
this patch combined them and catch Invalid only.

Change-Id: Id0c89e96637d2f82b5bd204eba2c6291b9a5dd38
Add standard constants for CPU architectures

Introduce a nova/compute/arch.py module to serve a similar role
to nova/compute/vmmode.py, by providing constants for known
CPU architectures and validation/canonicalization APIs

Related-bug: #1348624
Change-Id: Idd4907b5488dd0832c15783d87d57d77ae654519
libvirt: remove use of CONF.libvirt.virt_type in vif.py

The vif.py class imports the CONF.libvirt.virt_type parameter
from the main driver.py file. This sets up a circular dependency
which prevents driver.py importing vif.py in the top of the file,
since the CONF.libvirt.virt_type parameter won't have been
registered yet.

It is generally bad design practice to rely on global variables,
which is effectively what the CONF.* parameters are, so change
the VIF driver get_config() API to accept 'virt_type' as a
parameter instead.

Related-bug: #1302796
Change-Id: I023851df96640eeeb54124add8cc5c725b8d13be
libvirt: remove 'vif_driver' config parameter

The 'vif_driver' config parameter was deprecated in the
previous cycle, so can be deleted now.

The two places in the test file which were overriding
the 'vif_driver' config parameter were redundant as
mocking already ensured the real VIF driver module
was not exercised.

The FakeVIFDriver class is thus no longer required, and
the FakeIptablesFirewallDriver class was already not
required, so both are deleted.

DocImpact
Related-bug: #1302796
Change-Id: I75519266ffc65df92efd20badfeaf88ac0d66f8a
libvirt: remove pointless LibvirtBaseVIFDriver class

There is only one single VIF driver class these days, so it is
pointless having a LibvirtBaseVIFDriver base class to inherit
from.

Related-bug: #1302796
Change-Id: Ie132135f9b6cf5337782ee9d1611c05bd56557fe
EC2: fixed AttributeError when metadata is not found

Change-Id: I91e0141bd89cd18596ccae2ea213f641ba4d7f74
Closes-Bug: 1324091
Deprecate metadata_neutron_* configuration settings

Commit 5cacad3508570ce70b1f9ef620e0508169687fda created a
neutron configuration section. Move all of the metadata
neutron settings to that section.

DocImpact

The table below has the changes:

+-------------------------------------+------------------------------+
| 'DEFAULT' Section                   | 'neutron' Section            |
|-------------------------------------|------------------------------|
| service_neutron_metadata_proxy      | service_metadata_proxy       |
| neutron_metadata_proxy_shared_secret| metadata_proxy_shared_secret |
+-------------------------------------+------------------------------+

Change-Id: Id1b8b387e028c7752729f5c5b373c87397f7577a
Network: interface attach and detach raised confusing exception

When the aforementioned operations failed they raised an exception
that was not coherent - this is due to the fact that the instance object
was passed to the exception. That would print the whole instance object
which is really confusing to a openstack user. The exceptions should has passed
the instance ID and not the instance.

TrivialFix

Change-Id: If3fa89b17210c1db3540cd813157b51e786e1494
Closes-bug: #1335076
Boot an instance with multiple vnics on same network

If the same L2 network is requested multiple times for the same
instance then creating ports on same network and attaching them
to the same instance raises a DuplicateNetworks exception.
Similarly, attaching multiple existent ports on same L2 network
to the same instance raises a DuplicateNetworks exception. This
is the default behavior that is defaulted by a newly introduced
nova flag "allow_duplicate_networks" which is set to
False by default.

Not raising a DuplicateNetwork exception and allowing an instance
to have multiple vnics on same network is useful for NfV service
instances and in that case this newly introduced nova flag should
be set to True.

DocImpact: New neutron.allow_duplicate_networks configuration option
Implements blueprint multiple-if-1-net
Change-Id: Id4d633162c785c9b56b9c8426c0445770bc1352e
Closes-Bug: #1187244
Remove duplicate test of passing glance params

Simply removes a duplicate test of the passing of page size parameters
from the TestGlanceImageService and adds the page_size parameter and
check to the similar test in TestDetail.test_detail_params_passed() test
method.

Change-Id: I2f7fafbb5e33e0f81fd8d493babc0d47966c6791
Add ListOfDictOfNullableString field type

A new field type is added, it will be a list, with each item
in the list is a dict of string.

Implements: blueprint make-resource-tracker-use-objects

Change-Id: I4d3a310800afe344bc18c755658fc1dc3e792de8
Standardize logging for v2 api extensions

- add log translation hints for warning, error and info levels
- remove pointless debug message

Change-Id: I67c5e77e62f8ef89204b67426b88856ef91dfc26
Correct returned HTTP status code (Use 403 instead of 413)

The exception HTTPRequestEntityTooLarge should not be used as an
exception response based on RFC2616. Because of that, change
the returned response to HTTPForbidden.

Restore commit for Iab090c40c632a76b0528df8145ad0897c8b649bf
And add compute V3 changes.

Related Tempest commit to match this change:
If376eda0a7929ba2baa4ac4acbb457883bcfc96d

DocImpact: corrects HTTP return code from 413 to 403 for quota-related
limit faults

Closes-Bug: #1298131
Change-Id: I2bb8a60ef254afbfed514cfeebe75355d0de4475
Allow empty volumes to be created

The following patch allows an empty volume to be created and
attached to an instance at boot time. Today, you can define a
bootable volume that is sourced from an image, volume, or
snapshot. However, you cannot define a volume that is sourced
from 'blank', even though it is one of the source options
available. For example, the following command will not work:
nova boot --flavor m1.tiny
--block-device source=blank,dest=volume,size=1,bootindex=0 test.
This is because the method used to identify ephemeral volumes
(new_format_is_ephemeral) believes any block device with
source_type=blank is an ephemeral, and there is no logic to
handle volumes where source_type=blank. The following patch
fixes these bugs by properly identifying ephemeral block
devices and creating blank/empty volumes when source_type=blank
and destination_type=volume.

Change-Id: I5aa9684bfad1749fadff3018b13a225ed8f16fe8
Closes-Bug: #1347028
Closes-Bug: #1347499
Fix the error status code of duplicated agents

If posting the duplicated agent info to "create a agent" API, Nova v2
API returns a HTTP500 response(Internal Server Error).
The duplicated resources are not internal errors and this behavior has
been fixed on Nova v3 API. In this case, Nova v3 API returns a HTTP409
response(Conflict).
This patch fixes it for Nova v2 API.

Change-Id: I0b9424f9792270f6807cabcbf27a0f6e5ec42669
Merge BadRequest tests of "create a keypair" API

In test_attach_interfaces, there are BadRequest tests of "create a keypair".
Most parts of them are duplicated.
This patch merges them for the readability and clarifying their purposes.

Change-Id: I784f5d05969aa9a0c85b63f064190ab4e92ed7f7
Provide a quick way to run flake8

"run_tests.sh -p" always checks every file for errors even though you
have probably only changed a handful. This patch adds "run_tests.sh -8"
which only checks the files that were modified in the HEAD commit or
the current working tree.

An unscientific benchmark shows that this can be quite a bit faster:

Before:
$ time ./run_tests.sh -p
Running flake8 ...
./nova/compute/manager.py:77:1: F401 'periodic_task' imported but unused

real    0m37.493s
user    0m37.391s
sys     0m0.099s

After:
$ time ./run_tests.sh -8
Running flake8 on nova/compute/manager.py
./nova/compute/manager.py:77:1: F401 'periodic_task' imported but unused

real    0m0.667s
user    0m0.624s
sys     0m0.036s

Change-Id: I8dafaab40fd0427719040a76c69a8ab05e9e4386
docs - Fix errors,warnings from document generation

ERROR: Unexpected indentation.
WARNING: Field list ends without a blank line; unexpected unindent.
WARNING: Block quote ends without a blank line; unexpected unindent.

Partial-Bug: #1351350

Change-Id: If13b5c533cf9a1e8f543b8f679c464c0dc007a30