Allow configuring the wsgi pool size

Expose the number of greenthreads in wsgi pool to configuration. This
allows to put a limit to the number of resources like memcache
connections used.

DocImpact
Change-Id: I6e53d6e12661ed2f42a6b543891b13eec0c8300c
Fix LibvirtGenericVIFDriver.get_config() for quota

When Nova is used with Neutron and Open vSwitch, network quotas on the
bandwidth are ignored by libvirt: the "<bandwidth>" tag is not generated in
libvirt.xml of the virtual machine. This change fixes this issue but also for
the network types: "bridge", "network" and "direct".

Commands to create the flavor with download limited to 1 MB/sec and create a
virtual machine with this flavor:
---
nova flavor-create --ephemeral=1 victor_test_vif 50 256 1 1
nova flavor-key victor_test_vif set quota:vif_inbound_average=1000
nova boot --flavor=victor_test_vif --image=cirros-0.3.1-x86_64-uec victor_test
---

Commands to upload a file to this virtual machine with SCP to check that the
bandwidth is limited to 1 MB/sec (replace 10.0.0.3 with the IP of the virtual
machine):
---
ip netns
sudo ip netns exec qrouter-128db593-a0db-40c3-84c7-e6383d40c75f bash

dd if=/dev/urandom of=random10MB bs=1024 count=10240
scp random10M cirros@10.0.3:
---

Change-Id: I0f0111fc79fe90900e38df022034b208f1129088
Closes-Bug: #1254664
DB migration 209: Clean up child rows as well

Fixes bug 1252891

In order to add missing foreign key constraints, rows that reference a
nonexistant key need to be purged. Unfortunately, DB migration 209 does
not clean up any rows in child tables that themselves have a foreign key
constraint on the tables with the missing foreign keys. This can result
in an integrity violation causing the migration to fail.

When purging the rows, make sure to purge rows in any child tables first.

Change-Id: I25bd899383c72cb0670819b030759630ac89acaa
Update Instance from database after destroy

In cases where Nova falls back to simply locally deleting an
instance, the Instance object was not updated with the deleted_at
generated by the database implementation. Due to this, the
notification emitted to signify that the delete finished was
getting emitted without a deleted_at value.

This change uses the result from the instance_destroy call to
update the Instance object with the latest state from the database.
Thus, the compute.instance.delete.end notification is emitted
with the proper deleted_at value.

Closes-Bug: #1250206
Change-Id: I55ed37cf89dd6eb8ea901c6f813191a0ce1058e8
Add extra_resources field to compute_nodes table

The compute_nodes table has informtion about allocation and
availability of a predefined set of resources (e.g. memory,
disk, vcpu). The extra_resources field will contain a json
string of arbirtrary information describing an extensible set
of resources that will be tracked by resource tracker plug-ins.

This differs from other fields containing json strings that have
been added recently, including pci_stats and metrics. In particular,
pci_stats is used for information about pci devices, whereas metrics
contains continuously monitored usage data and is updated at different
times to extra_resources and according to a different trigger.
Accordingly, it makes sense to seperate these into different fields.

This is part of: blueprint extensible-resource-tracking

Change-Id: I75e339c2a2827875078c937e57c37e85d3ad1852
Get test_openAuth_can_refuse_None_uri to cleanup after itself

test_openAuth_can_refuse_None_uri sets the global
libvirt.allow_default_uri_connection to False but it leaves
it in this state which could cause test_tpool_execute_calls_libvirt
to fail if it ran immediately afterwards. This patch adds a cleanup
so _reset is called at the end of the test and not just if
another test in the same class is run after it.

Change-Id: I9762fcb81b2318beeb056c67bd9172056e1449ab
Closes-Bug: 1258553
libvirt: Custom disk_bus setting is being lost on hard_reboot

Currently, a non-default disk_bus setting of and image will not be used,
when powering on an instance (or _hard_reboot).

hard_reboot() should use image_meta, if it exist, when constructing
the disk_info

Closes-Bug: #1255449

Change-Id: I477f69c47515ad22165421b4d23ac865c7869873
Log exception in _heal_instance_info_cache

Previously, if errors occurred in _heal_instance_info_cache() they would
go unnoticed as their log level was debug and the stace trace was not
displayed. This patch changes that so the exception is logged so periodic
failures here can be more easily detected and fixed.

Change-Id: I15bb3e7e29462e930f6159be9f4bbf4980b1270c
Closes-bug: #1257940
Rename instance_type to flavor in test_utils and nova.tests.utils

Renames occurrences of 'instance_type' to 'flavor' in
nova.tests.test_utils and nova.tests.utils along with any impacted tests
using the changed parts.

Part of blueprint flavor-instance-type-dedup

Change-Id: Ie816356c176c2be61b4a346a4d47d8f3461ae84a
Prevent spoofing instance_id from neutron to nova

Previously, one could update a port's device_id in neutron to be
that of another tenant's instance_id and then be able to retrieve
that instance's metadata. This patch prevents this from occurring by
checking that X-Tenant-ID received from the metadata request matches
the tenant_id in the nova database.

DocImpact - This patch is dependent on another patch in neutron
            which adds X-Tenant-ID to the request. Therefore to
            minimize downtime one should upgrade Neutron first (then
            restart neutron-metadata-agent) and lastly update nova.

Change-Id: I93bf662797c3986324ca2099b403833c2e990fb4
Closes-Bug: #1235450
Use a dictionary to eliminate the inner loop in _choose_host_filters()

it looks like an o(n²) algorithm, we can use a dictionary to eliminate
the inner loop.

Change-Id: I32747120ae1f2f1bc0a3810a03e0cd9baa7a8d2c
Add IPAddress field type in object models

In object models, there are ipv4 address type and ipv6 address type.
However, for some fields in models.py, for instance, host_ip in compute node,
and a bunch of IPAddress variables in network, and address in other tables.

There is no field type corresponding to them, no matter whether it is v4 or
v6. So, add one more field type in object models to specify that type.

Change-Id: Ic53ed644f4bbc1e280b2f8cbb9b9c145c3d8a2e9
XenAPI: Synchronize on all VBD plug/unplug per VM

Dynamically adding or removing a device needs to be synchronized
to work around a bug in XenServer 6.1 and 6.2 where simultaneous
events can be partially ignored, leading to a failure to plug or
unplug.  The bug is between two internal components in XenServer
which pass messages - the first component will ignore all messages
from the second while a plug/unplug is in progress, expecting
only one message to be received.  If two plug/unplugs occur in
this time, an internal assertion is raised as XAPIs internal state
is inconsistent.

Change I7b86325714e38b00a2f51bbf083b5532a6c51fa9 attempted to
work around the issue by retrying - however that does not catch
all failures.  This change should eliminate some of the known
INTERNAL_ERROR failures, but not all, so I7b8 need not be
reverted.

Change Ie7a64c481a9e3874df9d5924fbb050ac7ab117db does the same
as this, but only for VBD.plug.

Change-Id: If78d4e5f1dd84f1d4a588f25b8f887a7b125bbea
Closes-bug: 1217972
Don't replace instance.info_cache on each save

If we are working on an Instance that came from an older client,
we should not replace an older nested .info_cache object with a
newer one for no reason during a save().

Related-Bug: #1258256
Change-Id: I8cf4a629d454de1a5de16ea4032f39e36aab505a
Update vnc virt driver api to take Instance object

Update the get_vnc_console() virt api method to take an Instance object
instead of a dict.  No real changes were needed in the drivers to make
this work.  While we're changing the API call, add the context, as well.

Change-Id: I2026ea725d61df4b88e7618f9fd4498c62843296
Pass Instance object to compute vnc rpc api

Pass a real Instance object to the get_vnc_console() method of the
compute rpc api.  The object_compat() decorator is used to provide
backwards compatibility with receiving data in the old format.

Change-Id: Ib615fd653e0ef6196241fde81f541b1e18eb3ced
Send Instance object to validate_console_port

Update the validate_console_port() method of the compute rpc api to take
in an Instance object instead of a dict.  The object_compat() decorator
is used to ensure the change is backwards compatible, and that the code
will still work if it receives an instance in dict format.

Change-Id: Ief8fabc8490315acb41bf85ad421683fea31daf9
Pass Instance object to console output virt driver api

Update the get_console_output() method of the virt driver API to be
defined to accept an Instance object.  While we're at it, pass the
context, as well.

Change-Id: If325a39f6314cff936e604fcc5816deefbb1199b
Merge "Use a dictionary to eliminate the inner loop in _choose_host_filters()"
Merge "XenAPI: Synchronize on all VBD plug/unplug per VM"