Allow configuring the wsgi pool size

Expose the number of greenthreads in wsgi pool to configuration. This
allows to put a limit to the number of resources like memcache
connections used.

DocImpact
Change-Id: I6e53d6e12661ed2f42a6b543891b13eec0c8300c
Update openstack/common/lockutils

The following commits are in this update:

79e6bc6 fix lockutils.lock() to make it thread-safe
ace5120 Add main() to lockutils that creates temp dir for locks
537d8e2 Allow lockutils to get lock_path conf from envvar

Change-Id: I2190d0aae72ef4a29daf8f7d960da82a33b331b4
Enforce metadata string type on key/value pairs

The compute API is currently enforcing quota limits for user metadata
and length restrictions on the dict entries, but is not enforcing that
they are string types. This patch adds the checks for string type on the
keys and values in the metadata dict.

Closes-Bug: #1240319

Change-Id: Ic4aacc342cf37d929871287e6e4917f4f48b63e8
Update openstack/common/periodic_task

The following commits are in this update:

1771a77 Adjust import order according to PEP8 imports rule
4bfb7a2 Apply six for metaclass
0ea069e Removed calls to locals()
1a2df89 Enable H302 hacking check

Change-Id: I5d15d9772002a2f764acef16e28bcfa16a9760c0
Remove deprecated instance_type alias from nova-manage

Was deprecated in Havana, so now that we are in Icehouse lets
remove it.

UpgradeImpact
Change-Id: Ieb3775e0a43084d00624f9e1d8b879c312e32b3c
Fixes get_vm_storage_paths issue for Hyper-V V2 API

Add get_vm_storage_paths function to vmutilsv2.py

get_vm_storage_paths implementation on Hyper-V V2 API changed
compared to previous V1 API. In previous V1 APIs, using
Msvm_ResourceAllocationSettingData class can get storage
paths, but in V2 API, only using Msvm_StorageAllocationSettingData
class can get the storage paths.

Closes-Bug: #1248463

Change-Id: Ica73221524f162a6ffcd9dc64ee33c85fb5ad31d
Handle UnicodeEncodeError in validate_integer

validate_integer can be potentially passed user
supplied data which causes a UnicodeEncodeError
exception during validation rather than just ValueError.
This change handles the UnicodeEncode exception
so validate_integer treats it as invalid input.

Change-Id: I5177789299b02448ed53e1b7fc23ccdaee1b70d3
Partial-Bug: 1243037
convert min_count and max_count to type int in nova v3 api

this converts min_count and max_count to type:int in nova
v3 api.
If the string type hasn't been converted, in the method
_check_num_instances_quota of nova/compute/api.py, something
like req_ram = '2'*128 will occur. That's why the issue in
bug 1254616 arise.

Change-Id: I567b249e7a30681980ed760d5d09aad4c8e603b9
Closes-bug: 1254616
Fix a lazy-load exception in security_group_update()

When security_group_api is nova, the update_security_group api raises http
500 error: group rules fails with lazy load exception. Because the rule
infos of the security group will be return in response body, but the rule
attribute hasn't been preloaded.

Closes-Bug: #1244018
Change-Id: Ie07879ac22e4add90f75968778bf334915e8349c
Delete instance faults when deleting instance

Instance delete should also delete related record from the
instance_faults table.

Change-Id: I9c845bce6cf39003858a1b2a726734db9fb9e584
Closes-bug: #1237155
Set is_public to False by default for volume backed snapshots

This update results in the same behavior that is already implemented
in _create_image where the meta_data for is_public is set to False.

Change-Id: Ie0720d5c847871527db7d9477ca6cb05abe7a1a5
Closes-Bug: #1255316
Check Neutron port quota during validate_networks in API

Unless ports are passed into Nova it will create ports on the
requested networks as part of the network allocation in
the compute manager. However if the user exceeds their
port quota the instance will end up in an Error state, having
first been re-scheduled a number of times.

It would be much better if the quota failure was detected as
part of the network validation in the API server, so that an
error can be reported to the user and the creation failed.

A full fix would include reserving or creating the ports at
this stage, but there is no reservation mechanism in the
Neutron API, and port creation depends in some cases
on mac addresses only available on the compute manager.

Instead this change just validates the quota and adjusts the
max_count to be consistent with that quota, which doesn't
guarantee that the create will work, but does catch the
majority of cases.

Refs bug: 1172808

Change-Id: Iaaee059a6746fad68049712f94b2f8cfea6ab8dc
Fix LibvirtGenericVIFDriver.get_config() for quota

When Nova is used with Neutron and Open vSwitch, network quotas on the
bandwidth are ignored by libvirt: the "<bandwidth>" tag is not generated in
libvirt.xml of the virtual machine. This change fixes this issue but also for
the network types: "bridge", "network" and "direct".

Commands to create the flavor with download limited to 1 MB/sec and create a
virtual machine with this flavor:
---
nova flavor-create --ephemeral=1 victor_test_vif 50 256 1 1
nova flavor-key victor_test_vif set quota:vif_inbound_average=1000
nova boot --flavor=victor_test_vif --image=cirros-0.3.1-x86_64-uec victor_test
---

Commands to upload a file to this virtual machine with SCP to check that the
bandwidth is limited to 1 MB/sec (replace 10.0.0.3 with the IP of the virtual
machine):
---
ip netns
sudo ip netns exec qrouter-128db593-a0db-40c3-84c7-e6383d40c75f bash

dd if=/dev/urandom of=random10MB bs=1024 count=10240
scp random10M cirros@10.0.3:
---

Change-Id: I0f0111fc79fe90900e38df022034b208f1129088
Closes-Bug: #1254664
DB migration 209: Clean up child rows as well

Fixes bug 1252891

In order to add missing foreign key constraints, rows that reference a
nonexistant key need to be purged. Unfortunately, DB migration 209 does
not clean up any rows in child tables that themselves have a foreign key
constraint on the tables with the missing foreign keys. This can result
in an integrity violation causing the migration to fail.

When purging the rows, make sure to purge rows in any child tables first.

Change-Id: I25bd899383c72cb0670819b030759630ac89acaa
Remove redundant body validation for createBackup

The body validation is redundant for createBackup, should remove it. In
fact that this extension is registered as WSGI action implies that the
body must contain a key with the same name as action.

Change-Id: I327688b2df50782bc38b97a4750f83a607623ce4
Closes-Bug: #1254988
Fix docstring on SnapshotController

Change-Id: I66a8b15ef700253bbad54ee3955550aeb2a522b7
Closes-Bug: #1257151
Update Instance from database after destroy

In cases where Nova falls back to simply locally deleting an
instance, the Instance object was not updated with the deleted_at
generated by the database implementation. Due to this, the
notification emitted to signify that the delete finished was
getting emitted without a deleted_at value.

This change uses the result from the instance_destroy call to
update the Instance object with the latest state from the database.
Thus, the compute.instance.delete.end notification is emitted
with the proper deleted_at value.

Closes-Bug: #1250206
Change-Id: I55ed37cf89dd6eb8ea901c6f813191a0ce1058e8
Add shutdown option to cleanup running periodic

Add a 'shutdown' option to the periodic task that cleans up instances
which are both deleted *and* running.

The option powers down instances and marks them as non-bootable.  This
is a flag for system administrators who would like to reclaim the
resources consumed by each instance, but leave the records around for
later examination/debugging.

DocImpact
Implements: blueprint cleanup-running-deleted-shutdown

Change-Id: Id95845eb995b62ed8858fe61c6fdb34045721674
LXC: Image device should be reset in mount() and teardown()

When fixing bug#1208387, image device name is returned in function
setup_container. But it's missed to reset in function mount() and
teardown().

This patch is to fix it. We now just return the device name when
mounting disk image during setting up container and remove the
member variable 'device' from Class _DiskImage added in last patch.

Closes-bug: #1243108
Related-bug: #1208387

Change-Id: I2aaed6f3367edce4fb23b78f0e979440954609a8
Add extra_resources field to compute_nodes table

The compute_nodes table has informtion about allocation and
availability of a predefined set of resources (e.g. memory,
disk, vcpu). The extra_resources field will contain a json
string of arbirtrary information describing an extensible set
of resources that will be tracked by resource tracker plug-ins.

This differs from other fields containing json strings that have
been added recently, including pci_stats and metrics. In particular,
pci_stats is used for information about pci devices, whereas metrics
contains continuously monitored usage data and is updated at different
times to extra_resources and according to a different trigger.
Accordingly, it makes sense to seperate these into different fields.

This is part of: blueprint extensible-resource-tracking

Change-Id: I75e339c2a2827875078c937e57c37e85d3ad1852