Check the length of aggregate metadata

PostgreSQL gives an error if you try to shove 256+ characters
into a 255-character field. So we need raise a HTTPBadRequest
explanation if the length of aggregate_metadata.keys or
aggregate_metadata.values is above 255.

Change-Id: I2c778f2237ba5bd2aa8335a0eae80f3aad3e9157
Closes-Bug: #1219693
Add unit tests for ipv4/ipv6 format validation

We found a bug that the other jsonschema format check('uri') did
not work for input validation. There was not any unit tests for
the format check, then we could not find it in long term.
The bug is fixed with Ibb2b7f27c374136e6865b28786d10ca1365e5709 .
And this patch adds unit tests for the format 'ipv4' and 'ipv6'.

Change-Id: I6f6636708e3f9a4eee14451f2baacb50002f1db8
Related-Bug: #1326599
Separate the PCI device object handling code

Currently the PCI device object includes several functions like
alloc/free/claim etc. However, the NovaObject should not be used
this way, and it makes the PCI device object really different
with other NovaObject implementations.

This patch duplicates the PCI device handling code as separated
functions. No logic changes but simply code copy. A later patch
will delete the code at PciDevice object.

Closes-Bug: #1273852

Change-Id: I2389d29218d8619998f8df973004af004e8b1b76
Use new pci device handling code in pci_manager

As we remove some pci device method from PCI device object,
update the pci manager accordingly.

Closes-Bug: #1273852

Change-Id: I0ec0e7b03625888a207918816b6e2d537b6d7d54
Remove the device handling from pci device object

Remove the device handling code at pci device object to make
it similar to other NovaObject.

Remove some usage in test cases.

Closes-Bug: #1273852

Change-Id: I4487ae0b36de18365d12593f480c05f0fe726769
Network: ensure that ports are 'unset' when instance is deleted

Commit a141206e9dfd31955b9b31d9e5a7f73bbd8510ca ensured that user defined
ports were not deleted when an instance is deleted or rescheduled. The
problem with this commit is that it did not reset the 'device_id' and
'device_owner' of the port. This leads to port leakage and eventually a tenant
is unable to launch any instances as their port quota has exceeded the
allocated amount.

The solution is for the deallocation to reset the 'device_id' and the
'device_owner' of the port. This is allocated when the port is actually assigned
to an instance.

In the case of minesweeper (which runs on OpenStack), the tenant was leaking ports.
The root cause was that the user defined ports were assigned to a specific instance,
which was deleted, as a result of a failed reschedule (which is legitimate).

When an instance is deleted all artificats of that instance should be reset on the
port.

Change-Id: Ie1516c299ce6d79faf2d5103a14d6d8931d3ede2
Closes-bug: #1321381
Deprecate neutron_* configuration settings

Create a new section in the configuration file called 'neutron'.
Move all of the neutron_* configuration settings to this section.

DocImpact

The table below has the changes:

+---------------------------------+-------------------------+
| 'DEFAULT' Section               | 'neutron' Section       |
|---------------------------------|-------------------------|
| neutron_url                     | url                     |
| neutron_url_timeout             | url_timeout             |
| neutron_admin_username          | admin_username          |
| neutron_admin_password          | admin_password          |
| neutron_admin_tenant_id         | admin_tenant_id         |
| neutron_admin_tenant_name       | admin_tenant_name       |
| neutron_region_name             | region_name             |
| neutron_admin_auth_url          | admin_auth_url          |
| neutron_api_insecure            | api_insecure            |
| neutron_auth_strategy           | auth_strategy           |
| neutron_region_name             | region_name             |
| neutron_ovs_bridge              | ovs_bridge              |
| neutron_extension_sync_interval | extension_sync_interval |
| neutron_ca_certificates_file    | ca_certificates_file    |
+---------------------------------+-----------------------=-+

UpgradeImpact
The changes are backward compatible.

Change-Id: I3308be33f60c4add3e2e78b75f238059a9b3d876
Add configurable HTTP timeout to cinder API calls

Some Nova API calls require synchronous use of cinder.  (e.g. volume
list)  If the call to cinder lands on a slow performing or otherwise bad
cinder API node, the user of Nova API will see poor performance.

This patch adds the ability to control HTTP timeouts when calling the
cinder API.  By configuring a timeout, nova will be able to more quickly
retry its cinder request and hopefully land on a better performing
cinder API node.  This will help make the API response times more
predictable.

For backward compatibility, the default timeout value is None.

Closes-Bug: 1329797
DocImpact

Change-Id: I5cccced524d9a9d69ed41f755781d0b7ae3dc970
Fix doc for service list

If a service is enabled, the "disabled_reason" field in response
for /v2/.{tenant_id}./os-services/detail is null, however, currently
it's specified as "" in the doc, thus may be confusing to users.

This patch change the template and doc, it should not cause issue to
existing applications.

Change-Id: Ia71ec4c97a355bcc1a7b63e6107db77f80a5d843
Close-bug: 1328382
Add test for HypervisorUnavailable on conductor

HypervisorUnavailable is added in excepted exception list in conductor
function migrate_server in commit d0a5e272, but the test cases are not added.
This patch adds test cases for it.

Change-Id: I83556e97cb151ff202b2377c00122a368b1c2892
Check the network_info obj type before invoke wait function

When instance schedule to host and it isn't first time schedule,
the instance may already allocated network. And then the code
will return an NetworkInfo object, not an NetworkInfoAsyncWrapper.
The checking code only assume it is NetworkInfoAsyncWrapper object.

Change-Id: I987902fc47e509e4fd08f762d66b16c09eaf5008
Partial-Bug: #1326207
Imported Translations from Transifex

Change-Id: I43633b01e6a9b93b4d291f2bc900ea9851469921
Sync "Prevent races in opportunistic db test cases"

This sync change Id82090743e8b93f6c960de066fd05d8a2f9fb934 from
oslo-incubator

Opportunistic db test cases create schemas on demand, so that each
test case which inherits the base test case class, will get its own
db schema (i. e. races between tests are not possible).

In order to do schema provisioning we have to connect to RDBMS server
first. So far we've been connecting to the openstack_citest database,
which is guaranteed to exist on CI nodes. It turns out, there are a
few test cases in Nova (maybe in other projects as well), that drop
and recreate the openstack_citest database. If they happen to do that
when the opportunistic db fixture is in the middle of provisioning a
schema, those tests will fail (as there is an open session to the
database and thus it can't be dropped).

This can be solved easily by changing the way we provision new
schemas in opportunistic db test cases as actually, we don't have to
connect to the openstack_citest database at all:

 - for MySQL we can use an empty db name to connect to MySQL server,
   but not to a particular database
 - PostgreSQL requires us to specify the database name. We can use
   the service postgres database here (PostgreSQL shell utils such
   as createdb, createuser, etc use it for the very same reason)

Partial-Bug: #1328997

Change-Id: Idc62b24b6ae51ea14f10e39d7b14115e36069637
Updated from global requirements

Change-Id: I4884005a65e9992447587588bac89b85b087bc4a
Merge "Network: ensure that ports are 'unset' when instance is deleted"
Merge "Sync "Prevent races in opportunistic db test cases""
Merge "Check the network_info obj type before invoke wait function"
VMware: update file permissions and mode

Commit 17c5f92b6ab552f1ac21f8ce332053e35d1de919 set the wrong
file permissions and mode.

Change-Id: Iffcd2961161cc6380bb434628183fe7162cf6e8a
Cleanup and gate on pep8 rules that are stricter in hacking 0.9

Fix up pep8 failures for rules we previously gated on, but that are now
stricter in there enforcement.

Ignore E251 due to https://github.com/jcrocholl/pep8/issues/301

Change-Id: I62b33e97c44c4a5be436b381cfbdaeb31cd2638b
Add bulk create/destroy functionality to FloatingIP

This adds bulk create and destroy functionality to the FloatingIP
object, which in turn requires that the DB API return the list of
IPs that it created.

This also fixes a bug in the load routine for FloatingIP, which
would previously try to load a FixedIP object with id=None,
which doesn't make sense.

Related to blueprint compute-manager-objects-juno

Change-Id: Ie3c7a5e672fcdb3e0df3d0e46b905b237dcfa182