Sync db.models.Instance* with migrations.

This patch syncs models with migrations for:
    - InstanceIdMapping
    - InstanceMetadata
    - InstanceSystemMetadata
    - InstanceTypeExtraSpecs
    - InstanceTypeProjects
    - InstanceTypes
    - InstanceGroupMember
    - InstanceGroupPolicy
    - InstanceGroupMetadata

Blueprint: db-sync-models-with-migrations

Change-Id: Ice28b9d545f60f7df381e3fcf2c86a2b8e7350d3
Sync db.models.Security* and db.models.Volume*.

This patch syncs models with migrations.

Change-Id: I2e4a4cf28376d933be47bc496933e6964e19a56e
Clarify flavorid vs instance_type_id in db

And add comment about future direction.

Partially implements bp flavor-instance-type-dedup

Change-Id: Ia6c1bad5c3cc35e772c32803c27a50a7728045e5
Sync 10 DB models and migrations.

This patch syncs models with migrations for:
- BlockDeviceMapping
- IscsiTarget
- FixedIp
- FloatingIp
- KeyPair
- Migration
- Network
- ProviderFirewallRule
- Reservation
- S3Image

Blueprint: db-sync-models-with-migrations

Change-Id: I8e16375fe48744513167272e956cdbfdbfd8af23
Moved tests for server._action_rebuild

Move tests to separate class as a preparation for code
deduplication.

This work continues in https://review.openstack.org/34556

bp nova-tests-code-duplication

Change-Id: I985232fa0a9b2d5b73034047b06138a8a3d7e390
Code dedup in tests for server._action_rebuild

Externed common parts to setUp():
- request body
- request object
- test image UUID
- test image URL
These parts were equal in the most part of the methods.

bp nova-tests-code-duplication

Change-Id: Ie29dd090075915a4065ee395b4e50fc178f72f3b
Code dedup in test_libvirt_volume

- externed common assertions to methods
- externed common parts to setUp():
  - test disk info
  - test connector
  - test volume name, location, iqn, data, uuid
  - test user
  - test fake connection

bp nova-tests-code-duplication

Change-Id: I83bfd8b597121c9de091283845b841680e150e76
Code dedup in class ImageXMLSerializationTest

Externed common parts to setUp():
- fixture with test image data
- test image
- dictionary with test images
- test serializer
These parts were equal in the most part of the methods.

bp nova-tests-code-duplication

Change-Id: I8394c7ccbd0d71ffcf5f35c64a5c307dbf579364
Add expected_errors for extension deferred_delete v3

Partially implements bp v3-api-extension-versioning

Change-Id: Ia63f6cd947c20a05e84385f2401d6b869ebeb456
Remove locals() from various places.

fixes bug 1171936

Remove usage of locals() for string formatting

Change-Id: Ib05538095086ddefdb486c84da506af662ec5c9b
Fixes wrong action comment 'lock' to 'unlock'

Change-Id: I2f285b405b598d69f80504b040b4a0e32d80020b
Remove project_id from alternate image link path

Removes the project id from the alternate link path
generated for the v3 API os-images extension.

Also fixes issue with os.path.join being used to generate
urls which will break on Windows.

Partially implements blueprint nova-v3-api

Change-Id: Icc34359cc935920e2d4780849d9310c4961c8f74
Add missing tests for console_* methods.

There was no tests in test_db_api for console_* methods.
Add tests to ensure that all works.

blueprint db-api-tests

Change-Id: Ic3d5a3dd60ba50b561126279c00ca07022aaa20e
Missed message -> msg_fmt conversion

We recently changed NovaException's to define `msg_fmt` instead of `message`
on the class. Looks like this exception was missed for some reason.

Fixes bug 1202255

Change-Id: If828bf42fa077665d1d92e08758fbddd342ab626
ec2-api: Disable describing of instances using deleted tags as filter.

Previously, one could describe instances using a deleted tag filter
that was once associated to the instance. The current change disables
describing instances via a deleted tag. Also,minor refactor in the
same function.

fixes bug #1197242

Change-Id: Iac6aa85b69d5a6dc563c00f3497cdab08e178ec1
Fix inconsistency between Nova-Net and Neutron

The neutron client raises NeutronClientException (status_code 404)
that is not unknown for nova, and is translated in 500 computeFault.
Catching this exception in nova/nova/network/neutronv2/api.py and
raising more generic exception as exception.NotFound solves the problem.

Added also an unitary test to test neutronv2 api

bug #1160309
bug #1200175

Change-Id: Ib2f1a0d49a12114b28f0cd5ce0bb7119f864d1a6
Send updated aggregate to compute on add/rm host

The XenAPI driver relies on getting an up-to-date version of the
aggregate during add_host_to_aggregate and remove_host_from_aggregate
This change ensures that happens.

Fixes bug 1161619
Change-Id: I764e1fc3b9ff20b6eda8f6ec00849f2164de0a49
Add ability to factor in per-instance overheads

Currently free memory calculations in the compute layer use a single
value to account for all memory overhead needed by a host.  This value
is: CONF.reserved_host_memory_mb

This patch adds the ability to factor in per-instance virtualization
overheads into usage calculations.  Virtualization overhead may differ
based on the flavor or other factors such as number of vcpus on a host.
This feature's goal is to provide the ability to do more accurate
computations of a host's overhead without having to pre-configure a
best guess in advance.

The actual calculation of overhead is left up to each virt driver to
implement.  This patch defaults to zero overhead for compatibility.

Change-Id: I4febfa2c837e0f80cee85ca74553829c960a3efc
Fix DB access when refreshing the network cache.

When calling ComputeManager.attach_interface(), the following happens:

1) self.network_api.allocate_port_for_instance() is called from within
   ComputeManager.attach_interface();
2) refresh_cache() is called, since it decorates allocate_port_for_instance();
3) refresh_cache calls update_instance_cache_with_nw_info(), with
   "conductor_api=None" since 'conductor_api' cannot be found in the kwargs;
4) update_instance_cache_with_nw_info() calls
   api.db.instance_info_cache_update();
5) Since the call to the database should be made through the conductor, a
   DBNotAllowed exception is raised, which prevents the cache from being updated.

This patch fixes the call to allocate_port_for_instance (step 1) so that
'conductor_api' can be found in the kwargs (step 3).

When using ComputeManager.detach_interface, the exact same problem arises (with
deallocate_port_for_instance instead of allocate_port_for_instance).

This is a first step towards fixing bug #1197192.

Change-Id: I647795907b22790759d3be74afcaf70b712cb338
Handle instance objects in conductor compute_stop

The compute manager is sending instance objects to
compute_stop in conductor but conductor's compute_stop
RPC proxy wasn't handling objects. This patch fixes that
and also handles compatibility with older installations
using the dict-style format in the conductor compute_stop
RPC proxy.

Fixes bug 1195849

Change-Id: Ib678df424a58a0d7023ce3b2bdd5001998daf398