Nova utils: add in missing translation

The method validating the hypervisor version would throw
an exception that did not have a translation.

Change-Id: I6284d313a553757b90bd6b2c4946c6aa84ea17cd
Scheduler: remove test scheduling methods that are not used

The test code no longer uses the following methods:
 - fake_get_group_filtered_hosts
 - fake_get_group_filtered_affinity_hosts

This patch removes those methods.

Change-Id: I0a005fb1301ce78cc2ec365429015c08d58f446e
Ignore errors when deleting non-existing vifs

If an instance fails during its network creation (for example
if the network-vif-plugged event doesn't arrive in time) a
subsequent delete will also fail when it tries to delete the
vif, leaving the instance in a Error(deleting) state.

This can be avoided by including the "--if-exists" option to
the ovs-vsctl command so that the command doesn't fail if the
vif doesn't exist.

Change-Id: Ic613c9d6e0619f51fcd931e62f6d6bce404ebe81
Closes-bug: #1308544
Fix wrong quota calculation when deleting a resizing instance

When deleting a resizing instance that has not yet finished resizing,
quotas should be adjusted for the old flavor type. Instead it
incorrectly use values from the new flavor.

This was previously fixed in bug 1099729, but has since resurfaced
with the change to use objects in delete methods.

Change-Id: I9bc68abc0d86784c642483dd0cc46c6b9c6a0c69
Closes-Bug: #1277422
Sync common db and db/sqlalchemy

To use db/sqlalchemy/test_base.py we need the test.py file, which does
not synced to Nova. Latest common db.sqlalchemy uses oslotest. The problem with
unexisting common/test.py is solved implicitly with synchronization of common
code.

Contains oslo-incubator commits:

54f7e7f Prevent races in opportunistic db test cases
8a0f581 Use oslotest instead of common test module
4a591ea Start ping listener also for postgresql
f0e50ed Add a warning to not use get_table for working with ForeignKeys
2fd457b Ignore migrate versioning tables in utf8 sanity check
9fed4ed Fix Keystone doc build errors with SQLAlchemy 0.9
f7705f3 Make table utf-8 charset checking be optional for DB migration
5b7e61c Dispose db connections pool on disconnect
295fcd9 Do not use the 'extend' method on a dict_items object
d1988b9 Set sql_mode callback on connect instead of checkout
a1a8280 Fix excessive logging from db.sqlalchemy.session
dc2d829 Add lockutils fixture to OpportunisticTestCase
d10f871 Adapt DB provisioning code for CI requirements
5920bed Make db utils importable without migrate
9933bdd Get mysql_sql_mode parameter from config
96a2217 Prevent incorrect usage of _wrap_db_error()
6cab37c Python3: define a __next__() method for ModelBase
20a7510 Add from_config() method to EngineFacade
fea119e Drop special case for MySQL traditional mode, update unit tests
a584166 Make TRADITIONAL the default SQL mode
5b9e9f4 Fix doc build errors in db.sqlalchemy

Closes-Bug: #1255532
Change-Id: If26b648dee95abc5aa0bf80d3111265b00141b6d
Fix CIDR values denoting hosts in PostgreSQL

We use PostgreSQL INET type to store both hosts IP addresses
and CIDR addresses of IP subnets. PostgreSQL actually has
INET and CIDR data types, which work a bit different.

The one thing they are different at, is the representation of
retrieved values denoting a single host in IP subnet (i. e. an
IP address followed by a netmask which has a value of /32 for
IPv4 address and /128 for IPv6 appropriately): if such value
is stored in column of type INET, it will be returned by SELECT
without a netmask part, so instead of 37.37.37.37/32 one will
receive 37.37.37.37.

For more details, please see:
http://www.postgresql.org/docs/8.2/static/datatype-net-types.html

While existing behaviour is completely OK, it depends on DB backend
which is used. To ensure, that CIDR values denoting a single host in IP
subnet, have the same representation in spite of chosen DB backend,
an SQLAlchemy model hook is provided, which adds a netmask value of /32
or /128 for retrieved values of IPv4 or IPv6 addresses appropriately.

Blueprint: db-api-tests-on-all-backends
Co-Authored-By: Roman Podolyaka <rpodolyaka@mirantis.com>

Change-Id: If36f0f9133b6c7cc6f20ba189a9f5b65001811d1
Allow admin user to get all tenant's floating IPs

When getting floatingips by Nova API, the results will be filtered
with the 'tenant_id'.
So, we can only get the floatingips belonging the tenant of current
context.
When ceilometer invokes novaclient to list floatingips, it will get an
empty list because the tenant is 'service'.
we should allow an admin user to index all tenants's floatingip by adding a
parameter 'all_tenants'.

Part1:
    part1 try to implement this and part2 will remove the unused codes.

Change-Id: I7ab1d5ff463fc29928f6811f846c9e204390a412
Closes-bug: #1262124
Call _validate_instance_group_policy in _build_and_run_instance

The _validate_instance_group_policy check was added to the boot process
in run_instance but was not added to build_and_run_instance which will
be replacing run_instance.  This ports it over to retain parity between
the two code paths.

Part of bp remove-cast-to-schedule-run-instance

Change-Id: I197794ffe04d7460cf67216b9ae67ed7c5f790d9
Add VIF mac address to fixed_ips in notifications

Currently the payload of the notification only contains fixed_ips from VIFs,
the mac address from VIFs is also a very important information and
should be included with the payload of the notification.

This patch add the VIF mac address to each fixed_ips as a new attribute
named "vif_mac".
Notification will looks like:
 	u'fixed_ips': [{u'address': u'192.168.44.1',
                           u'floating_ips': [],
                           u'label': u'publica',
                           u'meta': {},
                           u'type': u'fixed',
                           u'version': 4,
                           u'vif_mac': u'fa:16:3e:93:ec:d5'}],

Closes-Bug: #1320374
Change-Id: I071b8fa75fecdae556c7ad72878ca8f86cf57139
Fix tests/compute direct use of instance module objects

This switches the nova/tests/compute code to use nova.objects.Instance* vs
nova.objects.instance.Instance* to match the nova/compute changes in
the previous patch.

Partial-Blueprint: object-subclassing

Change-Id: I71f8705a22cbcb4ec171d80c1199bf28ccef3796
Fix direct use of aggregate module objects

This replaces all uses of nova.objects.aggregate.Aggregate* with
nova.objects.Aggregate*.

Partial-Blueprint: object-subclassing

Change-Id: Ib1abb5a3d85c2bc4aae9951da97bea24fe6327de
Fix ReservationTestCase on PostgreSQL

ReservationTestCase setUp() uses a hard coded id value
of quota usage, which might be different if a DB is not
recreated for each test case. Refactor tests a bit to use
a common setUp().

Blueprint: db-api-tests-on-all-backends

Co-Authored-By: Roman Podoliaka <rpodolyaka@mirantis.com>
Change-Id: I7e8ce94fa36392a508375ca3626ff0a8441a7066
Fix InstanceActionTestCase on PostgreSQL/MySQL

test_instance_action_get_by_instance_and_action() of
InstanceActionTestCase mistakenly uses the same value
of request_id and then relies on SQLite specific order
of retrieved rows to grab a required value. This causes
test to fail on MySQL and PostgreSQL.

Blueprint: db-api-tests-on-all-backends

Co-Authored-By: Roman Podoliaka <rpodolyaka@mirantis.com>
Change-Id: I6e3e100627ef7efc31c41aaa5a38d45e828fcf39
Merge "Nova utils: add in missing translation"
Catch InstanceNotFound exception if migration fails

When a VM with an attached iSCSI disk fails to migrate, the rollback methods
does not detach the disk from target host. What happens is _lookup_by_name()
fails, since the VM does not exist on the target host. In detach_volume(), it
is supposed to print a warning based on the correct error code being returned,
instead of throwing the exception. However, this is not happening, because
_lookup_by_name() throws an InstanceNotFound exception, rather than a
libvirt.libvirtError exception. So we also need to catch InstanceNotFound
exception, so that detach_volume() can continue to execute as expected.

Change-Id: I396c23d8137bf1e2834769c5483d2494506949ad
Closes-Bug: #1321082
Merge "Call _validate_instance_group_policy in _build_and_run_instance"
Merge "Fix tests/compute direct use of instance module objects"
VMware: spawn refactor - phase 1 - copy_virtual_disk

Move copy_virtual_disk to vm_util and change all copy_virtual_disk
in vmops to use it

partial blueprint vmware-spawn-refactor
Co-authored-by: Tracy Jones <tjones@vmware.com>

Change-Id: I0cda2bd465dc8a9de1b7ff4c1220e77b9c797471
Merge "Fix direct use of aggregate module objects"
Merge "Allow admin user to get all tenant's floating IPs"