No longer any need to pass admin context to aggregate DB API methods

No longer need to pass an elevated context because the check of
admin has been removed in following patch:
https://review.openstack.org/67026

This patch removes the admin context elevated
method in nova/scheduler sub-folder since it's not required any more.

Change-Id: Ib768c97c9e84a9db44d866fa5fbfc4665ad7c73e
Partial-Bug: #1279216
Nova utils: add in missing translation

The method validating the hypervisor version would throw
an exception that did not have a translation.

Change-Id: I6284d313a553757b90bd6b2c4946c6aa84ea17cd
Scheduler: remove test scheduling methods that are not used

The test code no longer uses the following methods:
 - fake_get_group_filtered_hosts
 - fake_get_group_filtered_affinity_hosts

This patch removes those methods.

Change-Id: I0a005fb1301ce78cc2ec365429015c08d58f446e
Ignore errors when deleting non-existing vifs

If an instance fails during its network creation (for example
if the network-vif-plugged event doesn't arrive in time) a
subsequent delete will also fail when it tries to delete the
vif, leaving the instance in a Error(deleting) state.

This can be avoided by including the "--if-exists" option to
the ovs-vsctl command so that the command doesn't fail if the
vif doesn't exist.

Change-Id: Ic613c9d6e0619f51fcd931e62f6d6bce404ebe81
Closes-bug: #1308544
Network: replace neutron check with decorator

Commit a581823ecee286c5c6329c9c373f0fea0b6b729f added in a
validator that would check if neutron was enabled.

It is much cleaner to do this as a decorator. The decorator
here needs to preserve the function signature.

Change-Id: I355e4bcee5e93a7f1459672e10ad7793d2bf8a49
Rename quota_injected_file_path_bytes

Rename quota_injected_file_path_bytes to
quota_injected_file_path_length, and change help message to
'Length of injected file path'.

DocImpact

Change-Id: Id6b9c03b37664601937e4bc9d4a8d8026c4d3b63
Closes-bug: 1258175
Downgrade log when attach interface can't find resources

Downgrade the log when a resource (e.g. instance is deleted concurrently)
can't be found, no need to LOG.exception because an exception
is already raised and no need to log again.

Change-Id: I260e89c8deab11bc9707abbbbc7bb9a4f88640e3
Closes-Bug: #1322410
Fix wrong quota calculation when deleting a resizing instance

When deleting a resizing instance that has not yet finished resizing,
quotas should be adjusted for the old flavor type. Instead it
incorrectly use values from the new flavor.

This was previously fixed in bug 1099729, but has since resurfaced
with the change to use objects in delete methods.

Change-Id: I9bc68abc0d86784c642483dd0cc46c6b9c6a0c69
Closes-Bug: #1277422
Sync common db and db/sqlalchemy

To use db/sqlalchemy/test_base.py we need the test.py file, which does
not synced to Nova. Latest common db.sqlalchemy uses oslotest. The problem with
unexisting common/test.py is solved implicitly with synchronization of common
code.

Contains oslo-incubator commits:

54f7e7f Prevent races in opportunistic db test cases
8a0f581 Use oslotest instead of common test module
4a591ea Start ping listener also for postgresql
f0e50ed Add a warning to not use get_table for working with ForeignKeys
2fd457b Ignore migrate versioning tables in utf8 sanity check
9fed4ed Fix Keystone doc build errors with SQLAlchemy 0.9
f7705f3 Make table utf-8 charset checking be optional for DB migration
5b7e61c Dispose db connections pool on disconnect
295fcd9 Do not use the 'extend' method on a dict_items object
d1988b9 Set sql_mode callback on connect instead of checkout
a1a8280 Fix excessive logging from db.sqlalchemy.session
dc2d829 Add lockutils fixture to OpportunisticTestCase
d10f871 Adapt DB provisioning code for CI requirements
5920bed Make db utils importable without migrate
9933bdd Get mysql_sql_mode parameter from config
96a2217 Prevent incorrect usage of _wrap_db_error()
6cab37c Python3: define a __next__() method for ModelBase
20a7510 Add from_config() method to EngineFacade
fea119e Drop special case for MySQL traditional mode, update unit tests
a584166 Make TRADITIONAL the default SQL mode
5b9e9f4 Fix doc build errors in db.sqlalchemy

Closes-Bug: #1255532
Change-Id: If26b648dee95abc5aa0bf80d3111265b00141b6d
Fix CIDR values denoting hosts in PostgreSQL

We use PostgreSQL INET type to store both hosts IP addresses
and CIDR addresses of IP subnets. PostgreSQL actually has
INET and CIDR data types, which work a bit different.

The one thing they are different at, is the representation of
retrieved values denoting a single host in IP subnet (i. e. an
IP address followed by a netmask which has a value of /32 for
IPv4 address and /128 for IPv6 appropriately): if such value
is stored in column of type INET, it will be returned by SELECT
without a netmask part, so instead of 37.37.37.37/32 one will
receive 37.37.37.37.

For more details, please see:
http://www.postgresql.org/docs/8.2/static/datatype-net-types.html

While existing behaviour is completely OK, it depends on DB backend
which is used. To ensure, that CIDR values denoting a single host in IP
subnet, have the same representation in spite of chosen DB backend,
an SQLAlchemy model hook is provided, which adds a netmask value of /32
or /128 for retrieved values of IPv4 or IPv6 addresses appropriately.

Blueprint: db-api-tests-on-all-backends
Co-Authored-By: Roman Podolyaka <rpodolyaka@mirantis.com>

Change-Id: If36f0f9133b6c7cc6f20ba189a9f5b65001811d1
Allow admin user to get all tenant's floating IPs

When getting floatingips by Nova API, the results will be filtered
with the 'tenant_id'.
So, we can only get the floatingips belonging the tenant of current
context.
When ceilometer invokes novaclient to list floatingips, it will get an
empty list because the tenant is 'service'.
we should allow an admin user to index all tenants's floatingip by adding a
parameter 'all_tenants'.

Part1:
    part1 try to implement this and part2 will remove the unused codes.

Change-Id: I7ab1d5ff463fc29928f6811f846c9e204390a412
Closes-bug: #1262124
Enforce query order for getting VIFs by instance

When spawn VM with multiple nics, during the phase of VM network allocations
using nova-network, the networks pass into driver layer will get reordered
by vifs queried by instance. We have to enforce the query order with vif
creation order rather than rely on the database backend's order.

Change-Id: I51333cc82c03e20d89d3ea548ff164fe71b5ba26
Remove spaces from SSH public key comment

An SSH key generated by Nova contains the comment 'Generated by Nova'. Older
versions (prior to 0.7.2) of cloud-init trip over the spaces in the comment
and as a result of that the key injected into the root account is not disabled
by cloud-init. Yes, it's a cloud-init bug but it's also a regression in the
sense that older OpenStack installations (Essex and older) don't contain
spaces in Nova generated key comments and thus older cloud-init's are not
affected in these environments.

This patch replaces the spaces with dashes, i.e., 'Generated-by-Nova'.

Spaces were introduced by commit: 114109dbf4094ae6b6333d41c84bebf6f85c4e48
Ubuntu cloud images with cloud-init < 0.7.2: 10.04 LTS and 12.04 LTS
cloud-init bug report: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1220273

Closes-Bug: #1297685
Change-Id: I1761f61dfbba58be98351ae4a51884b03268cf09
XenAPI: Add logging information for cache/download duration

We would like more explicit information on when images are cached.
This commit logs information on whether we are attempting to cache,
whether we downloaded, and how long it took to download.  This
log information can then be used for further processing elsewhere.

Change-Id: Ia971008fabec06c4ba8fc1db4b67474f52604b17
Cells: Pass instance objects to build_instances

When the cells scheduler creates instance records in its local db it
should keep the instance objects created.  build_instances expects a
list of instance objects so to keep parity between cells and non-cells
the cells scheduler needs to call build_instances with its list of
instance objects.

Part of bp remove-cast-to-schedule-run-instance

Change-Id: I4a1a96693f9c3cb92ffdccfebaeb2432584d4105
Add refresh=True to get_available_nodes call in build_and_run_instance

The run_instance code path passes refresh=True to get_available_nodes in
order to get fresh data from that call.  The build_and_run_instance
patch doesn't pass that argument and therefore may not get the latest
data.  This ports the argument over to that path to prepare for using
build_and_run_instance.

Part of bp remove-cast-to-schedule-run-instance

Change-Id: If857fdb4125d4af656f86de3e963987dd32c208b
Call _validate_instance_group_policy in _build_and_run_instance

The _validate_instance_group_policy check was added to the boot process
in run_instance but was not added to build_and_run_instance which will
be replacing run_instance.  This ports it over to retain parity between
the two code paths.

Part of bp remove-cast-to-schedule-run-instance

Change-Id: I197794ffe04d7460cf67216b9ae67ed7c5f790d9
Add VIF mac address to fixed_ips in notifications

Currently the payload of the notification only contains fixed_ips from VIFs,
the mac address from VIFs is also a very important information and
should be included with the payload of the notification.

This patch add the VIF mac address to each fixed_ips as a new attribute
named "vif_mac".
Notification will looks like:
 	u'fixed_ips': [{u'address': u'192.168.44.1',
                           u'floating_ips': [],
                           u'label': u'publica',
                           u'meta': {},
                           u'type': u'fixed',
                           u'version': 4,
                           u'vif_mac': u'fa:16:3e:93:ec:d5'}],

Closes-Bug: #1320374
Change-Id: I071b8fa75fecdae556c7ad72878ca8f86cf57139
Register objects in more services

We should be making sure we're registering objects in the services that
use them. In patches that follow, we'll stop importing the specific
object modules under nova/objects/. Also, we should register objects
after the config has been parsed, logging setup, and things monkey
patched. Also, also, also: let's import all objects and put the imports
in alphabetical order.

Change-Id: I07f4584c43bd88960d543962a162cd1dd8dea09a
Address issues with objects of same name

There are at least a couple of cases where we may (in the future) introduce
multiple objects that have the same name:

1) When we subclass an object to override internal behavior, presenting
the same version of the object.

2) When we want to introduce multiple versions of objects either via
subclassing or completely separate classes if differences are large
enough to warrant that.

There was a rough plan to support this, however we're currently very
broken. The code before this patch would happily track every one of the
objects with the same name. However, in the case where we see the same
version during 'object registration', we should actually always take the
newest one that we see, replacing the old one.

Also, throughout nova, we have code such as:

from nova.objects import instance as instance_obj

inst = instance_obj.Instance.<....>

This means we're always using the base object, which may or may not be
the correct ('newest') one.

This patch fixes the object registration and adds an attribute to the
'nova.objects' module for every object that is registered, setting the
attribute to the newest version we see.

This will allow us to drop a number of object module imports in favor
of simply importing nova.objects. The example above would become:

from nova import objects

inst = objects.Instance.<....>

Because the object registration ensures objects.Instance is the newest
version, we'll be golden.

Partial-Blueprint: object-subclassing

Change-Id: I04694ecd40e0d4ec40a87d319f2e689060b11651