No longer any need to pass admin context to aggregate DB API methods

No longer need to pass an elevated context because the check of
admin has been removed in following patch:
https://review.openstack.org/67026

This patch removes the admin context elevated
method in nova/scheduler sub-folder since it's not required any more.

Change-Id: Ib768c97c9e84a9db44d866fa5fbfc4665ad7c73e
Partial-Bug: #1279216
Nova utils: add in missing translation

The method validating the hypervisor version would throw
an exception that did not have a translation.

Change-Id: I6284d313a553757b90bd6b2c4946c6aa84ea17cd
Scheduler: remove test scheduling methods that are not used

The test code no longer uses the following methods:
 - fake_get_group_filtered_hosts
 - fake_get_group_filtered_affinity_hosts

This patch removes those methods.

Change-Id: I0a005fb1301ce78cc2ec365429015c08d58f446e
Fix AvailabilityZone check for hosts in multiple aggregates

Check for multiple AZ was distinct when adding an host to an aggregate
and when updating an aggregate (or its metadata), which was leading to
possible corner-cases where logics were different.

This patch proposes a single way for checking for both API methods and
also adds an extra control on the possibility to have an aggregate with
its metadata AZ set to the configuration default value (eg. 'nova') and
where it was possible to subsequently have one host in two disctinct AZs
Fixed that scenerio as it is a gap, Nova as of now preventing to have
one host in two distinct AZs.

Change-Id: Ida6aa507f34f36a436adaafe1f7609963c59a9bc
Partial-Bug: #1277230
Add specific doc build option to tox

Add doc venv to align with other OpenStack projects (such as keystone,
heat and nova-specs) who are using 'docs' for there doc builds.

Change-Id: I37771db339d864b2b938d1c02d15d38b07093493
Ignore errors when deleting non-existing vifs

If an instance fails during its network creation (for example
if the network-vif-plugged event doesn't arrive in time) a
subsequent delete will also fail when it tries to delete the
vif, leaving the instance in a Error(deleting) state.

This can be avoided by including the "--if-exists" option to
the ovs-vsctl command so that the command doesn't fail if the
vif doesn't exist.

Change-Id: Ic613c9d6e0619f51fcd931e62f6d6bce404ebe81
Closes-bug: #1308544
Set the volume access mode during volume attach

cinderclient supports an optional keyword argument named 'mode'
for volumes.attach(), which allows you to set the access mode of
a volume at attach time. The keyword argument is not passed in
by nova when it attaches a read-only volume, making the volume
access mode default to 'rw' instead of the desired 'ro'. This
patch finds the access mode of the volume and passes it to
cinderclient, so the correct access mode is used at attach
time.

Change-Id: Ib62f454292e60114f96debbad96c4a356dc32a21
Closes-Bug: #1307088
Fix a minor comments error

Seems a copy-paste error. This test case is test bigger than oversize
limitation.

Change-Id: Ie0ce557032b62a0d3a955a5b7ce8bbbe41fef7cd
Network: replace neutron check with decorator

Commit a581823ecee286c5c6329c9c373f0fea0b6b729f added in a
validator that would check if neutron was enabled.

It is much cleaner to do this as a decorator. The decorator
here needs to preserve the function signature.

Change-Id: I355e4bcee5e93a7f1459672e10ad7793d2bf8a49
Rename quota_injected_file_path_bytes

Rename quota_injected_file_path_bytes to
quota_injected_file_path_length, and change help message to
'Length of injected file path'.

DocImpact

Change-Id: Id6b9c03b37664601937e4bc9d4a8d8026c4d3b63
Closes-bug: 1258175
Fix wrong quota calculation when deleting a resizing instance

When deleting a resizing instance that has not yet finished resizing,
quotas should be adjusted for the old flavor type. Instead it
incorrectly use values from the new flavor.

This was previously fixed in bug 1099729, but has since resurfaced
with the change to use objects in delete methods.

Change-Id: I9bc68abc0d86784c642483dd0cc46c6b9c6a0c69
Closes-Bug: #1277422
Sync common db and db/sqlalchemy

To use db/sqlalchemy/test_base.py we need the test.py file, which does
not synced to Nova. Latest common db.sqlalchemy uses oslotest. The problem with
unexisting common/test.py is solved implicitly with synchronization of common
code.

Contains oslo-incubator commits:

54f7e7f Prevent races in opportunistic db test cases
8a0f581 Use oslotest instead of common test module
4a591ea Start ping listener also for postgresql
f0e50ed Add a warning to not use get_table for working with ForeignKeys
2fd457b Ignore migrate versioning tables in utf8 sanity check
9fed4ed Fix Keystone doc build errors with SQLAlchemy 0.9
f7705f3 Make table utf-8 charset checking be optional for DB migration
5b7e61c Dispose db connections pool on disconnect
295fcd9 Do not use the 'extend' method on a dict_items object
d1988b9 Set sql_mode callback on connect instead of checkout
a1a8280 Fix excessive logging from db.sqlalchemy.session
dc2d829 Add lockutils fixture to OpportunisticTestCase
d10f871 Adapt DB provisioning code for CI requirements
5920bed Make db utils importable without migrate
9933bdd Get mysql_sql_mode parameter from config
96a2217 Prevent incorrect usage of _wrap_db_error()
6cab37c Python3: define a __next__() method for ModelBase
20a7510 Add from_config() method to EngineFacade
fea119e Drop special case for MySQL traditional mode, update unit tests
a584166 Make TRADITIONAL the default SQL mode
5b9e9f4 Fix doc build errors in db.sqlalchemy

Closes-Bug: #1255532
Change-Id: If26b648dee95abc5aa0bf80d3111265b00141b6d
Fix CIDR values denoting hosts in PostgreSQL

We use PostgreSQL INET type to store both hosts IP addresses
and CIDR addresses of IP subnets. PostgreSQL actually has
INET and CIDR data types, which work a bit different.

The one thing they are different at, is the representation of
retrieved values denoting a single host in IP subnet (i. e. an
IP address followed by a netmask which has a value of /32 for
IPv4 address and /128 for IPv6 appropriately): if such value
is stored in column of type INET, it will be returned by SELECT
without a netmask part, so instead of 37.37.37.37/32 one will
receive 37.37.37.37.

For more details, please see:
http://www.postgresql.org/docs/8.2/static/datatype-net-types.html

While existing behaviour is completely OK, it depends on DB backend
which is used. To ensure, that CIDR values denoting a single host in IP
subnet, have the same representation in spite of chosen DB backend,
an SQLAlchemy model hook is provided, which adds a netmask value of /32
or /128 for retrieved values of IPv4 or IPv6 addresses appropriately.

Blueprint: db-api-tests-on-all-backends
Co-Authored-By: Roman Podolyaka <rpodolyaka@mirantis.com>

Change-Id: If36f0f9133b6c7cc6f20ba189a9f5b65001811d1
Defer applying iptable changes when nova-network start

When restarting nova-network service, nova-network will initialize iptables and
remove old rules, then add floating ip iptable forward rules one by one.

If there are a lot of floating ips on one host, the process of adding floating
ip rules could last several minutes, which will cause those floating ips that
last added iptable rules disconnected for same time.

This patch defers applying nova-network iptable rule changes until
initialization is done instead of adding floating ip forward rules one by one.

Change-Id: I12651850b44790aef3817fedacae050bff57e364
Closes-bug: 1308390
Allow admin user to get all tenant's floating IPs

When getting floatingips by Nova API, the results will be filtered
with the 'tenant_id'.
So, we can only get the floatingips belonging the tenant of current
context.
When ceilometer invokes novaclient to list floatingips, it will get an
empty list because the tenant is 'service'.
we should allow an admin user to index all tenants's floatingip by adding a
parameter 'all_tenants'.

Part1:
    part1 try to implement this and part2 will remove the unused codes.

Change-Id: I7ab1d5ff463fc29928f6811f846c9e204390a412
Closes-bug: #1262124
Enforce query order for getting VIFs by instance

When spawn VM with multiple nics, during the phase of VM network allocations
using nova-network, the networks pass into driver layer will get reordered
by vifs queried by instance. We have to enforce the query order with vif
creation order rather than rely on the database backend's order.

Change-Id: I51333cc82c03e20d89d3ea548ff164fe71b5ba26
Add a reference to the nova developer documentation

I've showed these docs to more people than I care to. It would be
extremely helpful to reference them in HACKING, which is generally
the first thing people read if they intend to hack.

Change-Id: I9194d98f5525e29711b4a1b26414a50b8ceba525
On rebuild check for null image_ref

While parsing the body passed in a rebuild request there is no check to
ensure that image_ref is not None.  This leads to a raised exception and
500 response by the API.

Change-Id: Ic9e51a761d0c1f30d55f4a167c876a6aa4f00a41
Closes-bug: 1312278
Merge "Add a reference to the nova developer documentation"
Make hacking test more accurate

Hacking check returns tuple or None, assertTrue/assertFalse doesn't
handle the return results properly. For example, non-tuple return value
also can pass the hacking unit test method.

  * use assertEqual or assertIsInstance instead of assertTrue
  * use assertIsNone instead of assertFalse
  * remove duplicated test data in method test_virt_driver_imports

Change-Id: Iee8b466c3ab2c6437144367cdb874bd2f8839984