Avoid traceback logs from simple tenant usage extension

Avoid generating traceback logs when invalid formatted datetime
parameters are passed to the simple tenant usage
extension. Exceptions have not been correctly handled by the
os-simple-tenant usage extension which results in tracebacks
being output to the log files. This patch correctly handles
exceptions as a result of badly formatted datetime strings.

Conflicts:
        nova/tests/api/openstack/compute/contrib/test_simple_tenant_usage.py

Change-Id: Iecd42239357b96450b6530c0486b27c68995c37f
Partial-Bug: 1300972
(cherry picked from commit f60c72fac04a97b321a7f2bd0865c1ae81250882)
Fix _parse_datetime in simple tenant usage extension

_parse_datetime in os-simple-tenant-usage incorrectly attempts
to parse the datetime string even when it is None. As a result
a 400 BadRequest is returned. This has the overall effect of making
the start and end datetime parameters compulsory when they are meant
to be optional.

Note that this restores the behavior where start/end are optional which is
in contradiction to the API docs, but is the long standing API behaviour.
This was an API change which was accidentally applied in commit
I8e0e870727d687da165c809ffb7a4456bff81122 as part of some nova internal
changes but not picked up by the unitests

Change-Id: I00427379dc7aa39770f9a16ff026addb6e311735
Closes-Bug: 1300972
(cherry picked from commit 99e6ec9131c2e3eeb5fe3e1a4dc5a26e6302ebd2)
libvirt: Use VIR_DOMAIN_AFFECT_LIVE for paused instances

When a volume is attached to a paused instance, it does not
appear in the instance's block devices list (i.e. lsblk)
after the instance has resumed. A similar situation
happens when detaching a volume from a paused instance; the
block device continues to exists in the block devices list,
where it should not.  It was found that the volume is only
persisted in the domain's config settings and not its active
settings, since only the VIR_DOMAIN_AFFECT_CONFIG flag was used
on attach and detach volume.  In order to affect the active
settings, the VIR_DOMAIN_AFFECT_LIVE flag has to be added when
attaching and detaching volumes.

This change adjusts little about the unit test:
add two imports.

Change-Id: I9c90a410a7ecb91f5a4de28acee21fe7da49242c
Closes-Bug: #1242366
Related-Bug: #1299331
(cherry picked from commit ead9cfca93c9b5ca55e3ba269213c98d5e6e1d38)
Fixes Hyper-V resize down exception

The Hyper-V driver does not support resize down and is currently
raising an exception if the user attempts to do that, causing the
instance to go in ERROR state.

The driver should use the recently introduced instance faults
"exception.InstanceFaultRollback" instead, which will leave the
instance in ACTIVE state as expected.

Closes-Bug: #1354448

(cherry picked from commit 3a5919fd4af6c3b772397a5e7d90eebdf9b371af)

Conflicts:
	nova/virt/hyperv/migrationops.py

Change-Id: Ibaf8482562094cd2b3165dc62a907fa9e0e56e19
Made unassigned networks visible in flat networking

This change fixes a bug in Nova's
GET /v2/{tenant_id}/os-networks

The doc
http://docs.openstack.org/api/openstack-compute/2/content/GET_os-networks-v2_ListNetworks__v2__tenant_id__os-networks_ext-os-networks.html
says that "Lists networks that are available to the tenant".

When invoked by a non-admin user it was returning only networks
assigned to the user's tenant.  But in flat and flat DHCP nova
networking, networks CAN NOT be assigned to tenants --- thus a
non-admin user would get zero networks from this operation.

The fix was to make this operation conditionally use an option already
present in the lower-level code to "allow_none" when fetching the list
of networks, meaning to include networks whose project_id field in the
DB held "none" (meaning the network is not assigned to a tenant).  The
condition under which this is done is that the Nova configuration
option named network_manager contains the string
"'nova.network.manager.Flat" --- which is true for flat and flat DHCP
nova networking and false for VLAN nova networking.

Conflicts:
        nova/network/api.py
        nova/tests/network/test_api.py

NOTE(mriedem): The conflicts are due to the objects conversion
in Juno. This cherry pick adds the related network_get_all unit
tests from Juno and makes them work with the DB API in Icehouse.
Also note the oslo.config import in test_api is removed since it
was erroneously merged into master and fixed with commit
0e88148907e1db5218b96e2fa54bf9fee1cba74f but rather than backport
that and squash it with this, I've just removed it.

Change-Id: I64c5a3f31c912cca6b5b9987152ba7a9b3f5987d
Closes-Bug: #1327406
(cherry picked from commit 46e88320e6e6231550f3e2b40312c51f55e059f5)
remove test_multiprocess_api

This test suite should be removed for the following reasons.

1) it's not a unit test

2) it's failing randomly, a lot, and depends on timing of the host for
process start and signal handling which are provably not predicable on
slower machines

3) it's triggering testr bugs in the ways that it fails which means we
don't actually get any subunit in the fail scenario, which makes it
actually impossible to move forward on fixes.

Conflicts:
	nova/tests/integrated/test_multiprocess_api.py

Change-Id: Ieb7c02115c12cd9f6c9fa67691a643aed7632784
Closes-Bug: #1357578
(cherry picked from commit f3c99ba1efb1c039ec47dfdf29781f2e05f32b81)
Ensure info cache updates don't overwhelm cells

Cells does its own healing of the info cache outside
of get_instance_nw_info, so we have double healing
happening if info_cache_cells_update is called for every
get_instance_nw_info. This also overwhelms cells'
ability to keep the global db updated at scale. Similar
functionality exists in nova/network/api.py.

Conflicts:
        nova/network/neutronv2/api.py

NOTE(mriedem): This is due to base_api not existing
until Juno, where the refresh_cache decorator now
lives.

Change-Id: I58f681f3674d826e3dce4fa956e2eb0e4c60f82b
(cherry picked from commit 446d433f6fea7624acae371189239a82c94ad455)
Neutron: Atomic update of instance info cache

In the Neutron network API implementation, there is a race condition
between a thread performing periodic task to read and heal instance
network info cache and another thread servicing interface-attach or
interface-detach calls. This patch ensures that instance info cache is
read and then updated in a synchronized block to ensure atomicity.

Please see the bug report for more details.

Conflicts:
        nova/network/base_api.py
        nova/network/neutronv2/api.py
        nova/tests/network/test_neutronv2.py

NOTE(mriedem): Notes on conflicts:

1. base_api didn't exist until Juno so that's where refresh_cache
   lives now.
2. nova.network.neutronv2.api is using oslo.i18n in Juno rather
   than the old gettextutils import.
3. test_neutronv2 didn't start using mock until Juno and the change
   that added the test class is probably something that we won't
   be backporting, so the test class definition is brought back
   with this change for the new test case.

Change-Id: Ifc76f2b1cce834b3c9927359ac9b957bc9f9c65f
Closes-Bug: #1326183
Co-Authored-By: Dan Smith <dansmith@redhat.com>
(cherry picked from commit 232cbfe67ffb7696f115830c711a960af5fa0828)
Merge "Made unassigned networks visible in flat networking" into stable/icehouse
Merge "remove test_multiprocess_api" into stable/icehouse
Merge "Fixes Hyper-V resize down exception" into stable/icehouse
Merge "libvirt: Use VIR_DOMAIN_AFFECT_LIVE for paused instances" into stable/icehouse
Merge "Ensure info cache updates don't overwhelm cells" into stable/icehouse
Merge "Neutron: Atomic update of instance info cache" into stable/icehouse
Merge "Avoid traceback logs from simple tenant usage extension" into stable/icehouse
Merge "Fix _parse_datetime in simple tenant usage extension" into stable/icehouse
Merge remote-tracking branch 'upstream/stable/icehouse' into stable/icehouse