Remove unneeded call to fetch network info on shutdown

There is no reason to fetch the network_info from the network api on shutdown
since we can just pull it out of the local cache. There are probably other
places where this call to the network api call can be removed but this patch
just handles the case for shutdown.

Note: the following test was removed 'test_terminate_no_fixed_ips' as
the call to the network api is no longer done so this test would be the
same as test_terminate_no_network.

Closes-bug: #1288392

Change-Id: Ifbf751739c215e566926719f481c03e2c064163a
(cherry picked from commit c47900064f2f3b4384cfa3fd6fe21f452514fa36)
Change errors_out_migration decorator to work with RPC

This decorator in nova/compute/manager.py was written to work with
positional arguments, but at run time the decorated methods are
called via RPC with keyword arguments.  This change fixes the
decorator and the associated test cases.

Closes-Bug: #1300380

Change-Id: I9f87151a71e5217e7eff97cdc5cad2c85b620fa5
(cherry picked from commit 72cd430e971589f7872a9d4e8f173c68b4955375)
Fix display of server group members

The previous code was failing to display the members of server
groups because it was filtering them all out.

Interestingly, the previous code passes the unit tests due to a
bug where we treat regex comparisons differently when using mysql
vs sqlite (tracked under bug 1298690).

Change-Id: I7be9c180f4284dd18cd02e510e77e3c1e3fc9925
Closes-bug: 1298494
(cherry picked from commit 8ce30b6ac7ea8d2e036a621a9904b35cb1563672)
Moved the registration of lifecycle event handler in init_host()

The event handler has to be registered before the libvirt connection
is used for the first time.  Otherwise it doesn't register and events
such as shut down initiated from within the instance are not registered
in nova until a period task compares the content of hypervisor with
content of database.

Closes bug: #1297485

Change-Id: I70887bea343734a977f621fedc634b6fab0e4ad5
(cherry picked from commit 8a430da045d86ccb8daee6e082c12890ce5c5276)
InvalidCPUInfo exception added to except block

When a live migration between two nodes with different
CPU info features is launched, an InvalidCPUInfo exception
is raised and not captured. That lets the vm in an
unusable state because the task_state rollback is not
executed.

Change-Id: Ib418fa324964cabd5f342d1b764f9ac7608b7ade
Closes-Bug: #1294210
(cherry picked from commit 7a3b4033f07f8c2144a36c1f12b1b9281cfb9a69)
Remove metadata's network-api dependence on the database

This makes the metadata server use instance.info_cache for getting
an instance's info cache instead of going direct to the network-api
(and thus, the database) for it. It also converts the network-api
implementation of get_fixed_ip_by_address() to use and return an
object. Since that translates to the same DB call in the end, none
of the network tests need to change.

Conflicts:

        nova/tests/network/test_api.py

Change-Id: Iab444b1669628604f7d96d9550eb684645c554cb
Closes-bug: #1306009
(cherry picked from commit 20d0cef99376ca622294536485a5987385a717e3)
Remove unnecessary call to fetch info_cache

The _heal_instance_info cache() always accesses instance['info_cache']
therefore we should always load the info_cache so we can avoid the extra
query to fetch it.

Change-Id: I3c2e005b98abf2a80276562cc05e88ba5ff199bc
Closes-bug: #1309239
(cherry picked from commit 5aa29b6b0a34d603e882a80cdc8e936f983b1892)
remove unneeded call to network_api on rebuild_instance

When rebuilding an instance we call the network_api and which results in
calling neutron and updating the info cache. We do not actually need to
do this as we can get the nw_info directly from the instance's info cache.

Change-Id: I2bcc2f15b1e5bb52d972098c7bc3e12f160b6feb
Closes-bug: 1294914
(cherry picked from commit 982850f0e912e1a72e501e456cb2b5ef94124ca6)
Merge "Fix display of server group members" into stable/icehouse
Pass configured auth strategy to neutronclient

When the token is invalid Nova emits a log trace about an
unknown auth strategy (caused by neutron client). This is a
red herring and is only masking the fact that the token was
unable to be validated in the first place.

Partial-bug: 1297309

Change-Id: Ia3801f80c7ca18674fcf46b5270aa04f6c8c9331
(cherry picked from commit fdd689c146713d18eb901d6f3ec986cdd11d13fe)
Merge "Remove unnecessary call to fetch info_cache" into stable/icehouse
Merge "remove unneeded call to network_api on rebuild_instance" into stable/icehouse
Merge "Remove unneeded call to fetch network info on shutdown" into stable/icehouse
Check for None or timestamp in availability zone api sample

test_availability_zone_detail depends on the services not having an
updated_at time set in the database.  This is set by a periodic task so
it's not guaranteed to be set, or not, shortly after the service starts.
On a fast system the test reliably passes, but adding a delay causes the
test to start failing.  Introducing a time.sleep(5) in the test seems to
make it reliably fail for me.  The test should pass if updated_at is set
to None or a timestamp.

Change-Id: I976ddf5354a52ca06a6419a92abcacb7d2379a37
Closes-bug: 1300810
(cherry picked from commit a70308a471c67386ad255e67e8971bbf96c31f18)
VMWare: add power off vm before detach disk during unrescue

Non Hot Plug type disk like IDE can only be detached when the VM is power off.

Change-Id: Ib1f387a41abe2b52357854e90c2535ebb7b43f18
Close-bug: #1279199
(cherry picked from commit 1e1915aaaca38b5691794e0e052a42b9d95dd3c2)
Revert "Use debug level logging during unit tests"

This change made the subunit_log 190 MB on every run (14MB
in the gz version). It also dramatically increases the
memory needed to run unit tests. We need to revert and
come at this problem with fresh eyes to address the problem
at hand without effectively overrunning people's environments.

This reverts commit f9090b0bd796f8c2fdf8d0a23173f3adee71b3df.

Change-Id: Ia1cfc3d874ddf4f8ee2037b487d22f968585854e
(cherry picked from commit f350f24cd2bad0d62a737c0f100e40cdfbbb36f6)
support local debug logging

Add OS_DEBUG as an environmental variable that will let you run
locally with debug logs when trying to debug a particular unit
test. Inspired by similar function in the neutron base tests.

History: I70790aa39c0774726de71e5fa5751c45e7e34ffd was enabled on
March 10th to address 2 issues, make it possible to see DEBUG logs
during test runs, and test that the LOG.debug functions that we happen
to execute as a side effect of unit tests did not have syntax errors
(it's important to note it was not guarunteed coverage for all debug
messages).

The net effect was largely a DDOS on the testing system. This
generated a 190MB subunit_log per test run. It meant on failures in
Nova unit tests now generated such an enormous dump that they became
non indexable in elastic recheck (it overloaded the indexers), so the
bug classification system was blinded to any nova unit test bugs.

It was thus reverted in: Ia1cfc3d874ddf4f8ee2037b487d22f968585854e

This patch solves the first concern, which is developer targetted
testing. It mirrors what is done in Neutron for cross project
consistency. There was a competing approach in requiring OS_LOG_LEVEL
to be set, however it had the same problem as the initial patch,
defaulting to DEBUG means that it's a DOS (at 15k unit tests, that
just can't be the default). Given that Neutron already established
some conventions here, using existing conventions also seems better.

This *does not* address concern #2. That is expected to be done in a
separate patch using an ephemeral string buffer logger. However
getting that correct has proved tricky enough that the approach to the
problem was:

 1. revert - Ia1cfc3d874ddf4f8ee2037b487d22f968585854e
 2. add targeted debug logging (this patch)
 3. add back side effect debug testing - being worked by joe

Change-Id: I04ff2f0650f1e6035d8de5c5802b96ce4294891b
(cherry picked from commit 31169bb26fc0a39486ef6cd6a5a154d570186886)
Use debug level logging in unit tests, but don't save them.

We want to try formatting debug level logs so we can gate on any
formatting errors, but we don't want to save the debug level logs as
there are just too many of them. So use fixtures.FakeLogger running at
INFO level and a custom NUllHandler at DEBUG level.

Not Using fixtures level parameter since that sets the log level for the
entire logger and not just the individual handlers.

Don't inherit from logging.NullHandler since it isn't in python2.6

Change-Id: I7317092c0a2bad8efd11ece3ba521d58c2acae42
(cherry picked from commit 158ac3d6c6ed29b3c3bed1448993d5276b0fe366)
VMWare - Check for compute node before triggering destroy

While booting an instance, if Claim on that compute node is not successful
compute manager triggers a call to destroy instance. Destroy fails since the
compute node (Cluster/ESX host) is set only after claim is successful.
Hence the check for node is made before destroy, to ensure that exception
does not get thrown. This ensures that the instance gets rescheduled
Closes-Bug: #1307408

Change-Id: Iaaf931e9c1e6cf046497e2c64952f92e802ad4be
(cherry picked from commit aadfffaaa69c6a8939f6c88cfbaeb026c7604163)
Use one query instead of two for quota_usages

The second query was unnecessary and not using a proper index. It
seemed to be contributing to a deadlock issue as well.

Change-Id: I1031451b25140d71380c8149d3c83827eec0d4a9
Closes-Bug: #1308715
Co-Authored-by: Chris Behrens <cbehrens@codestud.com>
(cherry picked from commit da1fa5dc93e6e64d397e8d00769e34828669e010)