Add error as not-in-progress migration status

When a migration status becomes "error" the migration is no longer
in progress, i.e. it is in a terminal state.  However, the
migration_get_in_progress_by_host_and_node method returns migrations
that have an "error" status, causing, among other things, the
Resource Tracker to continually log messages about how an
Instance is not resizing, creating excessive noise in the logs,
especially since the fact that a migration is entering the "error"
status is already logged.  This change causes "error"-status
migrations to not be included when retrieving "in progress"
migrations.

Closes-Bug: #1258275

Change-Id: I67bfec9f91f89ff38422469a5d86e14c4fffa40b
Update log message when remove pci device

When remove pci device, the log message does not include exception
message even the exception has been caught.

This patch was put the exception to the warning message.

Change-Id: I3c604a3c80b501dedf1f0dd9fab9f399ae73f481
VMware: improve connection issue diagnostic

This exception block can swallow all issues with a connection
adding something to log the exception thrown can aid in
the troubelshooting of a Nova node's configuration. This is
invaluable when guiding someone in the field through setup
because they may mis-type a key value or misconfigure an IP.

Change-Id: I27e5ed5c51926cfeb6747980df02a396f566dd77
libvirt: handle exception while get vcpu info

If an exception is raised while get a libvirt domain's vcpu info,
the update_available_resource periodic task will be failed, which
will result in the resource of this compute node will never be
reported.

This patch add an exception handling to avoid this situation.
Closes-bug: #1270008

Change-Id: I69109402416989fdaa421f8dbc72953bd067c407
Take a vm out of SNAPSHOTTING after Glance error

If a snapshot is attempted to be taken when Glance can't find the
underlying image (usually because it's been deleted), then ImageNotFound
is raised to nova.compute.manager.ComputeManager._snapshot_instance. The
error path for this leaves the instance in a snapshotting state; after
which "operator intervention" is required.

This simply reverts the instance state under that specific circumstance.

DocImpact: previously a snapshot that failed for this reason would leave
the instance in a spurious snapshotting task state indefinitely. Now,
the task state is reverted. At the moment that revert is 'silent'
(although the failure to snapshot will be evident to the user).

Change-Id: I9623cfaed31e8337ab65bc1eab85ad3b51b1a5ca
Fixes-bug: #1267097
Fix bug for neutron network-name

There may be cases when the network-name is not updated correctly
due to the requested networks not being owned by the tenant.

In the case when there is no matched network we use the details
from the used port.

A failed deletion of a port will raise an exception. If the
port is not found no exception will be raised.

Co-authored-by: Evgeny Fedoruk <EvgenyF@Radware.com>

Change-Id: Ie28e88ec9a9c7a180410ae5e57f84b1f653bbffc
Closes-Bug: #1246258
Add sort() method to ObjectListBase

This makes lists that inherit from ObjectListBase sort()'able

Related to blueprint icehouse-objects

Change-Id: I3cda27786a770fe98222edac95112708e4a8b645
Added a new scheduler filter for metrics

The scheduler filter MetricsFilter filters out those hosts which don't
have the metrics data available as required by metric settings.

This is part of the blueprint utilization-aware-scheduling.

DocImpact: Added a new metrics filter.

Change-Id: Ib4a898774daf683c4496ef3e9953d23027f11ac0
Sync log.py from oslo-incubator

We want to pick up change 55cd46af to avoid thousands of blank lines in
the nova logs, so sync log.py over to nova.

This includes the following changes:

  55cd46a remove extra newlines that eventlet seems to add
  d292f7d Add error type to unhandled exception log message
  0228b81 Logging excepthook: print exception info if debug=True
  71208fe Fix spelling errors in comments
  8b2b0b7 Use hacking import_exceptions for gettextutils._
  6d0a6c3 Correct invalid docstrings
  fe18bfa Remove keystone from default_log_levels default
  27b21bc Unify different names between Python2/3 with six.moves
  12bcdb7 Remove vim header
  a6f40ac Don't log to stdout when log_dir is set

Note that change 13eb01c "Adding domain to context and log" was
intentionally ommitted since it requires context.py to be synced and
changes nova.conf.sample, but nova uses it's own context module so we
don't want that change pulled in here.

Change-Id: Iebfdf484b9fcac5dbb689c4f7ec4f43a082a50c9
Fix bug with not implemented virConnect.registerCloseCallback

The unit test test_connection_to_primitive fails many times due to
"virConnect instance has no attribute 'registerCloseCallback'".

registerCloseCallback has been added since original(community's)
python-libvirt v1.0.0, and v1.2.0 also contains the method.
However ubuntu cloud's v1.0.1 does not contain the method, and current
version check of python-libvirt does not work.
This patch tries to operate the method and catch TypeError exception if
the method does not exist instead of the version check.

Co-Authored-By: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@cloudwatt.com>
Closes-Bug: #1266711
Change-Id: I4ec9ff9a684639ae5b146f400c90115c83afcda7
Update nova_ipam_lib to use VirtualInterface

Update nova.network.nova_ipam_lib to use the VirtualInterface object.
Also fix related tests.

Part of blueprint nova-network-objects

Change-Id: I5115eee1f2f6454688ea1d006c6a39e3096e5c29
Update linux_net to use VirtualInterface

Update nova.network.linux_net to use the VirtualInterface object.  Also
update tests as needed.

Part of blueprint nova-network-objects

Change-Id: I4bbf4c82efe8475d5e1f2823d52087362d851f3e
Add db.dnsdomain_get_all() method

Add a dnsdomain_get_all() method to the db API.  This method returns the
full model for each dnsdomain in the database.  There is a method
similar to this, dnsdomain_list(), but it doesn't return all of the
data, just one column from each of the rows.  That doesn't really map
well to objects, so make a new method that will be used by the
DNSDomainList object.

Part of blueprint nova-network-objects

Change-Id: I8753cf1e641a1a6040a32a9a43e8b29f2d32182f
Add DNSDomain object

Add a new object, DNSDomain, along with a DNSDomainList object.  This is
needed for converting nova-network to use objects so that we can have it
using conductor.

Part of blueprint nova-network-objects

Change-Id: I2e337ccdf9a23a3536ef6bdcd9bf3e4b825d4d74
Remove some dead dnsdomain code

Remove a method that was not used anywhere.  It also called a db API
method that doesn't exist.

Related to blueprint nova-network-objects

Change-Id: Ic4fafded3068e5d769ed68dce07f9c8b58954cc3
Update nova.network to use DNSDomain object

Update nova.network to make use of the DNSDomain and DNSDomainList
objects.  This removes more direct db API usage from nova-network.

This patch includes one really nice optimization, as well.  The method
get_dns_domains() used to call nova.db.dnsdomain_list(), which would
return a list of all domains in the database.  It would then call
_prepare_domain_entry() on each one, which would go back and query the
database again for each one to get the rest of the data associated with
it.  All of this data was actually returned from the db originally, but
the db API didn't return it.  Instead, it now gets all of this data
from the original db query for the list, eliminating N db queries each
time this code runs.

Part of blueprint nova-network-objects

Change-Id: I4b682affbbe988638c0c834b12a121e6a871080a
Remove get_network from nova.network.rpcapi

Remove the get_network() method from nova.network.rpcapi.  It's not
actually used, so help make that clear by removing it from the client
rpc class.

This also helps us clarify what changes we have to be careful about with
respect to rpc versioning.

Related to nova-network-objects

Change-Id: Ibfcc90499fb6c20b7f14597ea360ed55bb34e058
Remove get_all_networks from nova.network.rpcapi

Remove the get_all_networks() method from nova.network.rpcapi.  It's not
actually used, so help make that clear by removing it from the client
rpc class.

This also helps us clarify what changes we have to be careful about with
respect to rpc versioning.

Related to nova-network-objects

Change-Id: I21747958bff6330b100a9fd66d855a72aeabfab8
Merge "Added a new scheduler filter for metrics"
Merge "Add sort() method to ObjectListBase"