Sync filetuils from oslo-incubator

to support https://review.openstack.org/#/c/33409/

* nova/openstack/common/fileutils.py: Auto sync from oslo.
* nova/tests/virt/libvirt/test_libvirt.py (fake_rm_on_errror):
Adjust to match the new signature of delete_if_exists()
and handle mocking out the remove parameter to
remove_path_on_error() which is initialized at load time.

Change-Id: I756f1226853366e8f418d21309b0e8dcf8a24aa2
Limit instance fault messages to 255 characters

The instance_fault.message column is a VARCHAR(255).  By default,
MySQL silently truncates entries that are too long.  PostgreSQL
throws an error instead.  So we need to truncate the message field
before we try inserting into that column, to avoid errors from
very long messages in PostgreSQL (or MySQL configured to not allow
this).

Fixes bug #1217167

Change-Id: Ia959d64a8bc162a33ba818f1006e9779207b4796
add conf for number of conductor workers

Fixes bug #1213080 and implements blueprint condutor-workers.

Make it easy to launch a bunch of conductor processes on a host.

Deploying multiple conductor workers per host avoids serialization on
database accesses caused by libmysqlclient.so blocking eventlet's
single thread. In an experiment on a 24-core machine, when creating 20
VMs in parallel, maximum creation time was reduced by approx. 10s when
using 20 conductor processes vis-a-vis a single conductor process.
Profiling showed that all of the savings came from faster calls into
nova.db.sqlalchemy.api.

Note that there are alternative methods for preventing the eventlet
thread from blocking during database calls. However, none of these
alternatives performed as well as multiple nova-conductor processes.

    * Instead of using the native database driver like _mysql.so, you
      can use a pure-python driver, like pymysql by setting
      sql_connection=mysql+pymysql://... in the [DEFAULT] section of
      /etc/nova/nova.conf, which eventlet will monkeypatch to avoid
      blocking. The problem with this approach is the vastly greater
      CPU demand of the pure-python driver compared to the native
      driver. Since the pure-python driver is so much more CPU
      intensive, the eventlet thread spends most of its time talking to
      the database, which effectively the problem we had before!

    * Instead of making database calls from eventlet’s thread, you can
      submit them to eventlet’s pool of worker threads and wait for the
      results. Try this by setting dbapi_use_tpool=True in the
      [DEFAULT] section of /etc/nova/nova.conf.  The problem I found
      with this approach was the overhead of synchronizing with the
      worker threads. In particular, the time elapsed between the
      worker thread finishing and the waiting coroutine being resumed
      was typically several times greater than the duration of the
      database call itself.

Change-Id: I8698997d211d7617ee14a1c6113056a694d70620
Revert baremetal v3 API extension

Baremetal is scheduled to be deprecated in Nova soon. Since
the V3 API is experimental in Havana, we should revert the
extension to avoid maintenance of the extension and its tests
for no real gain.

Things have changed enough since it was added that a simple
git-revert won't work. This reverts the following changes:

  5de5ae1b5d3763dbcf14bf3b4d43199ad5255adf
  92c9975bd2d53624ae595b895c2e96991ee43778
  2382186311794591fb4d35893ffa774157d6131b

Mailing list discussion:

http://lists.openstack.org/pipermail/openstack-dev/2013-September/014614.html

Change-Id: I3bdc7234f91eb4f9078ee24d7ab093b28f9930c9
Fix inconsistent usages for network resources

If a normal user created a fixed_ip/floating_ip, a usage record with
column `in_use=1` will be created in the quota_usages table, and if an
admin user deleted the resource, a new record with column `in_use=-1`
will be created..This will cause the quota for the resources not
released.

Fixes bug 1220727

Change-Id: I705f2f9ef327a3198da6a47df772c3a1a28e1b4e
Add methods to get image metadata from instance

This patch adds a couple of utility functions that enclose all the logic
for getting and parsing the image metadata stored in the instance's
system metadata.

First, this will try to fetch the metadata from the real image and will
prevent it from failing if it is not available. It will be then merged
with the image metadata stored during the instance creation.

Related to bug #1039662

Change-Id: I2130caf19858585571b1199e27f0a98ad5f08701
Avoid errors on some actions when image not usable

Using the metadata saved on instance creation, we can now get all the
image related metadata we need from the instance itself.

This patch replace the logic for getting the image metadata on some
actions that shouldn't fail when the image is not accessible (create
an snapshot, resize, migrate, rescue an instance or attach an
interface).

Fixes bug 1039662

Change-Id: Id8c2f13ca96908317d271a84fd685104b64f87e9
Revert "Call safe_encode() instead of str()"

This reverts commit df44a7e25d47329fd863a5830b053c1cc3a0251f.

Not only did it not account for the object versioning changes
required, or all the other string attributes we have, but the
unicode "fix" was backwards anyway (we should be decoding, not
encoding there).

Change-Id: I2369a79944fa5cb3b076e09a92f5075ec3427f46
Merge "add conf for number of conductor workers"
Merge "Revert "Call safe_encode() instead of str()""
Don't stacktrace on ImageNotFound in image_snapshot

snapshot_instance stacktrace when nova.image.glance raises
ImageNotFound, but this is an expected error and no reason to stacktrace
for it.

Fix bug 1221925

Change-Id: I41af76f569d0b93b0d25750c529918328b9ea946
Merge "Limit instance fault messages to 255 characters"
Merge "Revert baremetal v3 API extension"
Merge "Sync filetuils from oslo-incubator"
Merge "Fix inconsistent usages for network resources"