ensure atomic manipulation of libvirt disk images

This pattern could probably be used elsewhere,
but only libvirt disk images are considered for now.
This change ensures there are no stale files left
anywhere in the path from glance, through the libvirt image cache.
These could cause subsequent operational errors either
directly or indirectly through disk wastage.

* nova/utils.py: Add a new remove_path_on_error() context manager
that is used to remove the passed PATH on a raised exception.
* nova/virt/images.py: Ensure temporary downloaded and
converted images are protected.
* nova/virt/libvirt/connection.py: Ensure all the images in
the image cache and instance dirs are protected.

Change-Id: I81a5407665a6998128c0dee41387ef00ebddeb4d
Clean up read_deleted support in host aggregates code

tl;dr - this is a cleaner fix for bug #949038

It seems clear to me that all of the DB APIs should not explicitly pass
read_deleted='no' or read_deleted='yes' to model_query without good
reason.

We want to allow callers to specify read_deleted via the context and
that only works if we don't explicitly pass it to model_query().

If we don't explicitly specify it to model_query(), we use the value
from the context which defaults to 'no'.

Given all that, there is no need to support read_deleted to any of the
DB API calls because they should support specifying the flag via the
context. There should also be no need to pass read_deleted='no' because
that is the default.

Really, the only place there should be any mention of read_deleted is
where we want read_deleted='yes' behaviour e.g.

  - In tests, where we want to check the operational_state of an
    aggregate after it has been deleted

  - Where we want to support undeleting an aggregate or aggreate host

Change-Id: I916a8d189a33d7f30838cccb26531a024066ef96
Support sql_connection_debug to get SQL diagnostic information

We've had a few reports of SQL issues; it'll be very helpful to be able to get SQL information
into our log.

Bug #964844

Change-Id: I9d08991a3677bc6cbed45a930ede9271d685cac1
Fix traceback when sending invalid data.

Running euca-terminate-instance with invalid data results
in a traceback and a hang. This fixes bug 965225

Change-Id: I0b547d527bca5fb86ff650188ac903066361d933
Allow self-referential groups to be created

 fixes bug 965674

Change-Id: I41db135f6437e1f1cb496528ae4d134c0d9cb744
Default scheduler to spread-first

 * Fixes bug 965732

Change-Id: I239b2b235905b30879974144263037eba6ed409f
Remove auth_uri, already have auth_host, auth_port

Remove to clarify config

From keystone docs:
`auth_uri``: (optional, defaults to `auth_protocol`://`auth_host`:`auth_port`)

Change-Id: I4349cd9545c571b37886ae38416c27b841e797f8
Fix marker behavior for flavors

Fixes Bug #956096

Change-Id: Ifa94a70f2aec3b9527c291e27d4710336a1d1834
Continue on the the next tenant_id on 400 codes

* Melange connection raises Exception if the response is > 400
* Fixes lp967261

Change-Id: I4592e5907276d290f4a54aa27dd4e91a62cc83e8
Minor xenapi driver cleanups

Remove reference to Deferred that has been incorrect for a while now.
Remove unnecessary __init__
Cleanup whitespace and some unnecessary +'s

Change-Id: Ice754ece564e88d46cd71d8ba56767d9df19dc0d
get_instance_uuids_by_ip_filter to QM

Adding a get_instance_uuids_by_ip_filter to the
QuantumManager. Also adds the supporting calls in
melange_ipam_lib an connector to support this. It
currently only works with Melange, I still need to
add a conditional that falls back on the original
implementation in the FlatManager if not using
Melange.

update: rebase

Change-Id: I9468de130c6cd1e0520f9d92c43f98aa6b36dce6
Support timestamps as prefixes for traceback log lines.

Traceback lines in the log are now formatted to match other log
lines. So, instead of getting something like:

2012-03-26 14:53:26 ERROR nova.rpc.common [-] AMQP server on 10.55.58.1:5672 is unreachable: Socket closed. Trying again in 1 secon
ds.
(nova.rpc.common): TRACE: Traceback (most recent call last):
(nova.rpc.common): TRACE:   File "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 446, in reconnect
(nova.rpc.common): TRACE:     self._connect()
(nova.rpc.common): TRACE:   File "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 423, in _connect
(nova.rpc.common): TRACE:     self.connection.connect()
[snip]

You would get something like:

2012-03-26 14:53:26 ERROR nova.rpc.common [-] AMQP server on 10.55.58.1:5672 is unreachable: Socket closed. Trying again in 1 secon
ds.
2012-03-26 14:53:26 TRACE nova.rpc.common Traceback (most recent call last):
2012-03-26 14:53:26 TRACE nova.rpc.common   File "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 446, in reconnect
2012-03-26 14:53:26 TRACE nova.rpc.common     self._connect()
2012-03-26 14:53:26 TRACE nova.rpc.common   File "/usr/lib/python2.7/dist-packages/nova/rpc/impl_kombu.py", line 423, in _connect
2012-03-26 14:53:26 TRACE nova.rpc.common     self.connection.connect()
[snip]

This is tracked by bug 967842.

Change-Id: Ie017317f7e0b636016a220cb21a3543ab569dea5
Base image signature files are not images.

Stop treating base image signature files as images. Resolves bug 967843.

Change-Id: Iad2f27e07a6a9887785bb4dab44476c62cd11a69
Touch in use image files when they're checked.

It was intended that FLAGS.remove_unused_resized_minimum_age_seconds
would indicate the number of seconds a base file was unused before it
was removed. This however isn't true at the moment. Because instances
are COW mostly, this will be the time since the resize occurred,
which is normally instantly true once the image isn't being used any
more. This small patch corrects that by touching the base image during
check cycles which find it in use. Resolves bug 967845.

Change-Id: I75bf06cf758c9a2d7a8bde5ce67c789d7e299174
Fix disassociate query to remove foreign keys

 * fixes bug 965333

Change-Id: I3f7605717825f91fc34d15c14a5fe86824bd3799
Update KillFilter to handle 'deleted' exe's.

Updates KillFilter so that it handles the case where the executable
linked to by /proc/PID/exe is updated or deleted.

Fixes LP Bug #967931.

Change-Id: I368a01383bf62b64b7579d573b8b84640dec03ae
When dnsmasq fails to HUP log an error.

Fixes LP Bug #967937.

Change-Id: I492de5094bfb740499a65b8483d6a6f26b070042
fixed bug lp:968019 ,fix network manager init floating ip  problem.

Change-Id: If2507256abcb609e7e3ee45d1585b7ba85624dee
Fix exception type in _get_minram_mindisk_params

This fixes bug #968066

Change-Id: I5bdb8602fe1d808f9af72252670b40b67b8d41c9
adjust logging levels for libvirt error conditions

Change-Id: I07d42169a9b3ef0bcaa0a3b033607669361f78e5