Failure during termination should always leave state as Error(Deleting)

Delete is a non-reversible operation for the user, and once the
user has indicated that they want to delete the instance from that
point on is the system's problem to complete the delete as soon as
possible.

If anything fails during the delete that the system cannot recover
from then the instance should be left in an Error(Deleting) state.
Anything else, in particular reverting to an Active(None) state, makes
it look like the system has ignored the request.

Currently InstanceTerminationFailure is explicitly caught, but the
exception does not propagate so the instance_fault wrapper does not
get a chance to log the failure.  Also terminate_instance is wrapped
by reverts_task_state which resets the state to Active(None)

This change removes the revert_task_state wrapper and catches all
exceptions so that unhandled exceptions always leave the instance
in Error(Deleting)

Change-Id: I5fb1bbd56035792f566a6e076edfe7a69df006ef
Closes-Bug: 1254122
Use utils method when getting instance metadata and system metadata

Change-Id: Idab1ba25a8ec4c7dcc9efbb9d86e7248ae3c351e
Recommend the right call instead of datetime.now()

Closes-Bug: #1258166
Change-Id: I92b94dbcd0405d8f2bb818ad1c44c4f4934c8bec
Add error as not-in-progress migration status

When a migration status becomes "error" the migration is no longer
in progress, i.e. it is in a terminal state.  However, the
migration_get_in_progress_by_host_and_node method returns migrations
that have an "error" status, causing, among other things, the
Resource Tracker to continually log messages about how an
Instance is not resizing, creating excessive noise in the logs,
especially since the fact that a migration is entering the "error"
status is already logged.  This change causes "error"-status
migrations to not be included when retrieving "in progress"
migrations.

Closes-Bug: #1258275

Change-Id: I67bfec9f91f89ff38422469a5d86e14c4fffa40b
Remove update_service_capabilities from nova

From the comments of update_service_capabilities, it is said that
once publish_service_capabilities was removed, then we can begin the
process of its removal.

publish_service_capabilities has been removed in patch
I31c21055163e94b712d337568b16b9b7a224b52f, so we can remove
update_service_capabilities now.

Change-Id: I46be33462321f5e3ff0904bdff8dbfe2fbe115cd
Closes-Bug: #1263569
Add alias as prefix for flavor_rxtx v3

As v3 api rules, the extended attribute should be with extension
alias as prefix. So fix this case for flavor-rxtx v3 extension.

Closes-bug: 1252986
DocImpact

Change-Id: I9f5984a406d039245db4bef2df53cfe10a331cc2
Hyper-V:Preserve config drive image after the instance is resized

If we configure config_drive_cdrom=True in nova.conf,
the config drive image will be lost after the instance
is resized on hyperV. This is because nova compute will
not copy the disk file if the file's ResourceSubType
is not 'Virtual Hard Disk'.

This commit adds 'Virtual CD/DVD Disk' into disk_resources
to preserve the config drive image after the instance is
resized.

Closes-Bug: #1250324

Change-Id: Ia125a8b132d194ddc2230754fd1b813698b410a8
Allow deleting instances while uuid lock is held

Avoid issues where locked up operations on the instance prevent it
from being deleted by the user. Reasons for the lock up are a separate
issue and will be handled elsewhere, but terminating the instance
should not affect any tasks anyway. Any modification should already
gracefully handle the instance going away.

For more discussion about the issue see:
http://lists.openstack.org/pipermail/openstack-dev/2013-October/017454.html

Partial-Bug: #1248563

Change-Id: Ife26ffefccea11d6e9ca5fbd970192b322fc04fd
Update class/function name for test_extended_availability_zone.py

In test_extended_availability_zone.py, it was using class name and
function name as following:
1) ExtendedServerAttributesTest
2) assertServerAttributes

This is not accurate as this test file is for availability zone,
this patch was change the above two names to:
1) ExtendedAvailabilityZoneTest
2) assertAvailabilityZone

Change-Id: I52d6ebcb1388fc10ac92d66cae166391d46e68c1
Improve error message in services API

The awkward grammar of the original error message was making i18n
translations hard.

Change-Id: I2ff98f2d3585d5f1ccf0deb552e73c452393f124
Closes-Bug: #1266620
Ensure that headers are utf8, not unicode

Although the api.openstack.wsgi.ResponseObject converted the headers to
utf8 strings to comply with RFC for HTTP this was not enough, since some
responses that contained unicode strings were returned, causing an error
when running nova behind an http server like Apache. This change ensures
that headers are always an utf8 encoded string.

Closes-Bug: 1254017

Change-Id: I980326f1f15907bf57060acdd132011254289079
Workers verification for WSGI service

When we start the nova-api service, the *_workers (metadata_workers,
ec2_workers, osapi_compute_workers) value are not verified if the value
is negative. If the user passes a negative value accidentally, and
InvalidInput exception will be threw.

Change-Id: I8337d672cb119efe700aa6be420fdac9864c1577
Closes-bug: #1269997
Allow _check_instance_build_time to hit db slave

If your deployment has a slave configured, _check_instance_build_time
will read against the db slave. If you have no slave configured
then behavior remains unchanged.

Change-Id: I2e19bef2b4231a711f473807a14a13398cbc7278
Partially-implements: blueprint periodic-tasks-to-db-slave
Allow run_image_cache_manager_pass to hit db slave

Deployments that have a slave_connection configured can offload
read queries there. For deployments with no slave behavior
remains the same.

Change-Id: Ic443913883d80bdbc95385f8d3e21c0d855bb7db
Partially-implements: blueprint periodic-tasks-to-db-slave
Fix race conditions between imagebackend and imagecache

The race may occur in the situation:
* There is a base file that is not used for a long time enough
  to be removed by imagecache.
* imagebackend is provisioning a virtual disk from the base file.
* imagecache is removing the base file.

Then, the base file is removed even though it is about to be used.

To fix this, these changes are in this patch:

* A new function imagecache.refresh_timestamp(base_file) updates
  the owner and mtime of the base file with the lock dedicated
  to the base file.
* imagebacked calls refresh_timestamp(base_file) before provision
  a disk from the base file.
* imagecache.ImageCacheManager._remove_base_file(base_file) uses
  the same lock as used by refresh_timestamp()

Closes-Bug: #1256838
Change-Id: I7c897cf6071d87a2c4532fb3a73863d649d02782
vmwareapi:remove unused variables in volumeops

Variables instance_name were not used in two methods,
Let us remove them.

Change-Id: Ib617aa82651923ebb2824edf5ff5008f61898bfb
hyperv: Retry after WMI query fails to find dev

During long stress runs the WMI query that is looking for
the iSCSI device number can incorrectly return no data.
If the query is retried the appropriate data can then be
obtained.

This commit adds a retry loop, calling
basevolumeutils.get_device_number_for_target to avoid this situation.
It also handles the case where the devices list returned in
get_device_number_for_target is empty.  The retry loop is
implemented with new mounted_disk_query_retry_count and
mounted_disk_query_retry_interval configuration options.

Unit tests have been added to check the good and bad paths for
get_mounted_disk_from_lun.

DocImpact
Closes-bug: 1247901
Change-Id: I082c4b1694efcd20cce65293cd330b7a0cf7d470
libvirt: move unnecesary comment

Remove a commented out line of code that was not used.

Change-Id: Ie4a331c8b0d1352be82a48d6a666aafb74f7dc6b
XenAPI: Tidy calls to get_all_ref_and_rec

get_all_records is more efficient than getting them individually in the
old get_all_ref_and_rec implementation.

Replaced some cases of get_all_records with a faster and more
specific API call.

Change-Id: I4725c3f1fdebdf1737941cb7db69b19afb22df9f
XenAPI: Speedup get_vhd_parent_uuid

In many cases the VDI record has already been retrieved.
Use it rather than expensively getting a new VDI record from XAPI.

Change-Id: I469e2a415c1f1924010f47d3927ca18c45a7b8f8