ensure zeros are written out when clearing volumes

Note O_DIRECT is _not_ used at the end of devices to
avoid issues with odd sized blocks etc. so instead
we arrange to have dd issue an fdatasync()
to ensure the data is persisted, lest it be discarded
from the write cache when the device is unprovisioned.

* nova/virt/libvirt/utils.py (clear_logical_volume): Add 'conv=fdatasync'
to the dd option list if O_DIRECT isn't used when clearing.

Fixes bug: 1100363
Change-Id: I76789557754ebaeb6d52bb34548a2ef17808fbf6
Cells: Add support for compute HostAPI()

This will allow the hosts API extension to work. Calls to get service
information from the DB are proxied via cells.

When using the host extension, the 'id' and 'host' fields for
service/compute will be modified to have the cell prepended to it.  One
can do a 'show' on a 'path!to!cell@id' to see details for a particular
host in the 'path!to!cell' cell, for instance.

Implements bp/nova-compute-cells

Change-Id: I8137fa64d2578acfc3d2457c231de2b0d69b96d8
Update en_US message translations.

Fixes bug 1101587.

* nova/locale/en_US/LC_MESSAGES/nova.po: This "translation" was
woefully outdated, causing KeyError exceptions under a number of
circumstances for users of this particular locale when format
strings wound up containing incorrect substitution variables. This
change attempts to sync up the msgstr entries to match msgid given
this is the source language for the pot file. I did not see any
obvious instances where en_US should differ from C locale for these.

Change-Id: I81d1729fee17653b089f3c47db20f3fc073ab363
Add support for memory overcommit in live-migration

Override the live-migration memory check in FilterScheduler and
make it use RamFilter for checking available memory on target
host.

Resolves bug 1068258.

Change-Id: I3002b8330e595ce71e2718b2091d3b0b99747707
Add api-samples to multinic extension

Partially implements bp nova-api-samples
Fixes bug 1070169

Change-Id: I923054bf92840a7fe81f78c97aa77a9201e94bdc
Fix boto capabilities check

In commit 4845fc27 a capability check for the boto library has been
accidentally broken. boto.connection existed in all versions and it
was HTTPResponse class that should cause the ImportError.
Exception will not happen on the import line anymore, so check the
capabilities using hasattr() instead.

Change-Id: Ic9bff71b91de685a958400f12836322c7ca996c4
Handle compute node not available for live migration

This patch handles exception.ComputeServiceUnavailable by restoring
instance's vm_state and instance's task_state after live migration
failure caused by unavailable source/dest compute node.

Raises detailed HTTPBadRequest explanation for this exception.

Fixes bug 973393 and bug 1051881

Change-Id: If825b61fad9c4e3030f2e6c5002907255eaf3661
Fix update quota with invalid value

If update quota with a value which is not integer, it will response
with http status code 500. Catch the exception and continue for
updating other quotas.

Fix LP# 1111327

Change-Id: I0ab3b10ff106a2e6c7677d1acdb55c2f35c492db
Remove hardcoded topic strings in network manager

Resolves bug 1107799.

Change-Id: I24e26fb53e9d4db709307280ef23241c6c4b09ae
Parse testr output through subunit2pyunit.

When running through run_tests.sh, filter the testr output through
subunit2pyunit so that incremental progress can be seen. Should be
a step towards folks seeing incremental test progress.

Change-Id: Ic90669acabaa1c8b64aae80349bff3d3deaf59fa
Add REST API to show availability_zone of instance.

Implement one workitem for bp:show-availability-zone

This show the availability zone and availability zone of its host in
API: GET /servers/details, GET /server/{uuid}.

When create instance if "availability_zone" doesn't specified, the
"availability_zone" of the instance will be None.  But actually the instance
is in a zone which its host is in. So I think show both of them will be fine.

Change-Id: I2d742ee2d291b514bf783fa79293785cca8ea2a5
Add API sample tests to os-network

Partially implements blueprint nova-api-samples
Bug 1070170

Change-Id: I24c708e59e4fe5fcc17be0ae9b7fcb294278a175
separate libvirt injection and configdrive config variables

This is mostly a refactoring so that if config drive is required,
variable lookup only needed for the injection case, is avoided.

There is a functional change here to make 'admin_pass' consistent
with the other variables in that the libvirt_inject_password flag
now does _not_ control whether the admin password is placed
in the config drive.

Change-Id: Ib79971aa050e56e5aa72fdd69aadd7539b2bd315
allow disabling file injection completely

With the libvirt_injection_partition config set to -2,
file injection is avoided completely.

This can be required if you want to make user specified files
available from the metadata server (and config drive is not enabled),
then you would want to avoid injection completely for performance reasons,
but also to avoid boot failure if the injection itself failed.

Flags: DocImpact
Change-Id: I604f4189114515c70c58cf183622542c4633e44a
Fix import for install_venv.py

On systems that don't use devstack the nova directory
doesn't automatically get added to the python path. This
breaks tools/install_venv_common.py which imports from
nova.openstack.common. This fixes the issue by manually
adding the nova root directory to the python path before
tools/install_venv_common.py is imported.

Change-Id: I8a3df02a51c2c777d8f47cb8bc72d0644ae47095
Default value of monkey_patch_modules is broken

The default value of monkey_patch_modules has not kept up with
the move of notifier to openstack.common

Without this change setting 'monkey_patch=True' in nova.conf
will cause runtime errors unless the monkey_patch_modules
is also specified pointing to the correct notifier

Change-Id: Ib22faa7404b09791799338de2edfe0ddcd662d5f
lxc: virDomainGetVcpus is not supported by driver

virDomainGetVcpus is not supported by the libvirt lxc
driver, so it results in a nasty traceback. Trap the
error and return the number of vcpus.

Fixes LP: #1106960

Change-Id: I4e81dedad74ce3c58fcdf3ee064dfa0b9aa9a755
Signed-off-by: Chuck Short <chuck.short@canonical.com>
Added Postgres CI opportunistic test case

Added test case and support functions for CI opportunistic testing
using a PostgreSQL database, as part of our postgresql as a first
class citizen effort.

rebased again on top of test migration with data framework

clean up debug, and ensure pg will fail to connect gracefully

fix previous migration test which was inserting a key by id, which
postgresql doesn't like, as it doesn't adjust the autoincrement
counter.

added driver to connection string for mysql

Updated comment to expose how to do this on pg easily

Fixes:
bug 1084567

Change-Id: If0bfe5cf748f6d5724d17082be6d18b6f96f6ee4
Fix check instance host for instance action

When instance has no host, actions such as get_console_output,
start_stop_instance cause HTTP 500 response. Here change to
HTTPConflict when action called before host set.

Fix LP# 1116012

Change-Id: I6153a03f449d9fad8d0d8fb7295bdea4d2b2c2b1
Use oslo database code

Bring in the new database code from oslo.

Uses get_session() from oslo as well as changing NovaBase to derive from
a common class.

Remove test_sqlalchemy.py now that this code is test in oslo.

Implements blueprint db-common.

Change-Id: I090754981c871250dd981cbbe1a08e7181440120