xenapi: send chunk terminator on subprocess exc

During chunk upload if a subprocess exception is thrown, a malformed
chunk upload occurs.

This patch adds logging for subprocess exceptions and sends the
terminator even in the event of a subprocess exception. The incomplete
file upload will then be cleaned up by the client.

Change-Id: I559f201430bcd4c36d669c7772a10bfe3cd54d41
Closes-Bug: 1365637
Fix service groups with zookeeper

Service groups using zookeeper don't work due to apparently improper
handling of the zookeeper session creation.

No additional unit tests as the mocking these interfaces probably
doesn't actually provide substantial future guaruntees. The correct
long term fix is to enable zk for unit testing upstream.

Change-Id: I21691843cb4936d10a0b82df41aef3afe5bf2519
Closes-bug: #1239864
Fix race condition with vif plugging in finish migrate

The tempest test that does a resize on an instance from time to time fails
with neutron with a virtual interface timout error. The reason why this
occurs is that nova-compute calls a vifs_unplug() then vifs_plug() fairly
quickly and the neutron-agent doesn't realize this happens because the way
it detects this is via polling. This patch fixings this problem by passing
vifs_already_plugged=True to _create_domain_and_network to avoid waiting
for the neutron notifcation.

Initially, I thought this problem could be solved by setting
vifs_already_plugged=True if migrate.source_host == migrate_desk.dest_host
but it turns out that this race condition can still occur if migrated to
another host though it's fairly unlikely. Always setting
vifs_already_plugged=True ensures we never hit this issue on migrate.

Change-Id: I90424ad50ac993c4abb049aa0654a94b225b5ebb
Closes-bug: 1357599
Closes-bug: 1357476
Merge "Reset task state 'migrating' on nova compute restart"
Update invalid state error message on reboot API

This change adds more detail to the error message
when a request to soft reboot an instance in an
invalid state fails, indicating that specifically
soft reboot is the request that failed, when
a hard reboot might still succeed.

Closes-Bug: 1365683

Change-Id: I4b12aa261d1c4fa05bacd5311793ed6e0478e59b
Merge "warn against sorting requirements"
Merge "Enabled qemu memory balloon stats"
Clean up LOG import in floating_ips_bulk v2 api

Remove useless code of LOG in floating_ips_bulk.py

Change-Id: I66f20a23d75b2c9c2e7cc2425f015a95fefb62d8
virt: add method get_serial_console to driver

Adds get_serial_console that returns a typed object with
all necessary information to connect to the instance.

Co-Authored-By: Ian Wells <iawells@cisco.com>
Co-Authored-By: Sushma Korati <sushma_korati@persistent.co.in>
Co-Authored-By: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@redhat.com>

Partial-Implements: blueprint serial-ports
Change-Id: Idc3ed6f3a8cf561c8fb7ac3cf600dbad96c4f662
compute: add get_serial_console in manager.py

Adds method get_serial_console and updates
validate_console_port to refer the new method in
virt/driver get_serial_console

Co-Authored-By: Ian Wells <iawells@cisco.com>
Co-Authored-By: Sushma Korati <sushma_korati@persistent.co.in>
Co-Authored-By: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@redhat.com>

Partial-Implements: blueprint serial-ports
Change-Id: Id1153e2d370463872eedd3988503457435b228f5
compute: add get_serial_console rpc and cells api calls

Co-Authored-By: Ian Wells <iawells@cisco.com>
Co-Authored-By: Sushma Korati <sushma_korati@persistent.co.in>
Co-Authored-By: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@redhat.com>

Partial-Implements: blueprint serial-ports
Change-Id: If79255be351e76fb4c4f671c929b71ce822a9db2
api: add serial console API calls v2

Adds a new REST APIv2 to call service serial console

Co-Authored-By: Ian Wells <iawells@cisco.com>
Co-Authored-By: Sushma Korati <sushma_korati@persistent.co.in>
Co-Authored-By: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@redhat.com>

Patial-Implements: blueprint serial-ports
Change-Id: I35d94c1924caa9cbc0fcdd9c637d28e95ff31ab2
Move to oslo.db

Replace common oslo code nova.openstack.common.db by usage
of oslo.db library and remove common code.

Replaced catching of raw sqlalchemy exceptions by catches
of oslo.db exceptions(such as DBError, DBDuplicateEntry, etc).

Co-Authored-By: Eugeniya Kudryashova  <ekudryashova@mirantis.com>

Closes-Bug: #1364986
Closes-Bug: #1353131
Closes-Bug: #1283987
Closes-Bug: #1274523
Change-Id: I0649539e071b2318ec85ed5d70259c949408e64b
Add user namespace support for libvirt-lxc

This change adds two config values that allow deployers
to enable user namespacing for libvirt-lxc containers.
User namespacing allows the mapping of uids and gids in
a container to separate uids and gids outside of the
container. The main advantage of this is to allow a
container's processes to appear to be running as root,
while actually running as a different user on the host.

Implements: bp libvirt-lxc-user-namespaces

DocImpact: This adds two config options to the libvirt
group, uid_maps and gid_maps. Default values are empty
strings and will leave user namespacing disabled for
libvirt-lxc containers.

Change-Id: I5f517f228b477f8238eccbb604b9ffd0ebaa5927
Add sqlite dev packages to devref env setup doc

While setting up a new Ubuntu Trusty development
system without devstack I ran into an undocumented
native package dependency for sqlite, so add those
here.

Change-Id: Ib82ea58e85f6ccab6008f2d19a31d0e890136ca7
Merge "Fixes Hyper-V boot from volume root device issue"
Merge "Fix race condition with vif plugging in finish migrate"
Merge "Make floatingip-ip-delete atomic with neutron"
Merge "Fix wrong command for _rescan_multipath"
Merge "Fix instance boot when Ceph is used for ephemeral storage"