Fix double reboot issue during soft reboot

Using the ID of domain in Libvirt to recognize the hard reboot has been
implemented or not, if the ID changed, we believe the domain has been rebooted,
return True and break from soft reboot.

Fixes: bug #1046356

Change-Id: Iec2f9e8225cfe2779f84d2095667f3c0e621e935
don't clear the database dicts in the tearDown method

This is superfluous as the two dicts (self.engines and
self.test_databases) are re-initialised during setUp anyway and aren't
used in between

Change-Id: Icafa9bdf3b0193f31752e6104ecac0e49476c812
Map libvirt error to InstanceNotFound in get_instance_disk_info

When getting instance disk info, the instance may be destroyed/deleted, and a
libvirtError will be raised in XMLDesc method, so catching and mapping it to
InstanceNotFound and the caller can handle it correctly.

Fixes bug #1064581

Change-Id: I07fed3e82e10dad4cb84ae5c8650ada351c24e78
Add Compute API validations for block device map

This patch adds validations to return 400 BadRequest to
Create server(s) with block device mappings where appropriate
and related unit tests

Fixes bug 1094244

Change-Id: I778d3ed4b0871561774594e950ce98b6a1ecfcb9
XenAPI: Fix volume detach

fixes bug 1101229

The code-cleanup and code-move changes from this patch:
https://review.openstack.org/#/c/19219/
introduced a problem: the volume_utils.find_sr_from_vbd method was
called after the vbd has already been destroyed. This change fixes the
issue by moving the sr discovery before the vbd destruction, and tests
the call order by using side effects.

Change-Id: Ide4f8ac810f98bb192909f5f0408affc940e7446
Generate coverage even if tests failed.

A test failing doesn't make coverage useless, so we should always
generate it.

Change-Id: I3ed9e54213c7fb6756906553920d19c5075e1adb
Add support for Option Groups in LazyPluggable

Use @markmc's suggestion to enhance LazyPluggable with an
optional config group. Also fix the baremetal database
backend option to use the "baremetal" config group.

Fixes LP #1093043

Change-Id: I28cf51a2962f516fcef4ced19e30c985220e86dc
Add 'ovs_interfaceid' to nova network VIF model

Instead of relying on the hypervisor VIF drivers to make up
the correct 'interfaceid' parameter based on the VIF uuid,
explicitly tell them what the ovs interfaceid value should
be.

Blueprint: libvirt-vif-driver
Change-Id: I77bb1bcb8f38c8c6e20e9ccbbbd490b251f32fef
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Move helper APIs for OVS ports into linux_net

The libvirt vif driver impls have some helper APIs for creating
and deleteing OVS bridge ports. These are not specific to libvirt,
so move them into the linux_net class which holds other general
Linux networking helper APIs.

With this change LibvirtHybridOVSBridgeDriver no longer needs
to inherit from LibvirtOpenVswitchDriver.

Blueprint: libvirt-vif-driver
Change-Id: I520c23f9dc9d64f834dec875fb49d17b43abd527
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Remove duplicated tapdev creation code from libvirt VIF

Move the 'create_tap_dev' method out of the linux_net class
QuantumLinuxBridgeInterfaceDriver, to be a standalone method.
Make the libvirt LibvirtOpenVswitchDriver call this method
instead of duplicating the tapdev creation code

Blueprint: libvirt-vif-driver
Change-Id: Ib9b5fb3aed5e339464c4f367179911166f2cddfb
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Ensure service's servicegroup API is created first.

We want to make sure to create the servicegroup API instance early,
before creating other things such as the manager, that will also
create a servicegroup API instance.  Internally, the servicegroup
only allocates a single instance of the driver API and we want to
make sure that the service's value of db_allowed is there when
it gets created. For that to happen, when the service class creates
a servicegroup API instance, it needs to be the first thing to do so.

Part of bp no-db-compute.

Change-Id: I41df39d34c4920fcbfcd50c8f0344ee1dd5547d1
Fix double reboot during resume_state_on_host_boot

Re-basing my commit introduced a mistake I did not catch before the
merge occurred.  resume_state_on_host_boot is calling both
_create_domain_and_network followed by _hard_reboot. It should only
call _hard_reboot.

Change-Id: I52be2c0d0c85abe6386842c980f26b3c5713e3e3
xenapi: Add cleanup_sm_locks script

XenServer 5.6 and 6.0 don't appear to be cleaning up sm locks when using
a FileSR. When enough locks accumulate, this causes image related
operations to fail.

This cleanup script is a stop-gap solution that can be run against all
of the dom0's in the cluster to prevent too many locks from
accumulating.

We can remove this script when the root-cause is fixed upstream.

References 1103158

Change-Id: I798bcfeb727a59003910b82d331fdbe031c4a2b4
Fixed nova-manage argument parsing error

In the dictionary CATEGORIES of file nova-manage, both 'flavor'
and 'instance_type' refer to InstanceTypeCommands, so the category
would be handled twice by add_command_parsers(). It would mess the
arguments if a command category is handled more than once by
function add_command_parsers().
Added a check to avoid the mess.

Change-Id: I7a314a9b389b3dea848a7111cedfc762bbb6b30e
Fixes: bug #1098733
Adding REST API to show all availability zones of an region

Returns a summary or detailed list of availability zone to client.
To keep consistent result with EC2 API and de-duplicate, I have
refactored EC2 cloud controllor code.

Implement one workitem for bp:show-availability-zone

Change-Id: I6c9c673c757c6c319a1cdd94d7420b8983f92bbb
Signed-off-by: zhiyanliu <zhiyanl@cn.ibm.com>
Add REST api to manage bare-metal nodes

* create/delete/list/show bare-metal nodes
* add/remove interfaces to/from bare-metal nodes

blueprint general-bare-metal-provisioning-framework

Change-Id: I1e76f7e3f7f74087e844cfb23dc92154f4c3e127
Correct a format string in virt/baremetal/ipmi.py

"%(err)%s" to "%(err)s"

Change-Id: I5ef1d3218c6121fed7c5d2b4d318da7c6d884776
Provide creating real unique constraints for columns

Main issue is that we can't create unique constraint for columns, because
we are using soft deletion of entries (set `deleted` column to True).

The main idea is to use `deleted` columns to create unique constraint for
columns. For example (`col1`, `deleted`).

To make (`col1`, `deleted`) unique after entry deletion, we should assign the
value of `id` to `deleted` column.

Change type of `deleted` column from Boolean to table.id.type for all tables.

Change models.soft_delete() method to assign table.id instead of True to
`deleted` column.

Change query.soft_delete() method to assign literal_column("id") instead of True

Change in db.models all occurrences of Table.deleted == False =>
Table.deleted == correct_type (0 or "")

Value of `deleted` property of entries is used in public nova api. To keep API
as is, we should change XMLDictSerializer by converting the type of
`deleted` property to string representation of boolean.

Change db.api.model_query() method to make it work with different types
of `deleted` column.
If we are using as model that is not subclass of NovaBase we should set
parameter base_model as subclass of NovaBase that corresponds to model.

Change in db.api all occurrences of using model_query with model that is not
subclass of NovaBase.

blueprint db-unique-keys

Change-Id: Ie1f67f49a5d085e6371efb63fc23a1c8b25d9464
Adds SSL support for API server.

Part of code in test for accessing with ipv6 and part of ssl
socket wrapping developed by Davanum Srinivas.
Original patch could be found here:
https://review.openstack.org/#/c/19559

DocImpact

Fixes bug 869884.

Change-Id: I6d60df77c3fb49552e597f94abae7f4d6f54044d
Remove network manager from compute manager.

Remove an instance of the configured network manager from the compute
manager.  It's not used anywhere.  This was also the only place the
network_manager config option was used, so remove the config import as
well.

Change-Id: I8cfad16e96018e649dd223c3a7132a3860928c23