Add API schema for v2.1/v3 migrate_server API

By defining the API schema, it is possible to separate the validation
code from the API method. The API method can be more simple.
In addition, a response of API validation error can be consistent for
the whole Nova API.

Partially implements blueprint v3-api-schema

Change-Id: I9f1eea8471af0f86dcf472ed8a4fdb634dfdb260
Add API schema for v2.1/v3 create_backup API

By defining the API schema, it is possible to separate the validation
code from the API method. The API method can be more simple.
In addition, a response of API validation error can be consistent for
the whole Nova API.

Partially implements blueprint v3-api-schema

Change-Id: Ie8a1ce6d0d7b90fb73555981fa974e9029344d3b
Add API schema for v2.1/v3 attach_interfaces API

By defining the API schema, it is possible to separate the validation
code from the API method. The API method can be more simple.
In addition, a response of API validation error can be consistent for
the whole Nova API.

Partially implements blueprint v3-api-schema

Change-Id: I0c4066c34f6da0934946aa9eda907b911dbede16
Add missing image to instance booted from volume

When booting an instance from a volume, the image_ref for
the instance will be None. And the API do not get the image
information for the instance also. So when we calling the
related API, we get "" for instance.image.

Change-Id: I3c35ab1d7c8bcec551fb5d67d0b44418266b32a4
Closes-bug: 1317880
add checksums to udp independent of /dev/vhost-net.

In newer kernel versions, the network devices added for lxc containers
do not get checksums added.  The result is the same as previously
occurred when vhost-net became prevalent in kvm guests.

Software that expects the checksums on packets will ignore them.  One example
of such software is udhcpc in cirros.  Without this change, cirros containers
in 3.13 kernels (Ubuntu 14.04) will fail to acquire an address via dhcp.

Closes-Bug: #1335193
Change-Id: Iba305fb7f8236955ca732e467db9e424535be93d
Fixed wrong assertion in test_vmops.py

There is wrong assert method in
nova.tests.virt.vmwareapi.vmwareapi.test_vmops.py:640
while checking a number of calls mocked method 'mkdir'.

We should use assertEqual(3, len(mock_mkdir.mock_calls))
instead of self.assertTrue(3, len(mock_mkdir.mock_calls))

But number of calling method 'mkdir' not always is 3.
If dict block_device_info contains key 'block_device_mapping'
with any information, method 'mkdir' wouldn't be called in method
self._vmops.spawn(). That is why corresponding check been added.

Change-Id: Ifc26e112b9c2974e0fb7a91ba39d5b2d887bd549
Closes-Bug: #1335859
VMware: remove duplicate key from test_instance dict

Remove the duplicate 'node' key from test_instance dict

Change-Id: I1199b5f9428f3fad196bd10331b268f9a3eddcf9
Add 'anon' kwarg to FakeDbBlockDeviceDict class

This will allow us to create fake block devices without the database id
field, so we can pass them to BDM objects in our test  which can then
call create(). create() would normally error out if there is a DB id set
on the object.

Change-Id: I2d1f975633c8b2cb9e2e744c5f769b8f753b5968
change the firewall debugging for clarity

When we are building rules ensure we log the instance['id'] so
we can actually correlate the iptables output to UUID for the
instance.

Also bundle up the security group to iptables translation to a
final view of the world instead of the piecemeal rule at a time
view.

Display what rules are being skipped in the add process, as the
skips seem to happen a lot. If this is completely normal we should
probably delete the bit entirely at some later point.

Related-Bug: #1298472

Change-Id: I0e90c3af9bf908b733ed895ad7c204b0a95ef786
Virt: fix typo (flavour should be flavor)

Commit 0c74df4962f256d9bc17b57c22ea45ece94e2f1a added support
which used the word flavour. Flavour is the preferred
spelling throughout the rest of the English-speaking world, but
flavor is preferrred spelling for OpenStack.

TrivialFix

Change-Id: I5fb3dd1601324c6be4f4d159166f5cacbec16a7e
Remove redundant code in Libvirt driver

Change-Id: I98c5b5bb5a8710d32fadafe9781736a963616fec
Co-Authored-By: Thomas Kaergel <kaergel@b1-systems.de>
Retry db.api.instance_destroy on deadlock

Update the instance_destroy method of the sqlalchemy db API to retry if
it fails because of a deadlock. The decorator that handles this was
introduced in I0293c62d2dd5ac036445bc639cabbd05ba016e83.

This patch does not include a unit test for this bug, as deadlocks are
very hard to reproduce.

Change-Id: I570d91e892a6e1ff4b3cdb8cf81acc483649835e
Closes-Bug: #1342086
Correct exception for flavor extra spec create/update

When create/update flavor extra spec failed for v2 api, it will not
raise MetadataLimitExceeded for but two exceptions as following:
1) FlavorExtraSpecUpdateCreateFailed
2) FlavorNotFound

When create/update flavor extra spec failed for v3 api, it will not
raise DBDuplicateEntry but raise FlavorExtraSpecUpdateCreateFailed.

This patch updates the exception handling logic for create/update
flavor extra spec.

Change-Id: If3310a1cb44952e8fe2efe0b3358783f729bc190
Small grammar fix in libvirt/driver.py. fix all occurrences.

Change wording to be more descriptive/correct for greenthread.sleep()
Change-Id: I074391eaa7cfd580b89d89c2dd587796112c4a98
libvirt: add version cap tied to gate CI testing

Limit the libvirt features that Nova will use in a default
configuration to only those which are tested in the CI gate.

The end user can override the limit if they wish to opt-in
to use of untested features via the 'version_cap' setting
in the 'libvirt' group.

Change-Id: Ib91164cc572b1dbde2a31beee5751c52b0445d3e
Cleanup and gate on hacking E713 rule

This fixes the offending instances of the rule
and removes it from the ignore list in tox.ini
so that we can gate on the E713 rule.

Change-Id: I1f9ef23a8569252b5b6fa660d0d200b1702056d0
libvirt: fix recent test changes to work on libvirt < 0.9.13

The newly used 'listAllDomains' method was only introduced in
libvirt 0.9.13. Although the driver code was written to cope
with older libvirt, the test suite code itself would fail
when faced with such an old libvirt because it was trying to
mock out methods which did not exist.

The fix here recognises that since '_list_instance_domains_fast'
and '_list_instance_domains_slow' both have explicit test cases,
there is no point in mocking out the low level 'listAllDomains'
method in all the other test cases. Instead we can mock out
the much higher level '_list_instance_domains' which means we
avoid any interaction with libvirt in these test cases. The
opportunity is taken to convert the tests in question to use
'mock' instead of 'mox' since it also simplifies the code.

Finally in the test for '_list_instance_domains_fast' simply
skip the test if libvirt is too old to have the required
constants defined.

Closes-bug: 1341729
Change-Id: I554352ea14e6b3306176e7524502e005cebd8c16
libvirt: add migrateToURI2 method to fakelibvirt

With the merge of

  commit ea7da5152cdca7ba674e2137c3899909995e2287
  Author: Solly Ross <sross@redhat.com>
  Date:   Tue Feb 4 14:27:51 2014 -0500

    Change listen address on libvirt live-migration

the libvirt driver uses migrateToURI2 instead of
migrateToURI. The fakelibvirt class did not define
this method so it was unable to be mocked out.

Change-Id: I74c28119c979e511cece6b839e25a40197b0e5d9
Closes-bug: 1342498
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
shelve doesn't work on nova-cells environment

The compute_api shelve/unshelve methods expect an Instance object,
but cell is still passing the sqlalchemy form.
So adding shelve/unshelve to the list of methods that will receive an
Instance object when using nova-cell.

When shelving instance with nova-cell environment,
instance object needs to contain 'metadata' and 'info_cache'.
So fetching the same from databse and adding it to
instance object before calling shelve/unshelve methods.

Closes-Bug: #1338451
Change-Id: Ia31b33c9a1d347f15d3e0a67fa31ef6b25a7407a
Merge "Virt: fix typo (flavour should be flavor)"