Remove pre-icehouse rpc client API compat

The compute rpc API was at version 3.23 for the Icehouse release.
Since we only support live upgrades from N-1 to N (Icehouse to Juno),
we can remove old rpc client support for versions that were
pre-Icehouse release.  This patch does that for the compute rpcapi.

Change-Id: Ice20b141dbe28de02e125fa5ced1dbc286ba2ee3
Fix wrong command for _rescan_multipath

The _run_multipath command is wrong in [1], it should be 'multipath -r'
not 'multipath - r'. And adds test for this.

[1]: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/volume.py#L590

Change-Id: I3c5d12cc11988d65a8fc9e6143d26c2d1220d253
Closes-bug: 1362916
Handle ExternalNetworkAttachForbidden exception

When creating an instance and assigning a public network
to it without admin authority, ExternalNetworkAttachForbidden
will be raised. But this exception is not handled in V3 api.

Change-Id: Id70d8eaa704fd82cef1afdb65756d75cb56fbdce
Closes-Bug: #1353949
libvirt: persist lxc attached volumes across reboots and power down

Currently, attached volumes to a LXC instance will be lost, when
hard rebooting or powering it down.
Adjusting the get_storage_config() to include LXC volumes in the
domain configuration.
The change will add all volumes to the instance config, a part from
the root device mapping.
When (re)creating a LXC instance, get_bdms_to_connect() generator
from block_device module, will be used to exclude the root device
mapping for a domain configuration.

Co-authored-by: Sam Morrison<sorrison@gmail.com>
Closes-Bug: #1269990
Change-Id: I0aa227ede5fca065c6da15506d8cc4391d993e20
Fixes Hyper-V boot from volume root device issue

Checks that instances are booting from volume by checking
the correct root device obtained from the block device info.

Change-Id: I94941dfe96c1596c8e9b91df3d3d19d33ae7fe92
Co-Authored-By: Zsolt Dudas <zdudas@cloudbasesolutions.com>
Closes-Bug: #1357972
Make floatingip-ip-delete atomic with neutron

The infra guys were noticing an issue where they were leaking floating ip
addresses. One of the reasons this would occur for them is they called
nova floating-ip-delete which first disassocates the floating-ip in neutron
and then deletes it. Because nova-api makes two calls to neutron if the
first one succeeds and the second fails it results in the instance no longer
being associated with the floatingip. They have retry logic but they base it on
the instance and when they go to retry cleaning up the instance the floatingip
is no longer on the instance so they never delete it.

This patch fixes this issue by creating a new method
disassocate_and_delete_floating_ip which contains the different logic
needed to atomically do this for neutron rather than putting the logic
in the api layer to do something differently if one is using neutron.

This patch also adds additional code coverage for nova-network in the
delete floatingip case.

Closes-bug: 1356157
Change-Id: I53b0c9d949404288e8687b304361a74b53d69ef3
Fix instance boot when Ceph is used for ephemeral storage

is_block_dev attribute was mistakenly set to True for RBD images
(we don't actually map RBD images to block devices in the *host*
system, but only in *guests*, so as far as the host system is
concerned an RBD image must not be treated like a system block
device). This led to a situation when mkfs was called for a non
existing file and failed when trying to create an ephemeral fs.

Closes-Bug: #1362221

Change-Id: I54c0d117a53bb61f278b2e137bd29a595548f5a1
Use NetworkRequest objects through to nova-network

Change-Id: Id99506cf6293ecab213764ef8b330c4d805d785b
Change v3 security_groups API to v2.1

This patch changes v3 security_groups API to v2.1 and makes v2 unit tests
share between v2 and v2.1.
This patch doesn't revert /os-security-groups, /os-security-group-rules
and /servers/server_id/os-security-groups because these features are
implemented in Networking API. On v2.1 API, we decide we don't need to
implement proxying features.

The differences between v2 and v3 are described on the wiki page
https://wiki.openstack.org/wiki/NovaAPIv2tov3 .

Partially implements blueprint v2-on-v3-api

Change-Id: I82660fd514d041bdd1fad0e689fa4fe90d441c47
Convert rescue v3 plugin to v2.1 API

Changes required to have v3 plugin natively support the V2.1
API.

The parameters admin_password and image_ref were reverted to
adminPass and rescue_image_ref

Partially implements blueprint v2-on-v3-api
Co-Authored-By: Chris Yeoh <cyeoh@au1.ibm.com>
Change-Id: Ibca991b5d6ec065ea9fbb0318c4ab51a3c5a3eba
Port v2 image_metadata extension to work in v2.1(v3) framework

Port v2 image_metadata extension and adapts it to the v2.1/v3 API
framework. API behaviour is identical with the exception that
there is no support for XML. Also

- unittest code modified to share existing testing with both
  v2/v2.1
- Adds expected error decorators for API methods
- Adds API samples

Partially implements blueprint v2-on-v3-api

Change-Id: Ibc8dc897f3449a1c70bc7ac1510445f48fddb291
Port image-size extension to work in v2.1/v3 framework

Ports v2 image-size extension and adapts it to the v2.1/v3 API
framework. API behaviour is identical with the exception
that there is no support for XML. Also

- unittest code modified to share testing with both v2/v2.1
- Adds expected error decorators for API methods

Partially implements blueprint v2-on-v3-api

Change-Id: I44f3d35b3d888e2f4429ca0788d74b9b0cce9494
Convert v3 ExtendedAvailabilityZone api to v2.1

This patch changes v3 ExtendedAvailabilityZone API to v2.1
and makes v2 unit tests share between v2 and v2.1.

The differences between v2 and v3 are described on the wiki page
https://wiki.openstack.org/wiki/NovaAPIv2tov3.

Partially implements blueprint v2-on-v3-api

Change-Id: I49baff38b7a9dabd40550267c2119fe84137594f
Add API schema for v2.1/v3 server_metadata API

By defining the API schema, it is possible to separate the validation
code from the API method. The API method can be more simple.
In addition, a response of API validation error can be consistent for
the whole Nova API.

Partially implements blueprint v3-api-schema

Change-Id: I4ef7d71f5d8eba13535f2c1a4aaf2b1f8d73a922
Add API schema for v2.1 "update a server" API

Some v2.1 APIs(create/update/rebuild server) contains the API
extension feature, it searches extension methods with its method
name and operates them automatically.
It is necessary to change the API schemas also automatically based
on the API extension loading condition, because the API extensions
add API parameters.

This patch adds API schema extensions which are implemented with the
same mechanism (stevedore library) as the API extension for "update
a server" extensions.

Partially implements blueprint v3-api-schema

Change-Id: I2f80a994bc69153bd40251827aad0479876a94ab
Add API schema for v2.1 "rebuild a server" API

Some v2.1 APIs(create/update/rebuild server) contains the API
extension feature, it searches extension methods with its method
name and operates them automatically.
It is necessary to change the API schemas also automatically based
on the API extension loading condition, because the API extensions
add API parameters.

This patch adds API schema extensions which are implemented with the
same mechanism (stevedore library) as the API extension for "rebuild
a server" extensions.

Partially implements blueprint v3-api-schema

Change-Id: Ibfdd99966335d062cde4cc1d9ab3b63bc82ed0b6
Add API schema for v2.1 access_ips extension

By defining the API schema, it is possible to separate the validation
code from the API method. The API method can be more simple.
In addition, a response of API validation error can be consistent for
the whole Nova API.

Partially implements blueprint v3-api-schema

Change-Id: I495e7538e5d8d67f19792dfa0427ab7a1b0068cf
Change v3 flavor_manage API to v2.1

This patch changes v3 flavor_manage API to v2.1 and makes v2 unit tests
share between v2 and v2.1.
This patch removes test_create_flavor_name_with_leading_trailing_whitespace
because v2.1 API contains consistent name rule for whole of API request and
the name which contain spaces at the start/end of name is against the rule.

The differences between v2 and v3 are described on the wiki page
https://wiki.openstack.org/wiki/NovaAPIv2tov3 .

Partially implements blueprint v2-on-v3-api

Change-Id: I57ca2562cb02ce5a35c4596e4c5a753d1268493f
VMware: image user functions for spawn()

Refactor each mode of using cached images when configuring a newly
created virtual machine into their own functions.

DocImpact:
vmware_linked_clone image property is ignored when spawning an
image that is an ISO.

partial-blueprint: vmware-spawn-refactor
Change-Id: I863fc0ef644bcf579e74d6f567f14aed98a23c84
VMware: spawn refactor enlist image

Refactor code to mark the cached image to be in active use into a
method in the image cache.

partial blueprint: vmware-spawn-refactor
Change-Id: Ib057f794f5a3a214811e55bb5f8c23d4767e4594