Handle IpAddressGenerationClient neutron

Handles IpAddressGenerationClient neutron exception which
can occur when allocating a floating ip fails because
the pool has been exhausted. Convert the exception
to a nova NoMoreFloatingIps exception which will be handled
by the api layer and converted to a 404.

Change-Id: Ib5320a13b5cc0c493ef0763d0e2b4575dc699bdf
Partial-Bug: 1210598
Fix service API and cells

The move to the object model broke the service API when used together
with cells. The underlying problem is how cells will overload the id
field with a cell path to uniquely identify each service in each cell.

Changing the id to a string would defeat the purpose of the object model
so this patch uses a proxy to modify the id on access to make the
difference transparent to the API.

Change-Id: I165b5d64b0c1556511eb02d179ca9fc6f101c008
Closes-bug: 1251043
Refactors nova.image.glance unit tests for show()

This patches continues a patch series to cleanup and streamline the unit
tests for the nova.image.glance module. Here, we create true unit tests
for the GlanceImageService.show() method, stressing all code paths and
edges but only for that specific method, using mocks and no fake glance
image service.

This patch removes "unit" tests from the TestGlanceImageService that
called glance.GlanceImageService.show(), as those tests used the fake
glance service and the code paths are now more than covered by these new
mock-using tests.

Change-Id: I200a9e9812d6de011fa6faa8fb8dc570075ce902
Related-bug: #1293938
Refactors unit tests of image service detail()

We do not need to have "unit" tests that test the things like
pagination work with the glanceclient.images.detail() call. glanceclient
and glance have unit tests for that. What the unit tests for the glance
image service detail() method in Nova need is to test only the code
boundary that exists within the detail() method -- i.e. verify that the
calls to the glanceclient and other helper methods are made with the
right parameters.

This patch pulls the various unit tests out of the
TestGlanceImageService test class and into their own test class that
uses mocks and verifies each code path within the detail() method.

Next up is the create, update and delete methods of the
GlanceImageService class. The final piece is to rework the download unit
tests (which aren't unit tests but more functional tests that take
around a second each to run :(

Change-Id: I94897ff54a36a1af4258ba4a3c88b45c079f4bb5
Related-bug: #1293938
Change exception type from HTTPBadRequest to HTTPForbidden

When user without admin permission wants to get a list of servers
which are in 'deleted' state, currently it raises  HTTPBadRequest.
This should be changed to HTTPForbidden exception.

Change-Id: I29da5f055c69e8aba66ffb068a3601ae69b2fc94
Closes-Bug: #1281904
Persist image format to a file, to prevent attacks based on changing it

The attack is based on creating a raw image that looks like a qcow2
image, and taking advantage of the code that used 'qemu-img info' to
autodetect the image format.

Now we store the image format to a 'disk.info' file, for Qcow2 and Raw
images, and only autodetect for images that have never been written to
that file.

SecurityImpact

Co-authored-by: Nikola Dipanov <ndipanov@redhat.com>
Closes-bug: #1221190
Change-Id: I2016efdb3f49a44ec4d677ac596eacc97871f30a
Updated from global requirements

Change-Id: I3296b2f6a12681199e35f05c288132347429eec3
Merge "Persist image format to a file, to prevent attacks based on changing it"
Disable volume attach/detach for suspended instances

As described in the bug - some hypervisors (libvirt) do not support
this. It is best to disable it in the API to provide a consistent user
experience.

Also adds a test to prevent an accidental regression.

Change-Id: I5b404cca22cffecbaf524e2511810e5341242052
Closes-bug: #1240922
Use binding:vif_details to control firewall

Neutron can include 'ovs_hybrid_plug' and 'port_filter' boolean keys in
the binding:vif_details port attribute. 'port_filter' indicates whether
or not neutron is handling port filtering for nova to determine if it needs
to filter for that port. 'ovs_hybrid_plug' can be set to True to indicate
that the neutron plugin still requires the bridge plugging strategy to attach
firewall rules.

Corresponding neutron patch:
Review -  https://review.openstack.org/#/c/83280/
Commit -  8c42ba115b58cc2c7486be9fee89e1facedb5f76

Closes-Bug: 1297469
Change-Id: I3bbcfc67036ab7389c82720add0bc0fc627bfee0
Add nova.conf.sample to gitignore

Now that we aren't storing this in tree and are recommending people
generate it locally add nova.conf.sample to .gitignore.

Change-Id: I0727c7552fead5a201116f9c2c6510fec3b92cd6
Merge "Use binding:vif_details to control firewall"
Add InstanceGroup.get_by_name()

Legacy behavior of the 'group' scheduler hint was that the value was
treated as a name for the group.  To preserve the old behavior, the
InstanceGroup object needs to be able to look up a group by name.

Since we're so close to doing an RC, just use the db API methods
available now but leave a note that this should be optimized later
with a db.api query that looks up the group by name.

Change-Id: I1b8c72b29a9a44a171aaa231aeb3533ccf41f7c2
Partial-bug: #1296913
Automatically create groups that do not exist

If the scheduler hint 'group' is specified and the contents are not a
UUID, automatically create the group.  This is to maintain backwards
compatibility with the old behavior of the 'group' scheduler hint.

The policy on this automatically created group is set as 'legacy'.
That way the scheduler knows that for this group it should emulate the
old group behavior for backwards compat purposes.

Partial-bug: #1296913

Change-Id: I0d4315980af7e47ae97dd4277784a74f086c8825
Add new style instance group scheduler filters

Prior to Icehouse, there was a different type of handling of the
'group' scheduler hint that got lost in the completion of the server
groups API.  This patch completes the code necessary to provide
backwards compatibility with the old behavior.

Previously, the policy for groups was simply based on what scheduler
filters you had enabled.  You could have either the affinity or
anti-affinity filter enabled and that would be applied to all groups.
These filters now act on groups with a policy type of 'legacy'.

New filters have been added that can be enabled simultaneously and act
based on the policy set on the group via the server group API.

DocImpact

Change-Id: Ia65c151396415ca48725cb3c756f33efa01d2fe5
Closes-bug: #1296913
Merge "Disable volume attach/detach for suspended instances"
Merge "Fix service API and cells"
Merge "Add nova.conf.sample to gitignore"
Merge "Add InstanceGroup.get_by_name()"
Merge "Automatically create groups that do not exist"