Add instanceset info to StopInstance response

Currently stopinstance response missing the instanceset
information with InstanceID, current state and previous
state details. It just returns the "True".

As per the AWS EC2 API reference document, the StopInstance
response elements should include the instanceset information
as below:
<StopInstancesResponse xmlns="http://ec2.amazonaws.com/doc/2014-02-01/">
  <requestId>req-a7326465-5ce2-4ed6-ab89-394b38cca85f</requestId>
  <instancesSet>
    <item>
      <instanceId>i-00000001</instanceId>
      <currentState>
        <code>80</code>
        <name>stopped</name>
      </currentState>
      <previousState>
        <code>16</code>
        <name>running</name>
      </previousState>
    </item>
  </instancesSet>
</StopInstancesResponse>

Included the instanceset into stopinstance response elements
and updated the test cases for stopinstance response elements
in nova/tests/api/ec2/test_cloud.py file.

Closes-bug: #1321239

Change-Id: I4d8a6faf2689a7df71920183682fd1e403ce2a42
Send compute.instance.create.end after launched_at is set

Downstream consumers of notifications want to know when an instance was
launched from the compute.instance.create.end notification.  This value
was previously set in the notification but commit
70196c4854e6cc1cb566dc9fc5b1a3397a4b69ab changed things so that it was
no longer set.  It will now be set, and is tested to be set.

Change-Id: If5ac00529d52e3898cda7fd942535ed3ba2039d5
Closes-bug: 1338736
Don't remove delete_on_terminate volumes on a reschedule

When cleaning up volumes before a reschedule if delete_on_terminate is
True the volume would be deleted.  That's not the desired behavior so
the volume cleanup has been moved to take place when a build is aborted.

Change-Id: I142370c0555495b4d51736f4f6b8070a3c112a59
Closes-bug: #1336127
VMware: remove local variable

Make use of the instance.root_gb instead of reading the value
into a local variable.

TrivialFix

Change-Id: I2ae4a9d2f5383ff05f47d00550c1c917fedc5db9
Make devref point to official devstack vagrant repo.

Now that we have an official devstack vagrant repo, lets point to that
instead of my own half maintained repo.

Change-Id: I944265d18f6e796608576ea71a7983aed0ece542
Properly handle snatting for external gateways

Previously, snatting for netowrks with external gateways had to be
handled by manually setting:
 * public_interface = ''
 * force_snat_range = snat_range
Whereas networks with a gateway managed by nova needed:
 * public_interface = eth0
 * force_snat_range = None or ['0.0.0.0/0']

In order to support both simultaneously, we calcualate whether the
gateway is external, and use the force_snat_range and skip interface
if it is an external gateway. Otherwise, we use the setting for public
interface and 0.0.0.0/0. This allows external and internal gateways
to co-exist.

Partially-implements blueprint better-support-for-multiple-networks

Change-Id: Id5ec66946c3a82841a6dd2d019404e822cdafe64
Fix _parse_datetime in simple tenant usage extension

_parse_datetime in os-simple-tenant-usage incorrectly attempts
to parse the datetime string even when it is None. As a result
a 400 BadRequest is returned. This has the overall effect of making
the start and end datetime parameters compulsory when they are meant
to be optional.

Note that this restores the behavior where start/end are optional which is
in contradiction to the API docs, but is the long standing API behaviour.
This was an API change which was accidentally applied in commit
I8e0e870727d687da165c809ffb7a4456bff81122 as part of some nova internal
changes but not picked up by the unitests

Change-Id: I00427379dc7aa39770f9a16ff026addb6e311735
Closes-Bug: 1300972
Parse unicode cpu_info as json before using it

If the extra specs are some of this:
- (capabilities:cpu_info:vendor, Intel)
- (capabilities:cpu_info:topology:cores, 2)
- (capabilities:cpu_info:features, <in> rdtscp)

It doesn't work because the cpu_info is loaded as unicode in HostState.
The patch fix these cases by parsing the cpu_info to dict.
Update: use six.string_types instead of unicode.

Change-Id: Ieabf95b1e3e5893ced16c0571c29b13b252df5d1
Closes-Bug: #1331176
Co-Authored-By: Pawel Koniszewski <pawel.koniszewski@intel.com>
Improve logging when python-guestfs/libguestfs isn't working

Bumped from debug to warn level, as most often refers to
missing python-guestfs package or other libguestfs misconfiguration,
and the fallback path is usually not desired.

Closes-bug: 1240339
Change-Id: I5d90a3ea109e79fe17221c001f94d49d599048ba
Update devref setup docs for latest libvirt on ubuntu

Adds instructions on how to enable the ubuntu cloud-archive
and update to the latest libvirt-dev from the icehouse repo.

This is needed to run unit tests in Juno if you're still on
ubuntu precise.

Change-Id: I8c93afdd72256b2bdc487e1413bdd570a6ea59e8
Use Ceph cluster stats to report disk info on RBD

Local disk statistics on compute nodes are irrelevant when ephemeral
disks are stored in RBD. With RBD, local disk space is not consumed when
instances are started on a compute node, yet it is possible for
scheduler to refuse to schedule an instance when combined disk usage of
instances already running on the node exceeds total disk capacity
reported by the hypervisor driver.

Change-Id: I9718c727db205b6f2191f8435583391584e96e6e
Closes-bug: #1332660
Signed-off-by: Dmitry Borodaenko <dborodaenko@mirantis.com>
Fix ownership checking in get_networks_by_uuid

The code was elevating context before requesting networks which
means that the project_only code is skipped and all networks
could be retrieved. This means that the default networks returned
by FlatDhcpManager could include networks that belong to other
projects.

This fixes the issue by requesting the network without elevating
the context, re-enabling the proper project checking. It includes
tests to verify that the proper exception is raised when an
illegal network is requested and that the context has not been
elevated by the compute manager.

Partially-implements blueprint better-support-for-multiple-networks

Change-Id: Icd3bc521003725cc3da9875dfd6532d5c5524f43
Closes-Bug: 1331092
docs - Fix docstring issues

fix identation, add line before block comment ends, remove
extra colon (:) to remove ERROR(s) and WARNING(s)

Change-Id: Iabb09867bf6068da35251f35f9f689a623420692
Import Ironic scheduler filters and host manager

This is an import of the Ironic scheduler changes as of
commit da967d77894be6f23d81fb5cc948f9d13898ba84

implements bp: add-ironic-driver

Co-authored-by: Adam Gandelman <adamg@ubuntu.com>
Co-authored-by: ChangBo Guo(gcb) <eric.guo@easystack.cn>
Co-authored-by: Chris Behrens <cbehrens@codestud.com>
Co-authored-by: Chris Krelle <nobodycam@gmail.com>
Co-authored-by: Devananda van der Veen <devananda.vdv@gmail.com>
Co-authored-by: Fengqian Gao <fengqian.gao@intel.com>
Co-authored-by: Hans Lindgren <hanlind@kth.se>
Co-authored-by: Jenkins <jenkins@review.openstack.org>
Co-authored-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
Co-authored-by: Michael Davies <michael@the-davies.net>
Co-authored-by: Rohan Kanade <openstack@rohankanade.com>
Co-authored-by: Zhongyue Luo <zhongyue.nah@intel.com>

Change-Id: I358d9c0485c5dcf81498871faa9150e3bf167c6b
docs - fix missing references

Change-Id: Ia33fc8a3495ef6d4b929aa88efccc002616475fc
Add a retry_on_deadlock to reservations_expire

Now that we have an index for the reservations_expire query, we
have uncovered a race with quota usage updates due to the foreign
key on the table, so just retry if there is a deadlock. This is
a fix to unblock the gate while we try to remove the deadlocks
completely.

Change-Id: I1993c3bd1f81facf1d98a4e29f9e8df4858a7d66
Partial-bug: #1350466
Merge "Fix _parse_datetime in simple tenant usage extension"
Merge "Properly handle snatting for external gateways"
Merge "Improve logging when python-guestfs/libguestfs isn't working"
Merge "Update devref setup docs for latest libvirt on ubuntu"