handle auto assigned flag on allocate floating ip

Set auto_assigned field to the right value on allocating floating ip to
avoid potential race condition in nova-network.

fix bug 1190585

Change-Id: I8ee34924439d2a29b1b5a966b92856e63cc1afdc
Clean destroy for project quota

    * Destroy user quotas under the project when deleting project quota.
    * Fixes bug 1206479

Change-Id: Id8391a2f6c25974b990c4a95a6bc99d696cd1c98
Don't inject files while resizing instance

Fixes bug 1208680

As what the code owner of method 'finish_migration' in libvirt driver
said, the files injecting operation is not necessary during resize,
because they have already been injected when creating the instance,
so added a parameter to _create_image to fix this issue, which is used
for identifying the injection is needed or not.

Change-Id: I01990de0ad2c7f1c2e499144034b7a4ac9faf024
Add an exception when doesn't have permissions to operate vm on hyper-v

This commit adds a wrapper checking method when creating/operating
vms on hyper-v, it will raise a HyperVAuthorizationException if the
windows account running nova-compute on this Hyper-V host doesn't
have the required permissions to create/operate the virtual machine.

Fixes bug #1195205

Change-Id: I2ef67a686c99dc83fcc3e9f47a71473929b01e9f
Fix FK violation in ConsoleTestCase

setUp() of ConsoleTestCase creates a few Console instances
that reference a non-existing instance with generated UUID
value. The tests work great on SQLite (because FKs are disabled),
but fail on MySQL and PostgreSQL.

Blueprint: db-api-tests-on-all-backends

Change-Id: I2417b6f6dad8c9c84af8b49a3c6db8b9bb715e56
Fix one port can be attached to more devices

a port attached to some device should not be attached to another device again

Change-Id: I60de0fa265358ac5b5e1b2c8f370862b32f9e724
Fixes: bug #1204850
Have tox install via setup.py develop

tox 1.6 was released, which means that we can now take advantage of the
feature we added to it - which is using setup.py develop to install the
code into the virtualenv. The logic was taken from run_tests.sh - so the
performance issues around using tox vs. using install_venv should now be
gone.

run_tests.sh is still needed to handle the ui switching between
parallel, single and debug modes to testr.

Change-Id: Ida5e440d1bdb9f8e9031277ea53a02d2ef171438
Unimplemented suspend/resume should not change vm state

When the underlying virt driver does not implement the suspend/resume API
and raises a NotImplementedError, the compute manager will error out and
change the instance's vm_state to ERROR. With this change, the compute
manager will handle the NotImplementedError and keep the instance
vm_state as it was before the error.

Change-Id: I8150a898fee91bbe66f1502638baca8050e2dd6d
Fixes: bug #1203986
baremetal: Log IPMI power on/off timeouts

Added log messages for cloud operators so that they can tell when an
attempt to turn a device on or off via IPMI has failed after the
configured number of attempts.

This should help operators diagnose issues on devices that respond
slowly to IPMI commands.  The failure to repeatedly change the device
power state is now logged precisely along with the cause, where
before operators had to diagnose the issue through trace log captures.

Partial-Bug: #1177584
Change-Id: Iecd6fadb2b653f4ccbe75882c2ef85812b334e8d
Add flag to make IsolatedHostsFilter less restrictive

This patch allows to run 'isolated' and 'non isolated'
images on isolated hosts by adding a flag
'restrict_isolated_hosts_to_isolated_images'.

If the flag is set to True then 'non isolated' images
can't be run on isolated hosts (same behaviour as the
current filter) otherwise they can be run on isolated
hosts.

DocImpact

blueprint improve-isolatedhostsfilter

Change-Id: I8c092caf32c05be88a547a7e8cb0530cc4925080
Port flavormanage extension to v3 API part 1

This is part 1 of the two part commit to port migrations to v3.
As part of this commit the api and test files are copied over to v3
plugins folder. The second changeset will have changes to port
migrations to v3.

Partially implements blueprint nova-v3-api

Change-Id: I2a51185f557c68022a2f3aaa006acb1e89b3c737
Adds Instance UUID to rsync debug logging

This commit adds the instance uuid to the rsync debug output in
the debug logs. The issue we were seeing is that when multiple
migrations are ongoing at the same time, it's not easy to
track the rsync progress of each migration.
Fixes: bug #1215629

Change-Id: Iec662bdc0aed1c6f54ecaa062335dc5504ac030f
emit warning while running flake8 without virtual env

run_tests.sh -N -p
it will call the flake8 installed on your host to detect PEP8, and
the flake8 plugin "OpenStack hacking" may not installed on your
host, so this command may not detect the OpenStack Style Commandment
supplied by hacking(e.g H202).

run_tests.sh -p
it will call the flake8 from virtual env, flake8 plugin "OpenStack
hacking" installed in virtual env will be triggered.

The result from "run_tests.sh -p" should be trusted, and jenkins uses
virtual env to run flake8 too.

When "-N" is enabled, emit warning to remind user.

Bug #1215404

Change-Id: Ie503fa34e6a3cf7b1f52f0b53019ddea188dabff
Add RBD supporting to libvirt for creating local volume

The block storage in OpenStack is divided into Nova and Cinder, Nova is
responsible for local volume such as root, swap and ephemeral partition. Cinder
is responsible for normal volume(global volume?). This mode will make difficult
for planning storage allocation. In order to improve the availability of
storage capacity and reduce the complexity of storage allocation, implementing
the unified storage for local volume in Nova and volumes in Cinder is
necessary.

Now Nova supports lvm backend for creating local volume. Ceph as a Cinder
backend we want to implement it too in Nova.

This patch add RBD backend to libvirt image backend, and users can use specify
rbd as the source of root, ephemeral and swap partition.

Implement: blueprint bring-rbd-support-libvirt-images-type
Change-Id: Id97de5c9af4ddc5c58493a5cef21b52386afac3e
Delete expired instance console auth tokens

Instance console auth tokens never get deleted from the cache
before the instnace is deleted, this is a waste of memory. now
we check if tokens are expired then remove them from the cache
before store new tokens.

Fix bug #1209134

Co-authored-by: Takashi Natsume <natsume.takashi@lab.ntt.co.jp>
Change-Id: I8dd5089ebaed3b3d91932f1f1558bbe302cd5675
XenAPI: Generic Fake.get_all_records_where implementation

Tests were incorrectly passing due to bad implementations.

Note that link to xenserver-core blueprint is due to patch following this
change

Implements blueprint xenserver-core

Change-Id: I051877551611e001295403af8369bcc97c8ba0fb
XenAPI: SR location should default to location stored in PBD

This value must be read from the PBD as it may change for different
installations.  The existing default is preserved as a fallback
for the cases where a PBD path is not set.

Implements blueprint xenserver-core

Change-Id: I055ebe8d7d79f238b0b9565277acac945bbe12e1
Fix unicode key of azcache can't be stored to memcache

The memcache library we are using assumes all strings passed to
it are plain ascii, so we need to convert unicode to string.

Fix bug #1212164

Change-Id: Ibf36e6b096893cb9f056371bb43cbb441cae6e32
Revert task state when terminate_instance fails

Calls to terminate_instance leave the instance task state as-is when
they raise an exception - this was introduced in Ie5701e5c as a
work-around for a bug with quota accounting, but introduced bugs with
power-state syncing. Ib72de1a4 removed the need for the original
work-around, and this change reverts Ie5701e5c.

Ie5701e5c worked around a bug in quota accounting by using the instance's
task state as a signal to communicate between the nova.compute.manager
and nova.compute.api modules: if the manager's terminate_instance method
failed, it would leave the instance task state as DELETING. If the user
retried the delete, the api would check the task state: if the
instance's task state was already DELETING, it would not decrement the
user's quota a second time.

Unfortunately, leaving an instance with an active task had the
side-effect of preventing the VM power-state from synchronizing, wedging
the instance (bug 1177584 and bug 1212420).

Ib72de1a4 changed the quota accounting mechanism, moving the quota
handling code out of the api module. The api code for the work-around to
check the DELETING task state was removed. However, the code for the
work-around in the manager was never cleaned up.

This change cleans up the nova.compute.manager work-around code, making
terminate_instance revert the task state to None upon failure.
Power-state syncing should work again, and quota handling is unaffected
by the change.

Fixes: bug 1177584, bug 1212420
Change-Id: I42c68d16dca56703d6b192bb452883e4d5ca80df
Fixes missing host in Hyper-V get_volume_connector

Fixes bug: #1215765

The Hyper-V driver is not returning the host in the
"get_volume_connector" method.

This commit fixes the issue by adding the "host" key to the dictionary
returned by the method.

Adds also a unit test for get_volume_connector.

Change-Id: Ic74882e54994f957bf6d2bb75bbd53c6a346042e