Opening stable/juno

Bump version to next stable release on juno branch, and set
defaultbranch in .gitreview for convenience.

Change-Id: I6616376d03a84b0cb0fc872f0d7c930f8ae40fa0
Fix XML UnicodeEncode serialization error

The generic Nova XMLSerializer code will currently attempt
to cast to str the value for all leaf nodes. This patch
ensures that no attempt is made to convert unicode which
can cause a UnicodeEncode error. We don't need to convert
unicode for XML text and regardless we encode to UTF-8 at
a later point.

Change-Id: I8135d2b9a67db62b0eafdd301b7fdb67a5dd72cc
Closes-Bug: #1279172
(cherry picked from commit 53fe8696314fb73ca9943fce998d96fa6d0414b4)
libvirt: use six.text_type when setting text node value in guest xml

Trying to spawn an instance with a unicode name using the libvirt driver
fails with a UnicodeDecodeError because the value is cast to str().

The fix is to use six.text_type for the cast.

Closes-Bug: #1382318

Conflicts:
        nova/virt/libvirt/config.py

Change-Id: I4628b94459a3c1e757d388916f1268884cb02038
(cherry picked from commit 73fcf4628089dd784889062e916b80d3fc9988a2)
VMware: attach config drive if booting from a volume

Ensure that the config drive is configured when booting from a
volume

Change-Id: Ia05021ff076c4546af181287a9ff6ae15ffb4857
Closes-bug: #1380624
(cherry picked from commit f274816f073cfe9d0071a99159309fee643dbdb8)
Merge "libvirt: use six.text_type when setting text node value in guest xml" into stable/juno
Add @_retry_on_deadlock to _instance_update()

This patch adds the @_retry_on_deadlock decorator
to _instance_update(), which has been observed to be
involved in some deadlock scenarios.  As it is the point
of transaction demarcation based on the presence of
get_session(), this is an appropriate point at which the
transaction can be re-attempted in the case of deadlock.

Change-Id: Id2cfdc129ea3cfc787b73f1f3e9aa286a7c0229a
Closes-bug: 1375467
Cherry-picked from: 68ed3c034ba1b0767694026000723e5255d24a64
Merge "Add @_retry_on_deadlock to _instance_update()" into stable/juno
Use response.text for returning unicode EC2 metadata

When serving meta-data that contains unicode, use the webob Response.text
instead of Response.body.  This also adds a test that triggers the bug in
addition to testing that all meta-data offered by the server can be served.

Closes-bug: #1380792

Change-Id: If2ddaa97bbc89cf574205c0327e3caa02dd503fc
(cherry picked from commit f0f07fdb6702f89f3800a5054e9365c3abc90f7a)
Translate 'powervm' hypervisor_type to 'phyp' for scheduling

This adds a temporary translation of the 'powervm' hypervisor type
image metadata to 'phyp' for scheduling.

The powervc-driver in stackforge uses the hypervisor type of 'powervm'
for scheduling so this is needed to make that work until the driver
can get a translation shim in place to convert powervm to phyp in
image metadata.

Closes-Bug: #1370613

Change-Id: I4b20ffcb9911806db821f2902e09248dfa89d845
(cherry picked from commit d05567cc3fcb8849402f7036895ea5f40f7e1a1f)
Keep migration status if instance still resizing

There is a race condition during finish_resize() where
poll_unconfirmed_resizes() can get called between the database save
of the migration record and instance record. The migration record
will be marked 'finished', enabling poll_unconfirmed_resizes() to pick
it up. But the instance will not have been saved yet to have the
vm_state of RESIZED, and task_state of None. The task_state will
still be RESIZE_FINISH. This will cause poll_unconfirmed_resizes() to
put the migration record in error state.

Instead, poll_unconfirmed_resizes() should ignore instances with
a task state of RESIZE_FINISH and not put the migration record
into error state.

Change-Id: I825234de5e4a4f0cc906ac05b715ed85cd953aa1
Closes-Bug: #1376933
(cherry picked from commit 0a84bb80c2ea66a806395964684e07d7bc6accbd)
Fixes DOS issue in instance list ip filter

Converts the ip filtering to filter the list locally based
on the network info cache instead of making an extremely expensive
call over to nova network where it attempts to retrieve a list
of every instance in the system.

Change-Id: I455f6ab4acdecacc5152b11a183027f933dc4475
Closes-bug: #1358583
Run build_and_run_instance in a separate greenthread

If we're doing a lot of build operations, we are using a large portion
of the limited rpc worker pool for long periods of time. Since we may wait
on external services (like neutron or glance) during those times, we could
fully deplete that pool.

This patch makes us spawn a new greenthread for that task and return the
rpc worker to the pool. Due to some funkiness with the stack of decorators,
this breaks the inner function out to an object method, which is probably
good anyway, given its size. This also moves the wrap_instance_event
decorator to the inner function so that the start and stop events properly
demarcate the actual task and not just the (now very quick) RPC call.

Change-Id: Ife712c43c5a61424bc68b2f5ab47cefdb46ac168
Closes-Bug: #1372049
(cherry picked from commit 1d8eddb2614de8daaddddd64ad1a8de4c215fe7a)
Merge "Keep migration status if instance still resizing" into stable/juno
Fix nova evacuate issues for RBD

For RBD scenario, there are some issues in Nova code
now against evacuate function:

1. Based on current implementation, nova evacuate and
nova rebuild are sharing some code. When user enables
the on_shared_storage option for nova evacuate, nova
will check if the instance path is accessible. For
the RBD scenario, the volume(block) is shared between
different hosts, though the path isn't shared at the
filesystem level. This patch fixes this issue and adds
test cases for that.

2. Missing the 'recreate' parameter for rebuild method.
Though the libvirt driver doesn't implement rebuild
method(only Ironic driver implements it), but we really
need to set 'recreate' in kwargs so it gets passed to
_rebuild_default_impl so we don't call driver.destroy
on evacuate for shared filesystem/block storage cases.
It is fixed in this patch and test case is added as well.

Closes-Bug: 1249319
Closes-Bug: 1340411

Change-Id: Idc8c45b055e986cf85730235d5d25777632ad1c1
(cherry picked from commit 91d3272b975572d9866b7d959547e438142dc4fb)
Fix nova-compute start issue after evacuate

After evacuated successfully, and restarting the failed
host to get it back, Nova will call init_host() and then
call method _destroy_evacuated_instances(). In method
_destroy_evacuated_instances(), nova will check again if
the storage is shared or not to decide if the storage
should be destroyed. Now nova is using temp file to check
if it's shared file system, but it's wrong for RBD case.
So Nova will attempt to delete the shared block storage,
which will fail since it's used by the new instance. This
patch fixes this issue and adds test cases for that.

Closes-Bug: 1385484

Change-Id: I71bb818f3c2930b3a2ddf1817dfd4bb61fae7e98
(cherry picked from commit 296d92bd44d1b8eb161f94f70cba5db4d17f8f65)
Clean up iSCSI multipath devices in Post Live Migration

When a volume is attached to a VM in the source compute node through
multipath, the related files in /dev/disk/by-path/ are like this

stack@ubuntu-server12:~/devstack$ ls /dev/disk/by-path/*24
/dev/disk/by-path/ip-192.168.3.50:3260-iscsi-iqn.1992-04.com.emc:cx.
fnm00124500890.a5-lun-24
/dev/disk/by-path/ip-192.168.4.51:3260-iscsi-iqn.1992-04.com.emc:cx.
fnm00124500890.b4-lun-24

The information on its corresponding multipath device is like this
stack@ubuntu-server12:~/devstack$ sudo multipath -l 3600601602ba034
00921130967724e411
3600601602ba03400921130967724e411 dm-3 DGC,VRAID
size=1.0G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
|-+- policy='round-robin 0' prio=-1 status=active
| `- 19:0:0:24 sdl 8:176 active undef running
`-+- policy='round-robin 0' prio=-1 status=enabled
  `- 18:0:0:24 sdj 8:144 active undef running

But when the VM is migrated to the destination, the related information is
like the following example since we CANNOT guarantee that all nodes are able
to access the same iSCSI portals and the same target LUN number. And the
information is used to overwrite connection_info in the DB before the post
live migration logic is executed.

stack@ubuntu-server13:~/devstack$ ls /dev/disk/by-path/*24
/dev/disk/by-path/ip-192.168.3.51:3260-iscsi-iqn.1992-04.com.emc:cx.
fnm00124500890.b5-lun-100
/dev/disk/by-path/ip-192.168.4.51:3260-iscsi-iqn.1992-04.com.emc:cx.
fnm00124500890.b4-lun-100

stack@ubuntu-server13:~/devstack$ sudo multipath -l 3600601602ba034
00921130967724e411
3600601602ba03400921130967724e411 dm-3 DGC,VRAID
size=1.0G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
|-+- policy='round-robin 0' prio=-1 status=active
| `- 19:0:0:100 sdf 8:176 active undef running
`-+- policy='round-robin 0' prio=-1 status=enabled
  `- 18:0:0:100 sdg 8:144 active undef running

As a result, if post live migration in source side uses <IP>, <IQN> and
<TARGET LUN Number> to find the devices to clean up, it may use 192.168.3.51,
iqn.1992-04.com.emc:cx.fnm00124500890.a5 and 100.
However, the correct one should be 192.168.3.50, iqn.1992-04.com.emc:cx.
fnm00124500890.a5 and 24.

Similar philosophy in (https://bugs.launchpad.net/nova/+bug/1327497) can be
used to fix it: Leverage the unchanged multipath_id to find correct devices
to delete.

Change-Id: I875293c3ade9423caa2b8afe9eca25a74606d262
Closes-Bug: #1357368
(cherry picked from commit aa9104ccedb3ff13cc34a498b11f5e8ff100fd99)
Merge "Translate 'powervm' hypervisor_type to 'phyp' for scheduling" into stable/juno
Merge "Clean up iSCSI multipath devices in Post Live Migration" into stable/juno
Merge "Run build_and_run_instance in a separate greenthread" into stable/juno
Merge "Fix nova evacuate issues for RBD" into stable/juno