Openstack - Nova Migration (non live migration) to Different Hypervisor

## Nova Migration Type

http://docs.openstack.org/admin-guide-cloud/compute-configuring-migrations.html

> **The migration types are:**
>
> 1. **Non-live migration** (sometimes referred to simply as ‘migration’)
> The instance is shut down for a period of time to be moved to another hypervisor. In this case, the instance recognizes that it was rebooted.
>
> 2. **Live migration** (or ‘true live migration’)
> Almost no instance downtime. Useful when the instances must be kept running during the migration.
> The different types of live migration are:
>     - 2-1. **Shared storage-based live migration.** <br>
>            Both hypervisors have access to shared storage.
>     - 2-2. **Block live migration** <br>
>            No shared storage is required. Incompatible with read-only devices such as CD-ROMs and Configuration Drive (config_drive).
>     - 2-3. **Volume-backed live migration** <br>
>            Instances are backed by volumes rather than ephemeral disk, no shared storage is required, and migration is supported (currently only available for libvirt-based hypervisors).

## What's this?

**1. Non-live migration** 

Nova instance (shutoff 状態) を別の Hypervisor (Compute node) へ移動

[Case]

1. Cinder volume を接続していない場合
2. Cinder volume を接続している場合

[条件]

* Nova instance は shutoff 状態
* Shared Storage 無し
* 移動する際に Target Hypervisor (Compute node) は指定出来ない (移動先は Nova scheduler しだい)

## [Case1] Migrate Nova instance (w/o Cinder volume)

### Initial state

#### Hypervisor (Compute node)

    # nova hypervisor-list
    +----+-----------------------+
    | ID | Hypervisor hostname   |
    +----+-----------------------+
    | 6  | node-27               |
    | 9  | node-28               |
    | 12 | node-29               |
    +----+-----------------------+

#### AZ

    # nova availability-zone-list | egrep -A 10 "^\| nova"
    | nova                     | available                              |
    | |- node-27               |                                        |
    | | |- nova-compute        | enabled :-) 2015-02-13T02:24:11.000000 |
    | |- node-28               |                                        |
    | | |- nova-compute        | enabled :-) 2015-02-13T02:24:09.000000 |
    | nova2                    | available                              |
    | |- node-29               |                                        |
    | | |- nova-compute        | enabled :-) 2015-02-13T02:24:10.000000 |
    +--------------------------+----------------------------------------+

#### Nova instance is on node-28

    # nova list --field name,status,,OS-EXT-STS:vm_state,OS-EXT-AZ:availability_zone,OS-EXT-SRV-ATTR:hypervisor_hostname
    +--------------------------------------+------------+---------+--+----------------------+------------------------------+--------------------------------------+
    | ID                                   | Name       | Status  |  | OS-EXT-STS: Vm State | OS-EXT-AZ: Availability Zone | OS-EXT-SRV-ATTR: Hypervisor Hostname |
    +--------------------------------------+------------+---------+--+----------------------+------------------------------+--------------------------------------+
    | 6235d670-a5f0-4658-aef5-1b9549613c38 | admin_vm01 | SHUTOFF |  | stopped              | nova                         | node-28                              |
    +--------------------------------------+------------+---------+--+----------------------+------------------------------+--------------------------------------+



### Migrate Nova instance

#### Migration

    # nova migrate --poll admin_vm01
    Server migrating... 100% complete
    Finished

#### Check the migration result

Nova instance is moved from node-28 to node-27

    # nova list --field name,status,,OS-EXT-STS:vm_state,OS-EXT-AZ:availability_zone,OS-EXT-SRV-ATTR:hypervisor_hostname
    +--------------------------------------+------------+---------------+--+----------------------+------------------------------+--------------------------------------+
    | ID                                   | Name       | Status        |  | OS-EXT-STS: Vm State | OS-EXT-AZ: Availability Zone | OS-EXT-SRV-ATTR: Hypervisor Hostname |
    +--------------------------------------+------------+---------------+--+----------------------+------------------------------+--------------------------------------+
    | 6235d670-a5f0-4658-aef5-1b9549613c38 | admin_vm01 | VERIFY_RESIZE |  | resized              | nova                         | node-27                              |
    +--------------------------------------+------------+---------------+--+----------------------+------------------------------+--------------------------------------+

### Confirm previous size

#### Resize confirm

    # nova resize-confirm admin_vm01

#### Check

The status is changed from VERIFY_RESIZE to SHUTOFF

    # nova list --field name,status,,OS-EXT-STS:vm_state,OS-EXT-AZ:availability_zone,OS-EXT-SRV-ATTR:hypervisor_hostname
    +--------------------------------------+------------+---------+--+----------------------+------------------------------+--------------------------------------+
    | ID                                   | Name       | Status  |  | OS-EXT-STS: Vm State | OS-EXT-AZ: Availability Zone | OS-EXT-SRV-ATTR: Hypervisor Hostname |
    +--------------------------------------+------------+---------+--+----------------------+------------------------------+--------------------------------------+
    | 6235d670-a5f0-4658-aef5-1b9549613c38 | admin_vm01 | SHUTOFF |  | stopped              | nova                         | node-27                              |
    +--------------------------------------+------------+---------+--+----------------------+------------------------------+--------------------------------------+


### Boot Nova instance on new hypervisor (Compute node)

#### Boot

    # nova start admin_vm01

#### Check

    # nova list --field name,status,,OS-EXT-STS:vm_state,OS-EXT-AZ:availability_zone,OS-EXT-SRV-ATTR:hypervisor_hostname
    +--------------------------------------+------------+--------+--+----------------------+------------------------------+--------------------------------------+
    | ID                                   | Name       | Status |  | OS-EXT-STS: Vm State | OS-EXT-AZ: Availability Zone | OS-EXT-SRV-ATTR: Hypervisor Hostname |
    +--------------------------------------+------------+--------+--+----------------------+------------------------------+--------------------------------------+
    | 34eb51cd-0585-4098-8b01-97f9c0ab8ea9 | admin_vm01 | ACTIVE |  | active               | nova                         | node-27                              |
    +--------------------------------------+------------+--------+--+----------------------+------------------------------+--------------------------------------+





## [Case2] Migrate Nova instance (w/ Cinder volume)

### Initial State

#### Hypervisor (Compute node)

    # nova hypervisor-list
    +----+-----------------------+
    | ID | Hypervisor hostname   |
    +----+-----------------------+
    | 6  | node-27               |
    | 9  | node-28               |
    | 12 | node-29               |
    +----+-----------------------+

#### AZ

    # nova availability-zone-list | egrep -A 10 "^\| nova"
    | nova                     | available                              |
    | |- node-27               |                                        |
    | | |- nova-compute        | enabled :-) 2015-02-13T02:24:11.000000 |
    | |- node-28               |                                        |
    | | |- nova-compute        | enabled :-) 2015-02-13T02:24:09.000000 |
    | nova2                    | available                              |
    | |- node-29               |                                        |
    | | |- nova-compute        | enabled :-) 2015-02-13T02:24:10.000000 |
    +--------------------------+----------------------------------------+

#### Nova instance is on node-27

    # nova list --field name,status,,OS-EXT-STS:vm_state,OS-EXT-AZ:availability_zone,OS-EXT-SRV-ATTR:hypervisor_hostname
    +--------------------------------------+------------+---------+--+----------------------+------------------------------+--------------------------------------+
    | ID                                   | Name       | Status  |  | OS-EXT-STS: Vm State | OS-EXT-AZ: Availability Zone | OS-EXT-SRV-ATTR: Hypervisor Hostname |
    +--------------------------------------+------------+---------+--+----------------------+------------------------------+--------------------------------------+
    | 6235d670-a5f0-4658-aef5-1b9549613c38 | admin_vm01 | SHUTOFF |  | stopped              | nova                         | node-27                              |
    +--------------------------------------+------------+---------+--+----------------------+------------------------------+--------------------------------------+

#### With Cinder volume

    # cinder list
    +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+
    |                  ID                  |   Status  | Display Name | Size | Volume Type | Bootable | Attached to |
    +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+
    | 0fd7299f-2862-493c-a795-fa7788985960 | available | admin_vol01  |  1   |     None    |  false   |             |
    +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+

    # nova show admin_vm01 | egrep "Property|volumes_attached"
    | Property                             | Value                                                    |
    | os-extended-volumes:volumes_attached | [{"id": "0fd7299f-2862-493c-a795-fa7788985960"}]         |



### Migrate Nova instance

#### Migration

    # nova migrate --poll admin_vm01
    Server migrating... 100% complete
    Finished

#### Check the migration result

Nova instance is moved from node-27 to node-28

    # nova list --field name,status,,OS-EXT-STS:vm_state,OS-EXT-AZ:availability_zone,OS-EXT-SRV-ATTR:hypervisor_hostname
    +--------------------------------------+------------+---------------+--+----------------------+------------------------------+--------------------------------------+
    | ID                                   | Name       | Status        |  | OS-EXT-STS: Vm State | OS-EXT-AZ: Availability Zone | OS-EXT-SRV-ATTR: Hypervisor Hostname |
    +--------------------------------------+------------+---------------+--+----------------------+------------------------------+--------------------------------------+
    | 6235d670-a5f0-4658-aef5-1b9549613c38 | admin_vm01 | VERIFY_RESIZE |  | resized              | nova                         | node-28                              |
    +--------------------------------------+------------+---------------+--+----------------------+------------------------------+--------------------------------------+


### Confirm previous size

#### Resize confirm

    # nova resize-confirm admin_vm01

#### Check

The status is changed from VERIFY_RESIZE to SHUTOFF

    # nova list --field name,status,,OS-EXT-STS:vm_state,OS-EXT-AZ:availability_zone,OS-EXT-SRV-ATTR:hypervisor_hostname
    +--------------------------------------+------------+---------+--+----------------------+------------------------------+--------------------------------------+
    | ID                                   | Name       | Status  |  | OS-EXT-STS: Vm State | OS-EXT-AZ: Availability Zone | OS-EXT-SRV-ATTR: Hypervisor Hostname |
    +--------------------------------------+------------+---------+--+----------------------+------------------------------+--------------------------------------+
    | 6235d670-a5f0-4658-aef5-1b9549613c38 | admin_vm01 | SHUTOFF |  | stopped              | nova                         | node-28                              |
    +--------------------------------------+------------+---------+--+----------------------+------------------------------+--------------------------------------+

### Boot Nova instance on new hypervisor (Compute node)

#### Boot

    # nova start admin_vm01

#### Check

* Nova

````
    # nova list --field name,status,,OS-EXT-STS:vm_state,OS-EXT-AZ:availability_zone,OS-EXT-SRV-ATTR:hypervisor_hostname
    +--------------------------------------+------------+--------+--+----------------------+------------------------------+--------------------------------------+
    | ID                                   | Name       | Status |  | OS-EXT-STS: Vm State | OS-EXT-AZ: Availability Zone | OS-EXT-SRV-ATTR: Hypervisor Hostname |
    +--------------------------------------+------------+--------+--+----------------------+------------------------------+--------------------------------------+
    | 6235d670-a5f0-4658-aef5-1b9549613c38 | admin_vm01 | ACTIVE |  | active               | nova                         | node-28                              |
    +--------------------------------------+------------+--------+--+----------------------+------------------------------+--------------------------------------+

    # nova show admin_vm01 | egrep "Property|volumes_attached"
    | Property                             | Value                                                    |
    | os-extended-volumes:volumes_attached | [{"id": "0fd7299f-2862-493c-a795-fa7788985960"}]         |
````

* Cinder

````
    # cinder list
    +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+
    |                  ID                  |   Status  | Display Name | Size | Volume Type | Bootable | Attached to |
    +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+
    | 0fd7299f-2862-493c-a795-fa7788985960 | available | admin_vol01  |  1   |     None    |  false   |             |
    +--------------------------------------+-----------+--------------+------+-------------+----------+-------------+
````

* Instance

````
    # ssh cirros@{INSTANCE} lsblk
    NAME   MAJ:MIN RM    SIZE RO TYPE MOUNTPOINT
    vda    253:0    0      1G  0 disk
    `-vda1 253:1    0 1011.9M  0 part /
    vdb    253:16   0      1G  0 disk
````

## Environment

* Openstack : Juno
* Ceph : Firefly

## Reference

[Nova Migration (non live migration) to Different Hypervisor][link-01]
[Nova Migration (non live migration) to Different Hypervisor between AZ][link-02]
[Nova Migration (true live migration) to Different Hypervisor][link-03]
[Nova Migration (true live migration) to Different Hypervisor between AZ][link-04]
[Nova Migration (Manually) between AZ][link-05]
[Nova Migration (Manually) between Region][link-06]
[Nova Migration (true live migration) to Different Hypervisor - Volume-backend live migration (instance)][link-07]
[Nova Migration (true live migration) to Different Hypervisor - Volume-backend live migration (instance) between AZ][link-08]

[link-01]:http://qiita.com/idzzy/items/a3032dce3110f959e11b
[link-02]:http://qiita.com/idzzy/items/3d67b3cedd42d39a8e2a
[link-03]:http://qiita.com/idzzy/items/a03b19e0c0289a63e491
[link-04]:http://qiita.com/idzzy/items/ed45aaa3cc6ecc144f90
[link-05]:http://qiita.com/idzzy/items/98967887a0d3771a7795
[link-06]:http://qiita.com/idzzy/items/01b32f52e7cf0e4c5f55
[link-07]:http://qiita.com/idzzy/items/515776f1461bb5c96eea
[link-08]:http://qiita.com/idzzy/items/5c54443c0d36f0a14e72