Package: fastforward
# FastForward [![GoDoc](https://godoc.org/github.com/nofdev/fastforward?status.svg)](https://godoc.org/github.com/nofdev/fastforward) [![Build Status](https://travis-ci.org/nofdev/fastforward.svg?branch=master)](https://travis-ci.org/nofdev/fastforward)FastForward is an ultimate DevOps platform.## Quick-start provision OpenStack(mitaka) on xenil or trusty### Requirements* The OpenStack bare metal hosts are in MAAS environment(recommend)* All hosts are two NICs at least(external and internal)* We assume that you have ceph installed, the cinder bachend default using ceph, the running instace default to using ceph as it's local storage. About ceph please visit: http://docs.ceph.com/docs/master/rbd/rbd-openstack/ or see the (Option)Ceph Guide below.* For resize instance nova user can be login to each compute node via ssh passwordless(include sudo), and all compute nodes need to restart libvirt-bin to enable live migration* The FastForward node is the same as ceph-deploy node where can be login to each openstack node passwordless and sudo-passwordless* The FastForward node default using ~/.ssh/id_rsa ssh private key to logon remote server* You need to restart the `nova-compute`, `cinder-volume` and `glance-api` services to finalize the installation if you have selected the ceph as that backend* FastForward support consistency groups for future use but the default LVM and Ceph driver does not support consistency groups yet because the consistency technology is not available at that storage level### Install FastForwardInstall FastForward on FASTFORWARD-NODEpip install fastforward### Prepare environmentPrepare the OpenStack environment.(NOTE) DO NOT setup eth1 in /etc/network/interfacesff --user ubuntu --hosts \HAPROXY1,\HAPROXY2,\CONTROLLER1,\CONTROLLER2,\COMPUTE1,\COMPUTE2,\COMPUTE3,\COMPUTE4,\COMPUTE5,\COMPUTE6,\COMPUTE7,\COMPUTE8,\COMPUTE9,\COMPUTE10 \environment \prepare-host --public-interface eth1### MySQL HADeploy to CONTROLLER1ff --user ubuntu --hosts CONTROLLER1 openstack mysql installff --user ubuntu --hosts CONTROLLER1 openstack mysql config --wsrep-cluster-address "gcomm://CONTROLLER1,CONTROLLER2" --wsrep-node-name="galera1" --wsrep-node-address="CONTROLLER1"Deploy to CONTROLLER2ff --user ubuntu --hosts CONTROLLER2 openstack mysql installff --user ubuntu --hosts CONTROLLER2 openstack mysql config --wsrep-cluster-address "gcomm://CONTROLLER1,CONTROLLER2" --wsrep-node-name="galera2" --wsrep-node-address="CONTROLLER2"Start the clusterff --user ubuntu --hosts CONTROLLER1 openstack mysql manage --wsrep-new-clusterff --user ubuntu --hosts CONTROLLER2 openstack mysql manage --startff --user ubuntu --hosts CONTROLLER1 openstack mysql manage --change-root-password changemeShow the cluster statusff --user ubuntu --hosts CONTROLLER1 openstack mysql manage --show-cluster-status --root-db-pass changeme### HAProxy HADeploy to HAPROXY1ff --user ubuntu --hosts HAPROXY1 openstack haproxy installDeploy to HAPROXY2ff --user ubuntu --hosts HAPROXY2 openstack haproxy installGenerate the HAProxy configuration and upload to target hosts(Do not forget to edit the generated configuration)ff openstack haproxygen-confff --user ubuntu --hosts HAPROXY1,HAPROXY2 openstack haproxy config --upload-conf haproxy.cfgConfigure Keepalivedff --user ubuntu --hosts HAPROXY1 openstack haproxy config --configure-keepalived --router_id lb1 --priority 150 --state MASTER --interface eth0 --vip CONTROLLER_VIPff --user ubuntu --hosts HAPROXY2 openstack haproxy config --configure-keepalived --router_id lb2 --priority 100 --state SLAVE --interface eth0 --vip CONTROLLER_VIP### RabbitMQ HADeploy to CONTROLLER1 and CONTROLLER2ff --user ubuntu --hosts CONTROLLER1,CONTROLLER2 openstack rabbitmq install --erlang-cookie changemechangeme --rabbit-user openstack --rabbit-pass changemeCreate cluster(Ensure CONTROLLER2 can access CONTROLLER1 via hostname)ff --user ubuntu --hosts CONTROLLER2 openstack rabbitmq join-cluster --name rabbit@CONTROLLER1### Keystone HACreate keystone databaseff --user ubuntu --hosts CONTROLLER1 openstack keystone create-keystone-db --root-db-pass changeme --keystone-db-pass changemeInstall keystone on CONTROLLER1 and CONTROLLER2ff --user ubuntu --hosts CONTROLLER1 openstack keystone install --admin-token changeme --connection mysql+pymysql://keystone:changeme@CONTROLLER_VIP/keystone --memcached-servers CONTROLLER1:11211,CONTROLLER2:11211 --populateff --user ubuntu --hosts CONTROLLER2 openstack keystone install --admin-token changeme --connection mysql+pymysql://keystone:changeme@CONTROLLER_VIP/keystone --memcached-servers CONTROLLER1:11211,CONTROLLER2:11211Create the service entity and API endpointsff --user ubuntu --hosts CONTROLLER1 openstack keystone create-entity-and-endpoint --os-token changeme --os-url http://CONTROLLER_VIP:35357/v3 --public-endpoint http://CONTROLLER_VIP:5000/v3 --internal-endpoint http://CONTROLLER_VIP:5000/v3 --admin-endpoint http://CONTROLLER_vip:35357/v3Create projects, users, and rolesff --user ubuntu --hosts CONTROLLER1 openstack keystone create-projects-users-roles --os-token changeme --os-url http://CONTROLLER_VIP:35357/v3 --admin-pass changeme --demo-pass changeme(OPTION) you will need to create OpenStack client environment scriptsadmin-openrc.shexport OS_PROJECT_DOMAIN_NAME=defaultexport OS_USER_DOMAIN_NAME=defaultexport OS_PROJECT_NAME=adminexport OS_TENANT_NAME=adminexport OS_USERNAME=adminexport OS_PASSWORD=changemeexport OS_AUTH_URL=http://CONTROLLER_VIP:35357/v3export OS_IDENTITY_API_VERSION=3export OS_IMAGE_API_VERSION=2export OS_AUTH_VERSION=3demo-openrc.shexport OS_PROJECT_DOMAIN_NAME=defaultexport OS_USER_DOMAIN_NAME=defaultexport OS_PROJECT_NAME=demoexport OS_TENANT_NAME=demoexport OS_USERNAME=demoexport OS_PASSWORD=changemeexport OS_AUTH_URL=http://CONTROLLER_VIP:5000/v3export OS_IDENTITY_API_VERSION=3export OS_IMAGE_API_VERSION=2export OS_AUTH_VERSION=3### Glance HACreate glance databaseff --user ubuntu --hosts CONTROLLER1 openstack glance create-glance-db --root-db-pass changeme --glance-db-pass changemeCreate service credentialsff --user ubuntu --hosts CONTROLLER1 openstack glance create-service-credentials --os-password changeme --os-auth-url http://CONTROLLER_VIP:35357/v3 --glance-pass changeme --public-endpoint http://CONTROLLER_VIP:9292 --internal-endpoint http://CONTROLLER_VIP:9292 --admin-endpoint http://CONTROLLER_VIP:9292Install glance on CONTROLLER1 and CONTROLLER2ff --user ubuntu --hosts CONTROLLER1 openstack glance install --connection mysql+pymysql://glance:GLANCE_PASS@CONTROLLER_VIP/glance --auth-uri http://CONTROLLER_VIP:5000 --auth-url http://CONTROLLER_VIP:35357 --glance-pass changeme --memcached-servers CONTROLLER1:11211,CONTROLLER2:11211 --populateff --user ubuntu --hosts CONTROLLER2 openstack glance install --connection mysql+pymysql://glance:GLANCE_PASS@CONTROLLER_VIP/glance --auth-uri http://CONTROLLER_VIP:5000 --auth-url http://CONTROLLER_VIP:35357 --glance-pass changeme --memcached-servers CONTROLLER1:11211,CONTROLLER2:11211### Nova HACreate nova databaseff --user ubuntu --hosts CONTROLLER1 openstack nova create-nova-db --root-db-pass changeme --nova-db-pass changemeCreate service credentialsff --user ubuntu --hosts CONTROLLER1 openstack nova create-service-credentials --os-password changeme --os-auth-url http://CONTROLLER_VIP:35357/v3 --nova-pass changeme --public-endpoint 'http://CONTROLLER_VIP:8774/v2.1/%\(tenant_id\)s' --internal-endpoint 'http://CONTROLLER_VIP:8774/v2.1/%\(tenant_id\)s' --admin-endpoint 'http://CONTROLLER_VIP:8774/v2.1/%\(tenant_id\)s'Install nova on CONTROLLER1ff --user ubuntu --hosts CONTROLLER1 openstack nova install --connection mysql+pymysql://nova:NOVA_PASS@CONTROLLER_VIP/nova --api-connection mysql+pymysql://nova:NOVA_PASS@CONTROLLER_VIP/nova_api --auth-uri http://CONTROLLER_VIP:5000 --auth-url http://CONTROLLER_VIP:35357 --nova-pass changeme --my-ip MANAGEMENT_IP --memcached-servers CONTROLLER1:11211,CONTROLLER2:11211 --rabbit-hosts CONTROLLER1,CONTROLLER2 --rabbit-user openstack --rabbit-pass changeme --glance-api-servers http://CONTROLLER_VIP:9292 --neutron-endpoint http://CONTROLLER_VIP:9696 --neutron-pass changeme --metadata-proxy-shared-secret changeme --populateInstall nova on CONTROLLER2ff --user ubuntu --hosts CONTROLLER2 openstack nova install --connection mysql+pymysql://nova:NOVA_PASS@CONTROLLER_VIP/nova --api-connection mysql+pymysql://nova:NOVA_PASS@CONTROLLER_VIP/nova_api --auth-uri http://CONTROLLER_VIP:5000 --auth-url http://CONTROLLER_VIP:35357 --nova-pass changeme --my-ip MANAGEMENT_IP --memcached-servers CONTROLLER1:11211,CONTROLLER2:11211 --rabbit-hosts CONTROLLER1,CONTROLLER2 --rabbit-user openstack --rabbit-pass changeme --glance-api-servers http://CONTROLLER_VIP:9292 --neutron-endpoint http://CONTROLLER_VIP:9696 --neutron-pass changeme --metadata-proxy-shared-secret changeme### Nova ComputeAdd nova computes(use `uuidgen` to generate the ceph uuid)ff --user ubuntu --hosts COMPUTE1 openstack nova-compute install --my-ip MANAGEMENT_IP --rabbit-hosts CONTROLLER1,CONTROLLER2 --rabbit-user openstack --rabbit-pass changeme --auth-uri http://CONTROLLER_VIP:5000 --auth-url http://CONTROLLER_VIP:35357 --nova-pass changeme --novncproxy-base-url http://CONTROLLER_VIP:6080/vnc_auto.html --glance-api-servers http://CONTROLLER_VIP:9292 --neutron-endpoint http://CONTROLLER_VIP:9696 --neutron-pass changeme --rbd-secret-uuid changeme-changeme-changeme-changeme --memcached-servers CONTROLLER1:11211,CONTROLLER2:11211ff --user ubuntu --hosts COMPUTE2 openstack nova-compute install --my-ip MANAGEMENT_IP --rabbit-hosts CONTROLLER1,CONTROLLER2 --rabbit-user openstack --rabbit-pass changeme --auth-uri http://CONTROLLER_VIP:5000 --auth-url http://CONTROLLER_VIP:35357 --nova-pass changeme --novncproxy-base-url http://CONTROLLER_VIP:6080/vnc_auto.html --glance-api-servers http://CONTROLLER_VIP:9292 --neutron-endpoint http://CONTROLLER_VIP:9696 --neutron-pass changeme --rbd-secret-uuid changeme-changeme-changeme-changeme --memcached-servers CONTROLLER1:11211,CONTROLLER2:11211The libvirt defaults to using ceph as shared storage, the ceph pool for running instance is vms. if you do not using ceph as it's bachend, you must remove the following param:images_type = rbdimages_rbd_pool = vmsimages_rbd_ceph_conf = /etc/ceph/ceph.confrbd_user = cinderrbd_secret_uuid = changeme-changeme-changeme-changemedisk_cachemodes="network=writeback"live_migration_flag="VIR_MIGRATE_UNDEFINE_SOURCE,VIR_MIGRATE_PEER2PEER,VIR_MIGRATE_LIVE,VIR_MIGRATE_PERSIST_DEST,VIR_MIGRATE_TUNNELLED"### Neutron HACreate nova databaseff --user ubuntu --hosts CONTROLLER1 openstack neutron create-neutron-db --root-db-pass changeme --neutron-db-pass changemeCreate service credentialsff --user ubuntu --hosts CONTROLLER1 openstack neutron create-service-credentials --os-password changeme --os-auth-url http://CONTROLLER_VIP:35357/v3 --neutron-pass changeme --public-endpoint http://CONTROLLER_VIP:9696 --internal-endpoint http://CONTROLLER_VIP:9696 --admin-endpoint http://CONTROLLER_VIP:9696Install Neutron for self-serviceff --user ubuntu --hosts CONTROLLER1 openstack neutron install --connection mysql+pymysql://neutron:NEUTRON_PASS@CONTROLLER_VIP/neutron --rabbit-hosts CONTROLLER1,CONTROLLER2 --rabbit-user openstack --rabbit-pass changeme --auth-uri http://CONTROLLER_VIP:5000 --auth-url http://CONTROLLER_VIP:35357 --neutron-pass changeme --nova-url http://CONTROLLER_VIP:8774/v2.1 --nova-pass changeme --public-interface eth1 --local-ip MANAGEMENT_INTERFACE_IP --nova-metadata-ip CONTROLLER_VIP --metadata-proxy-shared-secret changeme-changeme-changeme-changeme --memcached-servers CONTROLLER1:11211,CONTROLLER2:11211 --populateff --user ubuntu --hosts CONTROLLER2 openstack neutron install --connection mysql+pymysql://neutron:NEUTRON_PASS@CONTROLLER_VIP/neutron --rabbit-hosts CONTROLLER1,CONTROLLER2 --rabbit-user openstack --rabbit-pass changeme --auth-uri http://CONTROLLER_VIP:5000 --auth-url http://CONTROLLER_VIP:35357 --neutron-pass changeme --nova-url http://CONTROLLER_VIP:8774/v2.1 --nova-pass changeme --public-interface eth1 --local-ip MANAGEMENT_INTERFACE_IP --nova-metadata-ip CONTROLLER_VIP --metadata-proxy-shared-secret changeme-changeme-changeme-changeme --memcached-servers CONTROLLER1:11211,CONTROLLER2:11211### Neutron AgentInstall neutron agent on compute nodesff --user ubuntu --hosts COMPUTE1 openstack neutron-agent install --rabbit-hosts CONTROLLER1,CONTROLLER2 --rabbit-user openstack --rabbit-pass changeme --auth-uri http://CONTROLLER_VIP:5000 --auth-url http://CONTROLLER_VIP:35357 --neutron-pass changeme --public-interface eth1 --local-ip MANAGEMENT_INTERFACE_IP --memcached-servers CONTROLLER1:11211,CONTROLLER2:11211ff --user ubuntu --hosts COMPUTE2 openstack neutron-agent install --rabbit-hosts CONTROLLER1,CONTROLLER2 --rabbit-user openstack --rabbit-pass changeme --auth-uri http://CONTROLLER_VIP:5000 --auth-url http://CONTROLLER_VIP:35357 --neutron-pass changeme --public-interface eth1 --local-ip MANAGEMENT_INTERFACE_IP --memcached-servers CONTROLLER1:11211,CONTROLLER2:11211### Horizon HAInstall horizon on controller nodesff --user ubuntu --hosts CONTROLLER1,CONTROLLER2 openstack horizon install --openstack-host CONTROLLER_VIP  --memcached-servers CONTROLLER1:11211 --time-zone Asia/Shanghai### Cinder HACreate cinder databaseff --user ubuntu --hosts CONTROLLER1 openstack cinder create-cinder-db --root-db-pass changeme --cinder-db-pass changemeCreate cinder service creadentialsff --user ubuntu --hosts CONTROLLER1 openstack cinder create-service-credentials --os-password changeme --os-auth-url http://CONTROLLER_VIP:35357/v3 --cinder-pass changeme --public-endpoint-v1 'http://CONTROLLER_VIP:8776/v1/%\(tenant_id\)s' --internal-endpoint-v1 'http://CONTROLLER_VIP:8776/v1/%\(tenant_id\)s' --admin-endpoint-v1 'http://CONTROLLER_VIP:8776/v1/%\(tenant_id\)s' --public-endpoint-v2 'http://CONTROLLER_VIP:8776/v2/%\(tenant_id\)s' --internal-endpoint-v2 'http://CONTROLLER_VIP:8776/v2/%\(tenant_id\)s' --admin-endpoint-v2 'http://CONTROLLER_VIP:8776/v2/%\(tenant_id\)s'Install cinder-api and cinder-volume on controller nodes, the volume backend defaults to ceph (you must have ceph installed)ff --user ubuntu --hosts CONTROLLER1 openstack cinder install --connection mysql+pymysql://cinder:CINDER_PASS@CONTROLLER_VIP/cinder --rabbit-user openstack --rabbit-pass changeme --rabbit-hosts CONTROLLER1,CONTROLLER2 --auth-uri http://CONTROLLER_VIP:5000 --auth-url http://CONTROLLER_VIP:35357 --cinder-pass changeme --my-ip MANAGEMENT_INTERFACE_IP --glance-api-servers http://CONTROLLER_VIP:9292 --rbd-secret-uuid changeme-changeme-changeme-changeme --memcached-servers CONTROLLER1:11211,CONTROLLER2:11211 --populateff --user ubuntu --hosts CONTROLLER2 openstack cinder install --connection mysql+pymysql://cinder:CINDER_PASS@CONTROLLER_VIP/cinder --rabbit-user openstack --rabbit-pass changeme --rabbit-hosts CONTROLLER1,CONTROLLER2 --auth-uri http://CONTROLLER_VIP:5000 --auth-url http://CONTROLLER_VIP:35357 --cinder-pass changeme --my-ip MANAGEMENT_INTERFACE_IP --glance-api-servers http://CONTROLLER_VIP:9292 --rbd-secret-uuid changeme-changeme-changeme-changeme --memcached-servers CONTROLLER1:11211,CONTROLLER2:11211### Swift proxy HACreate the Identity service credentialsff --user ubuntu --hosts CONTROLLER1 openstack swift create-service-credentials --os-password changeme --os-auth-url http://CONTROLLER_VIP:35357/v3 --swift-pass changeme --public-endpoint 'http://CONTROLLER_VIP:8080/v1/AUTH_%\(tenant_id\)s' --internal-endpoint 'http://CONTROLLER_VIP:8080/v1/AUTH_%\(tenant_id\)s' --admin-endpoint http://CONTROLLER_VIP:8080/v1Install swift proxyff --user ubuntu --hosts CONTROLLER1,CONTROLLER2 openstack swift install --auth-uri http://CONTROLLER_VIP:5000 --auth-url http://CONTROLLER_VIP:35357 --swift-pass changeme --memcached-servers CONTROLLER1:11211,CONTROLLER2:11211### Swift storagePrepare disks on storage nodeff --user ubuntu --hosts OBJECT1,OBJECT2 openstack swift-storage prepare-disks --name sdb,sdc,sdd,sdeInstall swift storage on storage nodeff --user ubuntu --hosts OBJECT1 openstack swift-storage install --address MANAGEMENT_INTERFACE_IP --bind-ip MANAGEMENT_INTERFACE_IPff --user ubuntu --hosts OBJECT2 openstack swift-storage install --address MANAGEMENT_INTERFACE_IP --bind-ip MANAGEMENT_INTERFACE_IPCreate account ring on controller nodeff --user ubuntu --hosts CONTROLLER1 openstack swift-storage create-account-builder-file --partitions 10 --replicas 3 --moving 1ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage account-builder-add --region 1 --zone 1 --ip OBJECT1_MANAGEMENT_IP --device sdb --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage account-builder-add --region 1 --zone 1 --ip OBJECT1_MANAGEMENT_IP --device sdc --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage account-builder-add --region 1 --zone 1 --ip OBJECT1_MANAGEMENT_IP --device sdd --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage account-builder-add --region 1 --zone 1 --ip OBJECT1_MANAGEMENT_IP --device sde --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage account-builder-add --region 1 --zone 1 --ip OBJECT2_MANAGEMENT_IP --device sdb --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage account-builder-add --region 1 --zone 1 --ip OBJECT2_MANAGEMENT_IP --device sdc --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage account-builder-add --region 1 --zone 1 --ip OBJECT2_MANAGEMENT_IP --device sdd --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage account-builder-add --region 1 --zone 1 --ip OBJECT2_MANAGEMENT_IP --device sde --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage account-builder-rebalanceCreate container ring on controller nodeff --user ubuntu --hosts CONTROLLER1 openstack swift-storage create-container-builder-file --partitions 10 --replicas 3 --moving 1ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage container-builder-add --region 1 --zone 1 --ip OBJECT1_MANAGEMENT_IP --device sdb --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage container-builder-add --region 1 --zone 1 --ip OBJECT1_MANAGEMENT_IP --device sdc --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage container-builder-add --region 1 --zone 1 --ip OBJECT1_MANAGEMENT_IP --device sdd --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage container-builder-add --region 1 --zone 1 --ip OBJECT1_MANAGEMENT_IP --device sde --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage container-builder-add --region 1 --zone 1 --ip OBJECT2_MANAGEMENT_IP --device sdb --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage container-builder-add --region 1 --zone 1 --ip OBJECT2_MANAGEMENT_IP --device sdc --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage container-builder-add --region 1 --zone 1 --ip OBJECT2_MANAGEMENT_IP --device sdd --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage container-builder-add --region 1 --zone 1 --ip OBJECT2_MANAGEMENT_IP --device sde --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage container-builder-rebalanceCreate object ring on controller nodeff --user ubuntu --hosts CONTROLLER1 openstack swift-storage create-object-builder-file --partitions 10 --replicas 3 --moving 1ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage object-builder-add --region 1 --zone 1 --ip OBJECT1_MANAGEMENT_IP --device sdb --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage object-builder-add --region 1 --zone 1 --ip OBJECT1_MANAGEMENT_IP --device sdc --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage object-builder-add --region 1 --zone 1 --ip OBJECT1_MANAGEMENT_IP --device sdd --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage object-builder-add --region 1 --zone 1 --ip OBJECT1_MANAGEMENT_IP --device sde --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage object-builder-add --region 1 --zone 1 --ip OBJECT2_MANAGEMENT_IP --device sdb --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage object-builder-add --region 1 --zone 1 --ip OBJECT2_MANAGEMENT_IP --device sdc --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage object-builder-add --region 1 --zone 1 --ip OBJECT2_MANAGEMENT_IP --device sdd --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage object-builder-add --region 1 --zone 1 --ip OBJECT2_MANAGEMENT_IP --device sde --weight 100ff --user ubuntu --hosts CONTROLLER1 openstack swift-storage object-builder-rebalanceSync the builder file from controller node to each storage node and other any proxy nodeff --user ubuntu --host CONTROLLER1 openstack swift-storage sync-builder-file --to CONTROLLER2,OBJECT1,OBJECT2Finalize installation on all nodesff --user ubuntu --hosts CONTROLLER1,CONTROLLER2,OBJECT1,OBJECT2 openstack swift finalize-install --swift-hash-path-suffix changeme --swift-hash-path-prefix changeme### Ceph GuideFor more information about ceph backend visit:[preflight](http://docs.ceph.com/docs/jewel/start/quick-start-preflight/)[Cinder and Glance driver](http://docs.ceph.com/docs/jewel/rbd/rbd-openstack/)On Xenial please using ceph-deploy version 1.5.34Install ceph-deploy(1.5.34)wget -q -O- 'https://download.ceph.com/keys/release.asc' | sudo apt-key add -echo deb http://download.ceph.com/debian-jewel/ $(lsb_release -sc) main | sudo tee /etc/apt/sources.list.d/ceph.listsudo apt-get update && sudo apt-get install ceph-deployCreate ceph cluster directorymkdir ceph-clustercd ceph-clusterCreate cluster and add initial monitor(s) to the ceph.confceph-deploy new  CONTROLLER1 CONTROLLER2 COMPUTE1 COMPUTE2 BLOCK1 BLOCK2echo "osd pool default size = 2" | tee -a ceph.confInstall ceph client(Optionaly you can use `--release jewel` to install jewel version, the ceph-deploy 1.5.34 default release is jewel) and you can use `--repo-url http://your-local-repo.example.org/mirror/download.ceph.com/debian-jewel` to specify the local repository.ceph-deploy install PLAYBACK-NODE CONTROLLER1 CONTROLLER2 COMPUTE1 COMPUTE2 BLOCK1 BLOCK2Add the initial monitor(s) and gather the keysceph-deploy mon create-initialIf you want to add additional monitors, do thatceph-deploy mon add {additional-monitor}Add ceph osd(s)ceph-deploy osd create --zap-disk BLOCK1:/dev/sdbceph-deploy osd create --zap-disk BLOCK1:/dev/sdcceph-deploy osd create --zap-disk BLOCK2:/dev/sdbceph-deploy osd create --zap-disk BLOCK2:/dev/sdcSync admin keyceph-deploy admin PLAYBACK-NODE CONTROLLER1 CONTROLLER2 COMPUTE1 COMPUTE2 BLOCK1 BLOCK2sudo chmod +r /etc/ceph/ceph.client.admin.keyring # On all ceph clients nodeCreate osd pool for cinder and running instanceceph osd pool create volumes 512ceph osd pool create vms 512ceph osd pool create images 512Setup ceph client authenticationceph auth get-or-create client.cinder mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rwx pool=vms, allow rx pool=images'ceph auth get-or-create client.glance mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=images'Add the keyrings for `client.cinder` and `client.glance` to appropriate nodes and change their ownershipceph auth get-or-create client.cinder | sudo tee /etc/ceph/ceph.client.cinder.keyring # On all cinder-volume nodessudo chown cinder:cinder /etc/ceph/ceph.client.cinder.keyring" # On all cinder-volume nodesceph auth get-or-create client.glance | sudo tee /etc/ceph/ceph.client.glance.keyring # On all glance-api nodessudo chown glance:glance /etc/ceph/ceph.client.glance.keyring" # On all glance-api nodesNodes running `nova-compute` need the keyring file for the `nova-compute` processceph auth get-or-create client.cinder | sudo tee /etc/ceph/ceph.client.cinder.keyring # On all nova-compute nodesThey also need to store the secret key of the `client.cinder user` in `libvirt`. The libvirt process needs it to access the cluster while attaching a block device from Cinder.Create a temporary copy of the secret key on the nodes running `nova-compute`ceph auth get-key client.cinder | tee client.cinder.key # On all nova-compute nodesThen, on the `compute nodes`, add the secret key to `libvirt` and remove the temporary copy of the key(the uuid is the same as your --rbd-secret-uuid option, you have to save the uuid for later)uuidgen457eb676-33da-42ec-9a8c-9293d545c337# The following steps on all nova-compute nodescat > secret.xml <<EOF<secret ephemeral='no' private='no'><uuid>457eb676-33da-42ec-9a8c-9293d545c337</uuid><usage type='ceph'><name>client.cinder secret</name></usage></secret>EOFsudo virsh secret-define --file secret.xmlSecret 457eb676-33da-42ec-9a8c-9293d545c337 createdsudo virsh secret-set-value --secret 457eb676-33da-42ec-9a8c-9293d545c337 --base64 $(cat client.cinder.key) && rm client.cinder.key secret.xml(optional)Now on every compute nodes edit your Ceph configuration file, add the client section[client]rbd cache = truerbd cache writethrough until flush = truerbd concurrent management ops = 20[client.cinder]keyring = /etc/ceph/ceph.client.cinder.keyring(optional)On every glance-api nodes edit your Ceph configuration file, add the client section[client.glance]keyring= /etc/ceph/ceph.client.glance.keyring(optional)If you want to remove osdsudo stop ceph-mon-all && sudo stop ceph-osd-all # On osd nodeceph osd out {OSD-NUM}ceph osd crush remove osd.{OSD-NUM}ceph auth del osd.{OSD-NUM}ceph osd rm {OSD-NUM}ceph osd crush remove {HOST}(optional)If you want to remove monitorceph mon remove {MON-ID}Notes: you need to restart the `nova-compute`, `cinder-volume` and `glance-api` services to finalize the installation.### Shared File Systems serviceCreate manila database and service credentialsff --user ubuntu --hosts CONTROLLER1 openstack manila create-manila-db --root-db-pass CHANGEME --manila-db-pass CHANGEMEff --user ubuntu --hosts CONTROLLER1 openstack manila create-service-credentials --os-password CHANGEME --os-auth-url http://CONTROLLER_VIP:35357/v3 --manila-pass CHANGEME --public-endpoint-v1 "http://CONTROLLER_VIP:8786/v1/%\(tenant_id\)s" --internal-endpoint-v1 "http://CONTROLLER_VIP:8786/v1/%\(tenant_id\)s" --admin-endpoint-v1 "http://CONTROLLER_VIP:8786/v1/%\(tenant_id\)s" --public-endpoint-v2 "http://CONTROLLER_VIP:8786/v2/%\(tenant_id\)s" --internal-endpoint-v2 "http://CONTROLLER_VIP:8786/v2/%\(tenant_id\)s" --admin-endpoint-v2 "http://CONTROLLER_VIP:8786/v2/%\(tenant_id\)s"Install manila on CONTROLLER1 and CONTROLLER2ff --user ubuntu --hosts CONTROLLER1 openstack manila install --connection mysql+pymysql://manila:CHANGEME@CONTROLLER_VIP/manila --auth-uri http://CONTROLLER_VIP:5000 --auth-url http://CONTROLLER_VIP:35357 --manila-pass CHANGEME --my-ip CONTROLLER1 --memcached-servers CONTROLLER1:11211,CONTROLLER2:11211 --rabbit-hosts CONTROLLER1,CONTROLLER2 --rabbit-user openstack --rabbit-pass CHANGEME --populateff --user ubuntu --hosts CONTROLLER2 openstack manila install --connection mysql+pymysql://manila:CHANGEME@CONTROLLER_VIP/manila --auth-uri http://CONTROLLER_VIP:5000 --auth-url http://CONTROLLER_VIP:35357 --manila-pass CHANGEME --my-ip CONTROLLER2 --memcached-servers CONTROLLER1:11211,CONTROLLER2:11211 --rabbit-hosts CONTROLLER1,CONTROLLER2 --rabbit-user openstack --rabbit-pass CHANGEMEInstall manila share on CONTROLLER1 and CONTROLLER2ff --user ubuntu --hosts CONTROLLER1 openstack manila-share install --connection mysql+pymysql://manila:CHANGEME@CONTROLLER_VIP/manila --auth-uri http://CONTROLLER_VIP:5000 --auth-url http://CONTROLLER_VIP:35357 --manila-pass CHANGEME --my-ip CONTROLLER1 --memcached-servers CONTROLLER1:11211,CONTROLLER2:11211 --rabbit-hosts CONTROLLER1,CONTROLLER2 --rabbit-user openstack --rabbit-pass CHANGEME --neutron-endpoint http://CONTROLLER_VIP:9696 --neutron-pass CHANGEME --nova-pass CHANGEME --cinder-pass CHANGEMEff --user ubuntu --hosts CONTROLLER2 openstack manila-share install --connection mysql+pymysql://manila:CHANGEME@CONTROLLER_VIP/manila --auth-uri http://CONTROLLER_VIP:5000 --auth-url http://CONTROLLER_VIP:35357 --manila-pass CHANGEME --my-ip CONTROLLER2 --memcached-servers CONTROLLER1:11211,CONTROLLER2:11211 --rabbit-hosts CONTROLLER1,CONTROLLER2 --rabbit-user openstack --rabbit-pass CHANGEME --neutron-endpoint http://CONTROLLER_VIP:9696 --neutron-pass CHANGEME --nova-pass CHANGEME --cinder-pass CHANGEMECreate the service image for manilahttp://docs.openstack.org/mitaka/install-guide-ubuntu/launch-instance-manila.htmlCreate shares with share servers management supporthttp://docs.openstack.org/mitaka/install-guide-ubuntu/launch-instance-manila-dhss-true-option2.html