Project: Soda Foundation (Runtime / Cloud Native Storage)
File: OpenSDS_Aruba_POC_Plan.pdf

 
 OpenSDS Aruba POC Test Plan June 2018 Revision: 0.6 Author: OpenSDS   OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT ii Document Revision History Version Date Comments 0.1 6/12/2018 Initial revision. 0.2 6/15/2018 Added content to sections host-based replication, array-based replication, CLI guide, Cinder compatible APIs. 0.3 6/20/2018 Modified dates after reviewing it at OSS Summit Tokyo 0.4 6/26/2018 Add Dashboard section; Modify CLI section. 0.5 6/29/2018 Update Dashboard section and Installation section. 0.6 7/05/2018 Update the use cases of dashboard and replication with kubernetes          Related Documents Author Documents                  OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT iii Table of Contents 1 Overview................................................................................................................ 1 1.1 Project Scope and Objectives .................................................................................................. 1 1.2 POC Timeline ........................................................................................................................... 2 2 System requirements............................................................................................ 2 2.1 Hardware .................................................................................................................................. 2 2.2 Software .................................................................................................................................... 2 
 OS ....................................................................................................................................... 2 3 Features .................................................................................................................. 3 4 Installation............................................................................................................. 4 4.1 Prerequisite ............................................................................................................................... 4 
 Packages ............................................................................................................................. 4 
 Golang ................................................................................................................................ 4 
 docker ................................................................................................................................. 4 4.2 Kubernetes Local Cluster Deployment ................................................................................... 5 
 Install Etcd ......................................................................................................................... 5 
 kubernetes local cluster ..................................................................................................... 5 4.3 OpenSDS Deployment ............................................................................................................. 6 
 Pre-config ........................................................................................................................... 6 
 Download opensds-installer code .................................................................................... 6 
 Install ansible tool.............................................................................................................. 6 
 Configure OpenSDS cluster variables .............................................................................. 6 4.3.4.1 System environment ...................................................................................................... 6 4.3.4.2 LVM .............................................................................................................................. 7 4.3.4.3 Ceph .............................................................................................................................. 7 4.3.4.4 Cinder............................................................................................................................ 7 
 Check if the hosts can be reached ..................................................................................... 8 
 Run opensds-ansible playbook to start deploy ............................................................... 8 4.4 Test OpenSDS ........................................................................................................................... 8 
 Use OpenSDS CLI Tool ..................................................................................................... 8 OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT iv 
 Test CSI Plugin .................................................................................................................. 9 
 OpenSDS Dashboard......................................................................................................... 9 4.5 Cleanup OpenSDS .................................................................................................................... 9 
 Run opensds-ansible playbook to clean the environment .............................................. 9 
 Run ceph-ansible playbook to clean ceph cluster if ceph is deployed ......................... 10 
 Remove ceph-ansible source code (optional) ................................................................ 10 4.6 Troubleshooting ..................................................................................................................... 10 
 Problem Starting CSI Plugin ........................................................................................... 10 5 Use Cases ............................................................................................................. 11 5.1 Dashboard .............................................................................................................................. 11 
 Administrator configuration .......................................................................................... 11 5.1.1.1 Login ........................................................................................................................... 11 5.1.1.2 Create tenant ............................................................................................................... 12 5.1.1.3 Create user ................................................................................................................... 12 5.1.1.4 Create profile................................................................................................................ 13 5.1.1.5 View resources ............................................................................................................. 13 
 Tenant provision volume ................................................................................................ 14 5.1.2.1 Overview ..................................................................................................................... 14 5.1.2.2 Create volume .............................................................................................................. 14 5.1.2.3 Expand volume size...................................................................................................... 15 5.1.2.4 Create volume snapshot ................................................................................................ 15 5.1.2.5 Create volume from snapshot........................................................................................ 16 5.1.2.6 Create volume replication ............................................................................................. 16 5.1.2.7 Disable/Enable/Failover volume replication .................................................................. 17 5.1.2.8 Create volume group .................................................................................................... 17 5.1.2.9 Add volumes into group ............................................................................................... 18 5.1.2.10 Tenant isolation ....................................................................................................... 18 5.2 Kubernetes .............................................................................................................................. 20 5.3 OpenStack .............................................................................................................................. 20 
 Use OpenSDS to Manage Cinder Drivers ...................................................................... 21 5.3.1.1 Prepare ........................................................................................................................ 21 5.3.1.2 Install OpenStack using devstack ................................................................................. 21 5.3.1.3 Install ceph using ansible ............................................................................................. 21 5.3.1.4 Configuration .............................................................................................................. 22 5.3.1.4.1 OpenSDS ............................................................................................................. 22 OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT v 5.3.1.4.2 Set ceph as backend of Cinder ........................................................................... 23 5.3.1.5 Testing......................................................................................................................... 24 
 OpenSDS with Cinder Compatible API ......................................................................... 25 5.3.2.1 Installation .................................................................................................................. 25 5.3.2.2 Volume Types .............................................................................................................. 26 5.3.2.2.1 List all volume types (default policy) ................................................................ 26 5.3.2.2.2 Delete a volume type .......................................................................................... 26 5.3.2.2.3 List all volume types(0) ...................................................................................... 26 5.3.2.2.4 Create a volume type .......................................................................................... 27 5.3.2.2.5 Show volume type detail .................................................................................... 27 5.3.2.2.6 Create a volume type（2nd） ............................................................................ 27 5.3.2.2.7 List all volume types (2) ..................................................................................... 27 5.3.2.2.8 Update an encryption type ................................................................................. 28 5.3.2.2.9 Lists current volume types and extra specs. ..................................................... 28 5.3.2.2.10 Create or update extra specs for volume type ................................................ 29 5.3.2.2.11 Delete extra specification for volume type ...................................................... 29 5.3.2.3 Volumes ....................................................................................................................... 29 5.3.2.3.1 List accessible volumes with details (0) ............................................................. 29 5.3.2.3.2 Create a volume (1st) .......................................................................................... 30 5.3.2.3.3 List accessible volumes with details (1) ............................................................. 30 5.3.2.3.4 Show a volume’s details ..................................................................................... 30 5.3.2.3.5 Delete a volume .................................................................................................. 30 5.3.2.4 Snapshots..................................................................................................................... 30 5.3.2.4.1 Create a snapshot ................................................................................................ 31 5.3.2.4.2 List snapshots and details .................................................................................. 31 5.3.2.4.3 Show a snapshot’s details ................................................................................... 31 5.3.2.4.4 Delete a snapshot ................................................................................................ 31 5.3.2.5 Attachments ................................................................................................................ 31 5.3.2.5.1 Create attachment ............................................................................................... 31 5.3.2.5.2 Show attachment................................................................................................. 31 5.3.2.5.3 List attachment .................................................................................................... 32 5.3.2.5.4 Update attachment ............................................................................................. 32 5.3.2.5.5 Delete attachment ............................................................................................... 33 5.4 Array-based Replication using Dorado................................................................................ 34 
 Without Kubernetes ........................................................................................................ 34 5.4.1.1 Configuration .............................................................................................................. 34 5.4.1.2 Testing......................................................................................................................... 38 OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT vi 
 With Kubernetes .............................................................................................................. 41 5.4.2.1 Configuration .............................................................................................................. 41 5.4.2.2 Testing steps ................................................................................................................ 45 5.5 Host-based Replication using DRBD ................................................................................... 47 
 Prepare ............................................................................................................................. 47 
 Install DRBD .................................................................................................................... 47 
 Configuration .................................................................................................................. 47 
 Create Replication ........................................................................................................... 49 
 Check result ..................................................................................................................... 50 6 OpenSDS CLI Guide ......................................................................................... 51 6.1 List Docks ............................................................................................................................... 51 6.2 List Pools ................................................................................................................................ 52 6.3 Create/Delete Profile.............................................................................................................. 53 6.4 Create/Delete/Get/List Volume(s) ......................................................................................... 53 6.5 Create/Delete/Get/List Snapshot(s) ...................................................................................... 55 6.6 Create Volume from Snapshot............................................................................................... 56 6.7 Expand Volume....................................................................................................................... 57 6.8 Create/Update/Delete/Get/List Volume Groups .................................................................. 58 6.9 Replication ............................................................................................................................. 60 OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 1  1 Overview OpenSDS Aruba will be released in the week of June 27, 2018.  This document serves as the OpenSDS Aruba POC Test Plan.  It covers the following topics: 1. Overall project scope and objectives 2. Test objectives and success criteria 3. Test resources required 4. Test schedule 5. Use cases a. OpenStack/Kubernetes/bare-metal/mixed environment provisioning b. Host and storage replication, and local and remote replication c. Test cases for each use case 1.1 Project Scope and Objectives 
In the Zealand release, basic volume and snapshot CRUD functionalities were added and Kubernetes CSI/FlexVolume support was also added. 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 2 During the Aruba release, the focus has been on storage orchestration, building advanced automated storage and data services across traditional data centers, private and public clouds.  Functionalities in this release include basic OpenStack integration, integrating with Keystone for identity service, array-based and host-based replication, and storage profiles design based on Swordfish.  A deployment tool using Ansible is also available to install OpenSDS with Keystone and Dashboard.  1.2 POC Timeline June 15: POC plan draft ready for EUAC review June 20: Aruba release. POC plan approval. July 1-31: POC testing August 7: POC results/comments/testimonials  2 System requirements 2.1 Hardware The hardware requirements are described in this section. For array-based replication, two physical servers and two Dorado arrays are needed. For host-based replication, two physical servers are needed.  For other tests described in this POC, one physical server or one VM can be used for basic testing.  2.2 Software The software requirements are described in this section. 
 OS Ubuntu 16.04.2 has been used during the testing and therefore should be used in this POC: root@proxy:~# cat /etc/issue Ubuntu 16.04.2 LTS \n \l OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 3 For host-based replication, required DRBD software is described in the relevant section later.  Other required software is described in the installation section.  3 Features Features to be tested include the following: - Multitenancy using Keystone - Create/delete volume - Expand volume - Create/delete snapshot - Create volume from snapshot - Create volume group - Create/delete profile - Array-based replication - Host-based replication - Use Cinder-compatible API in OpenStack  Supported storage backends include the following: - LVM - Ceph - Dorado - IBM storage via Cinder driver? - Cinder stand alone with LVM - Cinder in an OpenStack deployment with LVM  Supported protocols: - iSCSI OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 4 - FC - RBD  Testing environment includes the following: - OpenSDS with Kubernetes - OpenSDS with OpenStack (full OpenStack deployment or Cinder stand-alone) - Hotpot only on bare-metal or a VM  4 Installation In the section, how to install OpenSDS using Ansible playbook will be discussed. Section 4.1 is prerequisites for Installation. If you are testing OpenSDS with Kubernetes, read section 4.2 Kubernetes Local Cluster Deployment first.  Otherwise, go to section 4.3 directly.  For OpenSDS with OpenStack, testing with Cinder stand-alone is part of the OpenSDS ansible installation in section 4.3, and testing with a separate Cinder deployment is discussed in section 5.3 OpenStack. 4.1 Prerequisite 
 Packages Install following packages: apt-get install vim git curl wget make gcc zip 
 Golang You can install golang by executing commands blow:  wget https://storage.googleapis.com/golang/go1.9.2.linux-amd64.tar.gz  tar -C /usr/local -xzf go1.9.2.linux-amd64.tar.gz  echo 'export PATH=$PATH:/usr/local/go/bin' >> /etc/profile  echo 'export GOPATH=$HOME/gopath' >> /etc/profile  source /etc/profile Check golang version information:  root@proxy:~# go version  go version go1.9.2 linux/amd64 
 docker Install docker: OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 5 wget https://download.docker.com/linux/ubuntu/dists/xenial/pool/stable/amd64/docker-ce_18.03.1~ce-0~ubuntu_amd64.deb dpkg -i docker-ce_18.03.1~ce-0~ubuntu_amd64.deb Version information: root@proxy:~# docker version Client:   Version:      18.03.1-ce   API version:  1.37   Go version:   go1.9.5   Git commit:   9ee9f40   Built:        Thu Apr 26 07:17:20 2018   OS/Arch:      linux/amd64   Experimental: false   Orchestrator: swarm  Server:   Engine:    Version:      18.03.1-ce    API version:  1.37 (minimum version 1.12)    Go version:   go1.9.5    Git commit:   9ee9f40    Built:        Thu Apr 26 07:15:30 2018    OS/Arch:      linux/amd64    Experimental: false  4.2 Kubernetes Local Cluster Deployment 
 Install Etcd You can install etcd by executing commands blow: cd $HOME wget https://github.com/coreos/etcd/releases/download/v3.3.0/etcd-v3.3.0-linux-amd64.tar.gz tar -xzf etcd-v3.3.0-linux-amd64.tar.gz cd etcd-v3.3.0-linux-amd64 sudo cp -f etcd etcdctl /usr/local/bin/ 
 kubernetes local cluster You can start the latest k8s local cluster by executing commands blow: cd $HOME git clone https://github.com/kubernetes/kubernetes.git cd $HOME/kubernetes git checkout v1.10.0 make echo alias kubectl='$HOME/kubernetes/cluster/kubectl.sh' >> /etc/profile ALLOW_PRIVILEGED=true FEATURE_GATES=CSIPersistentVolume=true,MountPropagation=true OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 6 RUNTIME_CONFIG="storage.k8s.io/v1alpha1=true" LOG_LEVEL=5 hack/local-up-cluster.sh 4.3 OpenSDS Deployment In this section, the steps to deploy an OpenSDS local cluster are described. 
 Pre-config  First download some system packages: apt-get install -y git curl wget Then config /etc/ssh/sshd_config file and change one line: PermitRootLogin yes Next generate ssh-token: ssh-keygen -t rsa ssh-copy-id -i ~/.ssh/id_rsa.pub <ip_address> # IP address of the target machine of the installation 
 Download opensds-installer code git clone https://github.com/opensds/opensds-installer.git cd opensds-installer/ansible 
 Install ansible tool To install ansible, run the commands below: # This step is needed to upgrade ansible to version 2.4.2 which is required for the "include_tasks" ansible command. chmod +x ./install_ansible.sh && ./install_ansible.sh ansible --version # Ansible version 2.4.x is required. 
 Configure OpenSDS cluster variables 4.3.4.1 System environment To integrate OpenSDS with cloud platform (for example k8s), modify nbp_plugin_type variable in group_vars/common.yml: nbp_plugin_type: hotpot_only # hotpot_only is the default integration method. Other available options are 'csi' and 'flexvolume'. OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 7 Note: If ‘csi’ is the selected nbp_plugin_type, make sure section 4.2 Kubernetes Local Cluster Deployment is followed before proceeding. Change opensds_endpoint to the actual IP address: opensds_endpoint: http://127.0.0.1:50040 # The IP (127.0.0.1) should be replaced with the opensds actual endpoint IP 4.3.4.2 LVM If lvm is chosen as the storage backend, there is no need to modify group_vars/osdsdock.yml because it is the default choice: enabled_backend: lvm # Change it according to the chosen backend. Supported backends include 'lvm', 'ceph', and 'cinder' Change tgtBindIp variable in group_vars/lvm/lvm.yaml to your real host IP address: tgtBindIp: 127.0.0.1 # change tgtBindIp to your real host ip, run 'ifconfig' to check 4.3.4.3 Ceph If ceph is chosen as storage backend, modify group_vars/osdsdock.yml: enabled_backend: ceph # Change it according to the chosen backend. Supported backends include 'lvm', 'ceph', and 'cinder'. Configure group_vars/ceph/all.yml with an example below: ceph_origin: repository ceph_repository: community ceph_stable_release: luminous # Choose luminous as default version public_network: "192.168.3.0/24" # Run 'ip -4 address' to check the ip address cluster_network: "{{ public_network }}" monitor_interface: eth1 # Change to the network interface on the target machine devices: # For ceph devices, append ONE or MULTIPLE devices like the example below:   - '/dev/sda' # Ensure this device exists and available if ceph is chosen   #- '/dev/sdb'  # Ensure this device exists and available if ceph is chosen osd_scenario: collocated 4.3.4.4 Cinder If cinder is chosen as storage backend, modify group_vars/osdsdock.yml: enabled_backend: cinder # Change it according to the chosen backend. Supported backends include 'lvm', 'ceph', and 'cinder' # Use block-box install cinder_standalone if true, see details in: OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 8 use_cinder_standalone: true Configure the auth and pool options to access cinder in group_vars/cinder/cinder.yaml. Do not need to make additional configure changes if using cinder standalone. 
 Check if the hosts can be reached ansible all -m ping -i local.hosts 
 Run opensds-ansible playbook to start deploy ansible-playbook site.yml -i local.hosts  4.4 Test OpenSDS 
 Use OpenSDS CLI Tool Configure OpenSDS CLI tool: sudo cp /opt/opensds-linux-amd64/bin/osdsctl /usr/local/bin export OPENSDS_ENDPOINT=http://{your_real_host_ip}:50040 export OPENSDS_AUTH_STRATEGY=keystone source /opt/stack/devstack/openrc admin admin osdsctl pool list # Check if the pool resource is available Create a default profile: osdsctl profile create '{"name": "default", "description": "default policy"}' Create a volume: osdsctl volume create 1 --name=test-001 List all volumes: osdsctl volume list Delete the volume: OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 9 osdsctl volume delete <your_volume_id> 
 Test CSI Plugin After running the ansible deployment tool in “csi” mode, three CSI plugin pods can be found by kubectl get pods like below: o csi-provisioner-opensdsplugin o csi-attacher-opensdsplugin o csi-nodeplugin-opensdsplugin More design details about CSI can be found  from CSI Volume Plugins in Kubernetes Design Doc. To test the OpenSDS CSI plugin, create an example nginx application: cd /opt/opensds-k8s-linux-amd64/ && kubectl create -f csi/server/examples/kubernetes/nginx.yaml This will create an OpenSDS volume and mount the volume at /var/lib/www/html. Use the following command to inspect the nginx container to verify it.  docker exec -it <nginx container id> /bin/bash Clean up example nginx application by the following commands: kubectl delete -f csi/server/examples/kubernetes/nginx.yaml 
 OpenSDS Dashboard OpenSDS UI dashboard is available at http://{your_host_ip}:8088, please login the dashboard using the default admin credentials: admin/opensds@123. Create tenant, user, and profiles as admin. Logout of the dashboard as admin and login the dashboard again as a non-admin user to create volume, snapshot, expand volume, create volume from snapshot, create volume group. 4.5 Cleanup OpenSDS 
 Run opensds-ansible playbook to clean the environment ansible-playbook clean.yml -i local.hosts This should clean up hotpot as well as nbp (including the CSI plugin). OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 10 
 Run ceph-ansible playbook to clean ceph cluster if ceph is deployed cd /opt/ceph-ansible sudo ansible-playbook infrastructure-playbooks/purge-cluster.yml -i ceph.hosts In addition, clean up the logical partition on the physical block device used by ceph, using the fdisk tool. 
 Remove ceph-ansible source code (optional) sudo rm -rf /opt/ceph-ansible 4.6 Troubleshooting 
 Problem Starting CSI Plugin If the CSI plugin cannot be started, check if OpenSDS endpoint IP is configured.  vi csi/server/deploy/kubernetes/csi-configmap-opensdsplugin.yaml The IP (127.0.0.1) should be replaced with the opensds and identity actual endpoint IP.  kind: ConfigMap  apiVersion: v1      metadata:  name: csi-configmap-opensdsplugin      data:      opensdsendpoint: http://127.0.0.1:50040      osauthurl: http://127.0.0.1/identity Manually create OpenSDS CSI pods:  kubectl create -f csi/server/deploy/kubernetes After this, three pods can be found by kubectl get pods like below: o csi-provisioner-opensdsplugin o csi-attacher-opensdsplugin OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 11 o csi-nodeplugin-opensdsplugin To test the OpenSDS CSI plugin, create an example nginx application:  kubectl create -f csi/server/examples/kubernetes/nginx.yaml This will mount an OpenSDS volume into /var/lib/www/html. Use the following command to inspect the nginx container to verify it.  docker exec -it <nginx container id> /bin/bash Clean up example nginx application and opensds CSI pods by the following commands. kubectl delete -f csi/server/examples/kubernetes/nginx.yaml kubectl delete -f csi/server/deploy/kubernetes   5 Use Cases 5.1 Dashboard 
 Administrator configuration 5.1.1.1 Login Log into dashboard as admin.  Password is opensds@123 OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 12   5.1.1.2 Create tenant Go to the tab: Identity/Tenant, click “Create” button and input the necessary information, then submit the request. 
 5.1.1.3 Create user Go to the tab: Identity/User, click “Create” button and input the necessary information, then submit the request. Notes: On the page of creation, you can specify tenants that the user belongs to. 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 13  5.1.1.4 Create profile Go to the tab: Profile, click “Create” button and input the necessary information, then submit the request. 
 5.1.1.5 View resources Go to Resource tab, check Availability Zone, Region and Storage resources. 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 14   
 Tenant provision volume 5.1.2.1 Overview Log into dashboard as user(user1).  The home page shows statistics of volumes, snapshots and replications. 
 5.1.2.2 Create volume Go to the tab: Volume/Volume, click “Create” button and input the necessary information, such as name, size, profile, etc., then submit the request.  
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 15  5.1.2.3 Expand volume size Go to the tab: Volume/Volume, Select a volume and click “More/Expand” button to extend the volume size. 
 5.1.2.4 Create volume snapshot Go to the tab: Volume/Volume, Select a volume and click “Create Snapshot” button to create volume snapshot. 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 16   5.1.2.5 Create volume from snapshot Go to the tab: Volume/Volume, Select a volume and click volume name to enter the volume detail page. Select a snapshot and click “Create volume” button to create volume. 
  5.1.2.6 Create volume replication Go to the tab: Volume/Volume, Select a volume and click “Create Replication” button to create replication. Input the secondary volume name, availability zone, profile, then submit the request. Note: To configure a storage backend with replication capabilities, see the section 5.4 Array-based Replication using Dorado. 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 17  5.1.2.7 Disable/Enable/Failover volume replication Go to the tab: Volume/Volume, Select the protected volume and click volume name to enter the volume detail page. In replication tab page, click “Disable/Enable/Failover” button to control replication. 
 5.1.2.8 Create volume group Go to the tab: Volume/Volume Group, click “Create” button to create volume group. 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 18  5.1.2.9 Add volumes into group Go to the tab: Volume/Volume Group, select a volume group and go into the volume group detail page. Click “Add” button to add volumes into volume group. 
 5.1.2.10 Tenant isolation Log out and log in as user2 and verify that user2 can view volumes created by user1. 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 19   Log out and log in as administrator(admin)  and can manage the volumes of all tenants. 
  
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 20   5.2 Kubernetes Kubernetes cluster runs on baremetal or VM using OpenSDS to provision storage, using the following drivers: - Native LVM driver - Native Ceph driver - Native Dorado driver - Cinder driver with Cinder stand-alone (LVM by default) Refer to the Installation section to see how to use the OpenSDS CSI plugin to provision storage for Kubernetes.  5.3 OpenStack There are two ways for OpenSDS to integrate with OpenStack. - OpenSDS provisions storage through the southbound Cinder driver.  Cinder can be Cinder stand-alone or part of an OpenStack deployment.  See the Installation section on how to install OpenSDS to test with Cinder driver. - OpenSDS provisions storage in an OpenStack deployment through the Cinder compatible API.  It can be southbound native driver or Cinder driver below OpenSDS in this case.  
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 21  
 Use OpenSDS to Manage Cinder Drivers As the backend of OpenSDS, Cinder supports many kinds of storage. Therefore, OpenSDS can manage other storage with the help of cinder. But the installer of OpenSDS only supports Cinder with LVM. In order to manage storage supported in Cinder, you need to configure Cinder backend manually. This section will show you an example using ceph as the cinder backend.  5.3.1.1 Prepare A recommend deployment would be like the graph blow. We need three hosts for this testing, say Host A (IP: 192.168.0.99), Host B (IP: 192.168.0.20) and Host C (IP: 192.168.0.21). Note: the keystone in Host A is used for OpenSDS authentication and the keystone in Host B is use for OpenStack authentication, there is no any relationship between them. 
  5.3.1.2 Install OpenStack using devstack You can reference this document https://docs.openstack.org/devstack/latest/  5.3.1.3 Install ceph using ansible You can reference this document http://docs.ceph.com/ceph-ansible/master/ 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 22 Install Kubernetes using local cluster 
 5.3.1.4 Configuration 5.3.1.4.1 OpenSDS There are two configurations we need to config for OpenSDS:  • /etc/opensds/opensds.conf • /etc/opensds/driver/cinder.yaml An example would be like this: 1. /etc/opensds/opensds.conf 2. /etc/opends/driver/cinder.conf   #/etc/opensds/opensds.conf [keystone_authtoken] memcached_servers = localhost:11211 signing_dir = /var/cache/cinder cafile = /opt/stack/data/ca-bundle.pem project_domain_name = Default project_name = service user_domain_name = Default password = opensds@123 username = opensds auth_url = http://192.168.0.99/identity auth_type = password  [cinder] name = cinder description = Cinder Test driver_name = cinder config_path = /etc/opensds/driver/cinder.yaml  [osdslet] api_endpoint = 0.0.0.0:50040 graceful = True log_file = /var/log/opensds/osdslet.log socket_order = inc auth_strategy = keystone  [osdsdock] api_endpoint = 192.168.0.99:50050 OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 23 Then you can restart the OPenSDS manually.  5.3.1.4.2 Set ceph as backend of Cinder Operation in Node C: 1. Create pool in ceph  ceph osd pool create rbd 64 2. Copy the ceph.conf to the Host B which contains cinder-volume server. ssh {your-openstack-server} sudo tee /etc/ceph/ceph.conf </etc/ceph/ceph.conf 3. Set the cinder authentication in ceph. ceph auth get-or-create client.cinder mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rwx pool=vms, allow rwx pool=images' 4. Generate the authentication file and copy it to Host B.  ceph auth get-or-create client.cinder | ssh {your-volume-server} sudo tee /etc/ceph/ceph.client.cinder.keyring ssh {your-cinder-volume-server} sudo chown stack:stack /etc/ceph/ceph.client.cinder.keyring # /etc/opensds/driver/opensds.conf authOptions:   endpoint: "http://192.168.0.20/identity"   domainName: "Default"   username: "admin"   password: "admin"   tenantName: "admin" pool:   ecs-351b@ceph#ceph:     storageType: block     availabilityZone: default     extras:       dataStorage:         provisioningPolicy: Thin         isSpaceEfficient: false       ioConnectivity:         accessProtocol: iscsi         maxIOPS: 7000000         maxBWS: 600       advanced:         diskType: SSD         latency: 3ms OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 24  Operation in Node B: 1. Intall python-rbd and ceph-common which is needed for cinder ceph backend. sudo apt-get install python-rbd sudo apt-get install ceph-common  2. Modified the cinder configuration file /etc/cinder/cinder.conf in Node B: 
3. Restart cinder-volume server sudo systemctl restart devstack@c-vol.service 4. delete default volume type cinder type-delete 1fd30cdc-63d0-4b1d-9e88-3b7b58f05d73  5.3.1.5 Testing Create volume [DEFAULT] ... default_volume_type = ceph enabled_backends = ceph ... [ceph] volume_driver = cinder.volume.drivers.rbd.RBDDriver volume_backend_name = rbd rbd_pool = volumes rbd_ceph_conf = /etc/ceph/ceph.conf rbd_flatten_volume_from_snapshot = false rbd_max_clone_depth = 5 rbd_store_chunk_size = 4 rados_connect_timeout = -1 glance_api_version = 2 OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 25      
 OpenSDS with Cinder Compatible API Cinder Compatible API adapter is not built in as part of the ansible deployment tool.  Follow the following instruction to install it. 5.3.2.1 Installation 1. The Cinder Compatible API only supports cinder's current Api(v3). You can use devstack to install cinder when testing, but in order to use cinder's current Api(v3), branch for devstack must be stable/queens. 2. When devstack is installed, kill all cinder processes. 3. Run the "source /opt/stack/devstack/openrc admin admin" command to execute the openstack's cli command. 4. Run the "openstack endpoint list" command to view the cinder endpoint. 5. Run the command "export CINDER_ENDPOINT=http://10.10.10.10:8776/v3". The actual value of CINDER_ENDPOINT is determined by the previous step. 6. Run the command export OPENSDS_ENDPOINT=http://127.0.0.1:50040. 7. Download the opensds source (https://github.com/opensds/opensds.git) and install opensds. 8. Run the command "go build -o ./build/out/bin/cindercompatibleapi github.com/opensds/opensds/contrib/cindercompatibleapi". 9. Execute the command "./build/out/bin/cindercompatibleapi". 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 26 10. Execute some cinder cli commands to see if the result is correct. For example, if you execute the command "cinder type-list", the results will show the profile of opensds.  5.3.2.2 Volume Types 5.3.2.2.1 List all volume types (default policy) cinder type-list   5.3.2.2.2 Delete a volume type cinder type-delete 
   5.3.2.2.3 List all volume types(0) cinder type-list    
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 27 5.3.2.2.4 Create a volume type cinder type-create type00 --description test_type_00   5.3.2.2.5 Show volume type detail  cinder type-show Id 
  5.3.2.2.6 Create a volume type（2nd） cinder type-create type01 --description test_type_01   5.3.2.2.7 List all volume types (2) cinder type-list 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 28   5.3.2.2.8 Update an encryption type cinder type-update 7abff35e-0cbb-4c48-8bab-4fe7c3286792 --name type0 --description test_type_0 --is-public true   If is-public is not set, false is the default which is not supported by opensds：   5.3.2.2.9 Lists current volume types and extra specs.  cinder extra-specs-list 
   
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 29 5.3.2.2.10 Create or update extra specs for volume type cinder type-key 7abff35e-0cbb-4c48-8bab-4fe7c3286792 set key1=value1 
  5.3.2.2.11 Delete extra specification for volume type cinder type-key 7abff35e-0cbb-4c48-8bab-4fe7c3286792 unset key1 
    5.3.2.3 Volumes 5.3.2.3.1 List accessible volumes with details (0) cinder list  
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 30 5.3.2.3.2 Create a volume (1st) cinder create 1 --name volume00 
  5.3.2.3.3 List accessible volumes with details (1) cinder list   5.3.2.3.4 Show a volume’s details cinder show <volume uuid>  5.3.2.3.5 Delete a volume cinder delete <volume uuid>  5.3.2.4 Snapshots  
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 31 5.3.2.4.1 Create a snapshot cinder snapshot-create <volume uuid> 5.3.2.4.2 List snapshots and details cinder snapshot-list  5.3.2.4.3 Show a snapshot’s details  cinder snapshot-show <snapshot uuid> 5.3.2.4.4 Delete a snapshot  cinder snapshot-delete <snapshot uuid>  5.3.2.5 Attachments 5.3.2.5.1 Create attachment cinder attachment-create cinder results:  Cinder compatible API results: 
  5.3.2.5.2 Show attachment Cinder attachment-show 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 32 cinder results： 
 Cinder compatible API results： 
 5.3.2.5.3 List attachment cinder attachment-list cinder results：  Cinder compatible API results：  5.3.2.5.4 Update attachment cinder attachment-update Cinder compatible API results： 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 33  5.3.2.5.5 Delete attachment cinder attachment-delete cinder results： 
 Cinder compatible API results： 
 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 34   5.4 Array-based Replication using Dorado 
 Without Kubernetes Test using Dashboard and CLI 5.4.1.1 Configuration In array-based replication scenario, we need to depoly opensds in two nodes. Node A includes dashboard, keystone, osdslet, osdsdock(provisioner) and etcd. For simplifying the testing scenario, node B includes just only includes osdsdock.  
NOTE: MGT IP means management ip, ETH IP is used for iscsi.  There are two configurations we need to config:  • /etc/opensds/opensds.conf • /etc/opensds/driver/dorado.yaml  An example in Node A (192.168.56.100) would be like this:  
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 35 1. /etc/opensds/opensds.conf  
 [keystone_authtoken] memcached_servers = 8.46.186.191:11211 signing_dir = /var/cache/opensds cafile = /opt/stack/data/ca-bundle.pem auth_uri = http://8.46.186.191/identity project_domain_name = Default project_name = service user_domain_name = Default password = opensds@123 username = opensds auth_url = http://8.46.186.191/identity auth_type = password [osdslet] api_endpoint = 0.0.0.0:50040 graceful = True log_file = /var/log/opensds/osdslet.log socket_order = inc auth_strategy = keystone  [osdsdock] api_endpoint = 192.168.56.100:50050 log_file = /var/log/opensds/osdsdock.log # Specify which backends should be enabled, sample,ceph,cinder,lvm and so on. enabled_backends = huawei_dorado  [database] endpoint = 192.168.56.100:62379,192.168.56.100:62380 driver = etcd  [huawei_dorado] name = huawei_dorado description = Huawei OceanStor Dorado driver_name = huawei_dorado config_path = /etc/opensds/driver/dorado.yaml support_replication = true  OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 36 2. /etc/opensds/driver/dorado.yaml  
 Then you can start opensds servers. Start etcd server: etcd --advertise-client-urls http://192.168.56.100:2379 --listen-client-urls http://192.168.56.100:2379 --listen-peer-urls http://127.0.0.1:2380  Start up osdslet: osdslet --logtostderr -v 8  Start up osdsdock(provisioner): osdsdock --logtostderr -v 8 An example in Node B(192.168.56.101) would be like this: authOptions:   endpoints: "https://192.168.56.200:8088/deviceManager/rest"   username: "opensds"   password: "opensds@123"   insecure: true replication:   remoteAuthOptions:     endpoints: "https://192.168.56.201:8088/deviceManager/rest"     username: "opensds"     password: "opensds@123"     insecure: true pool:   StoragePool001:     diskType: SSD     AZ: default     accessProtocol: iscsi     thinProvisioned: true     compressed: true     advanced:       deduped: true targetIp: 192.168.57.200 OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 37 1. /etc/opensds/opensds.conf  
 2. /etc/opensds/driver/dorado.yaml  
 In node B you just should only start up osdsdock(provisioner). Start up osdsdock(provisioner): osdsdock --logtostderr -v 8 authOptions:   endpoints: "https://192.168.56.201:8088/deviceManager/rest"   username: "opensds"   password: "opensds@123"   insecure: true replication:   remoteAuthOptions:     endpoints: "https://192.168.56.200/deviceManager/rest"     username: "opensds"     password: "opensds@123"     insecure: true  pool:   StoragePool_210038bc0177ae4f:     diskType: SSD     availabilityZone: secondary     accessProtocol: iscsi     thinProvisioned: true     compressed: true     advanced:       deduped: true targetIp: 192.168.57.201 [osdsdock] api_endpoint = 192.168.56.101:50050 log_file = /var/log/opensds/osdsdock.log # Specify which backends should be enabled, sample,ceph,cinder,lvm and so on. enabled_backends = huawei_dorado  [database] endpoint = 192.168.56.100:62379,192.168.56.100:62380 driver = etcd  [huawei_dorado] name = huawei_dorado_remote description = Huawei OceanStor Dorado Remote array driver_name = huawei_dorado config_path = /etc/opensds/driver/dorado.yaml support_replication = true  OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 38  5.4.1.2 Testing Here is the usage of replication CLI. 1. Create replication. Usage:   osdsctl replication create <primary volume id> <secondary volume id> [flags]  Flags:   -d, --description string             the description of created replication   -h, --help                           help for create   -n, --name string                    the name of created replication   -p, --primary_driver_data string     the primary replication driver data of created replication   -m, --replication_model string       the replication mode of created replication, value can be sync/async   -t, --replication_period int         the replication period of created replication, the value must greater than 0 (default 120)   -s, --secondary_driver_data string   the secondary replication driver data of created replication   2. List replication. Usage:   osdsctl replication list [flags]  Flags:   -h, --help   help for list  Global Flags:       --debug   shows debugging output.  OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 39  3. Show a replication Usage:   osdsctl replication show <replication id> [flags]  Flags:   -h, --help   help for show  Global Flags:       --debug   shows debugging output.   4. Enable replication. Usage:   osdsctl replication enable <replication id> [flags]  Flags:   -h, --help   help for enable  Global Flags:       --debug   shows debugging output.   5.disable replication Usage:   osdsctl replication disable <replication id> [flags]  Flags: OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 40   -h, --help   help for disable  Global Flags:       --debug   shows debugging output.  6. Failover replication Usage:   osdsctl replication failover <replication id> [flags]  Flags:   -a, --allow_attached_volume         whether allow attached volume when failing over replication   -h, --help                          help for failover   -s, --secondary_backend_id string   the secondary backend id of failoverr replication  Global Flags:       --debug   shows debugging output.   7. delete replication Usage:   osdsctl replication delete <replication id> [flags]  Flags:   -h, --help   help for delete  Global Flags:       --debug   shows debugging output. OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 41 
 With Kubernetes 5.4.2.1 Configuration OpenSDS provide storage to kubernetes using CSI plugin. OpenSDS replication feature also works in kubernetes, when an application pod crashes and the replication status is failedOver, the OpenSDS CSI plugin will switch to the secondary volume automatically. This is totally invisible for users. A simplest testing deployment would be like blow. 
NOTE: MGT IP means management ip, ETH IP is used for iscsi. There are two configurations we need to config:  • /etc/opensds/opensds.conf • /etc/opensds/driver/dorado.yaml  An example in Node A (192.168.56.100) would be like this:  1. /etc/opensds/opensds.conf  
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 42  [keystone_authtoken] memcached_servers = 8.46.186.191:11211 signing_dir = /var/cache/opensds cafile = /opt/stack/data/ca-bundle.pem auth_uri = http://8.46.186.191/identity project_domain_name = Default project_name = service user_domain_name = Default password = opensds@123 username = opensds auth_url = http://8.46.186.191/identity auth_type = password [osdslet] api_endpoint = 0.0.0.0:50040 graceful = True log_file = /var/log/opensds/osdslet.log socket_order = inc auth_strategy = keystone  [osdsdock] api_endpoint = 192.168.56.100:50050 log_file = /var/log/opensds/osdsdock.log # Specify which backends should be enabled, sample,ceph,cinder,lvm and so on. enabled_backends = huawei_dorado  [database] endpoint = 192.168.56.100:62379,192.168.56.100:62380 driver = etcd  [huawei_dorado] name = huawei_dorado description = Huawei OceanStor Dorado driver_name = huawei_dorado config_path = /etc/opensds/driver/dorado.yaml support_replication = true  OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 43 2. /etc/opensds/driver/dorado.yaml  
 Then you can start opensds servers. Start etcd server: etcd --advertise-client-urls http://192.168.56.100:2379 --listen-client-urls http://192.168.56.100:2379 --listen-peer-urls http://127.0.0.1:2380  Start up osdslet: osdslet --logtostderr -v 8  Start up osdsdock(provisioner): osdsdock --logtostderr -v 8  An example in Node B(192.168.56.101) would be like this: authOptions:   endpoints: "https://192.168.56.200:8088/deviceManager/rest"   username: "opensds"   password: "opensds@123"   insecure: true replication:   remoteAuthOptions:     endpoints: "https://192.168.56.201:8088/deviceManager/rest"     username: "opensds"     password: "opensds@123"     insecure: true pool:   StoragePool001:     diskType: SSD     AZ: default     accessProtocol: iscsi     thinProvisioned: true     compressed: true     advanced:       deduped: true targetIp: 192.168.57.200 OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 44 1. /etc/opensds/opensds.conf  
 2. /etc/opensds/driver/dorado.yaml  
 In node B you just should only start up osdsdock(provisioner). Start up osdsdock(provisioner): osdsdock --logtostderr -v 8 authOptions:   endpoints: "https://192.168.56.201:8088/deviceManager/rest"   username: "opensds"   password: "opensds@123"   insecure: true replication:   remoteAuthOptions:     endpoints: "https://192.168.56.200/deviceManager/rest"     username: "opensds"     password: "opensds@123"     insecure: true  pool:   StoragePool_210038bc0177ae4f:     diskType: SSD     availabilityZone: secondary     accessProtocol: iscsi     thinProvisioned: true     compressed: true     advanced:       deduped: true targetIp: 192.168.57.201 [osdsdock] api_endpoint = 192.168.56.101:50050 log_file = /var/log/opensds/osdsdock.log # Specify which backends should be enabled, sample,ceph,cinder,lvm and so on. enabled_backends = huawei_dorado  [database] endpoint = 192.168.56.100:62379,192.168.56.100:62380 driver = etcd  [huawei_dorado] name = huawei_dorado_remote description = Huawei OceanStor Dorado Remote array driver_name = huawei_dorado config_path = /etc/opensds/driver/dorado.yaml support_replication = true  OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 45  Startup kubernetes local cluster in Node A.If you get information like blow, your kubernetes local_cluster startup successfully. 
5.4.2.2 Testing steps 1. Run command kubectl get pod to confirm the OpenSDS CSI plugin server is up. There will be 3 pods.  2. Add the configuration item enableReplication: "true" at parameters section to enable the replication feature. 
# sc_pvc.yaml # This YAML file contains StorageClass and PVC # which are necessary to run nginx with csi opensds driver.  apiVersion: storage.k8s.io/v1 kind: StorageClass metadata:   name: csi-sc-opensdsplugin provisioner: csi-opensdsplugin parameters:   enableReplication: "true" --- apiVersion: v1 kind: PersistentVolumeClaim metadata:   name: csi-pvc-opensdsplugin spec:   accessModes:   - ReadWriteOnce   resources:     requests:       storage: 1Gi   storageClassName: csi-sc-opensdsplugin  OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 46 3. Create StorageClss and PVC.  You will find two volumes and a replication in OpenSDS.  4. Start up the nginx application pod. 
 5. Set the replication failed over. 
# nginx.yaml # This YAML file contains nginx  apiVersion: v1 kind: Pod metadata:   name: nginx spec:   containers:   - image: nginx     imagePullPolicy: IfNotPresent     name: nginx     ports:     - containerPort: 80       protocol: TCP     volumeMounts:       - mountPath: /var/lib/www/html         name: csi-data-opensdsplugin   volumes:   - name: csi-data-opensdsplugin     persistentVolumeClaim:       claimName: csi-pvc-opensdsplugin       readOnly: false  OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 47  6. Restart the nginx, you will find the storage which is used by nginx is switch to secondary. 
   5.5 Host-based Replication using DRBD 
 Prepare We need to prepare two hosts for this test, say HostA(IP: 192.168.0.131) and HostB(IP: 192.168.0.66).  And before we start, please make sure the OpenSDS is already installed on both hosts. And copy etcdctl, etcd, osdslet, osdsdock, osdsctl to /opt/opensds/bin/. 
 Install DRBD Install DRBD as the following steps on both hosts: • sudo add-apt-repository ppa:linbit/linbit-drbd9-stack • sudo apt-get update • sudoapt-get install drbd-utils python-drbdmanage drbd-dkms 
 Configuration Before do configuration, please stop opensds service first. That is find out the process id of etcd, osdslet and osdsdock, and kill them. Modify /etc/opensds/opensds.conf: • Add host_based_replication_driver for the osdsdock part on both hosts 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 48 • Change endpoint of database on hostB to the same as HostA. Here is the example:  [lvm] name = lvm description = LVM Test driver_name = lvm config_path = /etc/opensds/driver/lvm.yaml  [osdslet] api_endpoint = 0.0.0.0:50040 graceful = True log_file = /var/log/opensds/osdslet.log socket_order = inc auth_strategy = noauth  [osdsdock] api_endpoint = 192.168.0.131:50050 log_file = /var/log/opensds/osdsdock.log # Specify which backends should be enabled, sample,ceph,cinder,lvm and so on. enabled_backends = lvm host_based_replication_driver = drbd  [database] endpoint = 192.168.0.131:62379,192.168.0.131:62380 driver = etcd Add a new configuration file /etc/opensds/attacher.conf on both hosts, here is an example: [osdsdock] api_endpoint = 192.168.0.131:50051 log_file = /var/log/opensds/osdsdock.log bind_ip = 192.168.0.131 dock_type = attacher  [database] endpoint = 192.168.0.131:62379,192.168.0.131:62380 driver = etcd Note: both hosts have the same endpoint of database, but api_endpoint and bind_ip of osdsdock should be the host ip respectively. Add a new configuration file /etc/opensds/drbd.yaml on both hosts, the content is: #Minumum and Maximum TCP/IP ports used for DRBD replication PortMin: 7000 PortMax: 8000  #Exactly two hosts between resources are replicated. #Never ever change the Node-ID associated with a Host(name) Hosts:    - Hostname: ecs-37cc      IP: 192.168.0.66      Node-ID: 1     - Hostname: ecs-32bc      IP: 192.168.0.131      Node-ID: 0 Note: Hostname and IP should be the real value of each hosts. Modify /etc/opensds/driver/lvm.yaml on hostB, change availabilityZone to a new value. Here is an example: tgtBindIp: 192.168.0.66 OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 49 tgtConfDir: /etc/tgt/conf.d pool:   opensds-volumes-default:     diskType: NL-SAS     availabilityZone: secondary     extras:       dataStorage:         provisioningPolicy: Thin         isSpaceEfficient: false       ioConnectivity:         accessProtocol: iscsi         maxIOPS: 7000000         maxBWS: 600       advanced:         diskType: SSD         latency: 5ms  
 Create Replication Start services on HostA: • cd /opt/opensds/bin • ./etcd --advertise-client-urls http://192.168.0.131:62379 --listen-client-urls http://192.168.0.131:62379 --listen-peer-urls http://192.168.0.131:62380 --data-dir /opt/opensds/etcd/data  >> /var/log/opensds/etcd.log 2>&1 & • ./osdslet & • ./osdsdock & • ./osdsdock --config-file /etc/opensds/attacher.conf & Start services on HostB: • ./osdslet & • ./osdsdock & • ./osdsdock --config-file /etc/opensds/attacher.conf & Create volumes (run them on HostA or hostB): • ./osdsctl volume create 1 -n primary • ./osdsctl volume create 1 -n secondary -a secondary  Create replication: • ./osdsctl replication create e0b1c9e3-0c88-4601-b0e7-c09448a89e5c    3ea2e681-4884-4d84-a2e3-d5e3318763b2 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 50  
 Check result See the block device.  Create some data on HostA. • mkfs.ext4 /dev/drbd1 • mount /dev/drbd1   ./reptest/ • touch test • dd if=/dev/zero of=./2 bs=1M count=500 • touch test • …… Check the synchronous status on both hosts.        Check if the data is updated on HostB. • umount on HostA • mount on HostB • Check data on HostB, and you can see the data is updated. 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 51    6 OpenSDS CLI Guide  6.1 List Docks Use the following command to display the docks information. osdsctl dock list Sample results are as follows:   Display specific results by filter parameters. Filter parameters can be displayed by the following command. osdsctl dock list -h Results are as follows: 
  Example: 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 52  
    6.2 List Pools Use the following command to display the pools information. osdsctl pool list Sample results are as follows:   Display specific results by filter parameters. Filter parameters can be displayed by the following command. osdsctl pool list -h Results are as follows: 
  Example: 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 53    6.3 Create/Delete Profile Use the following command to create profile. osdsctl profile create * Example:  
  Use the following command to delete profile. osdsctl profile delete * Example:   6.4 Create/Delete/Get/List Volume(s) Use the following command to create volume. osdsctl volume create 3 Example: 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 54  Use the following command to display the volume details. osdsctl volume show * Example: 
 Use the following command to delete the volume. osdsctl volume delete * Example:   Display specific results by filter parameters. Filter parameters can be displayed by the following command. osdsctl volume list -h Results are as follows: 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 55  Example:   6.5 Create/Delete/Get/List Snapshot(s) Use the following command to create snapshot. osdsctl volume snapshot create * Example: 
 Use the following command to display snapshot details. osdsctl volume snapshot show * Example: 
 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 56 Use the following command to delete snapshot. osdsctl volume snapshot delete * Example:   Display specific results by filter parameters. Filter parameters can be displayed by the following command. osdsctl volume snapshot list -h Results are as follows: 
  6.6 Create Volume from Snapshot Use the following command to create volume from snapshot. osdsctl volume create 1 –s * Example: 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 57   6.7 Expand Volume Use the following command to expand volume size. osdsctl volume extend * * Example: 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 58  6.8 Create/Update/Delete/Get/List Volume Groups Use the following command to create volume group. osdsctl volume group create –-profiles * Example: 
 Use the following command to update volume group. osdsctl volume group update groupId -a “volumeId1, volumeId2” *  Use the following command to show volume group. osdsctl volume group show * 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 59 Example:  
  Display specific results by filter parameters. Filter parameters can be displayed by the following command. osdsctl volume group list -h Results are as follows: 
  Example:   Use the following command to update volume group. osdsctl volume group update * Example: 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 60  Use the following command to delete volume group.  osdsctl volume group delete * Example:     6.9 Replication Here is the usage of replication CLI. 1. Create replication. Usage:   osdsctl replication create <primary volume id> <secondary volume id> [flags]  Flags:   -d, --description string             the description of created replication   -h, --help                           help for create   -n, --name string                    the name of created replication   -p, --primary_driver_data string     the primary replication driver data of created replication   -m, --replication_model string       the replication mode of created replication, value can be sync/async 
OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 61   -t, --replication_period int         the replication period of created replication, the value must greater than 0 (default 120)   -s, --secondary_driver_data string   the secondary replication driver data of created replication   2. List replication. Usage:   osdsctl replication list [flags]  Flags:   -h, --help   help for list  Global Flags:       --debug   shows debugging output.   3. Show a replication Usage:   osdsctl replication show <replication id> [flags]  Flags:   -h, --help   help for show  Global Flags:       --debug   shows debugging output.   4. Enable replication. OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 62 Usage:   osdsctl replication enable <replication id> [flags]  Flags:   -h, --help   help for enable  Global Flags:       --debug   shows debugging output.   5.disable replication Usage:   osdsctl replication disable <replication id> [flags]  Flags:   -h, --help   help for disable  Global Flags:       --debug   shows debugging output.  6. Failover replication Usage:   osdsctl replication failover <replication id> [flags]  Flags:   -a, --allow_attached_volume         whether allow attached volume when failing over replication   -h, --help                          help for failover   -s, --secondary_backend_id string   the secondary backend id of failoverr replication OpenSDS  DATE: 07/09/18 
OpenSDS DRAFT 63  Global Flags:       --debug   shows debugging output.   7. delete replication Usage:   osdsctl replication delete <replication id> [flags]  Flags:   -h, --help   help for delete  Global Flags:       --debug   shows debugging output.