Package: cinderlib-csi
Cinderlib CSI driver====================.. image:: https://img.shields.io/pypi/v/cinderlib_csi.svg:target: https://pypi.python.org/pypi/cinderlib_csi.. image:: https://img.shields.io/pypi/pyversions/cinderlib_csi.svg:target: https://pypi.python.org/pypi/cinderlib_csi.. image:: https://pyup.io/repos/github/akrog/cinderlib_csi/shield.svg:target: https://pyup.io/repos/github/akrog/cinderlib_csi/:alt: Updates.. image:: https://img.shields.io/:license-apache-blue.svg:target: http://www.apache.org/licenses/LICENSE-2.0CSI Python driver that leverages all Cinder drivers to provide block volumeswithout needing to run any additional service, such as RabbitMQ, MariaDB,Cinder-API, Cinder-Scheduler, or Cinder-Volume.Current code is is a **Proof of Concept** only compatible with CinderOSP-12/Pike release.* Free software: Apache Software License 2.0* Documentation: PendingFeatures--------This CSI driver is up to date with latest CSI specs including the `newsnapshots feature<https://github.com/container-storage-interface/spec/pull/224>`_ recentlyintroduced.Currently supported features are:- Create block volume- Creating snapshots- Creating a block volume from a snapshot- Delete block volume- Deleting snapshots- Listing volumes with pagination- Listing snapshots with pagination- Attaching volumes- Detaching volumes- Reporting storage capacity- Probing the node- Retrieving the plugin infoRuntime Dependencies--------------------This driver requires that Cinder v11.0 (OSP-12/Pike) is already installed inthe system, how this is accomplished is left to the installer, as there aremultiple ways this can be accomplished:- From OSP repositories- From RDO repositories- From github- From other repositoriesAny other basic requirement is already handled by `cinderlib-csi` wheninstalling from PyPi.Besides the basic dependencies there are also some drivers that have additionalrequirements that must be met for proper operation of the driver and/orattachment/detachment operations, just like in Cinder.Some of these Python dependencies for the Controller servicer are:- DRBD: dbus and drbdmanage- HPE 3PAR: python-3parclient- Kaminario: krest- Pure: purestorage- Dell EMC VMAX, IBM DS8K: pyOpenSSL- HPE Lefthad: python-lefthandclient- Fujitsu Eternus DX: pywbem- IBM XIV: pyxcli- RBD: rados and rbd- Dell EMC VNX: storops- Violin: vmemclient- INFINIDAT: infinisdk, capacity, infy.dtypes.wwn, infi.dtypes.iqnOther backends may also require additional packages, for example LVM onCentOS/RHEL requires the `targetcli` package, so please check with yourhardware vendor.Besides the Controller requirements there are usually requirements for theNode servicer needed to handle the attaching and detaching of volumes to thenode based on the connection used to access the storage.  For example:- iSCSI: iscsi-initiator-tools and device-mapper-multipath- RBD/Ceph: ceph-common packageInstallation------------First we need to install the Cinder Python package, for example to install fromRDO on CentOS:.. code-block:: shell$ sudo yum install -y centos-release-openstack-pike$ sudo yum install -y openstack-cinder python-pipThen we just need to install the `cinderlib-csi` package:.. code-block:: shell$ sudo pip install cinderlib-csiNow we should install any additional package required by our backend.For iSCSI backends we'll want to install:.. code-block:: shell$ sudo yum install iscsi-initiator-utils$ sudo yum install device-mapper-multipath$ sudo mpathconf --enable --with_multipathd y --user_friendly_names n --find_multipaths yFor RBD we'll also need a specific package:.. code-block:: shell$ sudo yum install ceph-commonConfiguration-------------The CSI driver is configured via environmental variables, any value thatdoesn't have a default is a required value.+----------------------------+------------+---------------------------------------------------------------+----------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+| Name                       | Role       | Description                                                   | Default                                                                                                  | Example                                                                                                                                                                                                                                 |+============================+============+===============================================================+==========================================================================================================+=========================================================================================================================================================================================================================================+| `CSI_ENDPOINT`             | all        | IP and port to bind the service                               | [::]:50051                                                                                               | 192.168.1.22:50050                                                                                                                                                                                                                      |+----------------------------+------------+---------------------------------------------------------------+----------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+| `CSI_MODE`                 | controller | Role the service should perform: controller, node, all        | all                                                                                                      | controller                                                                                                                                                                                                                              |+----------------------------+------------+---------------------------------------------------------------+----------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+| `X_CSI_STORAGE_NW_IP`      | node       | IP address in the Node used to connect to the storage         | IP resolved from Node's fqdn                                                                             | 192.168.1.22                                                                                                                                                                                                                            |+----------------------------+------------+---------------------------------------------------------------+----------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+| `X_CSI_NODE_ID`            | node       | ID used by this node to identify itself to the controller     | Node's fqdn                                                                                              | csi_test_node                                                                                                                                                                                                                           |+----------------------------+------------+---------------------------------------------------------------+----------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+| `X_CSI_PERSISTENCE_CONFIG` | all        | Configuration of the `cinderlib` metadata persistence plugin. | {'storage': 'db', 'connection': 'sqlite:///db.sqlite'}                                                   | {'storage': 'db', 'connection': 'mysql+pymysql://root:stackdb@192.168.1.1/cinder?charset=utf8'}                                                                                                                                         |+----------------------------+------------+---------------------------------------------------------------+----------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+| `X_CSI_CINDERLIB_CONFIG`   | controller | Global `cinderlib` configuration                              | {'project_id': 'com.redhat.cinderlib-csi', 'user_id': 'com.redhat.cinderlib-csi', 'root_helper': 'sudo'} | {"project_id":"com.redhat.cinderlib-csi","user_id":"com.redhat.cinderlib-csi","root_helper":"sudo"}                                                                                                                                     |+----------------------------+------------+---------------------------------------------------------------+----------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+| `X_CSI_BACKEND_CONFIG`     | controller | Driver configuration                                          |                                                                                                          | {"volume_backend_name": "rbd", "volume_driver": "cinder.volume.drivers.rbd.RBDDriver", "rbd_user": "cinder", "rbd_pool": "volumes", "rbd_ceph_conf": "/etc/ceph/ceph.conf", "rbd_keyring_conf": "/etc/ceph/ceph.client.cinder.keyring"} |+----------------------------+------------+---------------------------------------------------------------+----------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+The only role that has been tested at the moment is the default one, whereController and Node servicer are executed in the same service (`CSI_MODE=all`),and other modes are expected to have issues at the moment.Staring the plugin------------------Once we have installed `cinderlib-csi` and required dependencies (for thebackend and for the connection type) we just have to run the `cinderlib-csi`service with a user that can do passwordless sudo... code-block:: shell$ cinderlib-csiTesting the plugin------------------There are several examples of running the CSI cinderlib driver in the`examples` directory both for a baremetal deployment and a containerizedversion of the driver.In all cases we have to run the plugin first before we can test it, and forthat we have to check the configuration provided as a test before starting theplugin.  By default all examples run the service on port 50051.Baremetal~~~~~~~~~For example to test with the LVM driver on our development environment we canjust run the following commands from the root of the `cinderlib-csi` project:.. code-block:: shell$ cd tmp$ sudo dd if=/dev/zero of=cinder-volumes bs=1048576 seek=22527 count=1$ lodevice=`sudo losetup --show -f ./cinder-volumes`$ sudo pvcreate $lodevice$ sudo vgcreate cinder-volumes $lodevice$ sudo vgscan --cache$ cd ../examples/baremetal$ ./run.sh lvmpy27 develop-inst-nodeps: /home/geguileo/code/reuse-cinder-drivers/cinderlib-csipy27 installed: ...___ summary ___py27: skipped testscongratulations :)Starting cinderlib CSI v0.0.1 (cinderlib: 0.1.0, cinder: 11.1.1.dev41)Running backend LVMVolumeDriver v3.0.0Now serving on [::]:50051...There is also an example of testing a Ceph cluster using a user called "cinder"and the "volumes" pool.  For the Ceph/RBD backend, due to a limitation inCinder, we need to have both the credentials and the configuration in`/etc/ceph` for it to work... code-block:: shell$ cd examples/baremetal$ ./run.sh rbdStarting cinderlib CSI v0.0.1 (cinderlib: 0.1.0, cinder: 11.1.0)Running backend RBDDriver v1.2.0Now serving on [::]:50051...There is also an XtremIO example that only requires the iSCSI connectionpackages.Containerized~~~~~~~~~~~~~There is a sample `Dockerfile` included in the project that has been used tocreate the `akrog/cinderlib-csi` container available in the docker hub.There are two bash scripts, one for each example, that will run the CSI driveron a container, be aware that the container needs to run as privileged to mountthe volumes.For the RBD example we need to copy our "ceph.conf" and"ceph.client.cinder.keyring" files, assuming we are using the "cinder" userinto the example/docker directory replacing the existing ones... code-block:: shell$ cd examples/docker$ ./rbd.shStarting cinderlib CSI v0.0.1 (cinderlib: 0.1.0, cinder: 11.1.0)Running backend RBDDriver v1.2.0Now serving on [::]:50051...CSC~~~Now that we have the service running we can use the `CSC tool<https://github.com/rexray/gocsi/tree/master/csc>`_ to runcommands simulating the Container Orchestration system.Due to the recent changes in the CSI spec not all commands are available yet,so you won't be able to test the snapshot commands.Checking the plugin info:.. code-block:: shell$ csc identity plugin-info -e tcp://127.0.0.1:50051"com.redhat.cinderlib-csi"      "0.0.1" "cinder-driver"="RBDDriver"     "cinder-driver-supported"="True"        "cinder-driver-version"="1.2.0" "cinder-version"="11.1.0"       "cinderlib-version"="0.1.0"     "persistence"="DBPersistence"Checking the node id:.. code-block:: shell$ csc node get-id -e tcp://127.0.0.1:50051localhost.localdomain$ hostname -flocalhost.localdomainChecking the current backend capacity:.. code-block:: shell$ csc controller get-capacity -e tcp://127.0.0.1:5005124202140712Creating a volume:.. code-block:: shell$ csc controller create-volume --cap SINGLE_NODE_WRITER,block --req-bytes 2147483648 disk -e tcp://127.0.0.1:50051"5ee5fd7c-45cd-44cf-af7b-06081f680f2c"  2147483648Listing volumes:.. code-block:: shell$ csc controller list-volumes -e tcp://127.0.0.1:50051"5ee5fd7c-45cd-44cf-af7b-06081f680f2c"  2147483648Store the volume id for all the following calls:.. code-block:: shell$ vol_id=`csc controller list-volumes -e tcp://127.0.0.1:50051|awk '{ print gensub("\"","","g",$1)}'Attaching the volume to `tmp/mnt/publish` on baremetal:.. code-block:: shell$ touch ../../tmp/mnt/{staging,publish}$ csc controller publish --cap SINGLE_NODE_WRITER,block --node-id `hostname -f` $vol_id -e tcp://127.0.0.1:50051"5ee5fd7c-45cd-44cf-af7b-06081f680f2c"  "connection_info"="{\"connector\": {\"initiator\": \"iqn.1994-05.com.redhat:aa532823bac9\", \"ip\": \"127.0.0.1\", \"platform\": \"x86_64\", \"host\": \"localhost.localdomain\", \"do_local_attach\": false, \"os_type\": \"linux2\", \"multipath\": false}, \"conn\": {\"driver_volume_type\": \"rbd\", \"data\": {\"secret_uuid\": null, \"volume_id\": \"5ee5fd7c-45cd-44cf-af7b-06081f680f2c\", \"auth_username\": \"cinder\", \"secret_type\": \"ceph\", \"name\": \"volumes/volume-5ee5fd7c-45cd-44cf-af7b-06081f680f2c\", \"discard\": true, \"keyring\": \"[client.cinder]\\n\\tkey = AQCQPetaof03IxAAoHZJD6kGxiMQfLdn3QzdlQ==\\n\", \"cluster_name\": \"ceph\", \"hosts\": [\"192.168.1.22\"], \"auth_enabled\": true, \"ports\": [\"6789\"]}}}"$ csc node stage --pub-info connection_info="irrelevant" --cap SINGLE_NODE_WRITER,block --staging-target-path `realpath ../../tmp/mnt/staging` $vol_id -e tcp://127.0.0.1:500515ee5fd7c-45cd-44cf-af7b-06081f680f2c$ csc node publish --cap SINGLE_NODE_WRITER,block --pub-info connection_info="irrelevant" --staging-target-path `realpath ../../tmp/mnt/staging` --target-path `realpath ../../tmp/mnt/publish` $vol_id -e tcp://127.0.0.1:500515ee5fd7c-45cd-44cf-af7b-06081f680f2cAttaching the volume to `tmp/mnt/publish` on container:.. code-block:: shell$ touch ../../tmp/mnt/{staging,publish}$ csc controller publish --cap SINGLE_NODE_WRITER,block --node-id `hostname -f` $vol_id -e tcp://127.0.0.1:50051"5ee5fd7c-45cd-44cf-af7b-06081f680f2c"  "connection_info"="{\"connector\": {\"initiator\": \"iqn.1994-05.com.redhat:aa532823bac9\", \"ip\": \"127.0.0.1\", \"platform\": \"x86_64\", \"host\": \"localhost.localdomain\", \"do_local_attach\": false, \"os_type\": \"linux2\", \"multipath\": false}, \"conn\": {\"driver_volume_type\": \"rbd\", \"data\": {\"secret_uuid\": null, \"volume_id\": \"5ee5fd7c-45cd-44cf-af7b-06081f680f2c\", \"auth_username\": \"cinder\", \"secret_type\": \"ceph\", \"name\": \"volumes/volume-5ee5fd7c-45cd-44cf-af7b-06081f680f2c\", \"discard\": true, \"keyring\": \"[client.cinder]\\n\\tkey = AQCQPetaof03IxAAoHZJD6kGxiMQfLdn3QzdlQ==\\n\", \"cluster_name\": \"ceph\", \"hosts\": [\"192.168.1.22\"], \"auth_enabled\": true, \"ports\": [\"6789\"]}}}"$ csc node stage --pub-info connection_info="irrelevant" --cap SINGLE_NODE_WRITER,block --staging-target-path /mnt/staging $vol_id -e tcp://127.0.0.1:500515ee5fd7c-45cd-44cf-af7b-06081f680f2c$ csc node publish --cap SINGLE_NODE_WRITER,block --pub-info connection_info="irrelevant" --staging-target-path /mnt/staging --target-path /mnt/publish $vol_id -e tcp://127.0.0.1:500515ee5fd7c-45cd-44cf-af7b-06081f680f2cDetaching the volume on baremetal:.. code-block:: shell$ csc node unpublish --target-path `realpath ../../tmp/mnt/publish` $vol_id -e tcp://127.0.0.1:500515ee5fd7c-45cd-44cf-af7b-06081f680f2c$ csc node unstage --staging-target-path `realpath ../../tmp/mnt/staging` $vol_id -e tcp://127.0.0.1:500515ee5fd7c-45cd-44cf-af7b-06081f680f2c$ csc controller unpublish --node-id `hostname -f` $vol_id -e tcp://127.0.0.1:500515ee5fd7c-45cd-44cf-af7b-06081f680f2cDetaching the volume on container:.. code-block:: shell$ csc node unpublish --target-path /mnt/publish $vol_id -e tcp://127.0.0.1:500515ee5fd7c-45cd-44cf-af7b-06081f680f2c$ csc node unstage --staging-target-path /tmp/mnt/staging $vol_id -e tcp://127.0.0.1:500515ee5fd7c-45cd-44cf-af7b-06081f680f2c$ csc controller unpublish --node-id `hostname -f` $vol_id -e tcp://127.0.0.1:500515ee5fd7c-45cd-44cf-af7b-06081f680f2cDeleting the volume:.. code-block:: shell$ csc controller delete-volume $vol_id -e tcp://127.0.0.1:50051Capable operational modes-------------------------The CSI spec defines a set of `AccessModes` that CSI drivers can support, suchas single writer, single reader, multiple writers, single writer and multiplereaders.This CSI driver currently only supports `SINGLE_MODE_WRITER`, although it willalso succeed with the `SINGLE_MODE_READER_ONLY` mode and mount it asread/write.Support-------For any questions or concerns please file an issue with the`cinderlib-csi <https://github.com/akrog/cinderlib-csi/issues>`_project or ping me on IRC (my handle is geguileo and I hang on the#openstack-cinder channel in Freenode).TODO----There are many things that need to be done in this POC driver, and here's a nonexhaustive list:- Support for NFS volumes- Support for mount filesystems- Support for Kubernetes CRDs as the persistence storage- Unit tests- Functional tests- Improve received parameters checking- Make driver more resilient- Test driver in Kubernetes- Review some of the returned error codes- Support volume attributes via cinder volume types- Look into multi-attaching- Support read-only mode- Report capacity based on over provisioning values- Configure the private data location=======History=======0.0.2 (2018-06-19)------------------* Use cinderlib v0.2.1 instead of github branch0.0.1 (2018-05-18)------------------* First release on PyPI.