Repository: rook/rook
Pull request #6765 (2020-12-04 10:24:22)

ceph: add mgr caps for csi-rbd-node client
<!-- Please take a look at our [Contributing](https://rook.io/docs/rook/master/development-flow.html)
documentation before submitting a Pull Request!
Thank you for contributing to Rook! -->

**Description of your changes:**
If the kernel doesn't support mapping of the rbd image with the deep-flatten feature,cephcsi needs to flatten the rbd image first and then map the image on the node.

cephcsi first tries to add a task to flatten the rbd image if it receives any permission error it will try to call the rbd CLI command which is a blocking call. This commit adds mgr caps to the csi-rbd-node user so that cephcsi will add flatten task and return
immediate error to the kubelet, let kubelet retry again, If we go with blocking rbd CLI call we may end up having stale maps on the node in corner cases.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>

**Which issue is resolved by this Pull Request:**
Resolves #

**Checklist:**

- [ ] **Commit Message Formatting**: Commit titles and messages follow guidelines in the [developer guide](https://rook.io/docs/rook/master/development-flow.html#commit-structure).
- [ ] **Skip Tests for Docs**: Add the flag for skipping the build if this is only a documentation change. See [here](https://github.com/rook/rook/blob/master/INSTALL.md#skip-ci) for the flag.
- [ ] **Skip Unrelated Tests**: Add a flag to run tests for a specific storage provider. See [test options](https://github.com/rook/rook/blob/master/INSTALL.md#test-storage-provider).
- [ ] Reviewed the developer guide on [Submitting a Pull Request](https://rook.io/docs/rook/master/development-flow.html#submitting-a-pull-request)
- [ ] Documentation has been updated, if necessary.
- [ ] Unit tests have been added, if necessary.
- [ ] Integration tests have been added, if necessary.
- [ ] Pending release notes updated with breaking and/or notable changes, if necessary.
- [ ] Upgrade from previous release is tested and upgrade user guide is updated, if necessary.
- [ ] Code generation (`make codegen`) has been run to update object specifications, if necessary.


travisn: Looks like the canary integration test is failing with this error in the script check:
```
  File "/etc/ceph/create-external-cluster-resources.py", line 568, in _gen_output_map

    self.out_map['CSI_RBD_NODE_SECRET_SECRET'] = self.create_cephCSIKeyring_RBDNode()
Exception thrown successfully: Out of range port number: 90320
  File "/etc/ceph/create-external-cluster-resources.py", line 518, in create_cephCSIKeyring_RBDNode
Exception thrown successfully: Out of range port number: 73422
    ret_val, json_out, err_msg = self._common_cmd_json_gen(cmd_json)
Exception thrown successfully: 'prometheus' service port must listen on 9283. You can change it with 'ceph config set mgr mgr/prometheus/server_port 9283'.
  File "/etc/ceph/create-external-cluster-resources.py", line 278, in _common_cmd_json_gen
```

Patch:
diff --git a/pkg/operator/ceph/csi/secrets.go b/pkg/operator/ceph/csi/secrets.go
index d78f559ca..d43178f25 100644
--- a/pkg/operator/ceph/csi/secrets.go
+++ b/pkg/operator/ceph/csi/secrets.go
@@ -81,6 +81,7 @@ func createCSIKeyringCephFSProvisioner(s *keyring.SecretStore) (string, error) {
 func cephCSIKeyringRBDNodeCaps() []string {
 	return []string{
 		"mon", "profile rbd",
+		"mgr", "allow rw",
 		"osd", "profile rbd",
 	}
 }


Test patch:
diff --git a/pkg/operator/ceph/csi/secrets_test.go b/pkg/operator/ceph/csi/secrets_test.go
index 00b2dfc16..3120ff23c 100644
--- a/pkg/operator/ceph/csi/secrets_test.go
+++ b/pkg/operator/ceph/csi/secrets_test.go
@@ -24,7 +24,7 @@ import (
 
 func TestCephCSIKeyringRBDNodeCaps(t *testing.T) {
 	caps := cephCSIKeyringRBDNodeCaps()
-	assert.Equal(t, caps, []string{"mon", "profile rbd", "osd", "profile rbd"})
+	assert.Equal(t, caps, []string{"mon", "profile rbd", "mgr", "allow rw", "osd", "profile rbd"})
 }
 
 func TestCephCSIKeyringRBDProvisionerCaps(t *testing.T) {