Chapter 6. NFS cluster and export management As a storage administrator, you can create an NFS cluster, customize it, and export Ceph File System namespace over the NFS protocol. 6.1. Prerequisites A running, and healthy Red Hat Ceph Storage cluster. Installation and configuration of the Ceph Metadata Server daemons ( ceph-mds ). Create and mount a Ceph File System. 6.2. Creating an NFS cluster Create an NFS cluster with the nfs cluster create command. This creates a common recovery pool for all NFS Ganesa daemons, new user based on the cluster name, and a common NFS Ganesha configuration RADOS object. Prerequisites A running, and healthy Red Hat Ceph Storage cluster. An existing Ceph File System. Root-level access to Ceph Monitor. Installation of the nfs-ganesha , nfs-ganesha-ceph , nfs-ganesha-rados-grace , and nfs-ganesha-rados-urls packages on the Ceph Manager hosts. Root-level access to the client. Procedure Log into the Cephadm shell: Example Enable the Ceph Manager NFS module: Example Create an NFS Ganesha cluster: Syntax Example In this example, the NFS Ganesha cluster name is nfs-cephfs and the daemon containers are deployed to host01 , and host02 . Important Red Hat only supports one NFS Ganesha daemon running per host. Verify the NFS Ganesha cluster information: Syntax Example Note Specifying the CLUSTER_NAME is optional. 6.3. Customizing an NFS configuration Customize an NFS cluster with the configuration file. With this, the NFS cluster uses the specified configuration and has precedence over the default configuration blocks. Prerequisites A running, and healthy Red Hat Ceph Storage cluster. Root-level access to a Ceph Metadata Server (MDS) node. An NFS cluster created using the ceph nfs cluster create command. Procedure Create a configuration file: Example Enable logging in the configuration file with the following block: Example Set the new configuration: Syntax Example View the customized NFS Ganesha configuration: Syntax Example This provides the output, if any, of the user-defined configuration. Optional: If you want to remove the user-defined configuration, run the following command: Syntax Example 6.4. Exporting Ceph File System namespaces over the NFS protocol (Limited Availability) Ceph File Systems (CephFS) namespaces can be exported over the NFS protocol using a NFS Ganesha file server. To export a CephFS namespace, you must first have a running NFS Ganesha cluster. Note This technology is Limited Availability. See the Deprecated functionality chapter for additional information. Important Red Hat supports only NFS version 4.0 or higher. Important NFS clients are unable to create CephFS snapshots through their native NFS mount. They must use server-side operator tooling for their snapshot needs. Prerequisites A running, and healthy Red Hat Ceph Storage cluster. An NFS cluster created using the ceph nfs cluster create command. Procedure Create a CephFS export: For Red Hat Ceph Storage 5.0, create the export as follows: Syntax Example For Red Hat Ceph Storage 5.1 and later, create the export as follows: Syntax Example In this example, the BINDING ( /ceph ) is the pseudo root path, which must be unique, and an absolute path. Note The --readonly option exports a path with the read-only permission, the default being read, and write permissions. Note The PATH_WITHIN_CEPHFS can be a subvolume. You can get the absolute subvolume path by using the following command: Syntax Example View the export block based on the pseudo root name: Syntax Example List the NFS exports: Syntax Example Get the information of the NFS export: Syntax Example On a client host, mount the exported Ceph File System: Syntax Example To automatically mount on boot, open and edit the /etc/fstab file by adding a new line: Syntax Example On a client host, to mount a exported NFS Ceph File System created with an ingress service: Syntax Replace VIRTUAL_IP_ADDRESS with --ingress --virtual-ip IP address used to create the NFS cluster. Replace BINDING with the pseudo root path. Replace LOCAL_MOUNT_POINT with the mount point to mount the export on. Example This example mounts the export nfs-cephfs that exists on a NFS cluster created with --ingress --virtual-ip 10.10.128.75 on the mount point /mnt . 6.5. Modifying the Ceph File System exports You can modify the following parameters in an export with a configuration file: access_type - This can be RW , RO , or NONE . squash - This can be No_Root_Squash , None , or Root_Squash . security_label - This can be true or false . Prerequisites A running, and healthy Red Hat Ceph Storage cluster. An NFS export created. Procedure View the export block based on the pseudo root name: Syntax Example Export the configuration file: Example Edit the export information: Syntax Example In the above example, the access_type is modified from RO to RW . Apply the specification: Syntax Example Get the updated export information: Syntax Example 6.6. Creating custom Ceph File System exports You can customize the Ceph File System (CepFS) exports and apply the configuration. Prerequisites A running, and healthy Red Hat Ceph Storage cluster. An NFS cluster created using the ceph nfs cluster create command. A CephFS created. Procedure Create a custom file: Example Create an export using the custom file: Syntax Example Apply the specification: Syntax Example Get the updated export information: Syntax Example 6.7. Deleting Ceph File System exports You can delete the Ceph File System (CephFS) NFS exports with the ceph export rm command. Prerequisites A running, and healthy Red Hat Ceph Storage cluster. A CephFS created. Procedure Delete a CephFS export: Syntax Example 6.8. Deleting an NFS cluster Delete an NFS cluster with the nfs cluster rm command. This deletes the deployed cluster. The removal of NFS daemons and the ingress service is asynchronous. Check the status of removal with ceph orch ls command. Prerequisites A running, and healthy Red Hat Ceph Storage cluster. Root-level access to a Ceph Metadata Server (MDS) node. NFS daemons deployed with ceph nfs cluster create command. Procedure Log into the Cephadm shell: Example Remove an NFS Ganesha cluster: Syntax Example

Commands:
cephadm shell
ceph mgr module enable nfs
ceph nfs cluster create CLUSTER_NAME [ PLACEMENT ] [--ingress] [--virtual_ip IP_ADDRESS ] [--ingress-mode {default|keepalive-only|haproxy-standard|haproxy-protocol}] [--port PORT ]
ceph nfs cluster create nfs-cephfs "host01 host02" NFS Cluster Created Successfully
ceph nfs cluster info [ CLUSTER_NAME ]
ceph nfs cluster info nfs-cephfs { "nfs-cephfs": [ { "hostname": "host01", "ip": "10.74.179.124", "port": 2049 }, { "hostname": "host02", "ip": "10.74.180.160", "port": 2049 } ] }
touch nfs-cephfs.conf
vi nfs-cephfs.conf LOG { COMPONENTS { ALL = FULL_DEBUG; } }
ceph nfs cluster config set CLUSTER_NAME -i PATH_TO_CONFIG_FILE
ceph nfs cluster config set nfs-cephfs -i nfs-cephfs.conf NFS-Ganesha Config Set Successfully
ceph nfs cluster config get CLUSTER_NAME
ceph nfs cluster config get nfs-cephfs LOG { COMPONENTS { ALL = FULL_DEBUG; } }
ceph nfs cluster config reset CLUSTER_NAME
ceph nfs cluster config reset nfs-cephfs NFS-Ganesha Config Reset Successfully
ceph nfs export create cephfs FILE_SYSTEM_NAME CLUSTER_NAME BINDING [--readonly] [--path= PATH_WITHIN_CEPHFS ]
ceph nfs export create cephfs cephfs01 nfs-cephfs /ceph --path=/
ceph nfs export create cephfs CLUSTER_NAME BINDING FILE_SYSTEM_NAME [--readonly] [--path= PATH_WITHIN_CEPHFS ]
ceph nfs export create cephfs nfs-cephfs /ceph cephfs01 --path=/
ceph fs subvolume getpath VOLUME_NAME SUBVOLUME_NAME [--group_name SUBVOLUME_GROUP_NAME ]
ceph fs subvolume getpath cephfs sub0
ceph nfs export get CLUSTER_NAME BINDING
ceph nfs export get nfs-cephfs /ceph { "export_id": 1, "path": "/", "cluster_id": "nfs-cephfs", "pseudo": "/ceph", "access_type": "RW", "squash": "no_root_squash", "security_label": true, "protocols": [ 4 ], "transports": [ "TCP" ], "fsal": { "name": "CEPH", "user_id": "cephnfs11", "fs_name": "cephfs", "sec_label_xattr": "" }, "clients": [] }
ceph nfs export ls CLUSTER_NAME [--detailed]
ceph nfs export ls nfs-cephfs [ "/ceph/" ] ceph nfs export ls nfs-cephfs --detailed [ { "export_id": 100, "path": "/", "cluster_id": "nfs-cephfs", "pseudo": "/ceph/", "access_type": "RW", "squash": "no_root_squash", "security_label": true, "protocols": [ 4 ], "transports": [ "TCP" ], "fsal": { "name": "CEPH", "user_id": "nfstest01", "fs_name": "cephfs", "sec_label_xattr": "" }, "clients": [] } ]
ceph nfs export info CLUSTER_NAME [ PSEUDO_PATH ]
ceph nfs export info nfs-cephfs /ceph { "export_id": 1, "path": "/", "cluster_id": "nfs-cephfs", "pseudo": "/ceph", "access_type": "RW", "squash": "none", "security_label": true, "protocols": [ 4 ], "transports": [ "TCP" ], "fsal": { "name": "CEPH", "user_id": "nfs.nfs-cephfs.1", "fs_name": "cephfs" }, "clients": [] }
mount -t nfs -o port= GANESHA_PORT HOST_NAME : BINDING LOCAL_MOUNT_POINT
mount -t nfs -o port=2049 host01:/ceph/ /mnt/nfs-cephfs
HOST_NAME : BINDING LOCAL_MOUNT_POINT nfs4 defaults,seclabel,vers=4.2,proto=tcp,port=2049 0 0
host01:/ceph/ /mnt/nfs-cephfs nfs4 defaults,seclabel,vers=4.2,proto=tcp,port=2049 0 0
mount -t nfs VIRTUAL_IP_ADDRESS : BINDING LOCAL_MOUNT_POINT
mount -t nfs 10.10.128.75:/nfs-cephfs /mnt
ceph nfs export get CLUSTER_NAME BINDING
ceph nfs export get nfs-cephfs /ceph { "export_id": 1, "path": "/", "cluster_id": "nfs-cephfs", "pseudo": "/ceph", "access_type": "RO", "squash": "none", "security_label": true, "protocols": [ 4 ], "transports": [ "TCP" ], "fsal": { "name": "CEPH", "user_id": "cephnfs11", "fs_name": "cephfs", "sec_label_xattr": "" }, "clients": [] }
ceph nfs export get nfs-cephfs /ceph > export.conf
{ "export_id": EXPORT_ID , "path": "/", "cluster_id": " CLUSTER_NAME ", "pseudo": " CLUSTER_PSEUDO_PATH ", "access_type": " RW/RO ", "squash": "SQUASH", "security_label": SECURITY_LABEL , "protocols": [ PROTOCOL_ID_ ], "transports": [ "TCP" ], "fsal": { "name": " NAME ", "user_id": " USER_ID ", "fs_name": " FILE_SYSTEM_NAME ", "sec_label_xattr": "" }, "clients": [] }
vi export.conf { "export_id": 1, "path": "/", "cluster_id": "nfs-cephfs", "pseudo": "/ceph", "access_type": "RW", "squash": "none", "security_label": true, "protocols": [ 4 ], "transports": [ "TCP" ], "fsal": { "name": "CEPH", "user_id": "cephnfs11", "fs_name": "cephfs", "sec_label_xattr": "" }, "clients": [] }
ceph nfs export apply CLUSTER_NAME PATH_TO_EXPORT_FILE
ceph nfs export apply nfs-cephfs -i export.conf Added export /ceph
ceph nfs export get CLUSTER_NAME BINDING
ceph nfs export get nfs-cephfs /ceph { "export_id": 1, "path": "/", "cluster_id": "nfs-cephfs", "pseudo": "/ceph", "access_type": "RW", "squash": "none", "security_label": true, "protocols": [ 4 ], "transports": [ "TCP" ], "fsal": { "name": "CEPH", "user_id": "cephnfs11", "fs_name": "cephfs", "sec_label_xattr": "" }, "clients": [] }
touch export_new.conf
EXPORT { Export_Id = EXPORT_ID ; Transports = TCP/UDP ; Path = PATH ; Pseudo = PSEUDO_PATH ; Protocols = NFS_PROTOCOLS ; Access_Type = ACCESS_TYPE ; Attr_Expiration_Time = EXPIRATION_TIME ; Squash = SQUASH ; FSAL { Name = NAME ; Filesystem = " CEPH_FILE_SYSTEM_NAME "; User_Id = " USER_ID "; } }
cat export_new.conf EXPORT { Export_Id = 2; Transports = TCP; Path = /; Pseudo = /ceph1/; Protocols = 4; Access_Type = RW; Attr_Expiration_Time = 0; Squash = None; FSAL { Name = CEPH; Filesystem = "cephfs"; User_Id = "nfs.nfs-cephfs.2"; } }
ceph nfs export apply CLUSTER_NAME -i PATH_TO_EXPORT_FILE
ceph nfs export apply nfs-cephfs -i new_export.conf Added export /ceph1
ceph nfs export get CLUSTER_NAME BINDING
ceph nfs export get nfs-cephfs /ceph1 { "export_id": 1, "path": "/", "cluster_id": "nfs-cephfs", "pseudo": "/ceph1", "access_type": "RW", "squash": "None", "security_label": true, "protocols": [ 4 ], "transports": [ "TCP" ], "fsal": { "name": "CEPH", "user_id": "nfs.nfs-cephfs.2", "fs_name": "cephfs", "sec_label_xattr": "" }, "clients": [] }
ceph nfs export rm CLUSTER_NAME BINDING
ceph nfs export rm nfs-cephfs /ceph
cephadm shell
ceph nfs cluster rm CLUSTER_NAME
ceph nfs cluster rm nfs-cephfs