Project: Rook (Runtime / Cloud Native Storage)
File: filesystem-mirroring.md

title: Filesystem Mirroring Ceph filesystem mirroring is a process of asynchronous replication of snapshots to a remote CephFS file system. Snapshots are synchronized by mirroring snapshot data followed by creating a snapshot with the same name (for a given directory on the remote file system) as the snapshot being synchronized. It is generally useful when planning for Disaster Recovery. Mirroring is for clusters that are geographically distributed and stretching a single cluster is not possible due to high latencies. This guide assumes you have created a Rook cluster as explained in the main The following will enable mirroring on the filesystem: ```yaml apiVersion: ceph.rook.io/v1 kind: CephFilesystem metadata: name: myfs namespace: rook-ceph spec: metadataPool: failureDomain: host replicated: size: 3 dataPools: name: replicated failureDomain: host replicated: size: 3 preserveFilesystemOnDelete: true metadataServer: activeCount: 1 activeStandby: true mirroring: enabled: true peers: secretNames: snapshotSchedules: path: / interval: 24h # daily snapshots snapshotRetention: path: / duration: "h 24" ``` Launch the `rook-ceph-fs-mirror` pod on the source storage cluster, which deploys the `cephfs-mirror` daemon in the cluster: ```console kubectl create -f deploy/examples/filesystem-mirror.yaml ``` Please refer to for more information. Once mirroring is enabled, Rook will by default create its own so that it can be used by another cluster. The bootstrap peer token can be found in a Kubernetes Secret. The name of the Secret is present in the Status field of the CephFilesystem CR: ```yaml status: info: fsMirrorBootstrapPeerSecretName: fs-peer-token-myfs ``` This secret can then be fetched like so: ```console eyJmc2lkIjoiOTFlYWUwZGQtMDZiMS00ZDJjLTkxZjMtMTMxMWM5ZGYzODJiIiwiY2xpZW50X2lkIjoicmJkLW1pcnJvci1wZWVyIiwia2V5IjoiQVFEN1psOWZ3V1VGRHhBQWdmY0gyZi8xeUhYeGZDUTU5L1N0NEE9PSIsIm1vbl9ob3N0IjoiW3YyOjEwLjEwMS4xOC4yMjM6MzMwMCx2MToxMC4xMDEuMTguMjIzOjY3ODldIn0= ``` The decoded secret must be saved in a file before importing. ```console ``` See the CephFS mirror documentation on . Further refer to CephFS mirror documentation to . To check the `mirror daemon status`, please run the following command from the pod. For example : ```console ``` ```json [ { "daemon_id": 906790, "filesystems": [ { "filesystem_id": 1, "name": "myfs", "directory_count": 1, "peers": [ { "uuid": "a24a3366-8130-4d55-aada-95fa9d3ff94d", "remote": { "client_name": "client.mirror", "cluster_name": "91046889-a6aa-4f74-9fb0-f7bb111666b4", "fs_name": "myfs" }, "stats": { "failure_count": 0, "recovery_count": 0 } } ] } ] } ] ``` Please refer to the `--admin-daemon` socket commands from the CephFS mirror documentation to verify and run the commands from the `rook-ceph-fs-mirror` pod: ```console ``` Fetch the `ceph-client.fs-mirror` daemon admin socket file from the `/var/run/ceph` directory: ```console ``` ```console ``` ```json { "rados_inst": "X.X.X.X:0/2286593433", "peers": { "a24a3366-8130-4d55-aada-95fa9d3ff94d": { "remote": { "client_name": "client.mirror", "cluster_name": "91046889-a6aa-4f74-9fb0-f7bb111666b4", "fs_name": "myfs" } } }, "snap_dirs": { "dir_count": 1 } } ``` For getting `peer synchronization status`: ```console ``` ```json { "/volumes/_nogroup/subvol-1": { "state": "idle", "lastsyncedsnap": { "id": 4, "name": "snap2" }, "snaps_synced": 0, "snaps_deleted": 0, "snaps_renamed": 0 } } ```