Oracle Cluster File System(OCFS2) setup for shared Block Volume
Tags: oci, ocfs2, cluster, blockvolume
Published: 2024-04-29 13:04:33

There are moments when simple tasks can slow down our progress, often because they're not part of our regular routine.  This post captured set of steps to enable Shared Block Volume for multiple OCI Compute Instances. Such a setup also needs Oracle Cluster File System(OCFS2) to be enabled. 
I so wish this enablement is possible by a click of Button, API call ...

Lets make our hands dirty.


**Starting point.**

1. Two OCI Instance with the provision to access through SSH keys


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xgbkc7o0trxpwm1hkafe.png)

2. Block Volume in the same Availability Domain



![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2hd3pamw2y4lqnyaeuz9.png)



**Attach the Block Volume to Instance**


1. Navigate to the Unattached Block Volume. Attached Instances >> Attach Instance

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/r837ac7objg1zs8ra55r.png)

Attach the First Instance.
Repeat the same for Second Instance


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5jxdn5pz6bi3mpnbwfnz.png)


**Setup the Oracle Cluster File System(OCFS2)**

1) Add stateful Ingress in Security List of the Instance VCN

Source: Subnet/VCN CIDR
IP Prototocol: TCP and UDP
Source Port: All
Target Port: 3260 and 7777


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/59dd22i1jl3yfdumgksn.png)

2) Open the Target ports on Local OS Firewall on **instance-1** and **instance-2**


```
sudo firewall-cmd --zone=public --permanent --add-port=7777/tcp
sudo firewall-cmd --zone=public --permanent --add-port=7777/udp
sudo firewall-cmd --zone=public --permanent --add-port=3260/tcp
sudo firewall-cmd --zone=public --permanent --add-port=3260/udp
sudo firewall-cmd --complete-reload
```

3) Install OCFS2 Packages on **instance-1** and **instance-2**

```
sudo dnf install -y ocfs2-tools
```

4) Create Cluster definition on **instance-1**

```
sudo o2cb add-cluster ocifs2
```

5) Add the nodes to Cluster on **instance-1**

```
sudo o2cb add-node ocifs2 instance-1 --ip <PRIVATE IP OF instance 1>
sudo o2cb add-node ocifs2 instance-2 --ip <PRIVATE IP OF instance 2>
```

6) Check the Cluster Config file on **instance-1**

```
[opc@instance-1 ~]$ sudo cat /etc/ocfs2/cluster.conf
cluster:
	heartbeat_mode = local
	node_count = 2
	name = ocifs2

node:
	number = 0
	cluster = ocifs2
	ip_port = 7777
	ip_address = 172.17.0.173
	name = instance-1

node:
	number = 1
	cluster = ocifs2
	ip_port = 7777
	ip_address = 172.17.0.154
	name = instance-2

```

7) Copy the content of Cluster Config file to instance-2 at the /etc/ocfs2/cluster.conf


8) Configure the Cluster Stack in **instance-1** and **instance-2** sequentially

```
[opc@instance-1 ~]$ sudo /sbin/o2cb.init configure
Configuring the O2CB driver.
...
...
Cluster to start on boot (Enter "none" to clear) [ocfs2]: ocifs2
...
```

9) Verify the settings for Cluster Stack and note the values are loaded, mounted and online. But the heart beat is not active

```
[opc@instance-1 ~]$ sudo /sbin/o2cb.init status
Driver for "configfs": Loaded
Filesystem "configfs": Mounted
Stack glue driver: Loaded
Stack plugin "o2cb": Loaded
Driver for "ocfs2_dlmfs": Loaded
Filesystem "ocfs2_dlmfs": Mounted
Checking O2CB cluster "ocifs2": Online
  Heartbeat dead threshold: 31
  Network idle timeout: 30000
  Network keepalive delay: 2000
  Network reconnect delay: 2000
  Heartbeat mode: Local
Checking O2CB heartbeat: Not active
Debug file system at /sys/kernel/debug: mounted
```

Repeat above steps in **instance-2**

10) Setup the Boot time startup of o2cb and ocfs2 services on **instance-1** and **instance-2**

```
sudo systemctl enable o2cb
sudo systemctl enable ocfs2
```

11) Configure Kernel for Cluster Operation on **instance-1** and **instance-2** using SSH


```
sudo sysctl kernel.panic=30
sudo sysctl kernel.panic_on_oops=1
```
Add the entries to sysctl configuration file for persistence

```
sudo vi /etc/sysctl.conf
kernel.panic=30
kernel.panic_on_oops=1

```

12) Create OCFS2 Volume on only **instance-1**

```
[opc@instance-1 ~]$ sudo mkfs.ocfs2 -L "ocfs2" /dev/sdb
mkfs.ocfs2 1.8.6
Cluster stack: classic o2cb
Label: ocfs2
...
Formatting Journals: 
...
Writing lost+found: done
mkfs.ocfs2 successful
```

Create a mount directory

```
[opc@instance-1 ~]$ sudo mkdir /ocfs2
```

Specify the netdev option in /etc/fstab to allow the system to mount ocfs2 volume during boot time

```
sudo vi /etc/fstab 
/dev/sdb /ocfs2 ocfs2   _netdev,defaults        0 0
```
Reload the sysctl and mount

```
sudo systemctl daemon-reload
sudo mount -a 
```

Check the mounted block volume

```
[opc@instance-1 ~]$ lsblk
NAME               MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
...
sdb                  8:16   0    1T  0 disk /ocfs2

```

13) OCFS2 Volume setup and mount on  **instance-2**

```
[opc@instance-2 ~]$ sudo mkdir /ocfs2
[opc@instance-2 ~]$ sudo vi /etc/fstab 
[opc@instance-2 ~]$ sudo systemctl daemon-reload
[opc@instance-2 ~]$ sudo mount -a 
[opc@instance-2 ~]$ lsblk
NAME               MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
...
sdb                  8:16   0    1T  0 disk /ocfs2
```
 

14) Test: Create a file on the mounted volume for eg. on **instance-2**

```
[opc@instance-2 ~]$ cd /ocfs2/
[opc@instance-2 ocfs2]$ sudo touch shared-cluster-file.txt
[opc@instance-2 ocfs2]$ ls -lart
total 8
drwxr-xr-x.  2 root root 3896 Apr 29 12:03 lost+found
dr-xr-xr-x. 19 root root  265 Apr 29 12:15 ..
-rw-r--r--.  1 root root    0 Apr 29 12:19 shared-cluster-file.txt
drwxr-xr-x.  3 root root 3896 Apr 29 12:19 .
```

15) Test: Access the file on the mounted volume of other instance eg. on **instance-1**

```
[opc@instance-1 ~]$ cd /ocfs2/
[opc@instance-1 ocfs2]$ ls -lart
total 8
drwxr-xr-x.  2 root root 3896 Apr 29 12:03 lost+found
dr-xr-xr-x. 19 root root  265 Apr 29 12:04 ..
-rw-r--r--.  1 root root    0 Apr 29 12:19 shared-cluster-file.txt
drwxr-xr-x.  3 root root 3896 Apr 29 12:19 .

```