Chapter 4. Install Pacemaker Refer to Configuring and managing high availability clusters documentation to set up a pacemaker cluster. Below is a sample procedure to install a Pacemaker cluster. It's recommended to work with a Red Hat consultant to install and configure Pacemaker in your environment. 4.1. Install Pacemaker rpms # yum -y install pcs pacemaker # passwd hacluster [provide a password] # systemctl enable --now pcsd.service 4.2. Create a cluster Create a cluster named s4ha , consisting of s4node1 and s4node2 , and start the cluster. Please note that at this point, the cluster is not yet configured to auto-start after reboot. # pcs cluster auth s4node1 s4node2 # pcs cluster setup --name s4ha s4node1 s4node2 # pcs cluster start --all 4.2.1. Define general cluster properties Set the resource stickiness: [root]# pcs resource defaults resource-stickiness=1 [root]# pcs resource defaults migration-threshold=3 4.3. Configure STONITH The fencing mechanism STONITH depends on the underlying platform. Please check the document Support Policies for RHEL High Availability Clusters - General Requirements for Fencing/STONITH to configure the STONITH. After configuring the STONITH, on s4node1 test by fencing s4node2 and vice-versa: [root@s4node1]# pcs stonith fence s4node2 The s4node2 should be properly fenced. After fencing, start cluster on s4node2 using the following command because the cluster has not yet been enabled to auto-start. Auto-start is enabled after initial testing. [root@s4node2 ~]# pcs cluster start 4.4. Install resource-agents-sap on all cluster nodes [root]# yum install resource-agents-sap 4.5. Configure cluster resources for shared filesystems Configure shared filesystem to provide following mount points on all the cluster nodes. /sapmnt /usr/sap/trans /usr/sap/S4H/SYS 4.5.1. Configure shared filesystems managed by the cluster The cloned Filesystem cluster resource can be used to mount the shares from external NFS server on all cluster nodes as shown below. [root]# pcs resource create s4h_fs_sapmnt Filesystem \ device='<NFS_Server>:<sapmnt_nfs_share>' directory='/sapmnt' \ fstype='nfs' --clone interleave=true [root]# pcs resource create s4h_fs_sap_trans Filesystem \ device='<NFS_Server>:<sap_trans_nfs_share>' directory='/usr/sap/trans' \ fstype='nfs' --clone interleave=true [root]# pcs resource create s4h_fs_sap_sys Filesystem \ device='<NFS_Server>:<s4h_sys_nfs_share>' directory='/usr/sap/S4H/SYS' \ fstype='nfs' --clone interleave=true After creating the Filesystem resources verify that they have started properly on all nodes. [root]# pcs status ... Clone Set: s4h_fs_sapmnt-clone [s4h_fs_sapmnt] Started: [ s4node1 s4node2 ] Clone Set: s4h_fs_sap_trans-clone [s4h_fs_sap_trans] Started: [ s4node1 s4node2 ] Clone Set: s4h_fs_sys-clone [s4h_fs_sys] Started: [ s4node1 s4node2 ] ... 4.6. Configure ASCS resource group 4.6.1. Create resource for virtual IP address [root]# pcs resource create s4h_vip_ascs20 IPaddr2 ip=192.168.200.201 \ --group s4h_ASCS20_group 4.6.2. Create resource for ASCS filesystem. Below is the example of creating resource for NFS filesystem [root]# pcs resource create s4h_fs_ascs20 Filesystem \ device='<NFS_Server>:<s4h_ascs20_nfs_share>' \ directory=/usr/sap/S4H/ASCS20 fstype=nfs force_unmount=safe \ --group s4h_ASCS20_group op start interval=0 timeout=60 \ op stop interval=0 timeout=120 \ op monitor interval=200 timeout=40 Below is the example of creating resources for HA-LVM filesystem [root]# pcs resource create s4h_fs_ascs20_lvm LVM \ volgrpname='<ascs_volume_group>' exclusive=true \ --group s4h_ASCS20_group [root]# pcs resource create s4h_fs_ascs20 Filesystem \ device='/dev/mapper/<ascs_logical_volume>' \ directory=/usr/sap/S4H/ASCS20 fstype=ext4 \ --group s4h_ASCS20_group 4.6.3. Create resource for ASCS instance [root]# pcs resource create s4h_ascs20 SAPInstance \ InstanceName="S4H_ASCS20_s4ascs" \ START_PROFILE=/sapmnt/S4H/profile/S4H_ASCS20_s4ascs \ AUTOMATIC_RECOVER=false \ meta resource-stickiness=5000 \ --group s4h_ASCS20_group \ op monitor interval=20 on-fail=restart timeout=60 \ op start interval=0 timeout=600 \ op stop interval=0 timeout=600 Note : meta resource-stickiness=5000 is here to balance out the failover constraint with ERS so the resource stays on the node where it started and doesn't migrate around the cluster uncontrollably. Add a resource stickiness to the group to ensure that the ASCS will stay on a node if possible: [root]# pcs resource meta s4h_ASCS20_group resource-stickiness=3000 4.7. Configure ERS resource group 4.7.1. Create resource for virtual IP address [root]# pcs resource create s4h_vip_ers29 IPaddr2 ip=192.168.200.202 \ --group s4h_ERS29_group 4.7.2. Create resource for ERS filesystem Below is the example of creating resource for NFS filesystem [root]# pcs resource create s4h_fs_ers29 Filesystem \ device='<NFS_Server>:<s4h_ers29_nfs_share>' \ directory=/usr/sap/S4H/ERS29 fstype=nfs force_unmount=safe \ --group s4h_ERS29_group op start interval=0 timeout=60 \ op stop interval=0 timeout=120 op monitor interval=200 timeout=40 Below is the example of creating resources for HA-LVM filesystem [root]# pcs resource create s4h_fs_ers29_lvm LVM \ volgrpname='<ers_volume_group>' exclusive=true --group s4h_ERS29_group [root]# pcs resource create s4h_fs_ers29 Filesystem \ device='/dev/mapper/<ers_logical_volume>' directory=/usr/sap/S4H/ERS29 \ fstype=ext4 --group s4h_ERS29_group 4.7.3. Create resource for ERS instance Create an ERS instance cluster resource. Note : In ENSA2 deployments the IS_ERS attribute is optional. To learn more about IS_ERS , additional information can be found in How does the IS_ERS attribute work on a SAP NetWeaver cluster with Standalone Enqueue Server (ENSA1 and ENSA2)? . [root]# pcs resource create s4h_ers29 SAPInstance \ InstanceName="S4H_ERS29_s4ers" \ START_PROFILE=/sapmnt/S4H/profile/S4H_ERS29_s4ers \ AUTOMATIC_RECOVER=false \ --group s4h_ERS29_group \ op monitor interval=20 on-fail=restart timeout=60 \ op start interval=0 timeout=600 \ op stop interval=0 timeout=600 4.8. Create constraints 4.8.1. Create colocation constraint for ASCS and ERS resource groups Resource groups s4h_ASCS20_group and s4h_ERS29_group should try to avoid running on the same node. Order of groups matters. [root]# pcs constraint colocation add s4h_ERS29_group with s4h_ASCS20_group \ -5000 4.8.2. Create order constraint for ASCS and ERS resource groups Prefer to start s4h_ASCS20_group before the s4h_ERS29_group [root]# pcs constraint order start s4h_ASCS20_group then start \ s4h_ERS29_group symmetrical=false kind=Optional [root]# pcs constraint order start s4h_ASCS20_group then stop \ s4h_ERS29_group symmetrical=false kind=Optional 4.8.3. Create order constraint for /sapmnt resource managed by cluster If the shared filesystem /sapmnt is managed by the cluster, then the following constraints ensure that resource groups with ASCS and ERS SAPInstance resources are started only once the filesystem is available. [root]# pcs constraint order s4h_fs_sapmnt-clone then s4h_ASCS20_group [root]# pcs constraint order s4h_fs_sapmnt-clone then s4h_ERS29_group

Commands:
yum -y install pcs pacemaker passwd hacluster systemctl enable --now pcsd.service
pcs cluster auth s4node1 s4node2 pcs cluster setup --name s4ha s4node1 s4node2 pcs cluster start --all
pcs resource defaults resource-stickiness=1 pcs resource defaults migration-threshold=3
pcs stonith fence s4node2
pcs cluster start
yum install resource-agents-sap
pcs resource create s4h_fs_sapmnt Filesystem device='<NFS_Server>:<sapmnt_nfs_share>' directory='/sapmnt' fstype='nfs' --clone interleave=true pcs resource create s4h_fs_sap_trans Filesystem device='<NFS_Server>:<sap_trans_nfs_share>' directory='/usr/sap/trans' fstype='nfs' --clone interleave=true pcs resource create s4h_fs_sap_sys Filesystem device='<NFS_Server>:<s4h_sys_nfs_share>' directory='/usr/sap/S4H/SYS' fstype='nfs' --clone interleave=true
pcs status Clone Set: s4h_fs_sapmnt-clone [s4h_fs_sapmnt] Started: [ s4node1 s4node2 ] Clone Set: s4h_fs_sap_trans-clone [s4h_fs_sap_trans] Started: [ s4node1 s4node2 ] Clone Set: s4h_fs_sys-clone [s4h_fs_sys] Started: [ s4node1 s4node2 ]
pcs resource create s4h_vip_ascs20 IPaddr2 ip=192.168.200.201 --group s4h_ASCS20_group
pcs resource create s4h_fs_ascs20 Filesystem device='<NFS_Server>:<s4h_ascs20_nfs_share>' directory=/usr/sap/S4H/ASCS20 fstype=nfs force_unmount=safe --group s4h_ASCS20_group op start interval=0 timeout=60 op stop interval=0 timeout=120 op monitor interval=200 timeout=40
pcs resource create s4h_fs_ascs20_lvm LVM volgrpname='<ascs_volume_group>' exclusive=true --group s4h_ASCS20_group pcs resource create s4h_fs_ascs20 Filesystem device='/dev/mapper/<ascs_logical_volume>' directory=/usr/sap/S4H/ASCS20 fstype=ext4 --group s4h_ASCS20_group
pcs resource create s4h_ascs20 SAPInstance InstanceName="S4H_ASCS20_s4ascs" START_PROFILE=/sapmnt/S4H/profile/S4H_ASCS20_s4ascs AUTOMATIC_RECOVER=false meta resource-stickiness=5000 --group s4h_ASCS20_group op monitor interval=20 on-fail=restart timeout=60 op start interval=0 timeout=600 op stop interval=0 timeout=600
pcs resource meta s4h_ASCS20_group resource-stickiness=3000
pcs resource create s4h_vip_ers29 IPaddr2 ip=192.168.200.202 --group s4h_ERS29_group
pcs resource create s4h_fs_ers29 Filesystem device='<NFS_Server>:<s4h_ers29_nfs_share>' directory=/usr/sap/S4H/ERS29 fstype=nfs force_unmount=safe --group s4h_ERS29_group op start interval=0 timeout=60 op stop interval=0 timeout=120 op monitor interval=200 timeout=40
pcs resource create s4h_fs_ers29_lvm LVM volgrpname='<ers_volume_group>' exclusive=true --group s4h_ERS29_group pcs resource create s4h_fs_ers29 Filesystem device='/dev/mapper/<ers_logical_volume>' directory=/usr/sap/S4H/ERS29 fstype=ext4 --group s4h_ERS29_group
pcs resource create s4h_ers29 SAPInstance InstanceName="S4H_ERS29_s4ers" START_PROFILE=/sapmnt/S4H/profile/S4H_ERS29_s4ers AUTOMATIC_RECOVER=false --group s4h_ERS29_group op monitor interval=20 on-fail=restart timeout=60 op start interval=0 timeout=600 op stop interval=0 timeout=600
pcs constraint colocation add s4h_ERS29_group with s4h_ASCS20_group -5000
pcs constraint order start s4h_ASCS20_group then start s4h_ERS29_group symmetrical=false kind=Optional pcs constraint order start s4h_ASCS20_group then stop s4h_ERS29_group symmetrical=false kind=Optional
pcs constraint order s4h_fs_sapmnt-clone then s4h_ASCS20_group pcs constraint order s4h_fs_sapmnt-clone then s4h_ERS29_group