InterSystems IRIS and Caché Application Consistent Backups with Azure Backup
Tags: azure, backup, cloud, cache
Published: 2020-12-01 12:36:30

Database systems have very specific backup requirements that in enterprise deployments require forethought and planning. For database systems, the operational goal of a backup solution is to create a copy of the data in a state that is equivalent to when application is shut down gracefully.  Application consistent backups meet these requirements and Caché provides a set of APIs that facilitate the integration with external solutions to achieve this level of backup consistency.<!--break-->

These APIs are **_ExternalFreeze_** and **_ExternalThaw_**. _ExternalFreeze_ temporarily pauses writes to disk and during this period Caché commits the changes in memory. During this period the backup operation must complete and be followed by a call to _ExternalThaw_. This call engages the write daemons to write the cached updated in the global buffer pool (database cache) to disk and resumes normal Caché database write daemon operations. This process is transparent to user processes with Caché.  The specific API class methods are:

[##Class(Backup.General).ExternalFreeze()](http://docs.intersystems.com/latest/csp/documatic/%25CSP.Documatic.cls?PAGE=CLASS&LIBRARY=%25SYS&CLASSNAME=Backup.General#ExternalFreeze)

[##Class(Backup.General).ExternalThaw()](http://docs.intersystems.com/latest/csp/documatic/%25CSP.Documatic.cls?PAGE=CLASS&LIBRARY=%25SYS&CLASSNAME=Backup.General#ExternalThaw)

These APIs in conjunction with the new capability of Azure Backup to execute a script prior and after the execution of a snapshot operation, provide a comprehensive backup solution for deployments of Caché on Azure. The [pre/post scripting capability of Azure Backup](https://azure.microsoft.com/en-us/blog/announcing-application-consistent-backup-for-linux-vms-using-azure-backup) is currently available only on Linux VMs.

## Prerequisites

At the high level, there are three steps that you need to perform before you can backup a VM using Azure Backup:

  1. Create a Recovery Services vault
  2. Install has the latest version of the VM Agent.
  3. Check network access to the Azure services from your VM. 

The Recovery Services vault manages the backup goals, policies and the items to protect. You can create a Recovery Services vault via the Azure Portal or via scripting using PowerShell.  Azure Backup requires an extension that runs in your VM, is controlled by the Linux VM agent and the latest version of the agent is also required.  The extension interacts with the external facing HTTPS endpoints of Azure Storage and the Recovery Services vault.  Secure access to those services from the VM can be configured using a proxy and network rules in an Azure Network Security Group. 

For more information about these steps visit [Prepare your environment to back up Resource Manager-deployed virtual machines](https://docs.microsoft.com/en-us/azure/backup/backup-azure-arm-vms-prepare).

## Pre and Post Scripting Configuration

The ability to call a script prior to the backup operation and after is, included in the latest version of the Azure Backup Extension (Microsoft.Azure.RecoveryServices.VMSnapshotLinux). For information about how to install the extension please check [the detailed feature documentation](https://docs.microsoft.com/en-us/azure/backup/backup-azure-linux-app-consistent).

By default, the extension included sample pre and pot scripts located in your Linux VM at: 

<pre>/var/lib/waagent/Microsoft.Azure.RecoveryServices.VMSnapshotLinux-1.0.9110.0/main/tempPlugin</pre>

And needs to be copied to the following locations respectively.

<pre>/etc/azure/prescript.sh
/etc/azure/postScript.sh</pre>

You can also download the script template from [GitHub](https://github.com/MicrosoftAzureBackup/Cache).

For Caché, the prescript.sh script where a call to the ExternalFreeze API can be implemented and the postScript.sh should contain the implementation that executes ExternalThaw.

**The following is a sample <u><em>prescript.sh</em></u> implementation for Caché.**

<pre><span style="color:#008000;">#!/bin/bash
# variables used for returning the status of the script</span>
success=0
error=1
warning=2
status=<span style="color:#0000CD;">$success</span>
log_path=<span style="color:#B22222;">"/etc/preScript.log"</span>   <span style="color:#008000;">#path of log file</span>
printf  <span style="color:#B22222;">"Logs:\n"</span> > <span style="color:#0000CD;">$log_path</span>
<span style="color:#008000;"># TODO: Replace &lt;CACHE INSTANCE> with the name of the running instance</span>
csession &lt;CACHE INSTANCE> -U%SYS <span style="color:#B22222;">"##Class(Backup.General).ExternalFreeze()"</span> >> <span style="color:#0000CD;">$log_path</span>
status=<span style="color:#0000CD;">$?</span>
<span style="color:#EE82EE;">if </span>[ <span style="color:#0000CD;">$status</span> -eq 5 ]; <span style="color:#EE82EE;">then</span>
echo <span style="color:#B22222;">"SYSTEM IS FROZEN"</span>
printf  <span style="color:#B22222;">"SYSTEM IS FROZEN\n"</span> >> <span style="color:#0000CD;">$log_path</span>
<span style="color:#EE82EE;">elif </span>[ <span style="color:#0000CD;">$status</span> -eq 3 ]; <span style="color:#EE82EE;">then</span>
echo <span style="color:#B22222;">"SYSTEM FREEZE FAILED"</span>
printf  <span style="color:#B22222;">"SYSTEM FREEZE FAILED\n"</span> >> <span style="color:#0000CD;">$log_path</span>
status=$error
csession &lt;CACHE INSTANCE> -U%SYS "##Class(Backup.General).ExternalThaw()"
<span style="color:#EE82EE;">fi
</span>
exit <span style="color:#0000CD;">$status</span></pre>

**The following is a sample <u><em>postScript.sh</em></u> implementation for Caché.**

<pre><span style="color:#008000;">#!/bin/bash
# variables used for returning the status of the script</span>
success=0
error=1
warning=2
status=<span style="color:#0000CD;">$success</span>
log_path=<span style="color:#B22222;">"/etc/postScript.log"</span>   <span style="color:#008000;">#path of log file</span>
printf  <span style="color:#B22222;">"Logs:\n"</span> > <span style="color:#0000CD;">$log_path</span>
<span style="color:#008000;"># TODO: Replace &lt;CACHE INSTANCE> with the name of the running instance</span>
csession &lt;CACHE INSTANCE> -U%SYS <span style="color:#B22222;">"##class(Backup.General).ExternalThaw()"</span>
status=$?
<span style="color:#EE82EE;">if </span>[ $status req 5]; <span style="color:#EE82EE;">then</span>
echo <span style="color:#B22222;">"SYSTEM IS UNFROZEN"</span>
printf  <span style="color:#B22222;">"SYSTEM IS UNFROZEN\n"</span> >> <span style="color:#0000CD;">$log_path</span>
<span style="color:#EE82EE;">elif </span>[ $status -eq 3 ]; <span style="color:#EE82EE;">then</span>
echo <span style="color:#B22222;">"SYSTEM UNFREEZE FAILED"</span>
printf <span style="color:#B22222;"> "SYSTEM UNFREEZE FAILED\n"</span> >> <span style="color:#0000CD;">$log_path</span>
status=<span style="color:#0000CD;">$error</span>
fi
exit <span style="color:#0000CD;">$status</span></pre>

## Executing a Backup

In the Azure Portal, you can trigger the first backup by navigating to the Recovery Service. Please consider that the VM snapshot time should be few seconds irrespective of first backup or subsequent backup. Data transfer of first backup will take longer but data transfer will start after executing post-script to thaw database and should not have any impact on the time between pre & post script.

It is highly recommended to regularly restore your backup in a non-production setting and [perform database integrity checks](http://docs.intersystems.com/latest/csp/docbook/DocBook.UI.Page.cls?KEY=GCDI_integrity#GCDI_integrity_verify_utility) to ensure your data protection operations are effective.

For more information about how to trigger the backup and other topics such as backup scheduling, please check [Back up Azure virtual machines to a Recovery Services vault](https://docs.microsoft.com/en-us/azure/backup/backup-azure-arm-vms-prepare).