Scaling Cloud Hosts and Reconfiguring InterSystems IRIS
Tags: cloud, aws, azure, beginners
Published: 2022-06-29 18:14:46

Like hardware hosts, virtual hosts in public and private clouds can develop resource bottlenecks as workloads increase. If you are using and managing InterSystems IRIS instances deployed in public or private clouds, you may have encountered a situation in which addressing performance or other issues requires increasing the capacity of an instance's host (that is, _vertically scaling_).

One common reason to scale is insufficient memory. As described in [Memory Management and Scaling for InterSystems IRIS](https://docs.intersystems.com/irislatest/csp/docbook/Doc.View.cls?KEY=GSCALE_vertical#GSCALE_vertical_memory) in the _Scalability Guide, _providing enough memory for all of the entities running on the host of an InterSystems IRIS instance under all normal operating circumstances is a critical factor in both performance and availability. In one common scenario, as the workload of an InterSystems IRIS instance increases, its working set becomes too large to be held by the instance's allocated database cache. This forces some queries to fall back to disk, greatly increasing the number of disk reads required and creating a major performance problem. Increasing the size of the cache solves that problem, but if doing so would leave insufficient memory remaining for other purposes, you also need to increase the total physical memory on the host to avoid pushing the bottleneck to another part of the system.

Fortunately, scaling a virtual host is typically a lot easier than scaling hardware. This post discusses the two stages of the process:

  * Scaling the cloud host's resources 

<p style="margin-left: 40px;">
  You can change the resource specification of a virtual host on AWS, GCP, and Azure, using the platform's command line, API, or portal. VMWare vSphere allows you to easily change a number of resource parameters for a VM through its vSphere Client interface..  
</p>

  * Reconfiguring InterSystems IRIS to take advantage of the scaled resources

<p style="margin-left: 40px;">
  There are a number of ways to reconfigure InterSystems IRIS to take advantage of scaled host resources. This document describes the use of the configuration merge feature, which merges new parameter values, specified in a merge file, into an instance's CPF. Configuration merge is an easy and effective method because it lets you address only the configuration settings you want to modify, make multiple changes to an instance's configuration in one operation, and easily make the same set of changes to multiple instances.
</p>

The procedures described here are manual, but in production they would very likely be automated, for example using a script that would apply a specific merge file in an accessible location to a list of instances.

## Scaling Cloud Host Resources

Public cloud platforms provide a range of resource templates to choose from that specify CPU, memory, network interfaces, and other resources for virtual hosts (storage is provisioned and sized separately). To resize a host, you change the template selected when the host was provisioned to one that specifies more of the resources you want to increase. On Amazon Web Services, the resource template is called an [instance type](http://aws.amazon.com/ec2/instance-types/); for example, the **t3.large** instance type specifies 2 CPUs and 8 GB of memory. On Google Cloud Platform it's a <a href="https://cloud.google.com/compute/docs/machine-types" rel="noopener">machine type</a><span>, such as the <strong>e2-standard-2</strong> (which also includes 2 CPUs and 8 GB), and on Microsoft Azure it's a</span> <a href="https://docs.microsoft.com/en-us/azure/virtual-machines/sizes" rel="noopener">size</a> (the **Standard_B2ms** calls for the same 2 CPUs and 8 GB). By redefining the instance type, machine type, or size of an existing public cloud host, you can scale its resource specifications. In a VMware vSphere private cloud, you can use the vSphere Client interface to the vCenter Server management console to directly modify one or more individual resource settings of an existing virtual machine. (You can also simultaneously scale groups of hosts on each platform.)

The following sections provide brief examples of resizing individual virtual hosts on the various platforms, with links to the documentation for all available methods. Please note that these methods (APIs, command line interfaces, and portal interfaces) are offered and maintained by the cloud vendors, and the examples included here are for informational purposes, to illustrate how easily you can adapt InterSystems IRIS to take advantage of increased resources 

### AWS

To modify the instance type of an AWS host (called an _instance_, not to be confused with an InterSystems IRIS instance), you can use the [**modify-instance-attribute** CLI command](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ec2/modify-instance-attribute.html), as shown in the following example:

<pre>$ aws ec2 describe-instances --instance-ids i-01519f663af48a55e
{
   "Instances": [
        {
            "AmiLaunchIndex": 0,
            "ImageId": "ami-0abcdef1234567890,
            "InstanceId": "i-1234567890abcdef0,
            "InstanceType": "m5n.large",
            ...
$ aws ec2 stop-instances --instance-ids i-01519f663af48a55e
{
    "StoppingInstances": [
        {
            "InstanceId": "i-1234567890abcdef0",
            ...
$ aws ec2 describe-instances --instance-ids i-01519f663af48a55e
{
   "Instances": [
        {
            ...
            "State": {
            "Code": 80, 
            "Name": "stopped"
            }
            ...
$ aws ec2 modify-instance-attribute --instance-ids i-01519f663af48a55e \
      --instance-type "{\"Value\": \"m5n.xlarge\"}"
$ aws ec2 start-instances --instance-ids i-01519f663af48a55e
{
    "StartingInstances": [
        {
            "InstanceId": "i-1234567890abcdef0",
            "CurrentState": {
                "Code": 0,
                "Name": "pending"
            },
            "PreviousState": {
                "Code": 80,
                "Name": "stopped"
            ...
$ aws ec2 describe-instances --instance-ids i-01519f663af48a55e
{
   "Instances": [
        {
            "AmiLaunchIndex": 0,
            "ImageId": "ami-0abcdef1234567890,
            "InstanceId": "i-1234567890abcdef0,
            "InstanceType": "m5n.xlarge",
            ...</pre>

 You can also make this change using the [ModifyInstanceAttribute](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_ModifyInstanceAttribute.html) AWS API call or the [AWS EC2 console](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-resize.html). 

### GCP

To modify the machine type of a GCP host (also known as an _instance_), you can use the [gcloud CLI](https://cloud.google.com/compute/docs/instances/changing-machine-type-of-stopped-instance#gcloud) to stop, modify, and restart the instance. For example, you would use the following commands to change the machine type of an instance named **scalingTest** to **n1-highmem-96**:

<pre>$ gcloud compute instances stop scalingTest
$ gcloud compute instances set-machine-type scalingTest --machine-type n1-highmem-32
$ gcloud compute instances start scalingTest</pre>

You can also make this change using the [Google Cloud Console](https://cloud.google.com/compute/docs/instances/changing-machine-type-of-stopped-instance#console) or [GCP API](https://cloud.google.com/compute/docs/instances/changing-machine-type-of-stopped-instance#api).

### Azure

When you use the [Azure CLI](https://docs.microsoft.com/en-us/azure/virtual-machines/linux/change-vm-size) to modify the size of a Linux VM , you can view a list of the sizes available on the hardware cluster where the VM is hosted using the **list-vm-resize-options** command, for example:

<pre>az vm list-vm-resize-options --resource-group testingGroup --name scalingTest --output table</pre>

You can then use the **resize** command to change the VM's size to one of the listed options, as shown. This command restarts the VM automatically.

<pre>az vm resize --resource-group <span style="font-size:14px;">testingGroup </span>--name scalingTest --size Standard_E32d_v4</pre>

If the size you want to change the VM to is not available, you can deallocate the VM, which can then be resized to any size supported by the region and restarted; the commands involved are illustrated below:

<pre>az vm deallocate --resource-group <span style="font-size:14px;">testingGroup </span>--name scalingTest
az vm resize --resource-group <span style="font-size:14px;">testingGroup </span>--name scalingTest --size Standard_M128s
az vm start --resource-group <span style="font-size:14px;">testingGroup </span>--name scalingTest</pre>

You can [resize a Windows VM](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/resize-vm) on Azure using either the Azure portal or Powershell commands.

### vSphere

To resize a VMware vSphere VM, do the following:

  1. Open [vSphere Client or Web Client](https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.vm_admin.doc/GUID-588861BB-3A62-4A01-82FD-F9FB42763242.html) and display the VM inventory.
  2. Right-click the VM you want to modify and select **Edit Settings.**
  3. On the **Virtual Hardware tab**, 
      * Expand **Memory** and change the amount of RAM configured for the VM.
      * Expand **CPU** and change the number of cores and optionally the number of cores per socket.
      * Make any other desired changes to the hardware resources allocated to the VM.

## Reconfiguring InterSystems IRIS for Scaled Resources

Once you have scaled the host, the next step is to reconfigure InterSystems IRIS to take advantage of the increased resources by changing one or more parameters in the instance's configuration parameter file (CPF). For example, to continue with the common scenario mentioned at the start of this post, now that you have increased the host's memory resources, you will want to take advantage of this by increasing the size of the InterSystems IRIS instance's database cache (which is done by changing the value of the **[globals](https://docs.intersystems.com/irislatest/csp/docbook/Doc.View.cls?KEY=RACS_globals)** parameter) so it can keep more data in memory.

An easy way to make such a change, and far the easiest and most repeatable way to make multiple changes to an instance's configuration in one operation or make the same changes to multiple instances, is to use the configuration merge feature, which is available on UNIX® and Linux systems. As described in [Using Configuration Merge to Deploy Customized InterSystems IRIS Instances](https://docs.intersystems.com/irislatest/csp/docbook/Doc.View.cls?KEY=ADOCK#ADOCK_iris_customizing) in _Running InterSystems Products in Containers_ and [Using the Configuration Merge Feature](https://docs.intersystems.com/irislatest/csp/docbook/Doc.View.cls?KEY=RACS_cpf#RACS_cpf_edit_merge) in the _Configuration Parameter File Reference_, configuration merge lets you specify a merge file containing the settings you want merged into an instance's CPF immediately prior to a restart.  (In release 2021.1 you'll be able to do this on a running instance without restarting it.) Not only is this more convenient than editing an instance's CPF directly, but it is highly repeatable across multiple instances, and supports reliable change management by enabling you to keep an accurate record of changes simply by versioning the configuration merge files you apply.

To execute a configuration merge, you need to do the following:

  1. Create the merge file with the parameters you want to modify.
  2. Stage the merge file in a location accessible to the instance. If the instance you are modifying is in a container (which is likely on a cloud host), you can stage the file in the instance's durable %SYS directory (see [Durable %SYS for Persistent Instance Data](https://docs.intersystems.com/irislatest/csp/docbook/Doc.View.cls?KEY=ADOCK#ADOCK_iris_durable) in _Running InterSystems Products in Containers_).
  3. Specify the merge file's location using the **ISC\_CPF\_MERGE_FILE** environment variable before restarting the instance.

For example, continuing with the case of the database cache that needs updating, suppose you wanted a containerized instance's database cache size increased to 100 GB. The setting for this, in the **[config]** section of the CPF, would be **globals=102400**, which sets the database cache for 8-kilobyte blocks to 102,400 MB, or 100 GB. (As explained in the **[globals](https://docs.intersystems.com/irislatest/csp/docbook/Doc.View.cls?KEY=RACS_globals)** description in the _Configuration Parameter File Reference,_ the parameter sets the size of the cache for multiple block sizes; if only one value is provided, however, it is applied to the 8-kilobyte block size, and **** [zero] is assumed for the other sizes; **globals=102400** is therefore the equivalent of **globals=0,0,102400,0,0,0**.)

To make this change, you might do the following on the cloud host:

1. <u>Create a configuration merge file</u>, called for example ******mergefile2021.06.30.cpf**, containing these lines:

<pre style="margin-left: 40px;">[config]
globals=102400</pre>

2. <u>Stage the merge file in the durable %SYS directory on the host's file system</u>, which if you mounted the external volume **/data** as **/external** in the container and used the **ISC\_DATA\_DIRECTORY** variable to specify **/external/iris_durable** as the durable %SYS directory for the instance, would be **/data/iris_durable**.

3. <u>Use the <strong>docker exec</strong> command on the host's command line to specify the variable and restart the instance with the <strong>iris</strong> command</u>; if the instance's container is named **iris** and the instance is named **IRIS**, for example, the command would look like this:

<pre style="margin-left: 40px;">docker exec iris ISC_CPF_MERGE_FILE=/data/iris_durable/mergefile2021.06.30.cpf iris stop IRIS restart</pre>

4. When the instance is restarted, you can <u>confirm the new <strong>globals </strong>setting</u> with this command:

<pre style="margin-left: 40px;">docker exec iris grep globals /data/iris_durable/iris.cpf</pre>