How to use Azure VM metadata service to automate post-provisioning metadata configuration in your IaC for VMSS
Tags: azure, cloudcomputing, vmss, azureservices
Published: 2023-08-10 06:57:50

> **TL;DR: How to use `cloud-init` for Linux VMs and [Azure Custom Script Extension](https://learn.microsoft.com/en-us/azure/virtual-machines/extensions/custom-script-windows) for Windows VMs to create a .env file on the VM containing VM metadata from [Azure VM metadata service](https://learn.microsoft.com/en-us/azure/virtual-machines/instance-metadata-service?tabs=windows) when using Azure VM Scale Sets**


When using [Virtual Machines](https://learn.microsoft.com/en-us/azure/virtual-machines/) or [Virtual Machine Scale Sets](https://learn.microsoft.com/en-us/azure/virtual-machine-scale-sets/overview) on Azure, it often becomes extremely useful to have certain VM metadata accessible to your applications. This type of metadata (like ID, name, private IP, etc.) gets normaly generated at the provisioning time, and having an automated way for applications to access these will come in handy.
 
Azure provides an amazing service called the [Azure VM metadata service](https://learn.microsoft.com/en-us/azure/virtual-machines/instance-metadata-service?tabs=windows), which can be accessed from within a VM to retrieve a all VM specific information.
 
 ```bash
 curl -s -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2021-02-01" | jq
 ```
 
While this command is useful, integrating it into your Infrastructure as Code (IaC) can automate the process and ensure scalability.


In this blog, we'll explore how to package the VM metadata service call into a script, store the metadata in a file, and incorporate this process into both Windows and Linux VMs in a VMSS setup. 

## Creating a Generalized Metadata Retrieval Script

When looking at the VM metadata service endpoint from Azure, everything other than the IP appears to be generic. However, upon closer reading of the Azure documentation, it is mentioned that this "magic" IP is the same for **all** VMs.

> "Azure's instance metadata service is a RESTful endpoint available to all IaaS VMs created via the new Azure Resource Manager. [..] The [VM metadata service] endpoint is available at a well-known non-routable IP address (169.254.169.254) that can be accessed only from within the VM."

This allows us to easily package the call up in a script and output the metadata in our needed format. For the sake of this blog, we will simply create a file that will contain the information we need.

Let's proceed with the implementation details for both Windows and Linux VMs. The full code can be found [here](https://github.com/MahrRah/vmss-vm-metatdata-retrival-sample).


### Windows VMs: Utilizing Azure Custom Script Extension

For Windows VMs, the [Azure Custom Script Extension](https://learn.microsoft.com/en-us/azure/virtual-machines/extensions/custom-script-windows) is a powerful tool to execute post-provisioning scripts. Within the script, we can use the VM metadata service to retrieve the VM name and store it in a file under `C:\` called `vm-metadata.env`.


```powershell
# vm-metadata.ps1vm-metadata.ps1
$vmName = Invoke-RestMethod -Headers @{"Metadata"="true"} -Method GET -Uri "http://169.254.169.254/metadata/instance/compute/name?api-version=2021-02-01&format=text"
"VM_NAME=$vmName" | Out-File -FilePath C:\vm-metadata.env -Append
```

In the IaC definition, the above script can be passed either via an Azure storage account or from GitHub.

```bicep
resource vmss 'Microsoft.Compute/virtualMachineScaleSets@2022-03-01' = {
  name: vmssName
  location: location
  ...
  properties: {
    singlePlacementGroup: null
    platformFaultDomainCount: 1
    virtualMachineProfile: {
      extensionProfile: {
        extensions: [ {
            name: 'CustomScriptExtension'
            properties: {
              publisher: 'Microsoft.Compute'
              type: 'CustomScriptExtension'
              typeHandlerVersion: '1.10'
              settings: {
                commandToExecute: 'powershell -ExecutionPolicy Unrestricted -File vm-metadata.ps1'
                fileUris: [ '<link-to-file>' ]
              }
            }
          } ]
      }
    }
    ...
  }
}
```

### Linux VMs: Harnessing cloud-init

For Linux VMs, leveraging the native [`cloud-init`](https://cloudinit.readthedocs.io/en/latest/) tool simplifies the process.

> Note: We could, however, also use the same [Azure Custom Script Extension](https://learn.microsoft.com/en-us/azure/virtual-machines/extensions/custom-script-windows) as we did for Windows here. Check out the docs for that [here](https://learn.microsoft.com/en-us/azure/virtual-machines/extensions/custom-script-linux).

Amongst many other things, the `cloud-init` definition allows you to specify one or more commands in the `runcmd` section, which should run after the initial startup. Just like for the PowerShell script, the VM metadata is called and the extracted VM name is stored in the `vm-metadata.env` file.

```yaml
#cloud-config
runcmd:
  -  vmName=$(curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance/compute/name?api-version=2021-02-01&format=text") && echo "VM_NAME=${vmName}" >> vm-metadata.env
```

Similar to regular VMs, the VMSS allows you to set the `customData` property when defining your OS profile. It behaves the same way as it does for a VM deployment with `cloud-init`, expecting the file to be passed as a base64-encoded string.


```bicep
param cloudInitScript string = loadFileAsBase64('./cloud-init.yaml')

...

resource vmss 'Microsoft.Compute/virtualMachineScaleSets@2022-03-01' = {
  name: '${prefix}-vmss'
  location: location
  dependsOn: [
    vmssLB
    vmssNSG
  ]
  sku: {
    name: 'Standard_DS1_v2'
    capacity: 1
  }
  properties: {
    singlePlacementGroup: null
    platformFaultDomainCount: 1
    virtualMachineProfile: {
      osProfile: {
        computerNamePrefix: 'vmss'
        adminUsername: 'azureuser'
        adminPassword: adminPassword
        customData: cloudInitScript
      }
      ...
      
    }
    ...
  }
}
```






And with that, you know how to retrieve VM metadata values for your applications from a VM in your VMSS pool in an automatic fashion :)