# Integrate Azure Key Vault with MongoDB Client-Side Field Level Encryption

When implementing MongoDB’s client-side field level encryption (CSFLE), you’ll find yourself making an important decision: Where do I store my customer master key? In another tutorial, I guided readers through the basics of CSFLE by using a locally-generated and stored master key. While this works for educational and local development purposes, it isn’t suitable for production! In this tutorial, we’ll see how to use Azure Key Vault to generate and securely store our master key.

## Prerequisites

* A MongoDB Atlas cluster running MongoDB 4.2 (or later) OR MongoDB 4.2 Enterprise Server (or later)—required for automatic encryption
* MongoDB .NET Driver 2.13.0 (or later)
* Mongocryptd
* An Azure Account with an active subscription and the same permissions as those found in any of these Azure AD roles (only one is needed):
    * Application administrator
    * Application developer
    * Cloud application administrator
* An Azure AD tenant (you can use an existing one, assuming you have appropriate permissions)
* Azure CLI
* Cloned sample application

## Quick Jump

**Prepare Azure Infrastructure**

* Register App in Azure Active Directory
* Create a Client Secret
* Create an Azure Key Vault
* Create and Add a Key to your Key Vault
* Grant Application Permissions to Key Vault

**Configure your Client Application to use Azure Key Vault and CSFLE**

* Integrate Azure Key Vault into Your Client Application
* The Results - What You Get After Integrating Azure Key Vault with MongoDB CSFLE

- - -

## Register App in Azure Active Directory

In order to establish a trust relationship between our application and the Microsoft identity platform, we first need to register it. 

1. Sign in to the Azure portal.
2. If you have access to multiple tenants, in the top menu, use the “Directory + subscription filter” to select the tenant in which you want to register an application.
3. In the main search bar, search for and select “Azure Active Directory.”
4. On the left-hand navigation menu, find the Manage section and select “App registrations,” then “+ New registration.”
5. Enter a display name for your application. You can change the display name at any time and multiple app registrations can share the same name. The app registration's automatically generated Application (client) ID, not its display name, uniquely identifies your app within the identity platform.
6. Specify who can use the application, sometimes called its sign-in audience. For this tutorial, I’ve selected “Accounts in this organizational directory only (Default Directory only - Single tenant).” This only allows users that are in my current tenant access to my application.
7. Click “Register.” Once the initial app registration is complete, copy the **Directory (tenant) ID** and **Application (client) ID** as we’ll need them later on.
8. Find the linked application under “Managed application in local directory” and click on it. 
9. Once brought to the “Properties” page, also copy the “**Object ID**” as we’ll need this too.

## Create a Client Secret

Once your application is registered, we’ll need to create a client secret for it. This will be required when authenticating to the Key Vault we’ll be creating soon.

1. On the overview of your newly registered application, click on “Add a certificate or secret”:
2. Under “Client secrets,” click “+ New client secret.”
3. Enter a short description for this client secret and leave the default “Expires” setting of 6 months.
4. Click “Ad." Once the client secret is created, be sure to copy the secret’s “**Value**” as we’ll need it later. It’s also worth mentioning that once you leave this page, the secret value is never displayed again, so be sure to record it at least once!

## Create an Azure Key Vault

Next up, an Azure Key Vault! We’ll create one so we can securely store our customer master key. We’ll be completing these steps via the Azure CLI, so open up your favorite terminal and follow along:

1. Sign in to the Azure CLI using the `az login` command. Finish the authentication steps by following the steps displayed in your terminal.
2. Create a resource group: 

    ``` bash
    az group create --name "YOUR-RESOURCE-GROUP-NAME" --location 
    ```

3. Create a key vault: 

    ``` bash
    az keyvault create --name "YOUR-KEYVAULT-NAME" --resource-group "YOUR-RESOURCE-GROUP-NAME" --location 
    ```

## Create and Add a Key to Your Key Vault

With a key vault, we can now create our customer master key! This will be stored, managed, and secured by Azure Key Vault.

Create a key and add to our key vault:

``` bash
az keyvault key create --vault-name "YOUR-KEYVAULT-NAME" --name "YOUR-KEY-NAME" --protection software
```

The `--protection` parameter designates the key protection type. For now, we'll use the `software` type. Once the command completes, take note of your key’s "**name**" as we‘ll need it later!

## Grant Application Permissions to Key Vault

To enable our client application access to our key vault, some permissions need to be granted:

1. Give your application the wrapKey and unwrapKey permissions to the keyvault. (For the `--object-id` parameter, paste in the Object ID of the application we registered earlier. This is the Object ID we copied in the last "Register App in Azure Active Directory" step.)

    ``` bash
    az keyvault set-policy --name "YOUR-KEYVAULT-NAME" --key-permissions wrapKey unwrapKey --object-id 
    ```

2. Upon success, you’ll receive a JSON object. Find and copy the value for the “**vaultUri**” key. For example, mine is `https://csfle-mdb-demo-vault.vault.azure.net`.

## Integrate Azure Key Vault into Your Client Application

Now that our cloud infrastructure is configured, we can start integrating it into our application. We’ll be referencing the sample repo from our prerequisites for these steps, but feel free to use the portions you need in an existing application.

1. If you haven’t cloned the repo yet, do so now! 

    ``` shell
    git clone https://github.com/adriennetacke/mongodb-csfle-csharp-demo-azure.git
    ```

2. Navigate to the root directory `mongodb-csfle-csharp-demo-azure` and open the `EnvoyMedSys` sample application in Visual Studio.
3. In the Solution Explorer, find and open the `launchSettings.json` file (`Properties` > `launchSettings.json`).
4. Here, you’ll see some scaffolding for some variables. Let’s quickly go over what those are:
    * `MDB_ATLAS_URI`: The connection string to your MongoDB Atlas cluster. This enables us to store our data encryption key, encrypted by Azure Key Vault.
    * `AZURE_TENANT_ID`: Identifies the organization of the Azure account.
    * `AZURE_CLIENT_ID`: Identifies the `clientId` to authenticate your registered application.
    * `AZURE_CLIENT_SECRET`: Used to authenticate your registered application.
    * `AZURE_KEY_NAME`: Name of the Customer Master Key stored in Azure Key Vault.
    * `AZURE_KEYVAULT_ENDPOINT`: URL of the Key Vault. E.g., `yourVaultName.vault.azure.net`.
5. Replace all of the placeholders in the `launchSettings.json` file with **your own information**. Each variable corresponds to a value you were asked to copy and keep track of:
    * `MDB_ATLAS_URI`: Your **Atlas URI****.**
    * `AZURE_TENANT_ID`: **Directory (tenant) ID**.
    * `AZURE_CLIENT_ID`: **Application (client) ID.**
    * `AZURE_CLIENT_SECRET`: Secret **Value** from our client secret.
    * `AZURE_KEY_NAME`: Key **Name**.
    * `AZURE_KEYVAULT_ENDPOINT`: Our Key Vault’s **vaultUri**.
6. Save all your files!

Before we run the application, let’s go over what’s happening: When we run our main program, we set the connection to our Atlas cluster and our key vault’s collection namespace. We then instantiate two helper classes: a `KmsKeyHelper` and an `AutoEncryptHelper`. The `KmsKeyHelper`’s `CreateKeyWithAzureKmsProvider()` method is called to generate our encrypted data encryption key. This is then passed to the `AutoEncryptHelper`’s `EncryptedWriteAndReadAsync()` method to insert a sample document with encrypted fields and properly decrypt it when we need to fetch it. This is all in our `Program.cs` file:

`Program.cs`

``` cs
using System;
using MongoDB.Driver;

namespace EnvoyMedSys
{
    public enum KmsKeyLocation
    {
        Azure,
    }

    class Program
    {
        public static void Main(string] args)
        {
            var connectionString = Environment.GetEnvironmentVariable("MDB_ATLAS_URI");
            var keyVaultNamespace = CollectionNamespace.FromFullName("encryption.__keyVaultTemp");

            var kmsKeyHelper = new KmsKeyHelper(
                connectionString: connectionString,
                keyVaultNamespace: keyVaultNamespace);
            var autoEncryptHelper = new AutoEncryptHelper(
                connectionString: connectionString,
                keyVaultNamespace: keyVaultNamespace);

            var kmsKeyIdBase64 = kmsKeyHelper.CreateKeyWithAzureKmsProvider().GetAwaiter().GetResult();

            autoEncryptHelper.EncryptedWriteAndReadAsync(kmsKeyIdBase64, KmsKeyLocation.Azure).GetAwaiter().GetResult();

            Console.ReadKey();
        }
    }
}
```

Taking a look at the `KmsKeyHelper` class, there are a few important methods: the `CreateKeyWithAzureKmsProvider()` and `GetClientEncryption()` methods. I’ve opted to include comments in the code to make it easier to follow along:

`KmsKeyHelper.cs` / `CreateKeyWithAzureKmsProvider()`

``` cs
public async Task CreateKeyWithAzureKmsProvider()
{
    var kmsProviders = new Dictionary>();

    // Pull Azure Key Vault settings from environment variables
    var azureTenantId = Environment.GetEnvironmentVariable("AZURE_TENANT_ID");
    var azureClientId = Environment.GetEnvironmentVariable("AZURE_CLIENT_ID");
    var azureClientSecret = Environment.GetEnvironmentVariable("AZURE_CLIENT_SECRET");
    var azureIdentityPlatformEndpoint = Environment.GetEnvironmentVariable("AZURE_IDENTIFY_PLATFORM_ENPDOINT"); // Optional, only needed if user is using a non-commercial Azure instance

   // Configure our registered application settings
   var azureKmsOptions = new Dictionary
    {
        { "tenantId", azureTenantId },
        { "clientId", azureClientId },
        { "clientSecret", azureClientSecret },
    };

    if (azureIdentityPlatformEndpoint != null)
    {
        azureKmsOptions.Add("identityPlatformEndpoint", azureIdentityPlatformEndpoint);
    }

    // Specify remote key location; in this case, Azure
    kmsProviders.Add("azure", azureKmsOptions);

    // Constructs our client encryption settings which
    // specify which key vault client, key vault namespace,
    // and KMS providers to use. 
    var clientEncryption = GetClientEncryption(kmsProviders);

    // Set KMS Provider Settings
    // Client uses these settings to discover the master key
    var azureKeyName = Environment.GetEnvironmentVariable("AZURE_KEY_NAME");
    var azureKeyVaultEndpoint = Environment.GetEnvironmentVariable("AZURE_KEYVAULT_ENDPOINT"); // typically .vault.azure.net
    var azureKeyVersion = Environment.GetEnvironmentVariable("AZURE_KEY_VERSION"); // Optional
    var dataKeyOptions = new DataKeyOptions(
        masterKey: new BsonDocument
        {
            { "keyName", azureKeyName },
            { "keyVaultEndpoint", azureKeyVaultEndpoint },
            { "keyVersion", () => azureKeyVersion, azureKeyVersion != null }
        });

    // Create Data Encryption Key
    var dataKeyId = clientEncryption.CreateDataKey("azure", dataKeyOptions, CancellationToken.None);
    Console.WriteLine($"Azure DataKeyId [UUID]: {dataKeyId}");

    var dataKeyIdBase64 = Convert.ToBase64String(GuidConverter.ToBytes(dataKeyId, GuidRepresentation.Standard));
    Console.WriteLine($"Azure DataKeyId [base64]: {dataKeyIdBase64}");

    // Optional validation; checks that key was created successfully
    await ValidateKeyAsync(dataKeyId);

    return dataKeyIdBase64;
}
```

`KmsKeyHelper.cs` / `GetClientEncryption()`

``` cs
private ClientEncryption GetClientEncryption(
Dictionary> kmsProviders)
{
    // Construct a MongoClient using our Atlas connection string
    var keyVaultClient = new MongoClient(_mdbConnectionString);

    // Set MongoClient, key vault namespace, and Azure as KMS provider
    var clientEncryptionOptions = new ClientEncryptionOptions(
     keyVaultClient: keyVaultClient,
        keyVaultNamespace: _keyVaultNamespace,
        kmsProviders: kmsProviders);

    return new ClientEncryption(clientEncryptionOptions);
}
```

With our Azure Key Vault connected and data encryption key encrypted, we’re ready to insert some data into our Atlas cluster! This is where the `AutoEncryptHelper` class comes in. The important method to note here is the `EncryptedReadAndWrite()` method:

`AutoEncryptHelper.cs` / `EncryptedReadAndWrite()`

``` cs
public async Task EncryptedWriteAndReadAsync(string keyIdBase64, KmsKeyLocation kmsKeyLocation)
{
    // Construct a JSON Schema
    var schema = JsonSchemaCreator.CreateJsonSchema(keyIdBase64);

    // Construct an auto-encrypting client
    var autoEncryptingClient = CreateAutoEncryptingClient(
        kmsKeyLocation,
        _keyVaultNamespace,
        schema);

    // Set our working database and collection to medicalRecords.patientData
    var collection = autoEncryptingClient
        .GetDatabase(_medicalRecordsNamespace.DatabaseNamespace.DatabaseName)
        .GetCollection(_medicalRecordsNamespace.CollectionName);

    var ssnQuery = Builders.Filter.Eq("ssn", __sampleSsnValue);

    // Upsert (update if found, otherwise create it) a document into the collection
    var medicalRecordUpdateResult = await collection
        .UpdateOneAsync(ssnQuery, new BsonDocument("$set", __sampleDocFields), new UpdateOptions() { IsUpsert = true });

    if (!medicalRecordUpdateResult.UpsertedId.IsBsonNull)
    {
        Console.WriteLine("Successfully upserted the sample document!");
    }

    // Query by SSN field with auto-encrypting client
    var result = await collection.Find(ssnQuery).SingleAsync();

    // Proper result in console should show decrypted, human-readable document
    Console.WriteLine($"Encrypted client query by the SSN (deterministically-encrypted) field:\n {result}\n");
}
```

Now that we know what’s going on, run your application!

## The Results: What You Get After Integrating Azure Key Vault with MongoDB CSFLE

If all goes well, your console will print out two `DataKeyIds` (UUID and base64) and a document that resembles the following: 

`Sample Result Document (using my information)`

``` bash
{
    _id:UUID('ab382f3e-bc79-4086-8418-836a877efff3'),
keyMaterial:Binary('tvehP03XhUsztKr69lxlaGjiPhsNPjy6xLhNOLTpe4pYMeGjMIwvvZkzrwLRCHdaB3vqi9KKe6/P5xvjwlVHacQ1z9oFIwFbp9nk...', 0),
    creationDate:2021-08-24T05:01:34.369+00:00,
    updateDate:2021-08-24T05:01:34.369+00:00,
    status:0,
    masterKey:Object,
    provider:"azure",
    keyVaultEndpoint:"csfle-mdb-demo-vault.vault.azure.net",
    keyName:"MainKey"
}
```

Here’s what my console output looks like, for reference:

![Screenshot of console output showing two Azure DatakeyIds and a non-formatted document

Seeing this is great news! A lot of things have just happened, and all of them are good:

* Our application properly authenticated to our Azure Key Vault.
* A properly generated data encryption key was created by our client application.
* The data encryption key was properly encrypted by our customer master key that’s securely stored in Azure Key Vault.
* The encrypted data encryption key was returned to our application and stored in our MongoDB Atlas cluster.

Here’s the same process in a workflow:

After a few more moments, and upon success, you’ll see a “Successfully upserted the sample document!” message, followed by the properly decrypted results of a test query. Again, here’s my console output for reference:

This means our sample document was properly encrypted, inserted into our `patientData` collection, queried with our auto-encrypting client by SSN, and had all relevant fields correctly decrypted before returning them to our console. How neat! 

And just because I’m a little paranoid, we can double-check that our data has actually been encrypted. If you log into your Atlas cluster and navigate to the `patientData` collection, you’ll see that our documents’ sensitive fields are all illegible:

## Let's Summarize

That wasn’t so bad, right? Let's see what we've accomplished! This tutorial walked you through:

* Registering an App in Azure Active Directory.
* Creating a Client Secret.
* Creating an Azure Key Vault.
* Creating and Adding a Key to your Key Vault.
* Granting Application Permissions to Key Vault.
* Integrating Azure Key Vault into Your Client Application.
* The Results: What You Get After Integrating Azure Key Vault with MongoDB CSFLE.

By using a remote key management system like Azure Key Vault, you gain access to many benefits over using a local filesystem. The most important of these is the secure storage of the key, reduced risk of access permission issues, and easier portability!

For more information, check out this helpful list of resources I used while preparing this tutorial:

* az keyvault command list
* Registering an application with the Microsoft Identity Platform
* MongoDB CSFLE and Azure Key Vault

And if you have any questions or need some additional help, be sure to check us out on the MongoDB Community Forums and start a topic!

A whole community of MongoDB engineers (including the DevRel team) and fellow developers are sure to help!