---
author: Steven Hoang
pubDatetime: 2024-10-12T12:00:00Z
title: "[Az] Day 03: Secret Management and Centralized Log Monitoring on Azure."
featured: false
draft: false
tags:
  - AKS
  - Private
  - Pulumi
description: "
In this tutorial, walk us through the process of establishing a secure and automated system for secret management using Azure Key Vault. Additionally, we will cover how to implement centralized log monitoring using Azure Log Analytics and Application Insights, enhancing observability and operational efficiency.
"
---

## Introduction

In this article, we will discuss the process of establishing a secure and automated system for secret management for the environment.

Additionally, we will cover how to implement centralized log monitoring enhancing observability and operational efficiency.

1. **Secret Management:**  
    Utilizes _Azure Key Vault_ to securely manage and store sensitive information such as credentials, certificates, and secrets. This guarantees consistent and secure access across all applications and services.

2. **Centralized Application Log Management:**  
    Implements a centralized logging system using _Azure Log Analytics_ and _Application Insights_ to collect and analyze logs from all applications. This setup allows for effective performance monitoring, issue troubleshooting, and operational insight maintenance across the environment.

3. **CI/CD flow:**
   ![secrets-management](/assets/az-03-pulumi-private-ask-credential-log-management/secrets-management.png)
    <p class="ml-44"><em>Illustration of the software CI/CD flow</em></p>

   - The purpose of this CI/CD flow is to ensure that environment secrets, especially for production, are automatically generated by Pulumi and stored securely in Azure Key Vault.
   - When developers need to deploy an application, they only need to specify the secret name in the deployment spec.
   - During the deployment process, the CI/CD pipeline will automatically replace the secret name with the actual secret from Key Vault.
   - This approach guarantees that sensitive production secrets remain secure and are not exposed to the team members.

## Table of Contents

## Configuration

Before we start coding, it's important to define our configuration settings. This involves specifying resource names and subnet address spaces that we'll use throughout the project.

### Resource Groups

We categorize our resources into different Azure resource groups for better organization and management:

| Resource Group                  | Description                                             |
|---------------------------------|---------------------------------------------------------|
| **Shared Resource Group**       | Where our Key Vault and logging components reside.      |
| **Hub VNet Resource Group**     | Contains our main VNet hub.                             |
| **AKS VNet Resource Group**     | Contains resources specific to the AKS cluster.         |
| **CloudPC VNet Resource Group** | For resources related to virtual desktops or cloud PCs. |

### Allocated Subnets

Again, this is the subnet Ip address spaces that we have defined in the [previous post](/posts/az-02-pulumi-private-ask-env-architecture#summary-of-allocated-subnets).

| VNet Name           | Subnet Name                    | Address Prefix      | Total | Usable |
|---------------------|--------------------------------|---------------------|-------|--------|
| **1. Hub VNet**     | 1.1 Firewall Subnet            | `192.168.30.0/26`   | 64    | 59     |
|                     | 1.2 Firewall Management Subnet | `192.168.30.64/26`  | 54    | 59     |
|                     | 1.3 General Subnet             | `192.168.30.128/27` | 32    | 27     |
| **2. AKS VNet**     | 2.1 AKS Subnet                 | `192.168.31.0/24`   | 256   | 251    |
| **3. CloudPC VNet** | 3.1 CloudPC Subnet             | `192.168.32.0/25`   | 128   | 123    |
|                     | 3.2 DevOps Subnet              | `192.168.32.128/27` | 32    | 27     |

### The configuration file
Here is the `config.ts` file will be used for all pulumi our projects:

<details><summary><em>View code:</em></summary>

[inline](https://github.com/baoduy/drunk-azure-pulumi-articles/blob/main/config.ts#1-21)

</details>

> **Note:** Adding a number as a prefix to the Azure resource group names helps keep them sorted in sequence, making them easier to find and navigate.

## The `Common` Project

To promote code reusability and maintainability, we create a common project named `az-commons`.
This library contains utilities and helper functions that we'll use across all our Pulumi projects.

### The `azEnv` Module

This module provides functions to retrieve Azure environment configurations:

- **Tenant ID**: Identifies the Azure Active Directory (EntraID) tenant.
- **Subscription ID**: Identifies the Azure subscription where resources will be deployed.
- **Current Principal**: The object ID of the user or service principal executing the scripts.
- **Region Code**: The Azure region.

<details><summary><em>View code:</em></summary>

[inline](https://github.com/baoduy/drunk-azure-pulumi-articles/blob/main/az-commons/azEnv.ts#1-1000)

</details>

### The `naming` Module

This module helps generate resource names with a consistent prefix based on the Pulumi stack name:

- **`getGroupName`**: Prepends the stack name to a resource group name.
- **`getName`**: uses to format the name with convention _{stack}-nameWithoutNumber-{suffix}_ and remove the numbers from the name.

<details><summary><em>View code:</em></summary>

[inline](https://github.com/baoduy/drunk-azure-pulumi-articles/blob/main/az-commons/naming.ts#1-1000)

</details>

### The `stackEnv` Module

This module provides functions to retrieve Pulumi stack environment configurations:

- **`isDryRun`**: Indicates whether the current execution is a dry run (preview) or an actual deployment.
- **Organization**: The Pulumi organization name.
- **Project Name**: The name of the Pulumi project.
- **Stack**: The name of the Pulumi stack.
- **StackReference**: This helper function ensures that a project correctly references stacks within the same organization and environment.
  For example, the `dev` stack of project `az-02-hub-vnet` will reference the `dev` stack of project `az-01-shared`.
  This mechanism prevents cross-environment resource referencing, ensuring that resources from different environments (e.g., dev and prod) are kept isolated and properly aligned within the intended environment.

<details><summary><em>View code:</em></summary>

[inline](https://github.com/baoduy/drunk-azure-pulumi-articles/blob/main/az-commons/stackEnv.ts#1-1000)

</details>

## The `Shared` Project

Following the instructions from [Day 01](/posts/az-01-pulumi-setup-developer-account), we create a new project named `az-01-shared`.
This project will include the following components:

### The `Vault` Module

Creating a `Azure Key Vault` is a secure storage solution for managing secrets, keys, and certificates. It helps safeguard cryptographic keys and secrets used by cloud applications and services.

- **Vault Options**:
  - **enablePurgeProtection**: This option enables purge protection for the Key Vault. When enabled, it prevents the permanent deletion of the vault and its contents for a specified retention period, even if a delete operation is performed. This is crucial for compliance and recovery scenarios.
  - **enabledForDiskEncryption**: This setting allows the Key Vault to be used for Azure Disk Encryption. It is necessary for encrypting virtual machine disks, ensuring that data at rest is protected.
  - **softDeleteRetentionInDays**: This specifies the number of days that deleted vault items (like keys, secrets, and certificates) are retained in a "soft deleted" state. During this period, they can be recovered. The minimum value is 7 days, and the maximum is 90 days.
  - **enableRbacAuthorization:** This enables Role-Based Access Control (RBAC) for managing access to the Key Vault. It requires authentication through EntraID, allowing for more granular and secure access management.

- **Vault Roles Management**: To implement the principle of least privilege, we create two EntraID groups:
  - **AZ ROL DEV-SHARED-VLT READONLY**: For read-only access to the Key Vault.
  - **AZ ROL DEV-SHARED-VLT WRITE**: For write access to the Key Vault.

<details><summary><em>View code:</em></summary>

[inline](https://github.com/baoduy/drunk-azure-pulumi-articles/blob/main/az-01-shared/Vault.ts#1-1000)

</details>

### The `Log` Module

This module provisions a Log Analytics Workspace, which is used for collecting and analyzing telemetry data from various sources, providing insights into resource utilization and performance.

- **Workspace Options**:
  - **immediatePurgeDataOn30Days**: which allows data to be purged immediately after 30 days.
  - **workspaceCapping**: Sets a daily data ingestion quota to control costs and manage data volume.
  - **sku**: Defines the pricing tier for the workspace, which affects cost and features.

<details><summary><em>View code:</em></summary>

[inline](https://github.com/baoduy/drunk-azure-pulumi-articles/blob/main/az-01-shared/Log/Workspace.ts#1-1000)

</details>

### The `AppInsight` module

This module provisions an Application Insights component for monitoring web applications, linking it to a Log Analytics Workspace for data ingestion.

- **AppInsights Options**:
  - **kind** and **applicationType**: Define the type of application being monitored, in this case, a web application.
  - **retentionInDays**: Sets the data retention period to 30 days.
  - **immediatePurgeDataOn30Days**: Allows data to be purged immediately after 30 days.
  - **ingestionMode**: Specifies that data ingestion is done through Log Analytics.

<details><summary><em>View code:</em></summary>

[inline](https://github.com/baoduy/drunk-azure-pulumi-articles/blob/main/az-01-shared/Log/AppInsight.ts#1-1000)

</details>

### The main `index.ts` module: 

This is a main module for the `shared` project, and a similar structure is maintained across all related projects. This file is tasked with:

- Establishing Resource Groups.
- Deploying all the Azure Resources above.
- Exporting essential resource details for future use by other projects.

<details><summary><em>View code:</em></summary>

[inline](https://github.com/baoduy/drunk-azure-pulumi-articles/blob/main/az-01-shared/index.ts#1-1000)

</details>

## Deployment and Cleanup

### Deploying the Stack

To deploy the stack, execute the `pnpm run up` command. This provisions the necessary Azure resources. We can verify the deployment as follows:

- EntraID groups configured for Key Vault access control:
  ![Entra-Group](/assets/az-03-pulumi-private-ask-credential-log-management/az-01-entra-groups.png)

- Successfully deployed Azure resources:
  ![Azure-Resources](/assets/az-03-pulumi-private-ask-credential-log-management/az-01-resources.png)

### Cleaning Up the Stack

To remove the stack and clean up all associated Azure resources, run the `pnpm run destroy` command. This ensures that any resources no longer needed are properly deleted.

## Conclusion

By following this guide, we have successfully automated the deployment of secure `secret management` and centralized `log management` using Azure Key Vault, Log Analytics, and Application Insights with Pulumi.

This setup ensures that sensitive data is securely stored and that we have real-time monitoring of application performance across our environment.

Implementing RBAC and the principle of least privilege enhances the security posture of our infrastructure. Centralized logging enables us to efficiently troubleshoot issues and gain operational insights.

## References

- [az-01-shared](https://github.com/baoduy/drunk-azure-pulumi-articles/blob/main/az-01-shared).
- [az-commons](https://github.com/baoduy/drunk-azure-pulumi-articles/blob/main/az-commons).

## Next

**[Day 04: Develops a Virtual Network Hub for Private AKS on Azure](/posts/az-04-pulumi-private-aks-hub-vnet-development)**

In the next article, We'll walk through the process of developing the first Hub VNet for a private AKS environment using Pulumi.
We will demonstrate how to seamlessly integrate a VNet with an Azure Firewall, along with configuring outbound public IP addresses.

## Thank You

Thank you for taking the time to read this guide! I hope it has been helpful, feel free to explore further, and happy coding! 🌟✨

**Steven** | _[GitHub](https://github.com/baoduy)_