Deploying Minimum Viable Dataspace to Azure: A Step-by-Step Guide
Tags: azure, edc
Published: 2023-04-19 04:39:30

## What is Minimum Viable Dataspace (MVD)?

The Minimum Viable Dataspace (MVD) is a sample implementation of a dataspace that leverages the Eclipse Dataspace Components (EDC). The main purpose is to demonstrate the capabilities of the EDC, make dataspace concepts tangible based on a specific implementation, and to serve as a starting point to implement a custom dataspace. 

Since I will not explain what EDC is in this article, I recommend that those who want to learn about EDC first refer to [the official Eclipse Foundation documents](https://projects.eclipse.org/projects/technology.edc).

## Azure deployment for MVD

In the past, there was an Azure deployment workflow on the MVD repository. However, it does not exist on the repo anymore.

There were some reasons for removing the deployment to Azure. Please refer to [this GitHub issue](https://github.com/eclipse-edc/MinimumViableDataspace/issues/128) if you want to know more details.

Some of you may still want to deploy MVD to Azure like me. In this article, how to achieve Azure deployment with the previous workflow will be illustrated.

### Table of Contents
- [Prerequisites](#prerequisites)
- [Initializing an Azure environment for CD](#initializing-an-azure-environment-for-cd)
  - [1. Create a service identity for GitHub Actions](#1-create-a-service-identity-for-github-actions)
      - [1-1. Create App Registration for GitHub Actions](#11-create-app-registration-for-github-actions)
      - [1-2. Configure main Branch Credentials](#12-configure-raw-main-endraw-branch-credentials)
      - [1-3. Configure Pull Request Credentials](#13-configure-pull-request-credentials)
      - [1-4. Create a new application secret](#14-create-a-new-application-secret)
      - [1-5. Grant Permissions for Azure Subscription](#15-grant-permissions-for-azure-subscription)
      - [1-6. Configure GitHub Secrets for GitHub Actions](#16-configure-github-secrets-for-github-actions)
  - [2. Create Service Identity for MVD Runtimes](#2-create-service-identity-for-mvd-runtimes)
      - [2-1. Create App Registration for MVD Runtimes](#21-create-app-registration-for-mvd-runtimes)
      - [2-2. Create Client Secret for MVD Runtimes](#22-create-client-secret-for-mvd-runtimes)
      - [2-3. Get Application Object ID](#23-get-application-object-id)
      - [2-4. Configure GitHub Secrets for MVD Runtimes](#24-configure-github-secrets-for-mvd-runtimes)
  - [3. Configure CD Settings](#3-configure-cd-settings)
  - [4. Deploying CD resources](#4-deploying-cd-resources)
- [Create a new dataspace instance](#create-a-new-dataspace-instance)
- [Ready to use MVD!](#ready-to-use-mvd)
- [Conclusion](#conclusion)

## Prerequisites

Before proceeding to the actual preparation of the environment, we must first revert to [the old commit where the Deploy pipeline for Azure existed](https://github.com/eclipse-edc/MinimumViableDataspace/commit/9362f36f09123d7ceca67b8239cae2b82bacca3a).

- A fork of the [MVD repository](https://github.com/eclipse-dataspaceconnector/MinimumViableDataspace) 

- Back to a previous commit

Please run the following commands on your local to reset the code in your fork to [a previous commit where the Deploy pipeline exists](https://github.com/eclipse-edc/MinimumViableDataspace/commit/9362f36f09123d7ceca67b8239cae2b82bacca3a). The commit id should be `9362f36f09123d7ceca67b8239cae2b82bacca3a`.

```console
git log
git reset --hard (COMMIT-ID)
git push REMOTE-NAME LOCAL-BRANCH-NAME:REMOTE-BRANCH-NAME
```
If a previous commit is successfully backed you will find that there are 6 yaml files (cd.yaml, check.yaml, cloud-cd.yaml, **deploy.yaml**, destroy.yaml, and initialize.yaml) in the workflow folder.

- Prepare an Azure subscription

A GitHub workflow then needs to be run to provision the Azure resources used for CD.

## Initializing an Azure environment for CD
### 1. Create a service identity for GitHub Actions

Further documentation for the following steps can be found in the Microsoft Docs to [Configure an app to trust a GitHub repo](https://docs.microsoft.com/azure/active-directory/develop/workload-identity-federation-create-trust-github).

#### 1-1. Create App Registration for GitHub Actions
Sign in to the [Azure Portal](https://portal.azure.com/), navigate to *App registrations* and select *New registration*.

Register a new application:

- **Name**: Provide a display name for the application, e.g. *"MVD GitHub Actions App"*
- In **Supported Account Types**, select **Accounts in this organizational directory only**.
- Don't enter anything for **Redirect URI (optional)**.

Select **Register** to create the app registration.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/822e77i1sc5az6b15mt7.png)

Take note of the *Application (client) ID* (will be required to configure a GitHub secret below).

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vhpfydbt1igna9du9f1y.png)

Next, we create two types of credentials: two federated credentials to authenticate GitHub Actions, and a client secret for Terraform (required as Terraform does not yet support Azure CLI login with a service principal).
Follow the instructions to [Configure a federated identity credential](https://docs.microsoft.com/azure/active-directory/develop/workload-identity-federation-create-trust-github?tabs=azure-portal#configure-a-federated-identity-credential) for the `main` branch and Pull requests.

#### 1-2. Configure main Branch Credentials

Select the previously created application (e.g. *"MVD GitHub Actions App"*) and navigate to **Certificates & secrets**.

Select **Federated credentials**, click **Add credential** and define the following values.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k75fvah1iofqyhezvk11.png)

- For **Federated credential scenario**, select **GitHub actions deploying Azure resources**.
- For **Organization**, add your GitHub organization. For example, your organization is *YourGitHubOrg* if the URL of your GitHub repository is [https://github.com/YourGitHubOrg/MinimumViableDataspace](https://github.com/YourGitHubOrg/MinimumViableDataspace).
- For **Entity Type**, select **Branch**.
- For **GitHub branch name**, enter `main`.
- For **Name**, enter a name for the credential, e.g. *"mvd-main-branch"*.

Click **Add** to create the credential.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rsoykq748er0h761xbvh.png)

>Note: You can add additional credentials to deploy from other branches.

#### 1-3. Configure Pull Request Credentials

Now, we set up a federated credential for pull requests, which allows to run a cloud deployment to validate pull requests.

>Note: This step is only required if you plan to create pull requests for the MVD.

Click **Add credential** to add another credential (for the previously created application (e.g. *"MVD GitHub Actions App"*), under **Certificates & secrets**, **Federated credentials**).

- For **Federated credential scenario**, select **GitHub actions deploying Azure resources**.
- For **Organization**, add your GitHub organization. For example, your organization is *YourGitHubOrg* if the URL of your GitHub repository is [https://github.com/YourGitHubOrg/MinimumViableDataspace](https://github.com/YourGitHubOrg/MinimumViableDataspace).
- For **Entity Type**, select **Pull Request**.
- For **Name**, enter a name for the credential, e.g. *"mvd-pull-requests"*.

Click **Add** to create the credential.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yuje19s09w0mv9s3linl.png)

#### 1-4. Create a new application secret

Create a client secret by following the section "Create a new application secret" in the page on [Creating a an Azure AD application to access resources](https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal#option-2-create-a-new-application-secret). 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6i3nxnlgi9fmfp99kn21.png)

Take note of the client secret and keep it safe.

#### 1-5. Grant Permissions for Azure Subscription

To allow GitHub Actions to deploy resources to your Azure subscription, grant the application created above Owner permissions on your Azure subscription.
Further documentation for the following steps can be found under [Assign Azure roles using the Azure portal](https://docs.microsoft.com/azure/role-based-access-control/role-assignments-portal).

Navigate to your **subscription** you want to deploy the MVD resources to, select **Access control (IAM)** and click on **Add role assignment**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tl8rtldd8oel0ymvx6jv.png)

On the **Add role assignment** page, select **Owner** and click **Next**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dcei27w4wogdmipgr7ts.png)

Now click on **Select members**, search for the application created above (e.g. *"MVD GitHub Actions App"*), click **Select** and then click on **Next** and then on **Review + assign** to assign the application Owner permissions on your Azure subscription.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6dmqxlibqzt8jipaorv2.png)

>You need to enter the full application name when searching for the application, the application will not show up if you only enter a partial name (e.g. *"MVD GitHub Act"* in the example above).

#### 1-6. Configure GitHub Secrets for GitHub Actions

Finally, the application (client) ID needs to be made available to your GitHub repository using [GitHub secrets](https://docs.github.com/en/actions/security-guides/encrypted-secrets).

To configure GitHub Secrets, navigate to your `MinimumViableDataspace` repository, select **Settings**, navigate to **Secrets** and then **Actions**, and click **New repository secret** to create a new secret.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xdyxrjkyr19e9sv6omxq.png)

Configure the following GitHub secrets with the value from the steps above:

| Secret name         | Value                          |
| ------------------- | ------------------------------ |
| `ARM_CLIENT_ID`     | The application (client) ID of the application created above (e.g. *"MVD GitHub Actions App"*). |
| `ARM_CLIENT_SECRET` | The application client secret. |

### 2. Create Service Identity for MVD Runtimes

Further documentation for the following steps can be found in the Microsoft Docs to [Create and configure an Azure AD application for the application runtimes](https://docs.microsoft.com/azure/active-directory/develop/workload-identity-federation-create-trust-github).

#### 2-1. Create App Registration for MVD Runtimes

Sign in to the [Azure Portal](https://portal.azure.com/), navigate to *App registrations* and select *New registration*.

Register a new application:

- **Name**: Provide a display name for the application, e.g. *"MVD Runtimes App"*
- In **Supported Account Types**, select **Accounts in this organizational directory only**.
- Don't enter anything for **Redirect URI (optional)**.

Select **Register** to create the app registration.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/829u4wq3enuyqx30tcm0.png)

Take note of the *Application (client) ID* (will be required to configure a GitHub secret below).

#### 2-2. Create Client Secret for MVD Runtimes

Navigate to **Certificates & secrets** and then to the **Client secrets** tab (for the previously created application (e.g. *"MVD Runtimes App"*), and select **New client secret**. Create a new client secret by entering a **Description** (e.g. *"mvd-runtimes-app-client-secret"*) and clicking **Add**.

Take note of the client secret (**Value**) and keep it safe (will be required to configure a GitHub secret below).

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h4dpctad0cfmydayodea.png)

#### 2-3. Get Application Object ID

Navigate to **Azure Active Directory** and select **Enterprise Applications**.

Take note of the enterprise application **Object ID**.

>Make sure you use the Object ID of the *Enterprise application*, and not the Object ID of the *App Registration*!

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9r8yf3kpr0xb3rvu5yzd.png)

#### 2-4. Configure GitHub Secrets for MVD Runtimes

Configure the following GitHub secrets with the values from the steps above:

| Secret name         | Value                          |
| ------------------- | ------------------------------ |
| `APP_CLIENT_ID`     | The application (client) ID.   |
| `APP_CLIENT_SECRET` | The application client secret. |
| `APP_OBJECT_ID`     | The ID of the service principal object associated with this application. |

See instructions under [Configure GitHub Secrets for GitHub Actions](#configure-github-secrets-for-github-actions) on how to configure GitHub secrets.

### 3. Configure CD Settings

Configure the following GitHub secrets which are required by the CD pipeline:

| Secret name                   | Value                                                        |
| ----------------------------- | ------------------------------------------------------------ |
| `ARM_TENANT_ID`               | The Azure Active Directory **Tenant ID**. Navigate to Azure Active Directory and copy the Tenant ID from the *Overview* page. |
| `ARM_SUBSCRIPTION_ID`         | The Azure **Subscription ID** to deploy resources to. Navigate to Subscriptions and copy the *Subscription ID* of your subscription. |
| `COMMON_RESOURCE_GROUP`       | The Azure resource group name to deploy common resources to, such as Azure Container Registry. Choose any valid resource group name, e.g. *rg-mvd-common*. |
| `COMMON_RESOURCE_GROUP_LOCATION` | The location where common resources should be deployed to, e.g. *eastus*. |
| `ACR_NAME`                    | The name of the Azure Container Registry to deploy. Use only lowercase letters and numbers. |
| `TERRAFORM_STATE_STORAGE_ACCOUNT` | The name of the storage account used to store the Terraform state container, e.g. *mvdterraformstates*. |
| `TERRAFORM_STATE_CONTAINER` | The name of the container used to store the Terraform state blob, e.g. *mvdterraformstates*. |

### 4. Deploying CD resources

Common resources need to be deployed once, these resources will be used by all CD pipelines.

Manually run the `Initialize CD` GitHub Actions workflow and make sure that it passes successfully.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xq2huo8zakugufweltz2.png)

>Since `workflow_dispatch` is already in deploy pipeline a ‘Run workflow’ button should be on the Actions tab, enabling you to easily trigger a run. ([GitHub Actions: Manual triggers with workflow_dispatch
](https://github.blog/changelog/2020-07-06-github-actions-manual-triggers-with-workflow_dispatch/)) If you cannot find the button like the screenshot below I would suggest you to try changing the file extension, from *deploy.yaml* to *deploy.yml*.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/w774tvaciqjfwmwrebv0.png)

Your infrastructure is now set up to run deployments, you can now run the `Deploy` GitHub Actions workflow in the next step.

## Create a new dataspace instance
Once your environment is set up, follow these steps to create a new dataspace instance:

Select the `Deploy` GitHub Actions workflow and provide your own resources name prefix. Please, use at most 3 characters, composed of lower case letters and numbers.

Click on `Run workflow` to manually run the `Deploy` GitHub Actions workflow.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/i859gk54cv5d9gyt3kpk.png)

Make sure that it passes successfully.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tnsduo8x3yjnankkcq7j.png)

## Ready to use MVD!
Once Deploy pipeline passes successfully then you can now access to MVD which just has been deployed with your own resources name prefix.

The Data Dashboard is a web application (development UI) on top of EDC's DataManagementAPI and is deployed for each participant. It can be accessed at the URLs provided in the GitHub workflow run page like below. 
For instance, if the name prefix was `dragon`:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/r4jpjjw25gqxwulz0on7.png)

You will have three Data Dashboards for three participants: company 1, company 2 and company 3 as below.

*company 1*
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8vjr80027bwkabvj35ql.png)

*company 2*
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mddfxqxc0kvcx6cfarjg.png)

*company 3*
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vlb1g7jjkzuzu66x0rgv.png)

If you want to demonstrate the possibilities of Dataspaces please refer to [Vision Demonstrator document](https://github.com/eclipse-edc/MinimumViableDataspace/blob/main/Vision%20Demonstrator/Vision%20Demonstrator%20Introduction.md).

## Conclusion

Azure deployment workflow has been removed from the MVD repo but it still can be deployed with the previous commit. Please be aware of that it will not be updated anymore.
Hope this step-by-step guide is helpful for some of you who want to try MVD on Azure.