Optimizely on Kubernetes
Published: 2021-10-25 12:55:16

## Introduction
This article shows you how to set up the Optimizely CMS sample site using Kubernetes. The article also covers topics such as scaling and load balancing, and some gotchas found when installing.

> The article is based on Google Cloud for hosting the database 
> and provisioning a Kubernetes 
> cluster, but you can of course use Azure or an on-premise
> installation if it suits you better.

## Pre-requisites
This assumes you has the following:
- Kubernetes Cluster (In this article i use Google Kubernetes Engine)
- Azure Servicebus (Standard or Premium tier)
- Azure Storage Account
- SQL Server (2012 or later)
- Linux or WSL2


## Getting the sample site
The sample site used in this article is [found here](https://github.com/episerver/netcore-preview).
Start by cloning the repository: 
```
git clone git@github.com:episerver/netcore-preview.git
cd netcore-preview
```  

The netcore-preview repository contains 2 sample web sites: Quicksilver and Alloy. This article focus on running the Alloy web site. To speed up building the solution, remove the Quicksilver project by running the following command in the same directory as the solution:
```
dotnet sln remove  Quicksilver/EPiServer.Reference.Commerce.Site/EPiServer.Reference.Commerce.Site.csproj
```

## Creating the database
To create a new Optimizely database we first need to install the new EPiserver CLI using the following command:

```
dotnet tool install EPiServer.Net.Cli --global --add-source https://pkgs.dev.azure.com/EpiserverEngineering/netCore/_packaging/beta-program/nuget/v3/index.json --version 1.0.0-pre-020034
```

When the CLI is done installing you can run the following command to create the Optimizely CMS database:

`dotnet-episerver create-cms-database Alloy/AlloyMvcTemplates.csproj -S {SQL SERVER ADDRESS} -U {USER} -P {PASSWORD}`

When the installer is done, the EPiServerDB connectionstring is appended to the appSettings.json 
configuration file located in the root of the project.

To create an admin user run the following command:

 `dotnet-episerver add-admin-user Alloy/AlloyMvcTemplates.csproj -u username -p password -e user@email.com -c EPiServerDB` 


## Configuring the website to run multiple instances
There are some things we must do in order to make the site able to run multiple instances
- Configuring EventProvider
- Configuring BlobStorage 
- Adding Dataprotection


### Adding Azure EventProvider & BlobProvider 
In a loadbalanced website events must be distributed to all instances, this is carried out by the EventProvider. 
In this article I will use Azure Servicebus to distribute the events. Optimizely provides a implementation for replicating events, AzureEventProvider found in the EPiServer.Azure NuGet package.

However we cannot use AzureEventProvider directly when we run on top of Kubernetes since Servicebus SubscriptionNames has a limitation of 50 characters and AzureEventProvider sets SubscriptionName using the following code snippet: `MachineName + Guid.NewGuid().ToString("N")`. 

When running a Kubernetes Deployment, pod names get a hash suffix. This name is used as the machine name, so when using AzureEventProvider the SubscriptionName would be: podname + hash + 32 character guid which is to many characters due to the 50 character restriction in [Azure Service Bus](https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-quotas).

To circumvent this, we create a custom implementation which inherits AzureEventProvider and uses System.Reflection magic to set SubscriptionName. The implementation can be found [here](https://github.com/crikke/epi-ha-k8s-article/blob/master/CustomAzureEventProvider.cs)


Install the required NuGet package by running the following command:
`dotnet add package EPiServer.Azure`

When the NuGet package(s) has been installed you need to add the following code shown below:
Register services in Startup.cs  
```
public void ConfigureServices(IServiceCollection services)
{
  ...
        // register blob storage
        services.AddAzureBlobProvider(o => {
            o.ConnectionString = Environment.GetEnvironmentVariable("BLOBSTORAGE_CONNECTIONSTRING");
        });
        
        // register event provider
        services.AddEventProvider<CustomAzureEventProvider>();
        services.Configure<CustomAzureEventProviderOptions>(o => {
            o.ConnectionString = Environment.GetEnvironmentVariable("SERVICEBUS_CONNECTIONSTRING");
            o.TopicName = "siteevents";
            o.SubscriptionName =  Guid.NewGuid().ToString("N");
        });
  
  ...
}
```

### Configuring Dataprotection 

In order to encrypt/decrypt and validate view-state, a shared key on each instance of our application needs to be used. This was previously done in ASP.NET Framework by setting the `<machineKey>` element under `<system.web>`. In ASP.NET Core, this has been replaced by DataProtection.

To distribute and persist our key to each instance we will use Azure Blobstorage.
Note that in production, you probably want to encrypt the keys at rest aswell. You can read more about this [here](https://docs.microsoft.com/en-us/aspnet/core/security/data-protection/configuration/overview?view=aspnetcore-5.0)

Install required NuGet by running the following command: `dotnet add package Azure.Extensions.AspNetCore.DataProtection.Blobs`
  
Add the code to Startup.cs
```
public void ConfigureServices(IServiceCollection services)
{
  ...
  services
    .AddDataProtection()
    .PersistKeysToAzureBlobStorage(Environment.GetEnvironmentVariable("BLOBSTORAGE_CONNECTIONSTRING"), "keys", "epidemo");
  
  ...
}
```

### Configuring License
Getting a developer license for Optimizely can be done [here](https://license.episerver.com/). 
![License](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uqwsmt8hd2wkbziuit4l.png)

Instead of bundling container specfic configuration with our application, a better way is to mount configuration either as files in an attached volume or as environment variables.
By default, Optimizely checks for the Licence.config file in the app root directory. We need to configure the application to check for the license in another directory where the license will be mounted. 
To configure this add the following to Startup.cs
  
```
public void ConfigureServices(IServiceCollection services)
{
    ...
  
    services.Configure<Licensing.LicensingOptions>(o => {
        o.LicenseFilePath = "/etc/app/License.config";
    });
  
    ...
}
```

## Dockerizing the project   
The final thing we need to do in order to run the site on Kubernetes is to dockerize the application. 


Create a new Dockerfile in the root directory 
```
FROM mcr.microsoft.com/dotnet/sdk:5.0-buster-slim AS build-env
WORKDIR /app

COPY NuGet.config .
COPY dependencies.props .
COPY ./Alloy/AlloyMvcTemplates.csproj   ./Alloy/
COPY *.sln .
RUN dotnet restore

COPY . .
RUN dotnet publish -c Release -o out

FROM mcr.microsoft.com/dotnet/aspnet:5.0-buster-slim
WORKDIR /app
COPY --from=build-env /app/out .

EXPOSE 80

ENTRYPOINT ["dotnet", "AlloyMvcTemplates.dll"]
```

## Deploying the site on Kubernetes

Since our docker image does not contain any ConnectionStrings or Licensefile we need to inject them into the container. This is done by using Kubernetes Secrets & ConfigMap. 
By separating configuration and application the configuration could be reused acros multiple applications. 

### Creating Connection string Secret
Create a new secret containg our connectionstrings by running the following command:
```
kubectl create secret generic epi-connectionstring \ 
--from-literal=eventprovider='{EVENT_PROVIDER_CONNECTION_STRING}' \
--from-literal=blobstorage='{BLOB_STORAGE_CONNECTION_STRING}' \
--from-literal=database='{DB_CONNECTION_STRING}'
```

### Creating a License ConfigMap
Create a new configmap by running the following command:
```
kubectl create configmap epi-demo-license --from-file=License.config=PATH_TO_License.config 
```

### Creating Deployment Manifest
Deployments are used to run a specified number of replicas at any given time. 

Create a new deployment by running the following command:
```
kubectl apply -f https://raw.githubusercontent.com/crikke/epi-ha-k8s-article/master/deployment.yml
```

Run `kubectl get deployments epi-k8s` to verify that the deploy is created

```
NAME      READY   UP-TO-DATE   AVAILABLE   AGE
epi-k8s   2/2     2            2           3h11m
```

### Loadbalancing and making the application reachable outside the cluster
Currently, the application is only reachable within the cluster. In order to expose it to the internet, a Service is needed which routes incomming traffic to pods. Selecting which pod a service should route traffic to is done by using matching Pod labels.

> Labels are key/value pairs that are attached to objects, such as pods. Labels are intended to be used to specify 
> identifying attributes of objects that are meaningful and relevant to users, but do not directly imply semantics to the
> core system. Labels can be used to organize and to select subsets of objects.
>
> Read more about labels at Kubernetes official documentation [here](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/)


Create a new service by running the following command:
```
kubectl apply -f https://github.com/crikke/epi-ha-k8s-article/blob/master/service.yml
```
Verify that the service is created by running `kubectl get service epi-k8s-demo`

You should get similar output as the example shown below:
```
NAME           TYPE           CLUSTER-IP   EXTERNAL-IP    PORT(S)                      AGE
epi-k8s-demo   LoadBalancer   10.44.7.49   34.88.54.254   80:30080/TCP,443:30443/TCP   155m
```

And thats it, you now have Optimizely CMS running in a Kubernetes cluster!

Your application should now be reachable by the `EXTERNAL-IP` field from your service. Note that if it´s `<PENDING>`, wait some minutes and check again. 

If you visit the site through `EXTERNAL-IP`, you could get an 404 error. This is because you first need to add a new site inside the Optimizely Admin UI. Visit EXTERNAL-IP/episerver/cms and go to Admin > Configuration > Manage Sites > Add Site

## Conclusion
The Optimizely Devteam has done massive work of migrating Optimizely from .NET Framework and all underlying dependencies. Not only does this open many possibilities on hosting your application, but brings performance boosts aswell.

I hope this post helped you get some insight how to run Optimizely CMS on Kubernetes. 
Happy coding!