Windows Containers in Azure Kubernetes Services
Tags: azure, kubernetes, devops, winops
Published: 2020-03-08 20:14:44

Kubernetes, or K8s, is the most popular tools to orchestrate containers. Kubernetes is widely available, you can deploy it on-premises or on any Cloud providers or use a managed service like AKS in Azure or Google Cloud.

But what happens when you want to orchestrate Windows Containers. There are many reasons to work with Windows Containers, libraries or languages are only available on Windows, you have too many dependencies or you may not have the knowledge or the time to use Linux. 

Windows containers are here. They are stable and run with dockers. You can publish your app, build your solutions with it and use it like Linux containers. But can you use them in Kubernetes?
Yes, it's possible to use them in Azure with AKS or in Google cloud GKE.

**How does it work in Azure AKS?**

Kubernetes core services, API Engine, DNS, … still need to run on Linux. Every Kubernetes cluster, including those with Windows Containers, need at least one Linux node to run core services. You can add Windows Server to run containers, but the first node needs to be a Linux VM.  

Windows Containers feature in AKS is in preview. Before starting to deploy AKS you will need to configure your workstation and your subscription. 
First, be sure to use the latest version of AZURE CLI. The 2.0.76 version is required to run the Windows Container feature. 

On windows, you can install the [latest MSI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-windows?view=azure-cli-latest#install-or-update) or use  Chocolatey to manage and update the installation.  

On Linux is you are on Debian like distro you can manage the update by APT. 

You will need also to add the AKS-Preview extension 
```
az extension add --name aks-preview
```
And you should try to update to be sure to have the latest version. 
```
az extension update --name aks-preview
```
If you never managed an AKS cluster, install the Kubernetes command-line tools (kubectl)
```
az aks install-cli
```

After that, you will need to update your subscription to register the Windows Container feature. This action changes the behavior of Azure for any new AKS cluster and not only those you want to use with Windows Containers. In other terms, any new cluster you deploy will be in preview mode, without any SLA from Microsoft. Do not choose a subscription with production clusters.

To perform this update. 

```
az feature register --name WindowsPreview --namespace Microsoft.ContainerService
```
You will have to wait a few minutes, enough to go out and grab a coffee or a pizza. To check the result you can use this command.

```
az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/WindowsPreview')].{Name:name,State:properties.state}"
```

Finally, after the registration is completed, you can register the new Microsoft.ContainerService

```
az provider register --namespace Microsoft.ContainerService
```

The AKS cluster configuration is slightly different than normal AKS cluster. The Windows Preview enables multi-node pools. You can have multiple kinds of VM in different pools. Each pool can have different VM Size and OS version. You need tp have a pool for the control plane with small VMs, a pool with bigger VMs on Linux and a pool with Windows 2019 VMs for Windows containers. The Windows Preview also enables a machine scale set for nodes.

There is another important change. If with a classic AKS cluster you have the choice between Azure CNI and KubeNet for the network plugin, the Windows preview limits the choice to Azure CNI only. Azure CNI is a little more complex than KubeNet.

Pools in the same cluster must share the same subnet and VNET and pool names have different requirements for Windows and Linux; 12 characters limit for Linux and 6 for Windows.

Knowing that we can deploy the cluster with the first Linux pool for the control plane.

```
WINPASS="YourRealPassWord"

az aks create \
    --resource-group omc-lab-akswin \
    --name akswin01 \
    --node-count 2 \
    --kubernetes-version 1.15.7 \
    --generate-ssh-keys \
    --windows-admin-password $WINPASS \
    --windows-admin-username adminomc \
    --vm-set-type VirtualMachineScaleSets \
    --load-balancer-sku standard \
    --network-plugin azure \
    --node-vm-size Standard_B2ms \
    --node-osdisk-size 80 \
    --nodepool-name coreaks \
    --docker-bridge-address 172.24.0.1/16 \
    --dns-service-ip 10.10.0.15 \
    --service-cidr 10.10.0.0/24 \
    --dns-name-prefix omc-aks-windows \
      --tags 'env=lab' 'app=Aks Windows'
```

A password is needed for Windows, even if the first action is to deploy the control plane in the Linux pool named coreaks. The password must comply with the Windows 2019 default password policy.

The command deploys a Resource Group, a public IP and a VNET. You can also use your own VNET by providing the subnet ID with the --vnet-subnet-id parameter. Be sure to have enough available IP. 

If you connect to the cluster and list pods, you will not find any Windows pods. 

```
kubectl get nodes
aks-coreaks-17338944-vmss000000   Ready    agent   18m   v1.15.7
aks-coreaks-17338944-vmss000001   Ready    agent   18m   v1.15.7
```
You need to add them 

```
az aks nodepool add \
    --resource-group omc-lab-akswin \
    --cluster-name akswin01 \
    --os-type Windows \
    --name win01 \
    --node-vm-size Standard_B2ms \
    --node-count 1 \
    --kubernetes-version 1.15.7
```

Be careful with the pool name, the maximum number of characters is 6.
Now you can see the Windows pools 

```
kubectl get nodes
NAME                              STATUS   ROLES   AGE     VERSION
aks-coreaks-17338944-vmss000000   Ready    agent   30m     v1.15.7
aks-coreaks-17338944-vmss000001   Ready    agent   30m     v1.15.7
akswin01000000                    Ready    agent   2m17s   v1.15.7
```

From the Azure Portal, you should be able to see theses two node pools in the Setting of your Aks cluster, coreaks and win01. 

Before deploying containers in the cluster, we need to create a namespace. It’s not a requirement and you can deploy whatever you want without a namespace (in the default namespace in fact). But namespace helps to organize resources and projects in a Kubernetes cluster. 

```
kubectl create namespace wintest
```

**Deploying Windows Applications** 

I have a simple windows container, a web site with IIS. Here's the Dockerfile

```dockerfile
FROM microsoft/iis
COPY /site/ /inetpub/wwwroot
```

To deploy the container to the AKS cluster we need to put it in a registry, public or private. Azure provides a private registry service, Azure Container Registry. 
To use it, login to your subscription using AZ Cli then login to the registry.

```
az acr login --name $acrname
```

Then tag the container 

```
docker tag webpage xxxx.azurecr.io/samples/webpage
```

and push it to the registry 

```
docker push xxxx.azurecr.io/samples/webpage
```

But if we need to login to the registry, how to manage login from the AKS cluster? During the cluster installation, the system created a Service Principal to manage VMs and networks in the AKS resource group. 
It’s possible to extract the Client ID using the command line to use it to manage rights in the containers registry. 

```powershell
$AKSClientID=az aks show --resource-group $rgname --name $AksClusterName --query "servicePrincipalProfile.clientId" --output tsv
```
We also need to get the Resource ID of the container registry 

```powershell
$AcrID= az acr show --name $acrname --resource-group $rgname --query "id" --output tsv
```

You can now manage permission on the container registry, pull and read are needed. 

```powershell
az role assignment create --assignee $AKSClientID --role acrpull --scope $AcrID
az role assignment create --assignee $AKSClientID --role reader --scope $AcrID
```

You can start to deploy applications and services in the cluster. For the webpage containers we need to create a deployment, how the container should run (number of pods, port, limit, …) and service (how can we access the application?). 
To be sure to run this application on a Windows server you only need to use nodeSelector property in the Deployment specification.

```yaml
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: webiis
  namespace: wintest
  labels:
    app: webiis
spec:
  replicas: 1
  template:
    metadata:
        name: webiis
        labels:
          app: webiis
    spec:
      nodeSelector:
        "beta.kubernetes.io/os": windows
      containers:
      - name: webiis
        image: xxxxxx.azurecr.io/samples/webpage:latest
        ports:
        - containerPort: 80
        resources:
          limits:
            cpu: 1
            memory: 800M
          requests:
            cpu: .1
            memory: 300M
  selector:
    matchLabels:
      app: webiis
---
apiVersion: v1
kind: Service
metadata:
  name: webiis
spec:
  type: LoadBalancer
  ports:
  - protocol: TCP
    port: 80
  selector:
    app: webiis
```

to apply 

```
kubectl apply -n wintest -f webapp.yaml
```

It should take a few minutes for the pod to be ready. You can monitor the pod with the get pods command. 

```
kubectl get pods -n wintest
```

To monitor the service, you can use the get services command. It will be necessary to get the load balancer IP.

```
kubectl get service -n wintest
```

As you can see, deploying Windows Application on Kubernetes is almost the same thing as deploying Linux applications. There is only one parameter to add to your deployments, the nodeSelector. Windows and Linux applications can coexist in the same cluster.