Cilium with Kubernetes Gateway API on Azure Cloud (BYOCNI)
Tags: azure, cilium, kubernetes
Published: 2023-08-16 19:20:18

## Spinning up the AKS Cluster
I selected the cheapest VM type that I found to keep the costs to minimum for this project

You will need AKS Preview enabled for the commands to work properly

We are planning to use cilium without kube-proxy, the flag ``--kube-proxy-config kube-proxy.json`` wil disable it upon creation.

kube-proxy json config

```json
{
  "enabled": false,
  "mode": "IPVS",
  "ipvsConfig": {
    "scheduler": "LeastConnection",
    "TCPTimeoutSeconds": 900,
    "TCPFINTimeoutSeconds": 120,
    "UDPTimeoutSeconds": 300
  }
}

```
In azure-cli or terminal execute the following command (make sure you are using an already authenticated user):
```bash
export NAME="cilium-sw"
export AZ_RESOURCEGROUP="resource group"
export AZ_LOCATION="ylocation"

az aks create \
    -n "${NAME}" \
    --network-plugin=none \
    -l "${AZ_LOCATION}" \
    -g "${AZ_RESOURCEGROUP}" \
    --kube-proxy-config="kube-proxy.json" \
    --enable-cluster-autoscaler \
    --min-count=1 \
    --max-count=2 \
    --no-wait \
    --node-count=1 \
    --node-vm-size="Standard_B2s" \
    --load-balancer-sku="basic" \
    --generate-ssh-keys
```

### Connecting to the AKS Cluster
```bash
az aks get-credentials --resource-group cilium-eu-1_group --name cilium-eu
```
Coredns & metrics services will be stuck because there is no CNI plugin installed in your cluster, you can verify that by running:
```kubectl get pods -n kube-system```

### Installing resources that are need for Gateway API support
Before we install cilium as we are planning to use the Kubernetes Gateway API we need to isntall following compoents to kubernetes (also you can reference the officials docs: [Gateway API Cilium](https://docs.cilium.io/en/latest/network/servicemesh/gateway-api/gateway-api/#prerequisites))

```bash
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v0.7.0/config/crd/standard/gateway.networking.k8s.io_gatewayclasses.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v0.7.0/config/crd/standard/gateway.networking.k8s.io_gateways.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v0.7.0/config/crd/standard/gateway.networking.k8s.io_httproutes.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v0.7.0/config/crd/standard/gateway.networking.k8s.io_referencegrants.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v0.7.0/config/crd/experimental/gateway.networking.k8s.io_tlsroutes.yaml
```
### Installing Cilium
There are different ways to install cilium according to this articles:

```bash
$ git clone https://github.com/cilium/cilium
$ cd cilium
# Then we will use helm to install cilium with the latest changes #from the main branch
$ cilium install --chart-directory ./install/kubernetes/cilium \
    --set kubeProxyReplacement=true \
    --set gatewayAPI.enabled=true
    --set azure.resourceGroup=${AZ_RESOURCEGROUP}
```
Upon successful installation you should see the following:
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ickv97l7nndx7f654ocj.png)
```kubectl get pods -n kube-system```
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/90njepzpvqg4nwjnkok4.png)

Now your cilium cluster is ready :)