Project: STUNner (Orchestration & Management / Service Proxy)
File: README.md

This guide shows how you can use inside a Kubernetes cluster, using STUNner as a WebRTC gateway. Neko uses WebRTC to stream a desktop inside of a docker container to your browser. However, integrating Neko into Kubernetes is far from trivial. In this demo you will learn the following steps to: integrate a typical WebRTC application server to be used with STUNner, deploy Neko into Kubernetes behind STUNner, Consult the to set up STUNner. The simplest way to deploy the demo is to clone the and deploy the packaged with STUNner. Install the STUNner gateway operator and STUNner (): ```console helm repo add stunner https://l7mp.io/stunner helm repo update helm install stunner-gateway-operator stunner/stunner-gateway-operator --create-namespace --namespace=stunner-system ``` Configure STUNner to act as a STUN server towards clients, and to let media reach the media server. ```console git clone https://github.com/l7mp/stunner cd stunner/docs/examples/neko kubectl apply -f stunner.yaml ``` This will expose STUNner on a public IP on UDP port 3478. A Kubernetes `LoadBalancer` assigns an ephemeral public IP address to the service, so first we need to learn the external IP. ```console kubectl get service udp-gateway -n default -o jsonpath='{.status.loadBalancer.ingress[0].ip}' STUNNERIP=$(kubectl get service udp-gateway -n default -o jsonpath='{.status.loadBalancer.ingress[0].ip}') ``` Note This IP should be accessible from your browser. If that "public IP" is behind a NAT, you can overwrite it with the actual public IP that routes to the service by hand (e.g. `STUNNERIP=<your public IP>`). We need to give this public IP the Neko configuration in the `NEKO_ICESERVERS` environment variable, inside the `json` content (basically this will tell you browser to use STUNner as a STUN/TURN server). You can do that by hand, or by this fancy `sed` command: ```console sed -i "s/turn:[\.0-9]*:3478/turn:$STUNNERIP:3478/g" neko.yaml ``` Now apply the Neko manifests and wait for the `neko` deployment to be available (should take a couple of seconds): ```console kubectl apply -f neko.yaml kubectl wait --for=condition=Available deployment neko --timeout 5m ``` In this setup we use `ingress` to expose the Neko UI. Feel free to customize the `ingress` resource to your setup. If you don't have an ingress controller, you can use the `neko-tcp` service with a `LoadBalancer` type. Ideally, by opening your ingress controller in your browser, you should see the Neko UI. You can log in with the `admin`:`admin` credentials. The WebRTC stream then should be relayed through STUNner. [!NOTE] Tested with Chromium/Google Chrome. STUNner development is coordinated in Discord, feel free to . ients with a valid TLS cert. This will have the unfortunate consequence that the majority of the below installation guide will be about securing client connections to LiveKit over TLS; as it turns out, once HTTPS is correctly working integrating LiveKit with STUNner is very simple. In the below example, STUNner will be installed into the identically named namespace, while LiveKit and the Ingress gateway will live in the default namespace. As mentioned above, the LiveKit server will need a valid TLS cert, which means it must run behind an existing DNS domain name backed by a CA signed TLS

This is simple if you have your own domain, but if you don't then provides a dead simple wildcard DNS for any IP address. We will use this to "own a domain" and obtain a CA signed certificate for LiveKit. This will allow us to point the domain name `client-<ingress-IP>.nip.io` to an ingress HTTP gateway in our Kubernetes cluster, which will then use some automation (namely, cert-manager) to obtain a valid CA signed cert. Note that public wildcard DNS domains might run into issues. If this occurs you can try instead of `nip.io`. The first step of obtaining a valid cert is to install a Kubernetes Ingress: this will be used during the validation of our certificates and to terminate client TLS encrypted contexts. Install an ingress controller into your cluster. We used the official , but this is not required. ```console helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx helm repo update helm install ingress-nginx ingress-nginx/ingress-nginx ``` Wait until Kubernetes assigns an external IP to the Ingress. ```console until [ -n "$(kubectl get service ingress-nginx-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}')" ]; do sleep 1; done ``` Store the Ingress IP address Kubernetes assigned to our Ingress; this will be needed later when we configure the validation pipeline for our TLS certs. ```console kubectl get service ingress-nginx-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}' export INGRESSIP=$(kubectl get service ingress-nginx-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}') export INGRESSIP=$(echo $INGRESSIP | sed 's/\./-/g') ``` We use the official to automate TLS certificate management. Add the Helm repository, which contains the cert-manager Helm chart, and install the charts: ```console helm repo add cert-manager https://charts.jetstack.io helm repo update helm install cert-manager jetstack/cert-manager --namespace cert-manager \ --create-namespace --set global.leaderElection.namespace=cert-manager \ --set installCRDs=true --timeout 600s ``` At this point we have all the necessary boilerplate set up to automate TLS issuance for LiveKit. Now comes the fun part. The simplest way to run this demo is to clone the and deploy the packaged with STUNner. Install the STUNner gateway operator and STUNner via : Legacy mode: ```console helm repo add stunner https://l7mp.io/stunner helm repo update helm install stunner-gateway-operator stunner/stunner-gateway-operator --create-namespace --namespace=stunner-system --set stunnerGatewayOperator.dataplane.mode=legacy helm install stunner stunner/stunner --create-namespace --namespace=stunner ``` Managed mode: ```console helm repo add stunner https://l7mp.io/stunner helm repo update helm install stunner-gateway-operator stunner/stunner-gateway-operator --create-namespace --namespace=stunner-system ``` Configure STUNner to act as a STUN/TURN server to clients, and route all received media to the LiveKit server pods. ```console git clone https://github.com/l7mp/stunner cd stunner kubectl apply -f docs/examples/livekit/livekit-call-stunner.yaml ``` The relevant parts here are the STUNner , which exposes the STUNner STUN/TURN server over UDP:3478 to the Internet, and the , which takes care of routing media to the pods running the LiveKit service. ```yaml apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: udp-gateway namespace: stunner spec: gatewayClassName: stunner-gatewayclass listeners: name: udp-listener port: 3478 protocol: UDP apiVersion: stunner.l7mp.io/v1 kind: UDPRoute metadata: name: livekit-media-plane namespace: stunner spec: parentRefs: name: udp-gateway rules: backendRefs: name: livekit-server ``` Once the Gateway resource is installed into Kubernetes, STUNner will create a Kubernetes LoadBalancer for the Gateway to expose the TURN server on UDP:3478 to clients. It can take up to a minute for Kubernetes to allocate a public external IP for the

Wait until Kubernetes assigns an external IP and store the external IP assigned by Kubernetes to STUNner in an environment variable for later use. ```console until [ -n "$(kubectl get svc udp-gateway -n stunner -o jsonpath='{.status.loadBalancer.ingress[0].ip}')" ]; do sleep 1; done export STUNNERIP=$(kubectl get service udp-gateway -n stunner -o jsonpath='{.status.loadBalancer.ingress[0].ip}') ``` The crucial step of integrating any WebRTC media server with STUNner is to ensure that the server instructs the clients to use STUNner as the STUN/TURN server. In order to achieve this, first we path the public IP address of the STUNner STUN/TURN server we have learned above into our LiveKit deployment manifest: ```console sed -i "s/stunner_ip/$STUNNERIP/g" docs/examples/livekit/livekit-server.yaml ``` Assuming that Kubernetes assigns the IP address 1.2.3.4 to STUNner (i.e., `STUNNERIP=1.2.3.4`), the relevant part of the LiveKit config would be something like the below: ```yaml ... rtc: ... turn_servers: host: 1.2.3.4 username: user-1 credential: pass-1 protocol: udp port: 3478 ``` This will make sure that LiveKit is started with STUNner as the STUN/TURN server. If unsure about the STUNner settings to use, you can always use the handy to dump the running STUNner configuration. ``` console stunnerctl -n stunner config udp-gateway Gateway: stunner/udp-gateway (loglevel: "all:INFO") Authentication type: static, username/password: user-1/pass-1 Listeners: Name: stunner/udp-gateway/udp-listener Protocol: TURN-UDP Public address:port: 34.118.88.91:3478 Routes: [stunner/iperf-server] Endpoints: [10.76.1.4, 10.80.4.47] ``` Note that LiveKit itself will not use STUNner (that would amount to a less efficient ); with the above configuration we are just telling LiveKit to instruct its clients to use STUNner to reach the LiveKit media servers. We also need the Ingress external IP address we have stored previously: this will make sure that the TLS certificate created by cert-manager will be bound to the proper `nip.io` domain and IP address. ```console sed -i "s/ingressserviceip/$INGRESSIP/g" docs/examples/livekit/livekit-server.yaml ``` Finally, fire up LiveKit. ```console kubectl apply -f docs/examples/livekit/livekit-server.yaml ``` The demo installation bundle includes a lot of resources to deploy LiveKit: a LiveKit-server, a web server serving the landing page using a cluster issuer for the TLS certificates, an Ingress resource to terminate the secure connections between your browser and the Kubernetes cluster. Wait until all pods become operational and jump right into testing! After installing everything, execute the following command to retrieve the URL of your fresh LiveKit demo app: ```console echo client-$INGRESSIP.nip.io ``` Copy the URL into your browser, and now you should be greeted with the LiveKit Video title. On the landing page you must set the LiveKit URL, which is the LiveKit server's IP address, or in our case the other subdomain we set earlier in the Ingress manifest. Executing the following command shall get you the required URL: ```console echo wss://mediaserver-$INGRESSIP.nip.io:443 ``` To obtain a valid token, install the and issue the below command. ```console livekit-cli create-token \ --api-key access_token --api-secret secretsecretsecretsecretsecretsecret \ --join --room room --identity user1 \ --valid-for 24h ``` Copy the access token into the token field and hit the Connect button. If everything is set up correctly, you should be able to connect to a room. If you repeat the procedure in a separate browser tab you can enjoy a nice video-conferencing session with yourself, with the twist that all media between the browser tabs is flowing through STUNner and the LiveKit-server deployed in you Kubernetes cluster.