Project: Kube-router (Runtime / Cloud Native Network)
File: generic.md

This guide is for running kube-router as the network provider for on premise and/or bare metal clusters outside of a cloud provider's environment. It assumes the initial cluster is bootstrapped and a networking provider needs configuration. All pod networking are allocated by kube-controller-manager. Kube-router provides service/pod networking, a network policy firewall, and a high performance based service proxy. The network policy firewall and service proxy are both optional but recommended. If you choose to run kube-router as daemonset, then both kube-apiserver and kubelet must be run with `--allow-privileged=true` option (see our ) Ensure your is configured to point its CNI configuration directory to `/etc/cni/net.d`. This is the default location for both `containerd` and `cri-o`, but can be set specifically if needed: Here is what the default containerd CNI plugin configuration looks like as of the writing of this document. The default containerd configuration can be retrieved using: ```sh containerd config default ``` ```toml [plugins] [plugins."io.containerd.grpc.v1.cri".cni] bin_dir = "/opt/cni/bin" conf_dir = "/etc/cni/net.d" conf_template = "" ip_pref = "" maxconfnum = 1 ``` cri-o CRI configuration can be referenced via their If a previous CNI provider (e.g. weave-net, calico, or flannel) was used, remove old configurations from `/etc/cni/net.d` on each kubelet. If you choose to use kube-router for pod-to-pod network connectivity then needs to be configured to allocate pod CIDRs by passing the `--allocate-node-cidrs=true` flag and providing a `cluster-cidr` (e.g. by passing `--cluster-cidr=10.32.0.0/12`) For example: ```sh --allocate-node-cidrs=true --cluster-cidr=10.32.0.0/12 --service-cluster-ip-range=10.50.0.0/22 ``` This runs kube-router with pod/service networking, the network policy firewall, and service proxy to replace kube-proxy. The example command uses `10.32.0.0/12` as the pod CIDR address range and `https://cluster01.int.domain.com:6443` as the address. Please change these to suit your cluster. ```sh CLUSTERCIDR=10.32.0.0/12 \ APISERVER=https://cluster01.int.domain.com:6443 \ sh -c 'curl -s https://raw.githubusercontent.com/cloudnativelabs/kube-router/master/daemonset/generic-kuberouter-all-features.yaml | \ sed -e "s;%APISERVER%;$APISERVER;g" -e "s;%CLUSTERCIDR%;$CLUSTERCIDR;g"' | \ kubectl apply -f - ``` If was ever deployed to the cluster, then you need to remove it when running kube-router in this capacity or they will conflict with each other. Remove any previously running kube-proxy and all iptables rules it created. Start by deleting the kube-proxy daemonset: ```sh kubectl -n kube-system delete ds kube-proxy ``` Any iptables rules kube-proxy left around will also need to be cleaned up. This command might differ based on how kube-proxy was setup or configured: To cleanup kube-proxy we can do this with docker, containerd, or cri-o: ```sh docker run --privileged -v /lib/modules:/lib/modules --net=host registry.k8s.io/kube-proxy-amd64:v1.28.2 kube-proxy --cleanup ``` ```sh ctr images pull k8s.gcr.io/kube-proxy-amd64:v1.28.2 ctr run --rm --privileged --net-host --mount type=bind,src=/lib/modules,dst=/lib/modules,options=rbind:ro \ registry.k8s.io/kube-proxy-amd64:v1.28.2 kube-proxy-cleanup kube-proxy --cleanup ``` ```sh crictl pull registry.k8s.io/kube-proxy-amd64:v1.28.2 crictl run --rm --privileged --net-host --mount type=bind,src=/lib/modules,dst=/lib/modules,options=rbind:ro registry.k8s.io/kube-proxy-amd64:v1.28.2 kube-proxy-cleanup kube-proxy --cleanup ``` This runs kube-router with pod/service networking and the network policy firewall. The Service proxy is disabled. ```sh kubectl apply -f https://raw.githubusercontent.com/cloudnativelabs/kube-router/master/daemonset/generic-kuberouter.yaml ``` In this mode kube-router relies on (or some other network service provider) to provide service networking. When service proxy is disabled kube-router will use to access APIserver through cluster-ip. Service networking must therefore be setup before deploying kube-router. kube-router supports setting log level via the command line -v or --v, To get maximal debug output from kube-router please start with `--v=3`