Project: Koordinator (Orchestration & Management / Scheduling & Orchestration)
File: koord-runtime-proxy-design-doc.md

* * * * * * * * KoordRuntimeProxy acts as a proxy between kubelet and containerd(dockerd under dockershim scenario), which is designed to intercept CRI request, and apply some resource management policies, such as setting different cgroup parameters by pod priorities under hybrid workload orchestration scenario, applying new isolation policies for latest Linux kernel, CPU architecture, and etc. There are two components involved, KoordRuntimeProxy and RuntimePlugins. Enhance resource management for QoS based Scheduling. Provide interface for new isolation features which are not supported by CRI. KoordRuntimeProxy is in charge of intercepting request during pod's lifecycle, such as RunPodSandbox, CreateContainer etc., and then calling RuntimePlugins to do resource isolation policies before transferring request to backend containerd(dockerd) and after transferring response to kubelet. KoordRuntimeProxy provides an isolation-policy-execution framework which allows customized plugins registered to do specified isolation policies, these plugins are called RuntimePlugins. KoordRuntimeProxy itself does NOT do any isolation policies. RuntimePlugins register events(RunPodSandbox etc.) to KoordRuntimeProxy and would receive notifications when events happen. RuntimePlugins should complete resource isolation policies basing on the notification message, and then response KoordRuntimeProxy, KoordRuntimeProxy would decide to transfer request to backend containerd or discard request according to plugins' response. If no RuntimePlugins registered, KoordRuntimeProxy would become a transparent proxy between kubelet and containerd. There are 4 main components for KoordRuntimeProxy. As a proxy between kubelet and containerd, KoordRuntimeProxy acts as a CRI server for kubelet(http server under dockershim scenario). It should intercept all request from kubelet, and generate protocols for talking with plugins before and after talking with backend containerd(dockerd) PluginsManager is in charge of parsing registered plugin info from `/etc/runtime/hookserver.d` dynamically. RuntimeDispatcher is designed to manage communications with plugins. As a proxy, KoordRuntimeProxy had better be designed as stateless, but sometimes it does NOT

Take StartContainer hook for example, there exists only containerID in CRI StartContainerRequest, which is not enough for plugins to adapt policies since plugins may not store pod/container info(such as meta, priority) locally. So KoordRuntimeProxy should store pod/container info during RunPodSandbox/CreateContainer Stage. When StartContainer request comes, KoordRuntimeProxy can get pod/container info by containerID, and then call plugins with pod/container info. With store, there would be pod/container info everytime KoordRuntimeProxy calls plugins, so there is no need for plugins to store pod/container info exceptionally, plugins can be designed as stateless. Considering performance, store locates in memory and does not generate external io to disk. All the plugin config files should be put to `/etc/runtime/hookserver.d` with `.json` suffix, for example: ``` $ cat /etc/runtime/hookserver.d/koordlet.json { "remote-endpoint": "/var/run/koordlet-runtimehookserver.sock", "failure-policy": "Ignore", "runtime-hooks": [ "PreRunPodSandbox" ] } ``` There are 3 fields involved: remote-endpoint: endpoint KoordRuntimeProxy talking with plugin, generated by plugin. failure-policy: policy when calling plugin fail, Fail or Ignore, default to Ignore. runtime-hooks: currently 5 hook points: PreRunPodSandbox, PreStartContainer, PostStartContainer, PreUpdateContainerResources, PostStopContainer. hook points with prefix 'Pre' means calling plugins before transferring request to contianerd(dockerd).<br> hook points with prefix 'Post' means calling plugins after receiving response from containerd(dockerd).<br> plugin provider can set any hook combinations to "runtime-hooks". get sources: `git clone https://github.com/koordinator-sh/koordinator.git` build: `cd koordinator; make build-koord-runtime-proxy` TODO Under containerd scenario, to make koord-runtime-proxy a proxy between kubelet and containerd, kubelet parameters should be altered as shown below: ``` kubelet <other options> --container-runtime=remote --container-runtime-endpoint=/var/run/koord-runtimeproxy/runtimeproxy.sock ``` Under docker scenario, to make koord-runtime-proxy a proxy between kubelet and dockerd, kubelet parameters should be altered as shown below: ``` kubelet <other options> --docker-endpoint=/var/run/koord-runtimeproxy/runtimeproxy.sock ``` Firstly, please make sure your runtime backend is containerd or dockerd. Under containerd scenario, koord-runtime-proxy can be setup with command: ``` koord-runtime-proxy --remote-runtime-service-endpoint=<runtime sockfile path> --remote-image-service-endpoint=<image sockfile path> ``` If containerd listening CRI request on default /var/run/koord-runtimeproxy/runtimeproxy.sock, koord-runtime-proxy can be setup by: ``` koord-runtime-proxy ``` Under docker scenario, koord-runtime-proxy should be setup with the additional parameter `--backend-runtime-mode Docker`, and without `remote-image-service-endpoint`: ``` koord-runtime-proxy --backend-runtime-mode=Docker --remote-runtime-service-endpoint=<runtime sockfile path> ```