Project: Contour (Orchestration & Management / Service Proxy)
File: endpoint-slice-support.md

Contour currently supports Kubernetes endpoints. This document proposes a process to enable support for k8s EndpointSlices and, eventually, migrate to exclusively using it. Contour will support k8s EndpointSlices. The EndpointSlices will be gated by a feature flag in the Contour config. Kubernetes endpoints will still be the default mode of operation. Removal of k8s endpoints. Support advanced features such as topology aware routing/hints offered by k8s endpoints. See Before the introduction of the EndpointSlices, the only method for a service to expose the IP addresses and ports of the pads it fronted was through Kubernetes endpoints. However, Kubernetes endpoints have a few challenges with scaling, such as transmission of the entire endpoint object even if one endpoint changes, leading to network contention and heavy recalculation on the consumers of this data. As of k8s v1.22, a hard limit of 1000 addresses per endpoint object. No topology aware information for more intelligent routing. To solve the above problems, EndpointSlices was introduced in 1.16 and, as of k8s v1.21 in . A feature flag will gate the usage of EndpointSlices, and the implementation will use the same pattern as used in the `endpointtranslator.go` with a few differences listed in the detailed design. EndpointSlices will be configured by a boolean feature flag, which defaults to false and allows users to opt to use EndpointSlices. ```yaml ... server: xds-server-type: contour use-endpoint-slice: true ... ``` ```go type Parameters struct { // useEndpointSlice configures contour to fetch endpoint data // from k8s EndpointSlices. defaults to false and reading endpoint // data from the k8s endpoints. useEndpointSlice bool } ``` The implementation of k8s EndpointSlices is fundamentally different from how EndpointSlices work. Below is an example of a Kubernetes service represented in as endpoint and EndpointSlice objects endpoints ``` NAME ENDPOINTS AGE ingress-conformance-echo 10.244.0.10:3000,10.244.0.153:3000,10.244.0.154:3000 + 7 more... 3d ``` EndpointSlice ``` NAME ADDRESSTYPE PORTS ENDPOINTS AGE ingress-conformance-echo-8bcrv IPv4 3000 10.244.0.156,10.244.0.157 18h ingress-conformance-echo-dr6d2 IPv4 3000 10.244.0.158,10.244.0.159 18h ingress-conformance-echo-dt27v IPv4 3000 10.244.0.153,10.244.0.154 2d14h ingress-conformance-echo-l2j48 IPv4 3000 10.244.0.10,10.244.0.155 3d ingress-conformance-echo-z4bcw IPv4 3000 10.244.0.160,10.244.0.161 18h ``` Because of the above mentioned differences, there is a need to adapt the cache logic from `endpointtranslator.go` such as; Instead of using `endpoints map[types.NamespacedName]*v1.Endpoints`, use the below format with a nested map `k,v` where `k` is the EndpointSlice name and the `v` is the EndpointSlice itself. ```go type EndpointSliceCache struct { endpointSlice: map[types.NamespacedName]map[string]*discoveryv1.EndpointSlice{}, } ``` The above structure lets us keep track of each EndpointSlice and change/update/delete an endpoint without shuffling all the existing endpoints. The EndpointSlice has information about the condition of the endpoint, namely, `ready`, `serving` and `terminating`. The EndpointSlice implementation will only support the `ready` condition. This means that only pods with `ready` status will be considered since `ready` on the EndpointSlice maps to `ready` status on the pods. This also keep contour inline with the existing implementation. refer FQDN address types on EndpointSlices will not be supported. Contour will continue to support `externalName` service types. If a service has disabled, it will not be processed by Contour. To ensure the feature is well tested, this feature will have e2e tests unit tests daily build run with EndpointSlices enabled. Once we have enough consensus on the correctness and performance, we can make the move to EndpointSlices permanent and deprecate use of endpoints. This can be a combination of daily builds, burn time and feedback from the community. N/A