Repository: kubernetes-sigs/gateway-api

Issue #98: Specify load-balancing behavior for each service that is being proxied
Created: 2020-02-15T00:27:42Z
What would you like to be added:
For any service that is exposed via service-apis, a service owner should be able to specify the load-balancing behavior that should be used. This can include a common set of algorithms like round-robin, weighted, hash based routing, cookie/session based routing.
Why is this needed:
Most services in Kubernetes are designed to scale horizontally. A lot of legacy services that are ported to run on Kubernetes have the requirement of stick sessions. Most proxies and load balancers support load-balancing as well and this is a widely adopted feature in most of current ingress implementations.

Duplicate issue #196: Service-level configuration
Created: 2020-05-19T18:21:05Z
Current Status
We are collecting issues and use cases that should be associated with core.Service.
Once we have a good understanding of problems, we will start discussing potential solutions, ultimately ending in a KEP.  Please feel free to add your comments to this thread and we will try to incorporate those here.
Summary
There are service-level configuration properties that cannot be specified anywhere in k8s or service-apis resources currently.
The community seems to have a consensus that core.Service is an overloaded resource with features concerning different areas and the more fields should not be added to further bloat the Service abstraction.
While these features are laid out with Ingress or Gateway in mind, some of these configurations are also valid in other areas such as clients of a Service running within the cluster perimeter or outside, Service Mesh deployments, etc.
Issues
Load balancing
Some examples of load balancing features are (not exhaustive):

Algorithm to use for load-balancing (L4/L7) and their properties
Retry behavior: If a request fails, it should be retried(L4/L7) a number of times, with delays, and to different/same endpoints
Cluster or region aware load-balancing of traffic. This also ties with Multicluster Service effort

core.Service has some aspects of load balancing using fields like SessionAffinity, and ExternalTrafficPolicy. The existing fields are (intentionally?) not exhaustive but also don’t provide any extension mechanisms.
Related issues:

#98
#184

Traffic management
Service upgrades are performed in a gradual way to avoid surprises. Canarying and Mirroring traffic are some of the most common ways this is performed. Ingress and mesh vendors provide such features using CRDs. Service-apis project has also seen a lot of interest in this area. While this spans multiple core.Service resources, it might be worth to explore how an end-user should think of Services and endpoints when it comes to upgrades.
Related issues:

#58

Health checking
While kubernetes performs health checking of pods that are associated with a Service, it is common for proxies and Load Balancer to perform health checking on their part as well.
There have been some implementations which derive the health checking info from the liveliness or readiness probes but those seem incorrect as the reasons/concerns for kubelet and the proxy are different.
Feature requests:

Active health-checking (L4/L7)
Passive health-checking a.k.a. circuit-breaking
Timeout policies (overlaps with #L4-connection)

Related issues:

#97
#99

L4 details
Some examples of such properties (not exhaustive):

Timeouts: connect, read, write timeouts
Capacity: number of requests per second, max number of simultaneous connections to an endpoint of the Service, maximum number of outstanding requests
keepalive settings for connection reuse
idle connection timeout policies, connection draining settings

Most of these properties have an overlap with other sections as these generic networking properties.
Related issues/PRs:

#184
#89

Existing workarounds
Currently, there is no good place to put these configurations anywhere.
This has led to various projects adding such properties into networking.Ingress like resources.

https://projectcontour.io/docs/v1.4.0/httpproxy/
https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/
https://docs.nginx.com/nginx-ingress-controller/configuration/virtualserver-and-virtualserverroute-resources/
https://github.com/Kong/kubernetes-ingress-controller/blob/master/docs/concepts/custom-resources.md#kongingress
https://istio.io/docs/reference/config/networking/destination-rule/
TODO: add more from other projects

Such duplication cause:

Ad-hoc annotations, the community has a consensus that extensions via annotations for common use cases as the ones mentioned here are not the way to move forward
Fragmentation in the ecosystem where common properties have to defined in multiple places
Defining service-level properties in Ingress or CRDs for configuring routing rules often collide when multiple routes are pointing to the same service. Proxies resort to creating an upstream pool for each route that points to a service, which defeats the purpose of having the Service abstraction.

Misc notes
Global default
While per-service configuration is required, cluster operators would like to configure a sane global default, which should be used in absence of the any configuration.
Client vs Server
Definitions:

Server A server is the one responding to client's request. This is represented by the core.Service abstraction in k8s. Server and Service is used interchangeably in this section.
Client A client is a software or human who is accessing the server. In case of service-apis, the Gateway can be considered a client of the Service.

Configurations such as timeouts are properties of the Server and client both. A client when attempting to connect to a server, will specify a timeout. On the other hand, the server wants to protect itself from too many idle connections and will have a timeout on connections from its side.
In the context of this issue, we are referring to client configurations and not Server.
That begs the question: Why then associate such configuration to a Service and not a client controlled resource such as HTTPRoute?
I(@hbagdi) think this is because we want to define the configuration in the context of a Service and want clients to use it whenever they are communicating with the Service. There is no way to enforce that clients will use these configurations but if a client is communicating with a Service, they should follow the above defined properties if they expect certain SLAs to be met (thereby making such properties part of the contract between the client and the server/service). This could be thought of how a client should use a published port if it wishes to communicate with a Service.
Note: When considering a proxy or gateway, there are connections on two sides, one between the user and the gateway and the other between the gateway and the Service. This issue discusses the latter.
Extensibility
While standardization of above features would provide a good experience for end-users, extensibility is important. Areas like Load balancing and L4 properties can vary widely between implementations.
If a solution lacks hooks for extensions, we risk devising a solution that only works for a small population of our users and #existing-workarounds will continue to exist. The goal should not be to eliminate such workarounds but minimize them as much as possible.
kube-proxy
We need to explore where does kube-proxy fit into this. It seems to be another internal-gateway or client of the service. There are some aspects of the above that kube-proxy does implement. Should it continue to implement those? Should those be deprecated instead? Should the scope of kube-proxy be limited to not further complicate it?

Closing comment (2020-12-07T19:18:42Z): @hbagdi: Closing this issue.

In response to this:

Update:
As Rob said, this umbrella issue has become large and served its purpose.
Some specifics before closing this issue:

Load balancing is tracked in #98
Traffic management (canarying, mirroring) are implement as HTTPRouteFilters as of v1alpha1
Healthcheck related features are already tracked in #97 #99 #89
Global defaults can be set in an implementation-speicfic CRD that is specified in the GatewayClass.Spec.ParametersRef.

Please let me know if something was missed here.
/close

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.