Project: Kuma (Orchestration & Management / Service Mesh)
File: proxy-template-redesign.md

The main goal of Kuma is to provide simple to use policies that abstract away complexity of the Envoy configuration. Unfortunately, not every field in Envoy can be abstracted by Kuma, sometimes a user needs to fine-tune properties of Envoy. Proxy Template is the policy that was suppose to fulfill this need, but it only allows omitting generating Envoy config by Kuma, add new Envoy resources or replace existing one. The problem is that what user really wants is to modify configuration that is generated by Kuma (ex. add a timeout on a cluster, add a filter to the filter chain) Add new resources on top of Kuma resources (new Cluster/Listener etc.). Remove all resources generated by Kuma. Remove some resources generated by Kuma. Modify resources generated by Kuma (replace field in the config) Add new network filter to listener. (take order into account) Add new HTTP filter to HTTP Connection Manager. (take order into account) Modify filters - although it's not a "top level" Envoy resource, it's very common need to modify it. Here is a full example of the configuration ```yaml type: ProxyTemplate mesh: default name: custom-template-1 selectors: match: service: backend conf: imports: default-proxy modifications: cluster: match: # optional side: inbound # optional name: backend # optional operation: add # remove / patch value: # Envoy YAML listener: match: # optional side: inbound # optional name: xyz # optional operation: add # remove / patch value: # Envoy YAML route: match: # optional side: inbound # optional name: xyz # optional operation: add # remove / patch value: # Envoy YAML virtualHost: match: # optional side: inbound # optional name: xyz # optional operation: add # remove / patch value: # Envoy YAML httpFilter: match: side: inbound # optional type: envoy.filters.http.buffer listenerName: outbound:127.0.0.1:3000 # optional operation: addBefore # addAfter / addFirst / addLast / remove / patch value: # YAML of filter networkFilter: operation: addAfter # addAfter / addFirst / addLast / remove / patch match: side: inbound # optional type: envoy.filters.network.rbac listenerName: inbound:127.0.0.1:1234 # optional value: # YAML of filter ``` cluster/listener/route etc. is `oneof` - you can only use one of them in the object in the array. `match` can differ across

As we provide this feature for users, we may see more need for matching (for example - match only EDS clusters or match listener with metadata). `side` can help you match resource that is generated on inbound or outbound side. In `httpFilter` and `networkFilter`: `addAfter`, `addFirst`, `patch` require `type` in match. `addFirst` and `addLast` you cannot use `name` in match. 1) Add a new Cluster on top of configuration generated by Kuma ```yaml conf: imports: default-proxy modifications: cluster: operation: add value: | connectTimeout: 5s name: localhost:9901 loadAssignment: clusterName: localhost:9901 endpoints: lbEndpoints: endpoint: address: socketAddress: address: 127.0.0.1 portValue: 9901 type: STATIC ``` There is no `match` section because it's not needed. 2) Remove route from the configuration generated by Kuma ```yaml conf: imports: default-proxy modifications: route: match: name: backend operation: remove ``` 3) Patch cluster Patch the cluster modifying connection timeout ```yaml conf: imports: default-proxy modifications: cluster: match: name: backend operation: patch value: connectTimeout: 3s ``` 4) Add HTTP JWT filter on one listener picked by name Use case: you want to add JWT but only on one inbound of your service ```yaml conf: imports: default-proxy modifications: httpFilter: match: listenerName: inbound:127.0.0.1:1234 operation: addFirst value: | name: envoy.filters.http.jwt_authn typedConfig: type.googleapis.com/envoy.config.filter.http.jwt_authn.v2alpha.JwtProvider issuer: https://example.com audiences: bookstore_android.apps.googleusercontent.com bookstore_web.apps.googleusercontent.com remote_jwks: http_uri: uri: https://example.com/.well-known/jwks.json cluster: examplejwkscluster cache_duration: seconds: 300 ``` Because of `listenerName: "inbound:127.0.0.1:1234"` it will be added only to listener with this name 5) Add network filters on every inbound listener ```yaml conf: imports: default-proxy modifications: networkFilter: match: side: inbound name: envoy.filters.network.tcp_proxy operation: addAfter value: | name: envoy.filters.network.local_ratelimit typedConfig: type.googleapis.com/envoy.config.filter.network.localratelimit.v2alpha.LocalRateLimit tokenBucket: maxTokens: 3 tokensPerFill: 1 fillInterval: 3s ``` Because there is no `listenerName` it matches all listeners. Because of `side: inbound` it matches on all inbound listeners. Because of `name: envoy.filters.network.tcpproxy` it will be inserted just after `envoy.filters.network.tcpproxy` 6) Remove network filter ```yaml conf: imports: default-proxy modifications: networkFilter: operation: remove match: name: envoy.filters.network.rbac listenerName: inbound:127.0.0.1:1234 ``` 7) Patch HTTP Connection Manager ```yaml conf: imports: default-proxy modifications: networkFilter: match: type: envoy.filters.network.httpconnectionmanager listenerName: inbound:127.0.0.1:1234 operation: patch value: requestTimeout: 3s ``` 8) Remove all clusters ```yaml conf: imports: default-proxy modifications: cluster: operation: remove ``` There is no `match` so it will match all With new `modifications`, the old `resources` field should be deprecated and then removed. Modifications will be executed in order specified in the array When you do `operation: add` with resource of the same name that was already generated it should be replaced (just like with current behaviour)