Project: Open Service Mesh (Orchestration & Management / Service Mesh)
File: how_osm_uses_envoy.md

OSM heavily uses envoy for proxying requests, applying mTLS, and exporting statistics. Envoy itself has 5 main configuration mechanisms for request manipulation: ListenerConfigs RouteConfigs EndpointConfigs ClusterConfigs SecretConfigs Envoy is able to intercept all inbound and outbound traffic through In addition to the address and port to listen on, a Listener is configured with a set of `filter_chains` which dictate what to look for and what to do with a matching request, and a `listener_filter`, which tells Envoy what data it needs to extract from the request. Each listener will have a single `filter_chain` per service, per port. That is, the inbound listener will have 1 filter chain for each port on each service that can reference its associated Pod, and the outbound filter will have 1 filter chain for each port on each service the Pod can reach. A filter chain is comprised of 2 parts, a `filterchainmatch`, and a list of `filters`. This gives us the following construct: ListenerFilters FilterChains Filter chain matches List of filters The main listener filter used in OSM is the "Original Destination" filter which tells the listener to pick up the address from a syscall `getsockopt` with the option `SOORIGINALDST` that gets the originally requested IP address from the socket (prior to the IP table redirect). Otherwise, the IP used in the filter chain match would be the the redirected IP referencing the sidecar itself. The flow of a request from `pod-1` backed by `service-a` to `service-b` is as follows: The pod performs a DNS lookup for `service-b`, which routes to the in-cluster DNS service (typically CoreDNS or KubeDNS) which will return the service ClusterIP address (we are currently ignoring the case for headless services, or ExternalName). This DNS request, which happens over UDP, is permitted and not redirected via the IP Tables redirects. Next, the pod will attempt to make a request to that IP address, on that port. The IP tables are setup to redirect the to port 15001, which the `outbound-listener` is listening on. The Listener is configured to extract the SOORIGINALDST, and passes that through the set of filter chains, looking for the most precise match. On a filter match, envoy will now apply that filter chain's full set of filters, 1 by 1, in order. The Filter we are interested in, is called the `HttpConnectionManager`, which uses rDS to pass apply a RouteConfiguration, which OSM calls `rds-outbound` to the request. Below is a significantly paired down Listener configuration, which depicts the relevant info we just went over. ```json { "name":"outbound-listener", "active_state":{ "listener":{ "address":{ "socket_address":{ "address":"0.0.0.0", // This tells envoy to listen on all hosts "port_value":15001 // The outbound listener port. The IP Tables must intercept all outbound traffic and redirect here. } }, "filter_chains":[ { "filterchainmatch":{ "prefix_ranges":[ { "addressprefix":"10.96.75.109", // This is populated by the Service, and is what is matched by the SOORIGINAL_DST. "prefix_len":32 } ], "destination_port":14001 // The destination port must be equal to this. }, "filters":[ { "name":"httpconnectionmanager", "typed_config":{ "@type":"type.googleapis.com/envoy.extensions.filters.network.httpconnectionmanager.v3.HttpConnectionManager", "rds":{ // This tells the listener which RouteConfiguration to use. "routeconfigname":"rds-outbound" }, } } ], "name":"outbound-mesh-http-filter-chain:bookwarehouse/bookwarehouse" } ], "listener_filters":[ { "name":"envoy.filters.listener.original_dst" } ], }, } } ``` The next step is the

The RoutesConfig is comprised of a series of "virtual hosts", one for each service. When the route config is applied to a request, it will iterate over the list of virtual hosts, looking for a match. A virtual host is comprised of a set of domains it applies to, and a set of routes, where each route has some match criteria (ie: inspecting headers, path, etc), and a set of clusters. Once a match is determined it will pick one of the available clusters, based on a load balancing scheme (ie: weighted round robin). Below is a significantly paired down Route configuration, which depicts the relevant info we just went over: ```json { "@type":"type.googleapis.com/envoy.config.route.v3.RouteConfiguration", "name":"rds-outbound", "virtual_hosts":[ { "name":"outbound_virtual-host|bookstore", "domains":[ // The set of domains that will match this virtual host. "bookstore", "bookstore.bookstore", "bookstore.bookstore.svc", "bookstore.bookstore.svc.cluster", "bookstore.bookstore.svc.cluster.local", "bookstore:14001", "bookstore.bookstore:14001", "bookstore.bookstore.svc:14001", "bookstore.bookstore.svc.cluster:14001", "bookstore.bookstore.svc.cluster.local:14001" ], "routes":[ { "match":{ // To apply these routes, all of the matches need to be satisfied. "headers":[ { "name":":method", "saferegexmatch":{ "regex":".*" } } ], }, "route":{ "weighted_clusters":{ "clusters":[ // There are multiple clusters when we create a TrafficSplit. Envoy will pick one of of these clusters to route to. { "name":"bookstore/bookstore-v1", "weight":50 }, { "name":"bookstore/bookstore-v2", "weight":50 } ], "total_weight":100 } } } ] }, { "name":"outbound_virtual-host|bookwarehouse.bookwarehouse", "domains":[ // The set of domains that will match this virtual host. "bookwarehouse.bookwarehouse", "bookwarehouse.bookwarehouse.svc", "bookwarehouse.bookwarehouse.svc.cluster", "bookwarehouse.bookwarehouse.svc.cluster.local", "bookwarehouse.bookwarehouse.svc.cluster.local:14001", "bookwarehouse.bookwarehouse.svc.cluster:14001", "bookwarehouse.bookwarehouse.svc:14001", "bookwarehouse.bookwarehouse:14001" ], "routes":[ { "match":{ "headers":[ { "name":":method", "saferegexmatch":{ "google_re2":{ }, "regex":".*" } } ], }, "route":{ "weighted_clusters":{ // Envoy will route to this cluster. "clusters":[ { "name":"bookwarehouse/bookwarehouse", "weight":100 } ], "total_weight":100 } } } ] } ], "validate_clusters":false }, "last_updated":"2021-06-03T16:30:49.174Z" } ``` With a cluster chosen from the routes above, Envoy can now apply the Cluster Configuration. The cluster configuration is relatively simple, which picks the server and client certs for mTLS, the SNI field for routing TLS, and the endpoints from the endpoints discovery service. An Envoy cluster is the closest concept to a Kubernetes or OSM Service. ```json { "@type":"type.googleapis.com/envoy.config.cluster.v3.Cluster", "name":"bookwarehouse/bookwarehouse", // The name of the cluster, which is matched against both the routes and the EDS. "type":"EDS", "edsclusterconfig":{ "eds_config":{ "ads":{ // This tells envoy to pick up endpoints from the Endpoint Discovery Service. // Could optionally set a "servicename" field here to tell Envoy to not use the clustername for the mapping }, } }, "transport_socket":{ "name":"envoy.transport_sockets.tls", "typed_config":{ "@type":"type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext", "commontlscontext":{ "tlscertificatesdssecretconfigs":[ { "name":"service-cert:bookstore/bookstore-v2", // Use this secret for the service cert. "sds_config":{ "ads":{ }, "resourceapiversion":"V3" } } ], "validationcontextsdssecretconfig":{ "name":"root-cert-for-mtls-outbound:bookwarehouse/bookwarehouse", // use the secret for mtls validation. "sds_config":{ "ads":{ }, "resourceapiversion":"V3" } } }, "sni":"bookwarehouse.bookwarehouse.svc.cluster.local" } }, }, ``` NOTE: Envoy heavily uses these mapping concepts, typically based on cluster name, to match from one entity to another, to allow for reuse. ie: the `rds-outbound` mapping above. The final step of the outbound flow, Envoy matches the set of endpoints based on the clustername, or the servicename field set above, and routes to one of the available endpoints. For OSM, the endpoints will contain the ip + port for every Pod backing the service. ```json { "@type":"type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment", "clustername": "bookwarehouse/bookwarehouse", // Name of the cluster to match, or servicename if provided. "endpoints":[ { "locality":{ "zone":"zone" }, "lb_endpoints":[ { "endpoint":{ "address":{ "socket_address":{ "address":"10.244.0.21", // Pod address "port_value":14001 } }, }, "health_status":"HEALTHY", "loadbalancingweight":100 } ] } ] } ```