Package: envoy-extproc-sdk
An Envoy ExternalProcessor SDK in pythonOverviewenvoy, one of the most powerful and widely used reverse proxies, is able to query anExternalProcessorgRPC service in it's filter chain. This functionality opens the door to quickly and robustly implemently customized functions at the edge, instead of in targeted services.Bond, for example, is using this functionality to implement authentication, API call logging, and write-request idempotency. While powerful, implementing these services still requires dealing with complicatedenvoyspecs, managing information sharing across request phases, and an understanding of gRPC, none of which are exactly straightforward.The purpose of this SDK is to make development of ExternalProcessors easy. This SDKcertainlywon't supply the mostperformantedge functions. With thedocker-composesetup here we see an overhead of about 20ms/req with 6 processors in the filter chain, or maybe about 4-5ms/req per filter. Without a doubt, much better performance will come from eschewing the ease-of-use functionality here, packing processor functions together in one filter, implementing your filter in a compiled language, or even more likelynot using an ExternalProcessor at allbut instead using aWASM pluginor registeredcustom filter binary. Optimal performance isn't our goal; usability, maintainability, and low time-to-functionality is, and those aspects can often be more important than minimal latency.UsageSpecifically we supply aBaseExtProcServicethat provides much of the boilerplate required to make this type of service. Here is a brief, untyped example of how to build one (based onexamples/decorated.py):import logging
from envoy_extproc_sdk import BaseExtProcService, serve

svc = BaseExtProcService(name="DecoratedExtProcService")

@svc.process("request_headers")
def start_digest(headers, context, request, response):
    ... # do stuff

@svc.process("request_body")
def complete_digest(body, context, request, response):
    ... # do stuff

if __name__ == "__main__":
    FORMAT = "%(asctime)s : %(levelname)s : %(message)s"
    logging.basicConfig(level=logging.INFO, format=FORMAT, handlers=[logging.StreamHandler()])
    serve(service=svc)In short, you can simply "decorate" methods (of the right signature) and form an ExternalProcessor. This "route decoration" is a pattern common topythonserver frameworks, and is probably the easiest way to get started. The primary pattern we use though is subclassing, as you'll see if you reviewexamples/*.py.class SomeExtProcService(BaseExtProcService):

    def process_request_headers(self, headers, context, request, response):
        ... # do stuff

    def process_request_body(self, body, context, request, response):
        ... # do stuff

if __name__ == "__main__":
    FORMAT = "%(asctime)s : %(levelname)s : %(message)s"
    logging.basicConfig(level=logging.INFO, format=FORMAT, handlers=[logging.StreamHandler()])
    serve(service=SomeExtProcService())Obviously there's more to it, but that's the basic idea: focus on behaviors more than lower level implementations.The providedserveinterface adopts thegrpc.aioparadigm, which we've found a bit cleaner to use here than the threading concurrency model. We also add an implementation of theHealthServicefor gRPC in order to run in a context (likekubernetes) with health probes.Really you'll still need to learn some details about howenvoyspecifies and types these services and their data, but it's much more limited here. BasicallyBaseExtProcServiceimplements the single RPCProcessdefined by the spec, pulls out the request phase data fromProcessingRequest, and wraps request phase specific handlers in the requisiteProcessingResponse. This enables a subclass (or decorated methods) to focus solely on the logic for handling the request phases. These phases are{request,response}_headers: process request or response headers{request,response}_body: process request or response bodies{request,response}_trailers: process request or response trailers (note: we've found this buggy inenvoy)Seethis PRDfor some extra illustration and discussion around the ExternalProcessor flow.TheBaseExtProcServicealsoimplements it's own "request context" (therequestargument in the decorated handlers above) to enable data passing between request phases. This is acriticalfeature for effective, powerful external processing.envoysends only request header data when asking to process request headers, only body data when asking to process request body, etc. But processor behaviors or computing can easily depend on the full known scope of request data.Storing and managing that inter-phase data is whatrequestis for; seeexamples/*.pyfor, well, examples.BaseExtProcServiceis largely unopinionated about what data should be contained in therequest, as that is highly use-case specific. As of now it only takes two default actions:In therequest_headersphase, it pulls a set of "standard" headers into therequest: themethod,path,content-type,content-length, and thex-request-id.In theresponse_headersphase, it does the same over writingcontent-typeandcontent-length.DistributionWe distribute this aspythonpackage (TBD)$ pip install envoy-extproc-sdkand as adockercontainer (TBD)$ docker pull envoy-extprox-sdk:latestNote we donotpackage generated code fromenvoy'sprotobufspecs in thepythonmodule. (Thegrpclibraries themselves are "broken" relative to newerprotobufbecause they embedd old generated code for health checks, which seem now unusable.) So if you use thepythonpackage you have to build and install theprotobufgenerated code fromenvoy(seebuf.yamlhere andmake codegen) for it to work. We recommend following our approach here, as we customize handling of the health check generated code.You can build on top of theenvoy_extproc_sdkdockerimage and avoid this, as wedopackage the generated code in images. This can be done in the normal way, actually as illustrated by the examples here. In fact,examples/Dockerfile(used in thedocker-compose.yaml) is only# syntax=docker/dockerfile:1.2
ARG IMAGE_TAG=latest
FROM envoy-extproc-sdk:${IMAGE_TAG}
COPY ./examples ./examplesTestingThere are also some testing utilities inenvoy_extproc_sdk.testing. These mainly help create and send payloads to a processor for unit testing.envoy_headers: return aHttpHeadersobject from adictof headers or alistof key-value pairsenvoy_body: return aHttpBodyobject from several types that could be bodiesenvoy_set_headers_to_dict: return adictof headers from aCommonResponseobject (useful for response modification assertions)AsEnvoyExtProcis a class that can be initialized with phase data and sent toBaseExtProcService.Processto mimic processing of a request; ieP = BaseExtProcService()
E = AsEnvoyExtProc(request_headers=headers, request_body=body)
async for response in P.Process(E, None):
    ... # parse ProcessResponse and execute assertions based on phaseEnvoy ConfigurationOf course, this service isn't useful outside anenvoydeployment configured to use it. This SDK doesn't help you configure yourenvoy, but seeenvoy.yamlfor example configurations and seethe configuration reference.A simple version is something like- name: envoy.filters.http.ext_proc
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.filters.http.ext_proc.v3.ExternalProcessor
    grpc_service: 
      envoy_grpc:
        cluster_name: my-extproc
      timeout: 30s
    failure_mode_allow: true
    message_timeout: 0.2s
    processing_mode: 
      request_header_mode: SEND
      response_header_mode: SKIP
      request_body_mode: BUFFERED
      response_body_mode: BUFFERED
      request_trailer_mode: SKIP
      response_trailer_mode: SKIPwheremy-extprocis a definedclusterpointing to the gRPC service.The key features to point out are:failure_mode_allowdeclares whether ExternalProcessor failures tobreakthe request flow (false) or should be ignored (true); if a processor's action is critical to request processing, this should befalse.message_timeoutis the per-message timeout within a stream. This should be tailored to how longany phasein request processing can take.grpc_service.timeoutis thefull requesttimeout of a whole stream. This should be tailored to how longthe whole requestcan take, including any upstream filters or the ultimate target.theprocessing_modes are important, describing what data an ExternalProcessor gets or doesn't. Seethe specificationfor details. The example service above will receive request headers butnotresponse headers, thefullrequest and response bodies in one pass (not streamed), and no trailers.InterfaceCommand Line InterfaceYou can run the package as module and invoke a CLI:$ python -m envoy_extproc_sdk --help
usage: __main__.py [-h] [-s SERVICE] [-p PORT] [-g GRACE_PERIOD] [-l]

optional arguments:
  -h, --help            show this help message and exit
  -s SERVICE, --service SERVICE
                        Processor to use, as an import spec
  -p PORT, --port PORT  Port to run service on
  -g GRACE_PERIOD, --grace-period GRACE_PERIOD
                        Grace period to finish requests on shutdown
  -l, --logging         Include logging setupUse-s/--serviceto tell the CLI what service to run (values should be apythonimport spec),-p/--portis the port to run the server on (by default50051),-g/--grace-periodis the time (in seconds) to wait for requests to finish after interrupt (by default5),-l/--loggingis a flag to setuploggingat runtime (you might not want this, preferring your own logging setup).Other or overlapping settings fromenvvars are insettings.py:GRPC_PORT(default50051): the server listerner portSHUTDOWN_GRACE_PERIOD(default5seconds): the time to wait for gracefull shutdown of the gRPC serviceREVEAL_EXTPROC_CHAIN(defaultTrue): whether to add a response header that builds a list of all ExternalProcessors used in handling a requestEXTPROCS_APPLIED_HEADER(defaultx-ext-procs-applied): the name of that headerUtilitiesCurrentlyBaseExtProcServicehas some@staticmethodhelpers for processing headers. Helpers for handling bodies may be introduced later.BaseExtProcService.get_headerGet a header from the request or response headers.Arguments:headersaHttpHeadersobjectname(str) the name of the header to look forlower_cased(bool, defaultFalse) whether the name isalreadylowercasedReturns the value of the header searched for, if it exists.Noneif it doesn't.BaseExtProcService.get_headersGet a set of headers from the request or response headers.Arguments:headersaHttpHeadersobjectnames(Union[Dict[str, str], List[Tuple[str, str]]]) the names of the headers to look for, byactualheader names, mapped to keys to use in the returned list (e.g.,x-api-keytoapikey)lower_cased(bool, defaultFalse) whether the name isalreadylowercasedReturns aDictwith mapped names as keys and header values (orNone) as values.BaseExtProcService.add_headerAdd a header to the request or response headers.Arguments:responseaCommonResponseobjectkey(str) the header to setvalue(str) the header valueReturns the updated response.BaseExtProcService.add_headersAdd a set of headers to the request or response headers.Arguments:responseaCommonResponseobjectheaders(Union[Dict[str, str], List[Tuple[str, str]]]) the headers, as adictor list of key-value pairs, to addReturns the updated response.BaseExtProcService.remove_headerRemove a header from the request or response headers.Arguments:responseaCommonResponseobjectname(str) the header to remove (if it exists)Returns the updated response.BaseExtProcService.remove_headersRemove a set of headers from the request or response headers.Arguments:responseaCommonResponseobjectname(List[str]) the headers, by name, to remove (if they exist)Returns the updated response.BaseExtProcService.form_immediate_responseConstruct anImmediateResponseobject, which tellsenvoyto stop processing the request and respond as described.Arguments:status(EnvoyHttpStatusCodea wrapper aroundenvoy'sStatusCode) the status code for the responseheaders(Dict[str, str]) the response headers to return to the callerbody(str) the body to return to the callerReturns the constructedImmediateResponse.Phase HandlersThe following documents how to implement the "request phase handlers". These describe what to do win each phase, how to read/write request context, and how to modify a request or response. In any phase you canraiseaenvoy_extproc_sdk.StopRequestProcessingExceptionto supply a response directly from the processor (without sending to the upstream processors or target). The constructor requires anImmediateResponseobject, which you can construct with the helper methodraise StopRequestProcessing(response=form_immediate_response(...))inBaseExtProcService.In the labels below we assumeP = BaseExtProcService(name="SomeExtProcService")@P.process("request_headers")ordef process_request_headersArguments:headers, anenvoyHttpHeadersobject describing the request headers.context, a gRPCServicerContextfrom the RPCrequest, a simpleDictfor supplying/supplementing request context across phasesresponse, aCommonResponseobject for tellingenvoyhow to mutate the request (if at all).Return the (possibly modified)responsepassed in, orraiseaStopRequestProcessing.@P.process("request_body")ordef process_request_bodyArguments:body, anenvoyHttpBodyobject describing the request body.context, a gRPCServicerContextfrom the RPCrequest, a simpleDictfor supplying/supplementing request context across phasesresponse, aCommonResponseobject for tellingenvoyhow to mutate the request (if at all).Return the (possibly modified)responsepassed in, orraiseaStopRequestProcessing.@P.process("response_headers")ordef process_response_headersArguments:headers, anenvoyHttpHeadersobject describing the request headers.context, a gRPCServicerContextfrom the RPCrequest, a simpleDictfor supplying/supplementing request context across phasesresponse, aCommonResponseobject for tellingenvoyhow to mutate the response (if at all).Return the (possibly modified)responsepassed in, orraiseaStopRequestProcessing.@P.process("response_body")ordef process_response_bodyArguments:body, anenvoyHttpBodyobject describing the response body.context, a gRPCServicerContextfrom the RPCrequest, a simpleDictfor supplying/supplementing request context across phasesresponse, aCommonResponseobject for tellingenvoyhow to mutate the response (if at all).Return the (possibly modified)responsepassed in, orraiseaStopRequestProcessing.TrailersTrailers handlers are similar, but less likely to be used. See the code for details.ExamplesThere are several examples inexamples/. These can be packaged in thedockerimage built fromexamples/Dockerfile(seemake build) and included as services in thedocker-compose.yaml. The basicenvoyconfigenvoy.yaml(used by thedocker-compose) sets each example up to be used.envoy_extproc_sdk.BaseExtProcService: TheBaseExtProcServiceis an example in it's own right, but does nothing to requests. UsingLOG_LEVEL=DEBUGwill print log lines describing the processing steps taken. Run withLOG_LEVEL=DEBUG make runFor any other examples you can use the same command to run, specifying theSERVICE. For example,make run SERVICE=examples.TrivialExtProcServicewill run our first example, the "trivial" processor.examples.TrivialExtProcService: This example adds an upstream header as well as a response header, both thex-request-idbut in the namex-extra-request-id.examples.TimerExtProcService: This example times the request, add an upstream header (which the request started), and add two response headers (when the request started and how long it took in nanoseconds).examples.DigestExtProcService: This example computes a SHA256 digest of the request method, path, and body, and add that as an upstream request header and a response headerx-request-digest. It demonstrates inter-phase data use.examples.DecoratedExtProcService: This example copiesDigestExtProcService, but implements the service using the@processdecorator instead of as a subclass service.examples.EchoExtProcService: This example demonstrates use ofStopRequestProcessingto respond immediately from an ExternalProcessor, instead of sending a request to the upstream processors or target.CtxExtProcService: This example allows for testing the request context. It reads a request headerx-context-id, adding that to the upstream request headers. If that header is missing, the service does nothing else. If it exists, it will also analyze the request body, which it expects to be exactly thex-context-idsupplied. The processor will fail if this doesn't match. The filter also processes the response body, which it expects to be JSON with the request path equal topath(as with our echo server intests/mocks/echo). The service checks that value matches thepathstored in the request context. These steps are largely to check that we canconcurrentlymake requests with different values and see consistency in the response headerx-context-id, which we will not get if the service's processing fails.DevelopmentRequirementspython3.9poetryfor package managementmakefor convenience commandsprotocandbuffor generating code fromprotobufschemas forenvoydockerfor testingQuickstartCLI/makeTheMakefileprovides a lot of helpful targets to get started. The simplest quickstart is probably$ make install format unit-test runThis will (a) install thepythondependencies, (b) usebufto generate code (and install it in the current virtual environment), (c) format the code, (d) run the unit tests, (e) and run theBaseExtProcService. However, running the service on it's own is only partially useful as the service is a gRPC service which isn't the easiest to justcurlat.Review theMakefilefor other commands, includingformat(isort,black, andflake8linting),types(formypystatic type analysis),build(fordocker build)package(for building apythonpackage)publish(for distributing thepythonpackage)dockerThedocker-composeis a setup withenvoy, a naive "echo" HTTP server (seetests/mocks/echo/echo.py), and the example ExternalProcessor services fromexamples/. This way you can make plain HTTP requests and actually see outcomes from the filters. The single upstreamechoserver responds to any request with a JSON payload containing the following keysmethod: the request method it sawpath: the request pathheaders: a nested JSON of all the request headers it receivedbody: any request body it receivedmessage: a message from theechoserverFor example, after running$ make upyou can try$ curl localhost:8080/something -D -
HTTP/1.1 200 OK
server: envoy
date: Sat, 16 Jul 2022 22:55:19 GMT
content-length: 524
content-type: application/json
x-envoy-upstream-service-time: 1
x-request-started: 2022-07-16T22:55:19.290822Z
x-duration-ns: 23589000
x-ext-procs-applied: TrivialExtProcService,TimerExtProcService,EchoExtProcService,DigestExtProcService,DecoratedExtProcService,CtxExtProcService
x-extra-request-id: 554c54e8-fac1-42e3-8ab8-1f2264f59664

{"method": "get", "path": "/something", "headers": {"host": "localhost:8080", "user-agent": "curl/7.64.1", "accept": "*/*", "x-forwarded-proto": "http", "x-request-id": "554c54e8-fac1-42e3-8ab8-1f2264f59664", "x-extra-request-id": "554c54e8-fac1-42e3-8ab8-1f2264f59664", "x-request-started": "2022-07-16T22:55:19.290822Z", "x-request-digest": "860d64d6465b9e9886050295087e8a547b3e7a3c40e79d26147b50a97b9ac2c6", "x-context-id": "", "x-envoy-expected-rq-timeout-ms": "15000"}, "body": "{\"hello\":\"hi\"}", "message": "Hello"}or$ curl localhost:8080/something -X PUT -H 'Content-type: application/json' -d '{"hello":"hi"}' -D -
HTTP/1.1 200 OK
server: envoy
date: Sat, 16 Jul 2022 22:54:49 GMT
content-length: 584
content-type: application/json
x-envoy-upstream-service-time: 0
x-request-digest: a794dbc467285567e4c2604c991938386366f6ab94b0b0e4fab5e27e0a932e60
x-request-started: 2022-07-16T22:54:49.660908Z
x-duration-ns: 25046000
x-ext-procs-applied: TrivialExtProcService,TimerExtProcService,EchoExtProcService,DigestExtProcService,DecoratedExtProcService,CtxExtProcService
x-extra-request-id: 7a983b59-d67c-44c8-a54a-2afae7069ac9

{"method": "put", "path": "/something", "headers": {"host": "localhost:8080", "user-agent": "curl/7.64.1", "accept": "*/*", "content-type": "application/json", "content-length": "14", "x-forwarded-proto": "http", "x-request-id": "7a983b59-d67c-44c8-a54a-2afae7069ac9", "x-extra-request-id": "7a983b59-d67c-44c8-a54a-2afae7069ac9", "x-request-started": "2022-07-16T22:54:49.660908Z", "x-request-digest": "a794dbc467285567e4c2604c991938386366f6ab94b0b0e4fab5e27e0a932e60", "x-context-id": "", "x-envoy-expected-rq-timeout-ms": "15000"}, "body": "{\"hello\":\"hi\"}", "message": "Hello"}For contrast, here are these two requestswithoutfilters:$ curl localhost:8080/something -X PUT -H 'Content-type: application/json' -d '{"hello":"hi"}' -D -
HTTP/1.1 200 OK
server: envoy
date: Sat, 16 Jul 2022 23:40:24 GMT
content-length: 362
content-type: application/json
x-envoy-upstream-service-time: 1

{"method": "put", "path": "/something", "headers": {"host": "localhost:8080", "user-agent": "curl/7.64.1", "accept": "*/*", "content-type": "application/json", "content-length": "14", "x-forwarded-proto": "http", "x-request-id": "0afcd2c4-6d3d-4513-a29b-40c7954f8942", "x-envoy-expected-rq-timeout-ms": "15000"}, "body": "{\"hello\":\"hi\"

$ curl localhost:8080/something -D -
HTTP/1.1 200 OK
server: envoy
date: Sat, 16 Jul 2022 23:40:30 GMT
content-length: 302
content-type: application/json
x-envoy-upstream-service-time: 1

{"method": "get", "path": "/something", "headers": {"host": "localhost:8080", "user-agent": "curl/7.64.1", "accept": "*/*", "x-forwarded-proto": "http", "x-request-id": "f8dfa254-157b-4f75-a7d0-121f3d245d6b", "x-envoy-expected-rq-timeout-ms": "15000"}, "body": "{\"hello\":\"hi\"}", "message": "Hello"}Note the additional response headers and the extra information about the upstream services request headers in the response body. That's the filter set working!