Package: opentelemetry_wrapper
opentelemetry_wrappera wrapper aroundopentelemetryandopentelemetry-instrumentation-*to make life a bit easierwhat this does (or is supposed to do)Make instrumentation (more) idempotentMake re-instrumentation of logging actually work with different format stringsMakeloggingprint as a one-line JSON dict by defaultProvide support for decorating functions and classesProvide support for instrumentation of dataclassesGlobal instrumentation needs to be runbeforeany dataclasses are initializedOtherwise, use the decorator on each class as usual (it's idempotent anyway)Add global instrumentation of FastAPIsometimes works even after apps are created for some reason, likely due to how Uvicorn runs in a new processbut somehow sometimes doesn't work in prod, for equally unknown reasonsprobably best to instrument each app instanceLogs some http headers received by fastapi as span attributesCreates OTLP exporter if specific env vars (below) are setusageTL;DR:callinstrument_all()to instrument logging and requestsinstrument your FastAPI app usinginstrument_fastapi_app(...)use@instrument_decorateon any function or class you want to monitorsetOTEL_WRAPPER_DISABLED=truein your CICD tests, especially if you're usingpytestenv varsVariable NameDescriptionDefault (if not set)OTEL_EXPORTER_OTLP_ENDPOINTLooks likehttp://tempo.localhost:4317.NA(traces are not exported to any OTLP endpoint)OTEL_EXPORTER_OTLP_HEADERLooks likeHeader-Name=header value, where values can contain space ('\x20'). To insert multiple headers, delimit by any other whitespace char.NA(no header sent to OTLP endpoint)OTEL_EXPORTER_OTLP_INSECURESet totrueto disable SSL for OTLP trace exports, orfalseto always verify.NA(follows OpenTelemetry default, which is secure for https and insecure for http)OTEL_HEADER_ATTRIBUTESList of HTTP headers to extract from incoming requests as span attributes, split by whitespace.x-pf-number, x-client-id, x-preferred-username, x-resource-accessOTEL_LOG_LEVELLog level used by the logging instrumentor (case-insensitive).infoOTEL_SERVICE_NAMESets the value of theservice.nameresource attribute.f'{k8s namespace}/{k8s pod name}' or f'{username}@{hostname}.{domain}:<{filename of main}>'OTEL_SERVICE_NAMESPACESets the value of theservice.namespaceresource attribute.f'{k8s namespace}' or NoneOTEL_WRAPPER_DISABLEDSet totrueto disable tracing globally (e.g. when running pytest).false(tracing is enabled)Note:Theservice.nameandservice.namespacecan also be set viaOTEL_RESOURCE_ATTRIBUTES, but any settings there
will be overwritten byOTEL_SERVICE_NAMEandOTEL_SERVICE_NAMESPACE. For more details, read the
originalOpenTelemetry docs@instrument_decoratedecorator for functions and classesdecorating a function creates a span whenever the function is calledthe span name is set to the function or class name, and attributes are added for the filename, line number, etcalso works for async functionsalso works for classes and dataclassescreates a span for every method call, includinginit(orpost_init),new, andcallcreates a span for every property get, set, or deleteif this is too verbose, feel free to decorate specific methods in the class insteadit is not recommended to use this on functions that are called thousands of times or classes you create thousands of
instances for (e.g. pydantic classes, usually), since it can be excessively noisyfromopentelemetry_wrapperimportinstrument_decorate@instrument_decoratedefsquare(x):returnx*x@instrument_decorateasyncdefcube(x):returnx*x*x@instrument_decorateclassThing:def__init__(self):self.x=1instrumenting the builtinloggingmodulesets a root logger handler (or more than one) that can output logs to the console or to a file pathlogs are output as json by default to make them easier to work with downstream, setprint_json=Falseto disabletheverboseparameter has no effect on json output, which is maximally verboseotherwise, it defaults toTrue- set toFalseto print less text to the consolethe log level can be specified via thelevelarg, but defaults to whatever was set via
theOTEL_LOG_LEVELenv varimportloggingfromopentelemetry_wrapperimportinstrument_logginginstrument_logging()logging.warning('...')instrumenting fastapiwhile theinstrument_fastapi()function will auto-instrument any app that is created, if your import order is wrong
then your app may end up not being instrumentedas such, it is safer to explicitly callinstrument_fastapi_appfromfastapiimportFastAPIfromopentelemetry_wrapperimportinstrument_fastapi_appapp=instrument_fastapi_app(FastAPI(...))