Package: llamatry
LlamatryLlamatry is a Python package that simplifies the process of instrumenting the OpenAI API using OpenTelemetry. It allows you to monitor and trace the interactions with the OpenAI API, providing insights into the performance and behavior of your code. By leveraging OpenTelemetry, Llamatry supports various output formats, making it easy to integrate with your existing observability stack.Why?Observability is essential for complex applications using large language models (LLMs), as it provides transparency, performance insights, and control over your data and even costs. By integrating observability into your LLM tooling, you can better understand their inner workings, optimize resource usage, and streamline your workflow. Owning your data and leveraging observability empowers you to take control of your AI application's performance.FeaturesOpenTelemetry instrumentation for OpenAI APISupports tracing and monitoring of OpenAI API interactionsCompatible with a wide range of output formats through OpenTelemetryEasy-to-use and straightforward setup processInstallationInstall Llamatry using pip:pipinstallllamatryUsageTo use Llamatry with the OpenAI API, follow these steps:Import the necessary packages:fromopentelemetryimporttracefromopentelemetry.sdk.traceimportTracerProviderfromopentelemetry.sdk.trace.exportimportConsoleSpanExporter,SimpleSpanProcessorfromllamatryimportOpenAIInstrumentorimportopenaiimportosSet up open telemetry:trace.set_tracer_provider(TracerProvider())console_exporter=ConsoleSpanExporter()span_processor=SimpleSpanProcessor(console_exporter)trace.get_tracer_provider().add_span_processor(span_processor)Set up OpenAI API:openai.api_key=os.environ["OPENAI_API_KEY"]Instrument the OpenAI API using Llamatry:OpenAIInstrumentor().instrument()Make API calls to the OpenAI API:response=openai.ChatCompletion.create(messages=[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":"Who won the world series in 2020?"},],max_tokens=50,temperature=0.5,)Console Export:{"name":"ChatCompletion.create","context":{"trace_id":"0x6026b10ff364a1954df343ac2e292fd7","span_id":"0x3f04991076717d88","trace_state":"[]"},"kind":"SpanKind.INTERNAL","parent_id":"0x14abcdaf2d49f177","start_time":"2023-05-09T20:32:10.674716Z","end_time":"2023-05-09T20:32:11.827791Z","status":{"status_code":"UNSET"},"attributes":{"openai.create.model":"gpt-3.5-turbo","openai.create.max_tokens":500,"openai.create.temperature":0.5,"openai.response.id":"chatcmpl-7EOIE4fVofq83WPl1HFNkDI6yBPZ8","openai.response.created":1683664330,"openai.response.model":"gpt-3.5-turbo-0301","openai.usage.completion_tokens":13,"openai.usage.prompt_tokens":30,"openai.usage.total_tokens":43},"events":[],"links":[],"resource":{"attributes":{"telemetry.sdk.language":"python","telemetry.sdk.name":"opentelemetry","telemetry.sdk.version":"1.17.0","service.name":"llamatry"},"schema_url":""}}Traces and other information related to the OpenAI API calls will be output to the console. By using Llamatry, you can easily switch to other exporters supported by OpenTelemetry, such as Jaeger or Zipkin, to visualize and analyze the data in different ways.Decorator and Context Manager helpersLlamatry provides a convenient tracing utility with both decorator and context manager support. This allows you to trace your functions and code blocks easily using the provided Trace class.Using the Trace decoratorTo use theTraceclass as a decorator, you can decorate your function using@Trace.trace. The function's name will be used as the span name by default. If you want to set a custom span name, you can provide it as an argument:@Trace.trace("custom_span_name"). By default if you decorate a function all arguments that are(str, int, float, bool)will be set as attributes.fromllamatryimportTrace@Trace.tracedefyour_function(a,b):# Your function implementation# a, b and automatically set as span attributespass@Trace.trace("custom_span_name")defanother_function():# Your function implementationpasswithTrace.span("custom_span_name")asspan:# Your code block herespan.set_attribute("foo","bar")passUsing the Trace context managerTo use the Trace class as a context manager, use thewithstatement followed byTrace.span("custom_span_name").fromllamatryimportTracewithTrace.span("custom_span_name")asspan:# Your code block herespan.set_attribute("foo","bar")passUsing theTraceclass in this way allows you to easily trace your functions and code blocks, providing better observability and understanding of the performance and behavior of your code.DocumentationFor more information about OpenTelemetry, visit theofficial OpenTelemetry Python documentation.For more information about the OpenAI API, visit theofficial OpenAI API documentation.LicenseLlamatry is released under theMIT License.