Package: graphsignal
Graphsignal: Observability for AI StackGraphsignal is an observability platform for AI agents and LLM-powered applications. It helps developers ensure AI applications run as expected and users have the best experience. With Graphsignal, developers can:Trace requests, runs, and sessions with full AI context.See latency breakdown by operations.Analyze model API costs for deployments, models, or users.Get notified about errors and anomalies.Monitor API, compute, and GPU utilization.Learn more atgraphsignal.com.InstallInstall Graphsignal library by running:pip install graphsignalOr clone and install theGitHub repository:git clone https://github.com/graphsignal/graphsignal-python.git
python setup.py installConfigureConfigure Graphsignal tracer by specifying your API key directly or viaGRAPHSIGNAL_API_KEYenvironment variable.importgraphsignalgraphsignal.configure(api_key='my-api-key',deployment='my-model-prod-v1')To get an API key, sign up for a free account atgraphsignal.com. The key can then be found in your account'sSettings / API Keyspage.Alternatively, you can add Graphsignal tracer at command line, when running your module or script. Environment variablesGRAPHSIGNAL_API_KEYandGRAPHSIGNAL_DEPLOYMENTmust be set.python-mgraphsignal<script>python-mgraphsignal-m<module>IntegrateAutomatic integrationGraphsignalauto-instrumentsand traces libraries and frameworks, such asOpenAI,LangChain,LlamaIndex,Hugging Face. Traces, errors, and data, such as prompts and completions, are automatically recorded and available for analysis atapp.graphsignal.com.Some integration examples are available inexamplesrepo.User trackingUser tracking allows grouping and visualization of user-related traces, interactions, metrics, and costs. It also enables detection of user interaction outliers and other events.To enable user tracking, set user identifier asuser_idtag for every request, e.g. in a request handler:graphsignal.set_context_tag('user_id',user_id)or directly, when tracing manually:withgraphsignal.start_trace(tags=dict(user_id=user_id)):...If you are running a single process per user and added Graphsignal at command line, you can set theuser_idtag in an environment variable.envGRAPHSIGNAL_TAGS="user_id=123"python-mgraphsignal<script>Tracing any operationTo measure and monitor operations that are not automatically instrumented, e.g. any model inference or inference API calls, wrap the code withstart_trace()method or use@trace_functiondecorator.withgraphsignal.start_trace('predict'):pred=model(x)@graphsignal.trace_functiondefpredict(x):returnmodel(x)Enable profiling to additionally record code-level statistics. Profiling is disabled by default due to potential overhead. To enable, provideTraceOptionsobject.withgraphsignal.start_trace('predict',options=graphsignal.TraceOptions(enable_profiling=True)):pred=model(x)The tracer will automatically choose a profiler depending on available modules. Currently, CProfile, PyTorch Kineto and Yappi are supported. The Kineto profiler is used iftorchmodule is detected and Yappi profiler is used ifyappimodule is detected. Otherwise, CProfile is used. To properly profileasynciocoroutines, simplypip install yappi.SeeAPI referencefor full documentation.Exception trackingFor auto-instrumented libraries, or when using@trace_functiondecorator,start_trace()method withwithcontext manager or callbacks, exceptions areautomaticallyrecorded. For other cases, useTrace.add_exception.Data monitoringData, such as prompts and completions, is automatically monitored for auto-instrumented libraries. To track data metrics and record data profiles for other cases,Trace.set_data()method can be used.withgraphsignal.start_trace('predict')asspan:span.set_data('input',input_data)The following data types are currently supported:list,dict,set,tuple,str,bytes,numpy.ndarray,tensorflow.Tensor,torch.Tensor.Raw data samples, such as prompts and completions, are recorded by default. To disable, setrecord_data_samples=Falseingraphsignal.configure. Note, that data statistics, such as size, shape or number of missing values will still be recorded.ObserveLog into Graphsignal to monitor and analyze your application and monitor for issues.OverheadGraphsignal tracer is very lightweight. The overhead per trace is measured to be less than 100 microseconds.Security and PrivacyGraphsignal tracer can only open outbound connections tosignal-api.graphsignal.comand send data, no inbound connections or commands are possible.Raw data samples, e.g. prompts, are recorded by default. This feature can be disabled at tracer initialization time, if necessary.TroubleshootingTo enable debug logging, adddebug_mode=Truetoconfigure(). If the debug log doesn't give you any hints on how to fix a problem, please report it to our support team via your account.In case of connection issues, please make sure outgoing connections tohttps://signal-api.graphsignal.comare allowed.