Hugging Face's logo

Spaces:
nonhuman
/
nnnn
Runtime error

App Files Community
nnnn / docs /my-website /docs /observability /langfuse_integration.md
nonhuman's picture
nonhuman
Upload 225 files
4ec8dba
preview code
|
raw
history blame contribute delete
No virus
3.03 kB

import Image from '@theme/IdealImage';

Langfuse - Logging LLM Input/Output

LangFuse is open Source Observability & Analytics for LLM Apps Detailed production traces and a granular view on quality, cost and latency

<Image img={require('../../img/langfuse.png')} />

:::info We want to learn how we can make the callbacks better! Meet the LiteLLM founders or join our discord :::

Pre-Requisites

Ensure you have run pip install langfuse for this integration

pip install langfuse litellm

Quick Start

Use just 2 lines of code, to instantly log your responses across all providers with Langfuse Open In Colab

Get your Langfuse API Keys from https://cloud.langfuse.com/

litellm.success_callback = ["langfuse"]

# pip install langfuse 
import litellm
import os

# from https://cloud.langfuse.com/
os.environ["LANGFUSE_PUBLIC_KEY"] = ""
os.environ["LANGFUSE_SECRET_KEY"] = ""
# Optional, defaults to https://cloud.langfuse.com
os.environ["LANGFUSE_HOST"] # optional

# LLM API Keys
os.environ['OPENAI_API_KEY']=""

# set langfuse as a callback, litellm will send the data to langfuse
litellm.success_callback = ["langfuse"] 
 
# openai call
response = litellm.completion(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "user", "content": "Hi 👋 - i'm openai"}
  ]
)

Advanced

Set Custom Generation names, pass metadata 

import litellm
from litellm import completion
import os

# from https://cloud.langfuse.com/
os.environ["LANGFUSE_PUBLIC_KEY"] = ""
os.environ["LANGFUSE_SECRET_KEY"] = ""


# OpenAI and Cohere keys 
# You can use any of the litellm supported providers: https://docs.litellm.ai/docs/providers
os.environ['OPENAI_API_KEY']=""

# set langfuse as a callback, litellm will send the data to langfuse
litellm.success_callback = ["langfuse"] 
 
# openai call
response = completion(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "user", "content": "Hi 👋 - i'm openai"}
  ],
  metadata = {
    "generation_name": "litellm-ishaan-gen", # set langfuse generation name
    # custom metadata fields
    "project": "litellm-proxy" 
  }
)
 
print(response)

Troubleshooting & Errors

Data not getting logged to Langfuse ?

  • Ensure you're on the latest version of langfuse pip install langfuse -U. The latest version allows litellm to log JSON input/outputs to langfuse

Support & Talk to Founders