Repository: traceloop/openllmetry
Issue category: Performance Issue

🐛 Bug Report: Performance issues with opentelemetry-instrumentation-openai
### Which component is this bug for?

Anthropic Instrumentation

### 📜 Description

Hi! First, thank you for opentelemetry-instrumentation-openai. It makes my life easier :-)

But, it's computationally extensive, significantly slowing the service of my team. I'm raising a bug report as I couldn't find a better issue type from the ones suggested. Feel free to change this as needed.

Overall, the impact of this issue is reduced throughput. In our production environment removing the library increased our throughput under load 6 folds! The minimal reproducible project [here](https://github.com/Nagasaki45/OTEL-OpenAI-performance-investigation) shows more than 4x throughput increase when dropping the library, without any other change.

### 👟 Reproduction steps

This is a bit elaborated, so I created a [minimal reproducible example](https://github.com/Nagasaki45/OTEL-OpenAI-performance-investigation) for demonstrating the problem and for going through the investigation steps. The README of that repo explains the steps in details. If more info is needed I'm happy to provide it.

### 👍 Expected behavior

Adding opentelemetry-instrumentation-openai to a project should have a minimal impact on a system's performance.

### 👎 Actual Behavior with Screenshots

Adding opentelemetry-instrumentation-openai to a project reduces the throughput under load by ~4-6 times.

### 🤖 Python Version

3.12.0

### 📃 Provide any additional context for the Bug.

_No response_

### 👀 Have you spent some time to check if this bug has been raised before?

- [x] I checked and didn't find similar issue

### Are you willing to submit PR?

Yes I am willing to submit a PR!
🐛 Bug Report: Performance issues with opentelemetry-instrumentation-openai
### Which component is this bug for?

Anthropic Instrumentation

### 📜 Description

Hi! First, thank you for opentelemetry-instrumentation-openai. It makes my life easier :-)

But, it's computationally extensive, significantly slowing the service of my team. I'm raising a bug report as I couldn't find a better issue type from the ones suggested. Feel free to change this as needed.

Overall, the impact of this issue is reduced throughput. In our production environment removing the library increased our throughput under load 6 folds! The minimal reproducible project [here](https://github.com/Nagasaki45/OTEL-OpenAI-performance-investigation) shows more than 4x throughput increase when dropping the library, without any other change.

### 👟 Reproduction steps

This is a bit elaborated, so I created a [minimal reproducible example](https://github.com/Nagasaki45/OTEL-OpenAI-performance-investigation) for demonstrating the problem and for going through the investigation steps. The README of that repo explains the steps in details. If more info is needed I'm happy to provide it.

### 👍 Expected behavior

Adding opentelemetry-instrumentation-openai to a project should have a minimal impact on a system's performance.

### 👎 Actual Behavior with Screenshots

Adding opentelemetry-instrumentation-openai to a project reduces the throughput under load by ~4-6 times.

### 🤖 Python Version

3.12.0

### 📃 Provide any additional context for the Bug.

_No response_

### 👀 Have you spent some time to check if this bug has been raised before?

- [x] I checked and didn't find similar issue

### Are you willing to submit PR?

Yes I am willing to submit a PR!


Functions to edit:
packages/opentelemetry-instrumentation-groq/opentelemetry/instrumentation/groq/utils.py:model_as_dict
packages/opentelemetry-instrumentation-openai/opentelemetry/instrumentation/openai/shared/__init__.py:model_as_dict
packages/opentelemetry-instrumentation-openai/opentelemetry/instrumentation/openai/utils.py:is_openai_v1

Patch:
diff --git a/packages/opentelemetry-instrumentation-groq/opentelemetry/instrumentation/groq/utils.py b/packages/opentelemetry-instrumentation-groq/opentelemetry/instrumentation/groq/utils.py
index e283307b9..f8d750d4c 100644
--- a/packages/opentelemetry-instrumentation-groq/opentelemetry/instrumentation/groq/utils.py
+++ b/packages/opentelemetry-instrumentation-groq/opentelemetry/instrumentation/groq/utils.py
@@ -9,6 +9,8 @@
 GEN_AI_SYSTEM = "gen_ai.system"
 GEN_AI_SYSTEM_GROQ = "groq"
 
+_PYDANTIC_VERSION = version("pydantic")
+
 
 def set_span_attribute(span, name, value):
     if value is not None and value != "":
@@ -68,7 +70,7 @@ def error_metrics_attributes(exception):
 
 
 def model_as_dict(model):
-    if version("pydantic") < "2.0.0":
+    if _PYDANTIC_VERSION < "2.0.0":
         return model.dict()
     if hasattr(model, "model_dump"):
         return model.model_dump()
diff --git a/packages/opentelemetry-instrumentation-openai/opentelemetry/instrumentation/openai/shared/__init__.py b/packages/opentelemetry-instrumentation-openai/opentelemetry/instrumentation/openai/shared/__init__.py
index e54902ac5..87965a3fe 100644
--- a/packages/opentelemetry-instrumentation-openai/opentelemetry/instrumentation/openai/shared/__init__.py
+++ b/packages/opentelemetry-instrumentation-openai/opentelemetry/instrumentation/openai/shared/__init__.py
@@ -23,6 +23,8 @@
 PROMPT_FILTER_KEY = "prompt_filter_results"
 PROMPT_ERROR = "prompt_error"
 
+_PYDANTIC_VERSION = version("pydantic")
+
 # tiktoken encodings map for different model, key is model_name, value is tiktoken encoding
 tiktoken_encodings = {}
 
@@ -236,7 +238,7 @@ def is_streaming_response(response):
 def model_as_dict(model):
     if isinstance(model, dict):
         return model
-    if version("pydantic") < "2.0.0":
+    if _PYDANTIC_VERSION < "2.0.0":
         return model.dict()
     if hasattr(model, "model_dump"):
         return model.model_dump()
diff --git a/packages/opentelemetry-instrumentation-openai/opentelemetry/instrumentation/openai/utils.py b/packages/opentelemetry-instrumentation-openai/opentelemetry/instrumentation/openai/utils.py
index 9030e3004..e0ab375a1 100644
--- a/packages/opentelemetry-instrumentation-openai/opentelemetry/instrumentation/openai/utils.py
+++ b/packages/opentelemetry-instrumentation-openai/opentelemetry/instrumentation/openai/utils.py
@@ -9,9 +9,11 @@
 import openai
 from opentelemetry.instrumentation.openai.shared.config import Config
 
+_OPENAI_VERSION = version("openai")
+
 
 def is_openai_v1():
-    return version("openai") >= "1.0.0"
+    return _OPENAI_VERSION >= "1.0.0"
 
 
 def is_azure_openai(instance):