Repository: DataDog/dd-trace-py
Pull request #13438: feat(litellm): [MLOB-2201] instrument router methods
Author: login: ncybul
Created: 2025-05-16T18:13:21Z | Merged: 2025-05-30T15:13:29Z

This PR makes the following changes to the LiteLLM integration
1. Updates logic for when to send LLM Obs spans from LiteLLM SDK methods to include client-side requests made to a proxy
2. Updates LiteLLM LLM Obs span names from `litellm.request` to their respective function names (e.g. `acompletion`, `router.completion`, etc)
3. Adds support for instrumenting the router methods which can be used on the client or server side
4. Adds metadata tags for routing information onto these new spans under the `router_settings` field
5. Removed token metrics from the workflow spans sent from LiteLLM (token metrics should be attached to the LLM span instead)

# Important
- We do not capture whether the response is a cached response in the trace (however, this can be inferred from the presence or lack of an LLM span for a particular request).
- I still need to revert the changes from this [PR](https://datadoghq.atlassian.net/browse/MLOB-2266) so we are consistent in our decision to send client-side requests that are directed to a proxy (which we originally determined based on the presence of a non-default base URL)
# Example Traces

## Using Proxy Server Only (curl requests)
[Chat completion request](https://dd.datad0g.com/llm/traces?query=%40ml_app%3Anicole-test%20%40event_type%3Aspan%20%40parent_id%3Aundefined&agg_m=count&agg_m_source=base&agg_t=count&fromUser=true&llmPanels=%5B%7B%22t%22%3A%22sampleDetailPanel%22%2C%22rEID%22%3A%22AwAAAZchkGuHnNvN5wAAABhBWmNoa0d1SEFBQWU5UGVDRTBSVkFBQUEAAAAkZjE5NzIxOTAtOTIwYi00YmI2LThjM2EtNDgwYzc5NDBhNmUxAAAI8g%22%7D%5D&sidepanelTab=trace&spanId=2052364930368031968&start=1748613920133&end=1748614820133&paused=false)
[Completion request](https://dd.datad0g.com/llm/traces?query=%40ml_app%3Anicole-test%20%40event_type%3Aspan%20%40parent_id%3Aundefined&agg_m=count&agg_m_source=base&agg_t=count&fromUser=true&llmPanels=%5B%7B%22t%22%3A%22sampleDetailPanel%22%2C%22rEID%22%3A%22AwAAAZchkSwC1ard1wAAABhBWmNoa1N3Q0FBQXg2eW5oR2hGNEFBQUEAAAAkZjE5NzIxOTEtM2UxNi00ZDE4LTkyNDktNjg4ZWVmOWRhOGFjAAAKdQ%22%7D%5D&sidepanelTab=trace&spanId=11245230139731748762&start=1748613961842&end=1748614861842&paused=false)

## Using LiteLLM SDK to route requests to the proxy
[Chat Completion request](https://dd.datad0g.com/llm/traces?query=%40ml_app%3Anicole-test%20%40event_type%3Aspan%20%40parent_id%3Aundefined&agg_m=count&agg_m_source=base&agg_t=count&fromUser=true&llmPanels=%5B%7B%22t%22%3A%22sampleDetailPanel%22%2C%22rEID%22%3A%22AwAAAZchkm2c-H7sSQAAABhBWmNoa20yY0FBQnhKdkVYSjJDTEFBQUEAAAAkZjE5NzIxOTItOGU1My00Yzk1LWFmMmUtMzQ1ODVjNjFhYzhmAAAKQQ%22%7D%5D&sidepanelTab=trace&spanId=16604607814639049896&start=1748614057156&end=1748614957156&paused=false)
[Async chat completion request](https://dd.datad0g.com/llm/traces?query=%40ml_app%3Anicole-test%20%40event_type%3Aspan%20%40parent_id%3Aundefined&agg_m=count&agg_m_source=base&agg_t=count&fromUser=true&llmPanels=%5B%7B%22t%22%3A%22sampleDetailPanel%22%2C%22rEID%22%3A%22AwAAAZchk5g-8Ng9AgAAABhBWmNoazVnLUFBQUdRa1lvNVlqWUFBQUEAAAAkZjE5NzIxOTMtYWEwZC00OWY5LWE5ZTYtMjRhZjI3YTBhMDY3AAAABA%22%7D%5D&sidepanelTab=trace&spanId=3471439022760264096&start=1748614147187&end=1748615047187&paused=false)
[Completion request](https://dd.datad0g.com/llm/traces?query=%40ml_app%3Anicole-test%20%40event_type%3Aspan%20%40parent_id%3Aundefined&agg_m=count&agg_m_source=base&agg_t=count&fromUser=true&llmPanels=%5B%7B%22t%22%3A%22sampleDetailPanel%22%2C%22rEID%22%3A%22AwAAAZchlJzxngTN5wAAABhBWmNobEp6eEFBREVoWUhzdHhvd0FBQUEAAAAkZjE5NzIxOTQtOWQzMC00MTdlLTllMzEtZDM1NjQwMTQxYTk0AAAACA%22%7D%5D&sidepanelTab=trace&spanId=13994069352827426916&start=1748614236859&end=1748615136859&paused=false)
[Async completion request](https://dd.datad0g.com/llm/traces?query=%40ml_app%3Anicole-test%20%40event_type%3Aspan%20%40parent_id%3Aundefined&agg_m=count&agg_m_source=base&agg_t=count&fromUser=true&llmPanels=%5B%7B%22t%22%3A%22sampleDetailPanel%22%2C%22rEID%22%3A%22AwAAAZchlorx0pH4sgAAABhBWmNobG9yeEFBQVVNTEhid1RZWkFBQUEAAAAkZjE5NzIxOTYtOGIzYy00ZDhhLTg1OTItMDRhOTc5NDg3YjYyAAAAEw%22%7D%5D&sidepanelTab=trace&spanId=14672684801939212401&start=1748614318293&end=1748615218293&paused=false)

## Using LiteLLM SDK without a proxy
[Chat completion request](https://dd.datad0g.com/llm/traces?query=%40ml_app%3Anicole-test%20%40event_type%3Aspan%20%40parent_id%3Aundefined&agg_m=count&agg_m_source=base&agg_t=count&fromUser=true&llmPanels=%5B%7B%22t%22%3A%22sampleDetailPanel%22%2C%22rEID%22%3A%22AwAAAZchmCNm-L7sSQAAABhBWmNobUNObUFBQlBaZ2s1c2hWb0FBQUEAAAAkZjE5NzIxOTgtMmRkMC00MmY0LTk3ZDQtODhlMzI4Y2I3MzdlAAAAHg%22%7D%5D&sidepanelTab=trace&spanId=14202682967795653964&start=1748614421912&end=1748615321912&paused=false)
[Async chat completion request](https://dd.datad0g.com/llm/traces?query=%40ml_app%3Anicole-test%20%40event_type%3Aspan%20%40parent_id%3Aundefined&agg_m=count&agg_m_source=base&agg_t=count&fromUser=true&llmPanels=%5B%7B%22t%22%3A%22sampleDetailPanel%22%2C%22rEID%22%3A%22AwAAAZchmCg8nhrN5wAAABhBWmNobUNnOEFBQ25UNjdCc21SVEFBQUEAAAAkZjE5NzIxOTgtMmRkMC00MmY0LTk3ZDQtODhlMzI4Y2I3MzdlAAAAHA%22%7D%5D&sidepanelTab=trace&spanId=4628372838232305416&start=1748614421912&end=1748615321912&paused=false)
[Completion request](https://dd.datad0g.com/llm/traces?query=%40ml_app%3Anicole-test%20%40event_type%3Aspan%20%40parent_id%3Aundefined&agg_m=count&agg_m_source=base&agg_t=count&fromUser=true&llmPanels=%5B%7B%22t%22%3A%22sampleDetailPanel%22%2C%22rEID%22%3A%22AwAAAZchmC3QnhvN5wAAABhBWmNobUMzUUFBQXo0Wk5oNkp0ZUFBQUEAAAAkZjE5NzIxOTgtMmRkMC00MmY0LTk3ZDQtODhlMzI4Y2I3MzdlAAAAGw%22%7D%5D&sidepanelTab=trace&spanId=14885379281700111172&start=1748614421912&end=1748615321912&paused=false)
[Async completion request](https://dd.datad0g.com/llm/traces?query=%40ml_app%3Anicole-test%20%40event_type%3Aspan%20%40parent_id%3Aundefined&agg_m=count&agg_m_source=base&agg_t=count&fromUser=true&llmPanels=%5B%7B%22t%22%3A%22sampleDetailPanel%22%2C%22rEID%22%3A%22AwAAAZchmDLSnhzN5wAAABhBWmNobURMU0FBQ0hMU3BZUUdZN0FBQUEAAAAkZjE5NzIxOTgtNWI2Yi00ZGE3LTg0YjMtMTU5ZmJmYzZlZjNhAAAAEw%22%7D%5D&sidepanelTab=trace&spanId=5286260739580077849&start=1748614461829&end=1748615361829&paused=false)

## Using LiteLLM Router SDK (no proxy)
[Chat Completion request](https://dd.datad0g.com/llm/traces?query=%40ml_app%3Anicole-test%20%40event_type%3Aspan%20%40parent_id%3Aundefined&agg_m=count&agg_m_source=base&agg_t=count&fromUser=true&llmPanels=%5B%7B%22t%22%3A%22sampleDetailPanel%22%2C%22rEID%22%3A%22AwAAAZchmYka5YgYCAAAABhBWmNobVlrYUFBQmE1WW1IeXJOSkFBQUEAAAAkMDE5NzIxOTktYmM5ZC00YmJlLWI1NGEtYzU1M2FlNzM4MThlAAAAFw%22%7D%5D&sidepanelTab=trace&spanId=12413067419586807102&start=1748614519880&end=1748615419880&paused=false)
[Async chat completion request](https://dd.datad0g.com/llm/traces?query=%40ml_app%3Anicole-test%20%40event_type%3Aspan%20%40parent_id%3Aundefined&agg_m=count&agg_m_source=base&agg_t=count&fromUser=true&llmPanels=%5B%7B%22t%22%3A%22sampleDetailPanel%22%2C%22rEID%22%3A%22AwAAAZchmY_cEXMG2QAAABhBWmNobVlfY0FBQldWMlNmMWI1TkFBQUEAAAAkMDE5NzIxOTktYmM5ZC00YmJlLWI1NGEtYzU1M2FlNzM4MThlAAAAFA%22%7D%5D&sidepanelTab=trace&spanId=11298459316548465352&start=1748614519880&end=1748615419880&paused=false)
[Completion request](https://dd.datad0g.com/llm/traces?query=%40ml_app%3Anicole-test%20%40event_type%3Aspan%20%40parent_id%3Aundefined&agg_m=count&agg_m_source=base&agg_t=count&fromUser=true&llmPanels=%5B%7B%22t%22%3A%22sampleDetailPanel%22%2C%22rEID%22%3A%22AwAAAZchmZWLniPN5wAAABhBWmNobVpXTEFBQmJfZnJoRU8xWkFBQUEAAAAkMDE5NzIxOTktYmM5ZC00YmJlLWI1NGEtYzU1M2FlNzM4MThlAAAADg%22%7D%5D&sidepanelTab=trace&spanId=2025915028460653697&start=1748614519880&end=1748615419880&paused=false)
[Async completion request](https://dd.datad0g.com/llm/traces?query=%40ml_app%3Anicole-test%20%40event_type%3Aspan%20%40parent_id%3Aundefined&agg_m=count&agg_m_source=base&agg_t=count&fromUser=true&llmPanels=%5B%7B%22t%22%3A%22sampleDetailPanel%22%2C%22rEID%22%3A%22AwAAAZchmZnV0qz4sgAAABhBWmNobVpuVkFBQ0o3T1hUdmtCMkFBQUEAAAAkMDE5NzIxOTktYmM5ZC00YmJlLWI1NGEtYzU1M2FlNzM4MThlAAAACg%22%7D%5D&sidepanelTab=trace&spanId=13968944998331803863&start=1748614519880&end=1748615419880&paused=false)

## Example Using Tag-Based Routing
Config yaml for tag based routing where providing tags in the request helps the proxy determine which model to use.

```
model_list:
  - model_name: gpt-3.5-turbo
    litellm_params:
      model: openai/gpt-4o
      api_key: "os.environ/OPENAI_API_KEY"
      temperature: 0.2
      tags: ["pick-me"]
  - model_name: gpt-3.5-turbo
    litellm_params:
      model: openai/gpt-3.5-turbo
      api_key: "os.environ/OPENAI_API_KEY"
      temperature: 0.2
      tags: ["dont-pick-me"]

router_settings:
  enable_tag_filtering: True
```

This [trace](https://dd.datad0g.com/llm/traces?query=%40ml_app%3Anicole-test%20%40event_type%3Aspan%20%40parent_id%3Aundefined&agg_m=count&agg_m_source=base&agg_t=count&fromUser=true&llmPanels=%5B%7B%22t%22%3A%22sampleDetailPanel%22%2C%22rEID%22%3A%22AwAAAZchnMPA6p9NogAAABhBWmNobk1QQUFBQjlmWEoxdnE1ckFBQUEAAAAkZjE5NzIxOWMtZjczYS00NTNiLTg3YTgtMTkxZjRjNzE5Y2I3AAAAJQ%22%7D%5D&sidepanelTab=trace&spanId=3165042724601171755&start=1748614734516&end=1748615634516&paused=false) results from the following request, where the model picked is gpt-4o because of the `pick-me` tag. The metadata on the workflow span indicates that tag filtering was enabled via the `enable_tag_filtering` setting and includes the fact that the `pick-me` tag was included in the request and also notes which tags are associated with which models in the model list.

```
curl -i http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {"role": "user", "content": "Heyyyyyy"}
    ],
    "tags": ["pick-me"]
  }
'
```

## Example Using Load Balancing
With the following config file:
```
model_list:
  - model_name: gpt-3.5-turbo
    litellm_params:
      model: openai/gpt-4o
      api_key: "os.environ/OPENAI_API_KEY"
      temperature: 0.2
  - model_name: gpt-3.5-turbo
    litellm_params:
      model: openai/gpt-3.5-turbo
      api_key: "os.environ/OPENAI_API_KEY"
      temperature: 0.2
```
The router will load balance across deployments with the same model group name (in this case `gpt-3.5-turbo`). With the following request:
```
curl -X POST 'http://0.0.0.0:4000/chat/completions' \
-H 'Content-Type: application/json' \
-d '{
  "model": "gpt-3.5-turbo",
  "messages": [
        {"role": "user", "content": "Hi there!"}
    ]
}'
```
The model`gpt-3.5-turbo` is selected ([trace](https://dd.datad0g.com/llm/traces?query=%40ml_app%3Anicole-test%20%40event_type%3Aspan%20%40parent_id%3Aundefined&agg_m=count&agg_m_source=base&agg_t=count&fromUser=false&llmPanels=%5B%7B%22t%22%3A%22sampleDetailPanel%22%2C%22rEID%22%3A%22AwAAAZchnpkwKmxlyQAAABhBWmNobnBrd0FBRDZnRkhRYlQ3YkFBQUEAAAAkZjE5NzIxOWUtYWI3Yi00NGY0LWFhNDctZTFlMTIzZmFmYjhjAAAAMw%22%7D%5D&sidepanelTab=trace&spanId=13225486193538532176&start=1748614937609&end=1748615837609&paused=false)).

However, mocking out a rate limit error with the following request causes `gpt-4o` to be selected ([trace](https://dd.datad0g.com/llm/traces?query=%40ml_app%3Anicole-test%20%40event_type%3Aspan%20%40parent_id%3Aundefined&agg_m=count&agg_m_source=base&agg_t=count&fromUser=true&llmPanels=%5B%7B%22t%22%3A%22sampleDetailPanel%22%2C%22rEID%22%3A%22AwAAAZchn3zv4jdLTQAAABhBWmNobjN6dkFBQUp1UmVra3ZPaEFBQUEAAAAkZjE5NzIxOWYtN2NmMC00OWUzLWFhYWMtOTM0NmVkODQxOTc2AAAACg%22%7D%5D&sidepanelTab=trace&spanId=14509004101071907461&start=1748614910619&end=1748615810619&paused=false)).
```
curl -X POST 'http://0.0.0.0:4000/chat/completions' \
-H 'Content-Type: application/json' \                                                                          
-d '{
  "model": "gpt-3.5-turbo",
  "messages": [
        {"role": "user", "content": "Hi there!"}
    ],
    "mock_testing_rate_limit_error": true
}'
```

## Example Using Budget Based Routing
The following config yaml includes provider budget configurations.
```
model_list:
    - model_name: gpt-3.5-turbo
      litellm_params:
        model: openai/gpt-3.5-turbo
        api_key: os.environ/OPENAI_API_KEY

router_settings:
  provider_budget_config: 
    openai: 
      budget_limit: 0.000000000001
      time_period: 1d 
```
Running the following curl command leads to this [trace](https://dd.datad0g.com/llm/traces?query=%40ml_app%3Anicole-test%20%40event_type%3Aspan%20%40parent_id%3Aundefined&agg_m=count&agg_m_source=base&agg_t=count&fromUser=true&llmPanels=%5B%7B%22t%22%3A%22sampleDetailPanel%22%2C%22rEID%22%3A%22AwAAAZchoWFj8GGXyQAAABhBWmNob1dGakFBQjZ1VnVIcnYxNEFBQUEAAAAkZjE5NzIxYTEtODExMC00NDMyLTg3MDYtYTQ4ZTdkODBmYTcxAAAAKA%22%7D%5D&spanId=13225486193538532176&start=1748615036888&end=1748615936888&paused=false) where you can see the provider budget config in the workflow span's metadata.
```
curl -i http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {"role": "user", "content": "Heyyyyyy"}
    ]
  }
'
```

A subsequent request will lead to the budget being exceeded and result in this [trace](https://dd.datad0g.com/llm/traces?query=%40ml_app%3Anicole-test%20%40event_type%3Aspan%20%40parent_id%3Aundefined&agg_m=count&agg_m_source=base&agg_t=count&fromUser=true&llmPanels=%5B%7B%22t%22%3A%22sampleDetailPanel%22%2C%22rEID%22%3A%22AwAAAZchoXYC8XQ9AgAAABhBWmNob1hZQ0FBQ1JZN04wV2pwbkFBQUEAAAAkZjE5NzIxYTEtODExMC00NDMyLTg3MDYtYTQ4ZTdkODBmYTcxAAAAJQ%22%7D%5D&sidepanelTab=trace&spanId=7587227003036292133&start=1748615036888&end=1748615936888&paused=false) where an error is present.

## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met 
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)


[](https://datadoghq.atlassian.net/browse/MLOB-2928)


Diff:
diff --git a/ddtrace/contrib/_litellm.py b/ddtrace/contrib/_litellm.py
index 4b735e72fd1..b5b95247fc6 100644
--- a/ddtrace/contrib/_litellm.py
+++ b/ddtrace/contrib/_litellm.py
@@ -1,5 +1,5 @@
 """
-The LiteLLM integration instruments the LiteLLM Python SDK's sync and async ``completion`` and ``text_completion`` methods.
+The LiteLLM integration instruments the LiteLLM Python SDK and proxy server.
 
 All traces submitted from the LiteLLM integration are tagged by:
 
diff --git a/ddtrace/contrib/internal/litellm/patch.py b/ddtrace/contrib/internal/litellm/patch.py
index 6aed01be783..250a2b27bd1 100644
--- a/ddtrace/contrib/internal/litellm/patch.py
+++ b/ddtrace/contrib/internal/litellm/patch.py
@@ -5,10 +5,12 @@
 from ddtrace import config
 from ddtrace.contrib.internal.litellm.utils import TracedLiteLLMAsyncStream
 from ddtrace.contrib.internal.litellm.utils import TracedLiteLLMStream
+from ddtrace.contrib.internal.litellm.utils import extract_host_tag
 from ddtrace.contrib.trace_utils import unwrap
 from ddtrace.contrib.trace_utils import with_traced_module
 from ddtrace.contrib.trace_utils import wrap
 from ddtrace.internal.utils import get_argument_value
+from ddtrace.llmobs._constants import LITELLM_ROUTER_INSTANCE_KEY
 from ddtrace.llmobs._integrations import LiteLLMIntegration
 from ddtrace.trace import Pin
 
@@ -23,87 +25,125 @@ def get_version() -> str:
 
 @with_traced_module
 def traced_completion(litellm, pin, func, instance, args, kwargs):
-    return _traced_completion(litellm, pin, func, instance, args, kwargs, False)
+    operation = func.__name__
+    integration = litellm._datadog_integration
+    model = get_argument_value(args, kwargs, 0, "model", None)
+    host = extract_host_tag(kwargs)
+    span = integration.trace(
+        pin,
+        operation,
+        model=model,
+        host=host,
+        submit_to_llmobs=integration.should_submit_to_llmobs(kwargs, model),
+    )
+    stream = kwargs.get("stream", False)
+    resp = None
+    try:
+        resp = func(*args, **kwargs)
+        if stream:
+            return TracedLiteLLMStream(resp, integration, span, kwargs)
+        return resp
+    except Exception:
+        span.set_exc_info(*sys.exc_info())
+        raise
+    finally:
+        # streamed spans will be finished separately once the stream generator is exhausted
+        if not stream:
+            if integration.is_pc_sampled_llmobs(span):
+                integration.llmobs_set_tags(span, args=args, kwargs=kwargs, response=resp, operation=operation)
+            span.finish()
 
 
 @with_traced_module
 async def traced_acompletion(litellm, pin, func, instance, args, kwargs):
-    return await _traced_acompletion(litellm, pin, func, instance, args, kwargs, False)
-
-
-@with_traced_module
-def traced_text_completion(litellm, pin, func, instance, args, kwargs):
-    return _traced_completion(litellm, pin, func, instance, args, kwargs, True)
+    operation = func.__name__
+    integration = litellm._datadog_integration
+    model = get_argument_value(args, kwargs, 0, "model", None)
+    host = extract_host_tag(kwargs)
+    span = integration.trace(
+        pin,
+        operation,
+        model=model,
+        host=host,
+        submit_to_llmobs=integration.should_submit_to_llmobs(kwargs, model),
+    )
+    stream = kwargs.get("stream", False)
+    resp = None
+    try:
+        resp = await func(*args, **kwargs)
+        if stream:
+            return TracedLiteLLMAsyncStream(resp, integration, span, kwargs)
+        return resp
+    except Exception:
+        span.set_exc_info(*sys.exc_info())
+        raise
+    finally:
+        # streamed spans will be finished separately once the stream generator is exhausted
+        if not stream:
+            if integration.is_pc_sampled_llmobs(span):
+                integration.llmobs_set_tags(span, args=args, kwargs=kwargs, response=resp, operation=operation)
+            span.finish()
 
 
 @with_traced_module
-async def traced_atext_completion(litellm, pin, func, instance, args, kwargs):
-    return await _traced_acompletion(litellm, pin, func, instance, args, kwargs, True)
-
-
-def _traced_completion(litellm, pin, func, instance, args, kwargs, is_completion):
+def traced_router_completion(litellm, pin, func, instance, args, kwargs):
+    operation = f"router.{func.__name__}"
     integration = litellm._datadog_integration
     model = get_argument_value(args, kwargs, 0, "model", None)
-    host = None
-    if "host" in kwargs.get("metadata", {}).get("headers", {}):
-        host = kwargs["metadata"]["headers"]["host"]
+    host = extract_host_tag(kwargs)
     span = integration.trace(
         pin,
-        func.__name__,
+        operation,
         model=model,
         host=host,
-        submit_to_llmobs=integration.should_submit_to_llmobs(kwargs, model),
+        submit_to_llmobs=True,
     )
     stream = kwargs.get("stream", False)
     resp = None
     try:
         resp = func(*args, **kwargs)
         if stream:
-            return TracedLiteLLMStream(resp, integration, span, kwargs, is_completion)
+            resp._add_router_span_info(span, kwargs, instance)
         return resp
     except Exception:
         span.set_exc_info(*sys.exc_info())
         raise
     finally:
-        # streamed spans will be finished separately once the stream generator is exhausted
         if not stream:
             if integration.is_pc_sampled_llmobs(span):
-                integration.llmobs_set_tags(
-                    span, args=args, kwargs=kwargs, response=resp, operation="completion" if is_completion else "chat"
-                )
+                kwargs[LITELLM_ROUTER_INSTANCE_KEY] = instance
+                integration.llmobs_set_tags(span, args=args, kwargs=kwargs, response=resp, operation=operation)
             span.finish()
 
 
-async def _traced_acompletion(litellm, pin, func, instance, args, kwargs, is_completion):
+@with_traced_module
+async def traced_router_acompletion(litellm, pin, func, instance, args, kwargs):
+    operation = f"router.{func.__name__}"
     integration = litellm._datadog_integration
     model = get_argument_value(args, kwargs, 0, "model", None)
-    host = None
-    if "host" in kwargs.get("metadata", {}).get("headers", {}):
-        host = kwargs["metadata"]["headers"]["host"]
+    host = extract_host_tag(kwargs)
     span = integration.trace(
         pin,
-        func.__name__,
+        operation,
         model=model,
         host=host,
-        submit_to_llmobs=integration.should_submit_to_llmobs(kwargs, model),
+        submit_to_llmobs=True,
     )
     stream = kwargs.get("stream", False)
     resp = None
     try:
         resp = await func(*args, **kwargs)
         if stream:
-            return TracedLiteLLMAsyncStream(resp, integration, span, kwargs, is_completion)
+            resp._add_router_span_info(span, kwargs, instance)
         return resp
     except Exception:
         span.set_exc_info(*sys.exc_info())
         raise
     finally:
-        # streamed spans will be finished separately once the stream generator is exhausted
         if not stream:
             if integration.is_pc_sampled_llmobs(span):
-                integration.llmobs_set_tags(
-                    span, args=args, kwargs=kwargs, response=resp, operation="completion" if is_completion else "chat"
-                )
+                kwargs[LITELLM_ROUTER_INSTANCE_KEY] = instance
+                integration.llmobs_set_tags(span, args=args, kwargs=kwargs, response=resp, operation=operation)
             span.finish()
 
 
@@ -129,10 +169,14 @@ def patch():
 
     wrap("litellm", "completion", traced_completion(litellm))
     wrap("litellm", "acompletion", traced_acompletion(litellm))
-    wrap("litellm", "text_completion", traced_text_completion(litellm))
-    wrap("litellm", "atext_completion", traced_atext_completion(litellm))
+    wrap("litellm", "text_completion", traced_completion(litellm))
+    wrap("litellm", "atext_completion", traced_acompletion(litellm))
     wrap("litellm", "get_llm_provider", traced_get_llm_provider(litellm))
     wrap("litellm", "main.get_llm_provider", traced_get_llm_provider(litellm))
+    wrap("litellm", "router.Router.completion", traced_router_completion(litellm))
+    wrap("litellm", "router.Router.acompletion", traced_router_acompletion(litellm))
+    wrap("litellm", "router.Router.text_completion", traced_router_completion(litellm))
+    wrap("litellm", "router.Router.atext_completion", traced_router_acompletion(litellm))
 
 
 def unpatch():
@@ -147,5 +191,8 @@ def unpatch():
     unwrap(litellm, "atext_completion")
     unwrap(litellm, "get_llm_provider")
     unwrap(litellm.main, "get_llm_provider")
-
+    unwrap(litellm.router.Router, "completion")
+    unwrap(litellm.router.Router, "acompletion")
+    unwrap(litellm.router.Router, "text_completion")
+    unwrap(litellm.router.Router, "atext_completion")
     delattr(litellm, "_datadog_integration")
diff --git a/ddtrace/contrib/internal/litellm/utils.py b/ddtrace/contrib/internal/litellm/utils.py
index 11b996891a4..58046297e9a 100644
--- a/ddtrace/contrib/internal/litellm/utils.py
+++ b/ddtrace/contrib/internal/litellm/utils.py
@@ -3,6 +3,7 @@
 import wrapt
 
 from ddtrace.internal.logger import get_logger
+from ddtrace.llmobs._constants import LITELLM_ROUTER_INSTANCE_KEY
 from ddtrace.llmobs._integrations.utils import openai_construct_completion_from_streamed_chunks
 from ddtrace.llmobs._integrations.utils import openai_construct_message_from_streamed_chunks
 
@@ -10,15 +11,41 @@
 log = get_logger(__name__)
 
 
+def extract_host_tag(kwargs):
+    if "host" in kwargs.get("metadata", {}).get("headers", {}):
+        return kwargs["metadata"]["headers"]["host"]
+    return None
+
+
 class BaseTracedLiteLLMStream(wrapt.ObjectProxy):
-    def __init__(self, wrapped, integration, span, kwargs, is_completion=False):
+    def __init__(self, wrapped, integration, span, kwargs):
         super().__init__(wrapped)
         n = kwargs.get("n", 1) or 1
         self._dd_integration = integration
-        self._dd_span = span
-        self._kwargs = kwargs
+        self._span_info = [(span, kwargs)]
         self._streamed_chunks = [[] for _ in range(n)]
-        self._is_completion = is_completion
+
+    def _add_router_span_info(self, span, kwargs, instance):
+        """Handler to add router span to this streaming object.
+
+        Helps to ensure that all spans associated with a single stream are finished and have the correct tags.
+        """
+        kwargs[LITELLM_ROUTER_INSTANCE_KEY] = instance
+        self._span_info.append((span, kwargs))
+
+    def _finish_spans(self):
+        """Helper to finish all spans associated with this stream."""
+        formatted_completions = None
+        for span, kwargs in self._span_info:
+            if not formatted_completions:
+                formatted_completions = _process_finished_stream(
+                    self._dd_integration, span, kwargs, self._streamed_chunks, span.resource
+                )
+            elif self._dd_integration.is_pc_sampled_llmobs(span):
+                self._dd_integration.llmobs_set_tags(
+                    span, args=[], kwargs=kwargs, response=formatted_completions, operation=span.resource
+                )
+            span.finish()
 
 
 class TracedLiteLLMStream(BaseTracedLiteLLMStream):
@@ -35,13 +62,12 @@ def __iter__(self):
                 yield chunk
                 _loop_handler(chunk, self._streamed_chunks)
         except Exception:
-            self._dd_span.set_exc_info(*sys.exc_info())
+            if self._span_info and len(self._span_info[0]) > 0:
+                span = self._span_info[0][0]
+                span.set_exc_info(*sys.exc_info())
             raise
         finally:
-            _process_finished_stream(
-                self._dd_integration, self._dd_span, self._kwargs, self._streamed_chunks, self._is_completion
-            )
-            self._dd_span.finish()
+            self._finish_spans()
 
     def __next__(self):
         try:
@@ -51,13 +77,12 @@ def __next__(self):
         except StopIteration:
             raise
         except Exception:
-            self._dd_span.set_exc_info(*sys.exc_info())
+            if self._span_info and len(self._span_info[0]) > 0:
+                span = self._span_info[0][0]
+                span.set_exc_info(*sys.exc_info())
             raise
         finally:
-            _process_finished_stream(
-                self._dd_integration, self._dd_span, self._kwargs, self._streamed_chunks, self._is_completion
-            )
-            self._dd_span.finish()
+            self._finish_spans()
 
 
 class TracedLiteLLMAsyncStream(BaseTracedLiteLLMStream):
@@ -74,13 +99,12 @@ async def __aiter__(self):
                 yield chunk
                 _loop_handler(chunk, self._streamed_chunks)
         except Exception:
-            self._dd_span.set_exc_info(*sys.exc_info())
+            if self._span_info and len(self._span_info[0]) > 0:
+                span = self._span_info[0][0]
+                span.set_exc_info(*sys.exc_info())
             raise
         finally:
-            _process_finished_stream(
-                self._dd_integration, self._dd_span, self._kwargs, self._streamed_chunks, self._is_completion
-            )
-            self._dd_span.finish()
+            self._finish_spans()
 
     async def __anext__(self):
         try:
@@ -90,13 +114,12 @@ async def __anext__(self):
         except StopAsyncIteration:
             raise
         except Exception:
-            self._dd_span.set_exc_info(*sys.exc_info())
+            if self._span_info and len(self._span_info[0]) > 0:
+                span = self._span_info[0][0]
+                span.set_exc_info(*sys.exc_info())
             raise
         finally:
-            _process_finished_stream(
-                self._dd_integration, self._dd_span, self._kwargs, self._streamed_chunks, self._is_completion
-            )
-            self._dd_span.finish()
+            self._finish_spans()
 
 
 def _loop_handler(chunk, streamed_chunks):
@@ -110,9 +133,10 @@ def _loop_handler(chunk, streamed_chunks):
         streamed_chunks[0].insert(0, chunk)
 
 
-def _process_finished_stream(integration, span, kwargs, streamed_chunks, is_completion=False):
+def _process_finished_stream(integration, span, kwargs, streamed_chunks, operation):
     try:
-        if is_completion:
+        formatted_completions = None
+        if integration.is_completion_operation(operation):
             formatted_completions = [
                 openai_construct_completion_from_streamed_chunks(choice) for choice in streamed_chunks
             ]
@@ -120,10 +144,10 @@ def _process_finished_stream(integration, span, kwargs, streamed_chunks, is_comp
             formatted_completions = [
                 openai_construct_message_from_streamed_chunks(choice) for choice in streamed_chunks
             ]
-        operation = "completion" if is_completion else "chat"
         if integration.is_pc_sampled_llmobs(span):
             integration.llmobs_set_tags(
                 span, args=[], kwargs=kwargs, response=formatted_completions, operation=operation
             )
     except Exception:
         log.warning("Error processing streamed completion/chat response.", exc_info=True)
+    return formatted_completions
diff --git a/ddtrace/llmobs/_constants.py b/ddtrace/llmobs/_constants.py
index c8a0e855370..d1d4f953300 100644
--- a/ddtrace/llmobs/_constants.py
+++ b/ddtrace/llmobs/_constants.py
@@ -28,6 +28,7 @@
 
 GEMINI_APM_SPAN_NAME = "gemini.request"
 LANGCHAIN_APM_SPAN_NAME = "langchain.request"
+LITELLM_APM_SPAN_NAME = "litellm.request"
 OPENAI_APM_SPAN_NAME = "openai.request"
 VERTEXAI_APM_SPAN_NAME = "vertexai.request"
 CREWAI_APM_SPAN_NAME = "crewai.request"
@@ -82,3 +83,5 @@
 # Tool call arguments are used to lookup the associated tool call info.
 # When there are no tool call args, we use this as a place-holder lookup key
 OAI_HANDOFF_TOOL_ARG = "{}"
+
+LITELLM_ROUTER_INSTANCE_KEY = "_dd.router_instance"
diff --git a/ddtrace/llmobs/_integrations/litellm.py b/ddtrace/llmobs/_integrations/litellm.py
index 80107500afe..2d5df81940f 100644
--- a/ddtrace/llmobs/_integrations/litellm.py
+++ b/ddtrace/llmobs/_integrations/litellm.py
@@ -5,11 +5,17 @@
 from typing import Tuple
 
 from ddtrace.internal.utils import get_argument_value
+from ddtrace.llmobs._constants import INPUT_MESSAGES
 from ddtrace.llmobs._constants import INPUT_TOKENS_METRIC_KEY
+from ddtrace.llmobs._constants import INPUT_VALUE
+from ddtrace.llmobs._constants import LITELLM_ROUTER_INSTANCE_KEY
+from ddtrace.llmobs._constants import METADATA
 from ddtrace.llmobs._constants import METRICS
 from ddtrace.llmobs._constants import MODEL_NAME
 from ddtrace.llmobs._constants import MODEL_PROVIDER
+from ddtrace.llmobs._constants import OUTPUT_MESSAGES
 from ddtrace.llmobs._constants import OUTPUT_TOKENS_METRIC_KEY
+from ddtrace.llmobs._constants import OUTPUT_VALUE
 from ddtrace.llmobs._constants import SPAN_KIND
 from ddtrace.llmobs._constants import TOTAL_TOKENS_METRIC_KEY
 from ddtrace.llmobs._integrations.base import BaseLLMIntegration
@@ -20,6 +26,27 @@
 from ddtrace.trace import Span
 
 
+TEXT_COMPLETION_OPERATIONS = (
+    "text_completion",
+    "atext_completion",
+    "router.text_completion",
+    "router.atext_completion",
+)
+
+MODEL_LIST_KEYS = ("api_base", "model", "organization", "max_tokens", "temperature", "tags")
+METADATA_KEYS = (
+    "deployment",
+    "endpoint",
+    "headers",
+    "litellm_api_version",
+    "model_group",
+    "model_group_size",
+    "model_info",
+    "tags",
+)
+PROXY_SERVER_REQUEST_KEYS = ("body", "url", "method")
+
+
 class LiteLLMIntegration(BaseLLMIntegration):
     _integration_name = "litellm"
     # maps requested model name to parsed model name and provider
@@ -45,19 +72,137 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if self.is_completion_operation(operation):
             openai_set_meta_tags_from_completion(span, kwargs, response)
         else:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
         )
 
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base")
+        # select certain keys within metadata to avoid sending sensitive data
+        if "metadata" in metadata:
+            inner_metadata = {}
+            for key in METADATA_KEYS:
+                value = metadata["metadata"].get(key)
+                if key == "headers" and value:
+                    value = {"host": value.get("host")}
+                if value:
+                    inner_metadata[key] = value
+            metadata["metadata"] = inner_metadata
+        if "proxy_server_request" in metadata:
+            metadata["proxy_server_request"] = self._select_keys(
+                metadata["proxy_server_request"], PROXY_SERVER_REQUEST_KEYS
+            )
+
+        if base_url and "model" in kwargs:
+            metadata["model"] = kwargs["model"]
+            span._set_ctx_items({METADATA: metadata})
+            return
+        if base_url or "router" not in operation:
+            span._set_ctx_items({METADATA: metadata})
+            return
+
+        llm_router = kwargs.get(LITELLM_ROUTER_INSTANCE_KEY)
+        if not llm_router:
+            span._set_ctx_items({METADATA: metadata})
+            return
+
+        metadata["router_settings"] = {
+            "router_general_settings": getattr(llm_router, "router_general_settings", None),
+            "routing_strategy": getattr(llm_router, "routing_strategy", None),
+            "routing_strategy_args": getattr(llm_router, "routing_strategy_args", None),
+            "provider_budget_config": getattr(llm_router, "provider_budget_config", None),
+            "retry_policy": getattr(llm_router, "retry_policy", None),
+            "enable_tag_filtering": getattr(llm_router, "enable_tag_filtering", None),
+        }
+        if hasattr(llm_router, "get_model_list"):
+            metadata["router_settings"]["model_list"] = self._construct_litellm_model_list(llm_router.get_model_list())
+
+        span._set_ctx_items({METADATA: metadata})
+
+    def _construct_litellm_model_list(self, model_list: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+        new_model_list = []
+        for model in model_list:
+            litellm_params = model.get("litellm_params", {})
+            litellm_params_dict = self._select_keys(litellm_params, MODEL_LIST_KEYS)
+            new_model_list.append(
+                {
+                    "model_name": model.get("model_name"),
+                    "litellm_params": litellm_params_dict,
+                }
+            )
+        return new_model_list
+
+    def _update_input_output_value(self, span: Span, span_kind: str = ""):
+        if span_kind == "llm":
+            return
+        input_messages = span._get_ctx_item(INPUT_MESSAGES)
+        output_messages = span._get_ctx_item(OUTPUT_MESSAGES)
+        if input_messages:
+            span._set_ctx_item(INPUT_VALUE, input_messages)
+        if output_messages:
+            span._set_ctx_item(OUTPUT_VALUE, output_messages)
+
+    def _has_downstream_openai_span(self, kwargs: Dict[str, Any], model: Optional[str] = None) -> bool:
+        """
+        Determine whether an LLM span will be submitted for the given request from outside the LiteLLM integration.
+
+        Currently, this only applies to the OpenAI integration when the following conditions are met:
+            - request is not streamed
+            - model provider is OpenAI/AzureOpenAI
+            - the OpenAI integration is enabled
+        """
+        stream = kwargs.get("stream", False)
+        model_lower = model.lower() if model else ""
+        # best effort attempt to check if Open AI or Azure since model_provider is unknown until request completes
+        is_openai_model = any(prefix in model_lower for prefix in ("gpt", "openai", "azure"))
+        return is_openai_model and not stream and LLMObs._integration_is_enabled("openai")
+
+    def _get_span_kind(
+        self, kwargs: Dict[str, Any], model: Optional[str] = None, operation: Optional[str] = None
+    ) -> str:
+        """
+        Workflow span should be submitted to LLMObs if:
+            - span represents a router operation OR
+            - base_url is set (indicates a request to the proxy)
+        LLM spans should be submitted to LLMObs if:
+            - base_url is not set AND an LLM span will not be submitted elsewhere
+        """
+        base_url = kwargs.get("api_base")
+        if self.is_router_operation(operation) or base_url:
+            return "workflow"
+        return "llm"
+
+    def _select_keys(self, data: Dict[str, Any], keys_to_select: Tuple[str, ...]) -> Dict[str, Any]:
+        new_data = {}
+        for key in keys_to_select:
+            value = data.get(key)
+            if value:
+                new_data[key] = value
+        return new_data
+
+    def is_completion_operation(self, operation: str) -> bool:
+        return operation in TEXT_COMPLETION_OPERATIONS
+
+    def is_router_operation(self, operation: Optional[str]) -> bool:
+        return "router" in operation if operation else False
+
     @staticmethod
-    def _extract_llmobs_metrics(resp: Any) -> Dict[str, Any]:
-        if not resp:
+    def _extract_llmobs_metrics(resp: Any, span_kind: str) -> Dict[str, Any]:
+        if not resp or span_kind != "llm":
             return {}
         if isinstance(resp, list):
             token_usage = _get_attr(resp[0], "usage", None)
@@ -75,22 +220,11 @@ def _extract_llmobs_metrics(resp: Any) -> Dict[str, Any]:
 
     def should_submit_to_llmobs(self, kwargs: Dict[str, Any], model: Optional[str] = None) -> bool:
         """
-        Span should be NOT submitted to LLMObs if:
-            - base_url is not None: is a proxy request and we will capture the LLM request downstream
-            - non-streamed request and model provider is OpenAI/AzureOpenAI and the OpenAI integration
-                is enabled: this request will be captured in the OpenAI integration instead
+        LiteLLM spans will be submitted to LLMObs unless the following are true:
+            - base_url is not set AND
+            - the LLM request will be submitted elsewhere (e.g. OpenAI integration)
         """
-        base_url = kwargs.get("api_base", None)
-        if base_url is not None:
-            return False
-        stream = kwargs.get("stream", False)
-        model_lower = model.lower() if model else ""
-        # model provider is unknown until request completes; therefore, this is a best effort attempt to check
-        # if model provider is Open AI or Azure
-        if (
-            any(prefix in model_lower for prefix in ("gpt", "openai", "azure"))
-            and not stream
-            and LLMObs._integration_is_enabled("openai")
-        ):
+        base_url = kwargs.get("api_base")
+        if not base_url and self._has_downstream_openai_span(kwargs, model):
             return False
         return True
diff --git a/ddtrace/llmobs/_integrations/utils.py b/ddtrace/llmobs/_integrations/utils.py
index 7a202bf1be4..8ee39ff0ad7 100644
--- a/ddtrace/llmobs/_integrations/utils.py
+++ b/ddtrace/llmobs/_integrations/utils.py
@@ -19,6 +19,7 @@
 from ddtrace.llmobs._constants import DISPATCH_ON_TOOL_CALL_OUTPUT_USED
 from ddtrace.llmobs._constants import INPUT_MESSAGES
 from ddtrace.llmobs._constants import INPUT_TOKENS_METRIC_KEY
+from ddtrace.llmobs._constants import LITELLM_ROUTER_INSTANCE_KEY
 from ddtrace.llmobs._constants import METADATA
 from ddtrace.llmobs._constants import OAI_HANDOFF_TOOL_ARG
 from ddtrace.llmobs._constants import OUTPUT_MESSAGES
@@ -32,6 +33,24 @@
 
 ACCEPTED_OPENAI_DEFAULT_HOSTNAMES = ("api.openai.com", "api.deepseek.com")
 AZURE_URL_REGEX_PATTERN = "^[\\w.-]*openai\\.azure\\.com$"
+OPENAI_SKIPPED_COMPLETION_TAGS = (
+    "model",
+    "prompt",
+    "api_key",
+    "user_api_key",
+    "user_api_key_hash",
+    LITELLM_ROUTER_INSTANCE_KEY,
+)
+OPENAI_SKIPPED_CHAT_TAGS = (
+    "model",
+    "messages",
+    "tools",
+    "functions",
+    "api_key",
+    "user_api_key",
+    "user_api_key_hash",
+    LITELLM_ROUTER_INSTANCE_KEY,
+)
 
 
 def is_openai_default_base_url(base_url: Optional[str] = None) -> bool:
@@ -297,9 +316,7 @@ def openai_set_meta_tags_from_completion(span: Span, kwargs: Dict[str, Any], com
     prompt = kwargs.get("prompt", "")
     if isinstance(prompt, str):
         prompt = [prompt]
-    parameters = {
-        k: v for k, v in kwargs.items() if k not in ("model", "prompt", "api_key", "user_api_key", "user_api_key_hash")
-    }
+    parameters = {k: v for k, v in kwargs.items() if k not in OPENAI_SKIPPED_COMPLETION_TAGS}
     output_messages = [{"content": ""}]
     if not span.error and completions:
         choices = getattr(completions, "choices", completions)
@@ -321,11 +338,7 @@ def openai_set_meta_tags_from_chat(span: Span, kwargs: Dict[str, Any], messages:
         if tool_call_id:
             core.dispatch(DISPATCH_ON_TOOL_CALL_OUTPUT_USED, (tool_call_id, span))
         input_messages.append({"content": str(_get_attr(m, "content", "")), "role": str(_get_attr(m, "role", ""))})
-    parameters = {
-        k: v
-        for k, v in kwargs.items()
-        if k not in ("model", "messages", "tools", "functions", "api_key", "user_api_key", "user_api_key_hash")
-    }
+    parameters = {k: v for k, v in kwargs.items() if k not in OPENAI_SKIPPED_CHAT_TAGS}
     span._set_ctx_items({INPUT_MESSAGES: input_messages, METADATA: parameters})
 
     if span.error or not messages:
diff --git a/ddtrace/llmobs/_utils.py b/ddtrace/llmobs/_utils.py
index c7c9a7de0f2..c00b1e12641 100644
--- a/ddtrace/llmobs/_utils.py
+++ b/ddtrace/llmobs/_utils.py
@@ -16,6 +16,7 @@
 from ddtrace.llmobs._constants import INTERNAL_QUERY_VARIABLE_KEYS
 from ddtrace.llmobs._constants import IS_EVALUATION_SPAN
 from ddtrace.llmobs._constants import LANGCHAIN_APM_SPAN_NAME
+from ddtrace.llmobs._constants import LITELLM_APM_SPAN_NAME
 from ddtrace.llmobs._constants import ML_APP
 from ddtrace.llmobs._constants import NAME
 from ddtrace.llmobs._constants import OPENAI_APM_SPAN_NAME
@@ -34,6 +35,7 @@
     GEMINI_APM_SPAN_NAME,
     LANGCHAIN_APM_SPAN_NAME,
     VERTEXAI_APM_SPAN_NAME,
+    LITELLM_APM_SPAN_NAME,
 )
 
 
diff --git a/docs/integrations.rst b/docs/integrations.rst
index 41fa315c10a..fa9a6babccd 100644
--- a/docs/integrations.rst
+++ b/docs/integrations.rst
@@ -269,6 +269,12 @@ LangChain
 ^^^^^^^^^
 .. automodule:: ddtrace.contrib._langchain
 
+.. _litellm:
+
+LiteLLM
+^^^^^^^
+.. automodule:: ddtrace.contrib._litellm
+
 .. _logbook:
 
 Logbook
diff --git a/docs/spelling_wordlist.txt b/docs/spelling_wordlist.txt
index 3d677ec34ef..8ae74702847 100644
--- a/docs/spelling_wordlist.txt
+++ b/docs/spelling_wordlist.txt
@@ -151,6 +151,7 @@ libdatadog
 libddwaf
 lifecycle
 linters
+litellm
 LLM
 lockfiles
 logbook
diff --git a/releasenotes/notes/litellm-router-tracing-04d28e5b3fc73702.yaml b/releasenotes/notes/litellm-router-tracing-04d28e5b3fc73702.yaml
new file mode 100644
index 00000000000..79383352336
--- /dev/null
+++ b/releasenotes/notes/litellm-router-tracing-04d28e5b3fc73702.yaml
@@ -0,0 +1,4 @@
+---
+features:
+  - |
+    litellm: Adds APM and LLM Observability tracing support for LiteLLM's synchronous and asynchronous ``completion`` and ``text_completion`` router methods.
\ No newline at end of file
diff --git a/tests/contrib/litellm/conftest.py b/tests/contrib/litellm/conftest.py
index bd7be20fd48..1a5d96d80ac 100644
--- a/tests/contrib/litellm/conftest.py
+++ b/tests/contrib/litellm/conftest.py
@@ -1,3 +1,4 @@
+from litellm import Router
 import pytest
 
 from ddtrace.contrib.internal.litellm.patch import patch
@@ -5,6 +6,7 @@
 from ddtrace.llmobs import LLMObs as llmobs_service
 from ddtrace.trace import Pin
 from tests.contrib.litellm.utils import get_request_vcr
+from tests.contrib.litellm.utils import model_list
 from tests.llmobs._utils import TestLLMObsSpanWriter
 from tests.utils import DummyTracer
 from tests.utils import override_global_config
@@ -75,3 +77,8 @@ def request_vcr():
 @pytest.fixture
 def request_vcr_include_localhost():
     return get_request_vcr(ignore_localhost=False)
+
+
+@pytest.fixture
+def router():
+    yield Router(model_list=model_list)
diff --git a/tests/contrib/litellm/test_litellm.py b/tests/contrib/litellm/test_litellm.py
index 5ddee4173e6..958bd6d64c8 100644
--- a/tests/contrib/litellm/test_litellm.py
+++ b/tests/contrib/litellm/test_litellm.py
@@ -103,3 +103,73 @@ def test_litellm_completion_different_models(litellm, snapshot_context, request_
                 stream=False,
                 n=1,
             )
+
+
+@pytest.mark.parametrize("stream,n", [(True, 1), (True, 2), (False, 1), (False, 2)])
+def test_litellm_router_completion(litellm, snapshot_context, request_vcr, router, stream, n):
+    with snapshot_context(token="tests.contrib.litellm.test_litellm.test_litellm_router_completion"):
+        with request_vcr.use_cassette(get_cassette_name(stream=stream, n=n)):
+            messages = [{"content": "Hey, what is up?", "role": "user"}]
+            resp = router.completion(
+                model="gpt-3.5-turbo",
+                messages=messages,
+                stream=stream,
+                n=n,
+            )
+            if stream:
+                for _ in resp:
+                    pass
+
+
+@pytest.mark.parametrize("stream,n", [(True, 1), (True, 2), (False, 1), (False, 2)])
+async def test_litellm_router_acompletion(litellm, snapshot_context, request_vcr, router, stream, n):
+    with snapshot_context(
+        token="tests.contrib.litellm.test_litellm.test_litellm_router_completion", ignores=["resource"]
+    ):
+        with request_vcr.use_cassette(get_cassette_name(stream=stream, n=n)):
+            messages = [{"content": "Hey, what is up?", "role": "user"}]
+            resp = await router.acompletion(
+                model="gpt-3.5-turbo",
+                messages=messages,
+                stream=stream,
+                n=n,
+            )
+            if stream:
+                async for _ in resp:
+                    pass
+
+
+@pytest.mark.parametrize("stream,n", [(True, 1), (True, 2), (False, 1), (False, 2)])
+def test_litellm_router_text_completion(litellm, snapshot_context, request_vcr, router, stream, n):
+    with snapshot_context(
+        token="tests.contrib.litellm.test_litellm.test_litellm_router_completion", ignores=["resource"]
+    ):
+        with request_vcr.use_cassette(get_cassette_name(stream=stream, n=n)):
+            prompt = "What color is the sky?"
+            resp = router.text_completion(
+                model="gpt-3.5-turbo",
+                prompt=prompt,
+                stream=stream,
+                n=n,
+            )
+            if stream:
+                for _ in resp:
+                    pass
+
+
+@pytest.mark.parametrize("stream,n", [(True, 1), (True, 2), (False, 1), (False, 2)])
+async def test_litellm_router_atext_completion(litellm, snapshot_context, request_vcr, router, stream, n):
+    with snapshot_context(
+        token="tests.contrib.litellm.test_litellm.test_litellm_router_completion", ignores=["resource"]
+    ):
+        with request_vcr.use_cassette(get_cassette_name(stream=stream, n=n)):
+            prompt = "What color is the sky?"
+            resp = await router.atext_completion(
+                model="gpt-3.5-turbo",
+                prompt=prompt,
+                stream=stream,
+                n=n,
+            )
+            if stream:
+                async for _ in resp:
+                    pass
diff --git a/tests/contrib/litellm/test_litellm_llmobs.py b/tests/contrib/litellm/test_litellm_llmobs.py
index cfb8cc59a00..e0d05d6395f 100644
--- a/tests/contrib/litellm/test_litellm_llmobs.py
+++ b/tests/contrib/litellm/test_litellm_llmobs.py
@@ -2,12 +2,15 @@
 
 from ddtrace._monkey import patch
 from ddtrace._trace.pin import Pin
+from ddtrace.llmobs._utils import safe_json
 from tests.contrib.litellm.utils import async_consume_stream
 from tests.contrib.litellm.utils import consume_stream
+from tests.contrib.litellm.utils import expected_router_settings
 from tests.contrib.litellm.utils import get_cassette_name
 from tests.contrib.litellm.utils import parse_response
 from tests.contrib.litellm.utils import tools
 from tests.llmobs._utils import _expected_llmobs_llm_span_event
+from tests.llmobs._utils import _expected_llmobs_non_llm_span_event
 from tests.utils import flaky
 
 
@@ -200,17 +203,82 @@ async def test_atext_completion(self, litellm, request_vcr, llmobs_events, mock_
             tags={"ml_app": "<ml-app-name>", "service": "tests.contrib.litellm"},
         )
 
-    @flaky(until=1748750400, reason="Patching Open AI to be used within the LiteLLM library appears to be flaky")
-    def test_completion_openai_enabled(self, litellm, request_vcr, llmobs_events, mock_tracer, stream, n):
+    def test_completion_proxy(self, litellm, request_vcr_include_localhost, llmobs_events, mock_tracer, stream, n):
+        with request_vcr_include_localhost.use_cassette(get_cassette_name(stream, n, proxy=True)):
+            messages = [{"content": "Hey, what is up?", "role": "user"}]
+            resp = litellm.completion(
+                model="gpt-3.5-turbo",
+                messages=messages,
+                stream=stream,
+                n=n,
+                stream_options={"include_usage": True},
+                api_base="http://0.0.0.0:4000",
+            )
+            if stream:
+                output_messages, _ = consume_stream(resp, n)
+            else:
+                output_messages, _ = parse_response(resp)
+
+        span = mock_tracer.pop_traces()[0][0]
+        assert len(llmobs_events) == 1
+        assert llmobs_events[0] == _expected_llmobs_non_llm_span_event(
+            span,
+            span_kind="workflow",
+            input_value=safe_json(messages, ensure_ascii=False),
+            output_value=safe_json(output_messages, ensure_ascii=False),
+            metadata={
+                "stream": stream,
+                "n": n,
+                "stream_options": {"include_usage": True},
+                "api_base": "http://0.0.0.0:4000",
+                "model": "gpt-3.5-turbo",
+            },
+            tags={"ml_app": "<ml-app-name>", "service": "tests.contrib.litellm"},
+        )
+
+    def test_router_completion(self, litellm, request_vcr, llmobs_events, mock_tracer, router, stream, n):
         with request_vcr.use_cassette(get_cassette_name(stream, n)):
-            patch(openai=True)
-            import openai
+            messages = [{"content": "Hey, what is up?", "role": "user"}]
+            resp = router.completion(
+                model="gpt-3.5-turbo",
+                messages=messages,
+                stream=stream,
+                n=n,
+                stream_options={"include_usage": True},
+            )
+            if stream:
+                output_messages, _ = consume_stream(resp, n)
+            else:
+                output_messages, _ = parse_response(resp)
 
-            pin = Pin.get_from(openai)
-            pin._override(openai, tracer=mock_tracer)
+        trace = mock_tracer.pop_traces()[0]
+        assert len(trace) == 2
+        router_span = trace[0]
+
+        assert len(llmobs_events) == 2
+        router_event = llmobs_events[1]
+        llm_event = llmobs_events[0]
 
+        assert llm_event["meta"]["span.kind"] == "llm"
+        assert llm_event["name"] == "completion"
+        assert router_event == _expected_llmobs_non_llm_span_event(
+            router_span,
+            span_kind="workflow",
+            input_value=safe_json(messages, ensure_ascii=False),
+            output_value=safe_json(output_messages, ensure_ascii=False),
+            metadata={
+                "stream": stream,
+                "n": n,
+                "stream_options": {"include_usage": True},
+                "router_settings": expected_router_settings,
+            },
+            tags={"ml_app": "<ml-app-name>", "service": "tests.contrib.litellm"},
+        )
+
+    async def test_router_acompletion(self, litellm, request_vcr, llmobs_events, mock_tracer, router, stream, n):
+        with request_vcr.use_cassette(get_cassette_name(stream, n)):
             messages = [{"content": "Hey, what is up?", "role": "user"}]
-            resp = litellm.completion(
+            resp = await router.acompletion(
                 model="gpt-3.5-turbo",
                 messages=messages,
                 stream=stream,
@@ -218,14 +286,121 @@ def test_completion_openai_enabled(self, litellm, request_vcr, llmobs_events, mo
                 stream_options={"include_usage": True},
             )
             if stream:
-                for _ in resp:
-                    pass
+                output_messages, _ = await async_consume_stream(resp, n)
+            else:
+                output_messages, _ = parse_response(resp)
 
-        assert len(llmobs_events) == 1
-        assert llmobs_events[0]["name"] == "OpenAI.createChatCompletion" if not stream else "litellm.request"
+        trace = mock_tracer.pop_traces()[0]
+        assert len(trace) == 2
+        router_span = trace[0]
+
+        assert len(llmobs_events) == 2
+        router_event = llmobs_events[1]
+        llm_event = llmobs_events[0]
+
+        assert llm_event["meta"]["span.kind"] == "llm"
+        assert llm_event["name"] == "acompletion"
+        assert router_event == _expected_llmobs_non_llm_span_event(
+            router_span,
+            span_kind="workflow",
+            input_value=safe_json(messages, ensure_ascii=False),
+            output_value=safe_json(output_messages, ensure_ascii=False),
+            metadata={
+                "stream": stream,
+                "n": n,
+                "stream_options": {"include_usage": True},
+                "router_settings": expected_router_settings,
+            },
+            tags={"ml_app": "<ml-app-name>", "service": "tests.contrib.litellm"},
+        )
+
+    def test_router_text_completion(self, litellm, request_vcr, llmobs_events, mock_tracer, router, stream, n):
+        with request_vcr.use_cassette(get_cassette_name(stream, n)):
+            prompt = "Hey, what is up?"
+            resp = router.text_completion(
+                model="gpt-3.5-turbo",
+                prompt=prompt,
+                stream=stream,
+                n=n,
+                stream_options={"include_usage": True},
+            )
+            if stream:
+                output_messages, _ = consume_stream(resp, n, is_completion=True)
+            else:
+                output_messages, _ = parse_response(resp, is_completion=True)
+
+        trace = mock_tracer.pop_traces()[0]
+        assert len(trace) == 2
+        router_span = trace[0]
+
+        assert len(llmobs_events) == 2
+        router_event = llmobs_events[1]
+        llm_event = llmobs_events[0]
+
+        assert llm_event["meta"]["span.kind"] == "llm"
+        assert llm_event["name"] == "text_completion"
+        assert router_event == _expected_llmobs_non_llm_span_event(
+            router_span,
+            span_kind="workflow",
+            input_value=safe_json([{"content": prompt}], ensure_ascii=False),
+            output_value=safe_json(output_messages, ensure_ascii=False),
+            metadata={
+                "stream": stream,
+                "n": n,
+                "stream_options": {"include_usage": True},
+                "router_settings": expected_router_settings,
+            },
+            tags={"ml_app": "<ml-app-name>", "service": "tests.contrib.litellm"},
+        )
+
+    async def test_router_atext_completion(self, litellm, request_vcr, llmobs_events, mock_tracer, router, stream, n):
+        with request_vcr.use_cassette(get_cassette_name(stream, n)):
+            prompt = "Hey, what is up?"
+            resp = await router.atext_completion(
+                model="gpt-3.5-turbo",
+                prompt=prompt,
+                stream=stream,
+                n=n,
+                stream_options={"include_usage": True},
+            )
+            if stream:
+                output_messages, _ = await async_consume_stream(resp, n, is_completion=True)
+            else:
+                output_messages, _ = parse_response(resp, is_completion=True)
+
+        trace = mock_tracer.pop_traces()[0]
+        assert len(trace) == 2
+        router_span = trace[0]
+
+        assert len(llmobs_events) == 2
+        router_event = llmobs_events[1]
+        llm_event = llmobs_events[0]
+
+        assert llm_event["meta"]["span.kind"] == "llm"
+        assert llm_event["name"] == "atext_completion"
+        assert router_event == _expected_llmobs_non_llm_span_event(
+            router_span,
+            span_kind="workflow",
+            input_value=safe_json([{"content": prompt}], ensure_ascii=False),
+            output_value=safe_json(output_messages, ensure_ascii=False),
+            metadata={
+                "stream": stream,
+                "n": n,
+                "stream_options": {"include_usage": True},
+                "router_settings": expected_router_settings,
+            },
+            tags={"ml_app": "<ml-app-name>", "service": "tests.contrib.litellm"},
+        )
+
+    @flaky(until=1748750400, reason="Patching Open AI to be used within the LiteLLM library appears to be flaky")
+    def test_completion_openai_enabled(self, litellm, request_vcr, llmobs_events, mock_tracer, stream, n):
+        with request_vcr.use_cassette(get_cassette_name(stream, n)):
+            patch(openai=True)
+            import openai
+
+            pin = Pin.get_from(openai)
+            pin._override(openai, tracer=mock_tracer)
 
-    def test_completion_proxy(self, litellm, request_vcr_include_localhost, llmobs_events, mock_tracer, stream, n):
-        with request_vcr_include_localhost.use_cassette(get_cassette_name(stream, n, proxy=True)):
             messages = [{"content": "Hey, what is up?", "role": "user"}]
             resp = litellm.completion(
                 model="gpt-3.5-turbo",
@@ -233,10 +408,10 @@ def test_completion_proxy(self, litellm, request_vcr_include_localhost, llmobs_e
                 stream=stream,
                 n=n,
                 stream_options={"include_usage": True},
-                api_base="http://0.0.0.0:4000",
             )
             if stream:
-                consume_stream(resp, n)
+                for _ in resp:
+                    pass
 
-        # client side requests made to the proxy are not submitted to LLMObs
-        assert len(llmobs_events) == 0
+        assert len(llmobs_events) == 1
+        assert llmobs_events[0]["name"] == "OpenAI.createChatCompletion" if not stream else "litellm.request"
diff --git a/tests/contrib/litellm/utils.py b/tests/contrib/litellm/utils.py
index 1779f5dfda9..753370fd3e3 100644
--- a/tests/contrib/litellm/utils.py
+++ b/tests/contrib/litellm/utils.py
@@ -1,6 +1,7 @@
 import json
 import os
 
+from litellm import RouterGeneralSettings
 import vcr
 
 
@@ -138,6 +139,23 @@ def parse_response(resp, is_completion=False):
     return output_messages, token_metrics
 
 
+model_list = [
+    {
+        "model_name": "gpt-3.5-turbo",
+        "litellm_params": {
+            "model": "gpt-3.5-turbo",
+            "api_key": "<not-a-real-key>",
+        },
+    },
+    {
+        "model_name": "gpt-4",
+        "litellm_params": {
+            "model": "gpt-4",
+            "api_key": "<not-a-real-key>",
+        },
+    },
+]
+
 tools = [
     {
         "type": "function",
@@ -158,3 +176,26 @@ def parse_response(resp, is_completion=False):
         },
     }
 ]
+
+expected_router_settings = {
+    "router_general_settings": RouterGeneralSettings(async_only_mode=False, pass_through_all_models=False),
+    "routing_strategy": "simple-shuffle",
+    "routing_strategy_args": {},
+    "provider_budget_config": None,
+    "retry_policy": None,
+    "enable_tag_filtering": False,
+    "model_list": [
+        {
+            "model_name": "gpt-3.5-turbo",
+            "litellm_params": {
+                "model": "gpt-3.5-turbo",
+            },
+        },
+        {
+            "model_name": "gpt-4",
+            "litellm_params": {
+                "model": "gpt-4",
+            },
+        },
+    ],
+}
diff --git a/tests/snapshots/tests.contrib.litellm.test_litellm.test_litellm_router_completion.json b/tests/snapshots/tests.contrib.litellm.test_litellm.test_litellm_router_completion.json
new file mode 100644
index 00000000000..06e41c11c27
--- /dev/null
+++ b/tests/snapshots/tests.contrib.litellm.test_litellm.test_litellm_router_completion.json
@@ -0,0 +1,45 @@
+[[
+  {
+    "name": "litellm.request",
+    "service": "tests.contrib.litellm",
+    "resource": "router.completion",
+    "trace_id": 0,
+    "span_id": 1,
+    "parent_id": 0,
+    "type": "",
+    "error": 0,
+    "meta": {
+      "_dd.p.dm": "-0",
+      "_dd.p.tid": "682c9a2b00000000",
+      "language": "python",
+      "litellm.request.model": "gpt-3.5-turbo",
+      "runtime-id": "c3947e7c67484e44af5d4c7c20d9216c"
+    },
+    "metrics": {
+      "_dd.measured": 1,
+      "_dd.top_level": 1,
+      "_dd.tracer_kr": 1.0,
+      "_sampling_priority_v1": 1,
+      "process_id": 92661
+    },
+    "duration": 68491000,
+    "start": 1747753515704152000
+  },
+     {
+       "name": "litellm.request",
+       "service": "tests.contrib.litellm",
+       "resource": "completion",
+       "trace_id": 0,
+       "span_id": 2,
+       "parent_id": 1,
+       "type": "",
+       "error": 0,
+       "meta": {
+         "litellm.request.model": "gpt-3.5-turbo"
+       },
+       "metrics": {
+         "_dd.measured": 1
+       },
+       "duration": 66297000,
+       "start": 1747753515705671000
+     }]]


Reviews:
user: login: emmettbutler
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-05-21T10:20:15Z
user: login: wantsui
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-21T21:26:48Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T13:35:52Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T17:07:31Z
user: login: Yun-Kim
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T03:05:13Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T14:28:18Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T14:52:24Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T14:53:59Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T14:56:53Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T15:09:04Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T15:29:50Z
user: login: datadog-datadog-prod-us1[bot]
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T16:08:37Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T16:13:38Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T16:14:55Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T16:21:05Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T16:22:48Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T16:25:04Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T16:33:13Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T16:45:43Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T16:49:52Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T16:52:13Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T17:12:26Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T17:16:36Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T17:23:02Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T17:27:38Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T17:57:54Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T19:15:20Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-27T13:42:11Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-27T13:43:41Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-27T13:45:04Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-27T13:56:14Z
user: login: rachelyangdog
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-28T19:45:57Z
user: login: Kyle-Verhoog
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-28T20:00:07Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-28T20:32:04Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-28T20:39:05Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-28T20:40:56Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-28T20:52:45Z
user: login: Kyle-Verhoog
body: just a couple things then good to go
state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-29T19:05:44Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-29T20:03:42Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-29T20:35:55Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-29T20:51:28Z
user: login: ncybul
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-29T21:18:49Z

Review comments:
diff_hunk: @@ -107,6 +88,68 @@ async def _traced_acompletion(litellm, pin, func, instance, args, kwargs, is_com
             span.finish()
 
 
+@with_traced_module
+def traced_router_completion(litellm, pin, func, instance, args, kwargs):
+    operation = f"router.{func.__name__}"
+    integration = litellm._datadog_integration
+    model = get_argument_value(args, kwargs, 0, "model", None)
+    host = extract_host_tag(kwargs)
+    span = integration.trace(
+        pin,
+        operation,
+        model=model,
+        host=host,
+        submit_to_llmobs=True,
+    )
+    stream = kwargs.get("stream", False)
+    resp = None
+    try:
+        resp = func(*args, **kwargs)
+        if stream:
+            resp.add_router_span_info(span, kwargs, instance)
+        return resp
+    except Exception:
+        span.set_exc_info(*sys.exc_info())
+        raise
+    finally:
+        if not stream:
+            if integration.is_pc_sampled_llmobs(span):
+                kwargs["router_instance"] = instance
+                integration.llmobs_set_tags(span, args=args, kwargs=kwargs, response=resp, operation=operation)
+            span.finish()
path: ddtrace/contrib/internal/litellm/patch.py
user: login: wantsui
body: What are the possible values of stream? Just `True` or `False`?

Asking because it looks like we only finish the span if there isn't a stream.

Is there a scenario in which we keep a bunch of these spans open?
created_at: 2025-05-21T21:26:48Z
author_association: COLLABORATOR
diff_hunk: @@ -107,6 +88,68 @@ async def _traced_acompletion(litellm, pin, func, instance, args, kwargs, is_com
             span.finish()
 
 
+@with_traced_module
+def traced_router_completion(litellm, pin, func, instance, args, kwargs):
+    operation = f"router.{func.__name__}"
+    integration = litellm._datadog_integration
+    model = get_argument_value(args, kwargs, 0, "model", None)
+    host = extract_host_tag(kwargs)
+    span = integration.trace(
+        pin,
+        operation,
+        model=model,
+        host=host,
+        submit_to_llmobs=True,
+    )
+    stream = kwargs.get("stream", False)
+    resp = None
+    try:
+        resp = func(*args, **kwargs)
+        if stream:
+            resp.add_router_span_info(span, kwargs, instance)
+        return resp
+    except Exception:
+        span.set_exc_info(*sys.exc_info())
+        raise
+    finally:
+        if not stream:
+            if integration.is_pc_sampled_llmobs(span):
+                kwargs["router_instance"] = instance
+                integration.llmobs_set_tags(span, args=args, kwargs=kwargs, response=resp, operation=operation)
+            span.finish()
path: ddtrace/contrib/internal/litellm/patch.py
user: login: ncybul
body: Yep, stream can be True or False. If stream is True, then we return `TracedLiteLLMStream` or `TracedAsyncLiteLLMStream` which finishes the span once the stream has been entirely consumed.
created_at: 2025-05-22T13:35:52Z
author_association: CONTRIBUTOR
diff_hunk: @@ -107,6 +88,68 @@ async def _traced_acompletion(litellm, pin, func, instance, args, kwargs, is_com
             span.finish()
 
 
+@with_traced_module
+def traced_router_completion(litellm, pin, func, instance, args, kwargs):
+    operation = f"router.{func.__name__}"
+    integration = litellm._datadog_integration
+    model = get_argument_value(args, kwargs, 0, "model", None)
+    host = extract_host_tag(kwargs)
+    span = integration.trace(
+        pin,
+        operation,
+        model=model,
+        host=host,
+        submit_to_llmobs=True,
+    )
+    stream = kwargs.get("stream", False)
+    resp = None
+    try:
+        resp = func(*args, **kwargs)
+        if stream:
+            resp.add_router_span_info(span, kwargs, instance)
path: ddtrace/contrib/internal/litellm/patch.py
user: login: ncybul
body: `router.completion` will call `litellm.completion` or another SDK method internally. Therefore, when `stream = True`, `resp` will be a `TracedLiteLLMStream`. Originally, I thought I could wrap `resp` in another `TracedLiteLLMStream`; however, this does not work as is because of the way `wrapt.ObjectProxy` handles nested wrapped objects (the attributes of the inner proxy object get overwritten causing the inner span to be unfinished and the chunks to be repeated in the resulting streamed chunks). 

The solution that I came up with here is to return the original `TracedLiteLLMStream` with some additional information included (notably, the router span) such that when this stream is fully consumed, all relevant spans can be finished simultaneously.
created_at: 2025-05-22T17:07:31Z
author_association: CONTRIBUTOR
diff_hunk: @@ -23,30 +24,10 @@ def get_version() -> str:
 
 @with_traced_module
 def traced_completion(litellm, pin, func, instance, args, kwargs):
-    return _traced_completion(litellm, pin, func, instance, args, kwargs, False)
-
-
-@with_traced_module
-async def traced_acompletion(litellm, pin, func, instance, args, kwargs):
-    return await _traced_acompletion(litellm, pin, func, instance, args, kwargs, False)
-
-
-@with_traced_module
-def traced_text_completion(litellm, pin, func, instance, args, kwargs):
-    return _traced_completion(litellm, pin, func, instance, args, kwargs, True)
-
-
-@with_traced_module
-async def traced_atext_completion(litellm, pin, func, instance, args, kwargs):
-    return await _traced_acompletion(litellm, pin, func, instance, args, kwargs, True)
-
-
-def _traced_completion(litellm, pin, func, instance, args, kwargs, is_completion):
+    is_completion = "text" in func.__name__
path: ddtrace/contrib/internal/litellm/patch.py
user: login: Yun-Kim
body:  `text_completion` is a completion, but `completion` is not?
created_at: 2025-05-23T01:02:27Z
author_association: CONTRIBUTOR
diff_hunk: @@ -23,30 +24,10 @@ def get_version() -> str:
 
 @with_traced_module
 def traced_completion(litellm, pin, func, instance, args, kwargs):
-    return _traced_completion(litellm, pin, func, instance, args, kwargs, False)
-
-
-@with_traced_module
-async def traced_acompletion(litellm, pin, func, instance, args, kwargs):
-    return await _traced_acompletion(litellm, pin, func, instance, args, kwargs, False)
-
-
-@with_traced_module
-def traced_text_completion(litellm, pin, func, instance, args, kwargs):
-    return _traced_completion(litellm, pin, func, instance, args, kwargs, True)
-
-
-@with_traced_module
-async def traced_atext_completion(litellm, pin, func, instance, args, kwargs):
-    return await _traced_acompletion(litellm, pin, func, instance, args, kwargs, True)
-
-
-def _traced_completion(litellm, pin, func, instance, args, kwargs, is_completion):
+    is_completion = "text" in func.__name__
path: ddtrace/contrib/internal/litellm/patch.py
user: login: Yun-Kim
body: Don't see this being used anywhere..?
created_at: 2025-05-23T01:03:31Z
author_association: CONTRIBUTOR
diff_hunk: @@ -107,6 +88,68 @@ async def _traced_acompletion(litellm, pin, func, instance, args, kwargs, is_com
             span.finish()
 
 
+@with_traced_module
+def traced_router_completion(litellm, pin, func, instance, args, kwargs):
+    operation = f"router.{func.__name__}"
+    integration = litellm._datadog_integration
+    model = get_argument_value(args, kwargs, 0, "model", None)
+    host = extract_host_tag(kwargs)
+    span = integration.trace(
+        pin,
+        operation,
+        model=model,
+        host=host,
+        submit_to_llmobs=True,
+    )
+    stream = kwargs.get("stream", False)
+    resp = None
+    try:
+        resp = func(*args, **kwargs)
+        if stream:
+            resp.add_router_span_info(span, kwargs, instance)
+        return resp
+    except Exception:
+        span.set_exc_info(*sys.exc_info())
+        raise
+    finally:
+        if not stream:
path: ddtrace/contrib/internal/litellm/patch.py
user: login: Yun-Kim
body: What if `stream=True` but there is an error? 
created_at: 2025-05-23T02:29:49Z
author_association: CONTRIBUTOR
diff_hunk: @@ -107,6 +88,68 @@ async def _traced_acompletion(litellm, pin, func, instance, args, kwargs, is_com
             span.finish()
 
 
+@with_traced_module
+def traced_router_completion(litellm, pin, func, instance, args, kwargs):
+    operation = f"router.{func.__name__}"
+    integration = litellm._datadog_integration
+    model = get_argument_value(args, kwargs, 0, "model", None)
+    host = extract_host_tag(kwargs)
+    span = integration.trace(
+        pin,
+        operation,
+        model=model,
+        host=host,
+        submit_to_llmobs=True,
+    )
+    stream = kwargs.get("stream", False)
+    resp = None
+    try:
+        resp = func(*args, **kwargs)
+        if stream:
+            resp.add_router_span_info(span, kwargs, instance)
+        return resp
+    except Exception:
+        span.set_exc_info(*sys.exc_info())
+        raise
+    finally:
+        if not stream:
+            if integration.is_pc_sampled_llmobs(span):
+                kwargs["router_instance"] = instance
path: ddtrace/contrib/internal/litellm/patch.py
user: login: Yun-Kim
body: Is kwargs guaranteed to contain a `router_instance`? Can we be defensive? 
created_at: 2025-05-23T02:30:21Z
author_association: CONTRIBUTOR
diff_hunk: @@ -35,13 +63,11 @@ def __iter__(self):
                 yield chunk
                 _loop_handler(chunk, self._streamed_chunks)
         except Exception:
-            self._dd_span.set_exc_info(*sys.exc_info())
+            for span in self._dd_spans:
+                span.set_exc_info(*sys.exc_info())
path: ddtrace/contrib/internal/litellm/utils.py
user: login: Yun-Kim
body: Why do we need to set the same exception info on both spans? 
created_at: 2025-05-23T02:32:10Z
author_association: CONTRIBUTOR
diff_hunk: @@ -10,15 +10,43 @@
 log = get_logger(__name__)
 
 
+def extract_host_tag(kwargs):
+    if "host" in kwargs.get("metadata", {}).get("headers", {}):
+        return kwargs["metadata"]["headers"]["host"]
+    return None
+
+
 class BaseTracedLiteLLMStream(wrapt.ObjectProxy):
-    def __init__(self, wrapped, integration, span, kwargs, is_completion=False):
+    def __init__(self, wrapped, integration, span, kwargs):
         super().__init__(wrapped)
         n = kwargs.get("n", 1) or 1
         self._dd_integration = integration
-        self._dd_span = span
-        self._kwargs = kwargs
+        self._dd_spans = [span]
+        self._kwargs = [kwargs]  # each span is associated with a different kwargs
         self._streamed_chunks = [[] for _ in range(n)]
-        self._is_completion = is_completion
+
+    def add_router_span_info(self, span, kwargs, instance):
+        """Handler to add router span to this streaming object.
+
+        Helps to ensure that all spans associated with a single stream are finished and have the correct tags.
+        """
+        self._dd_spans.append(span)
+        kwargs["router_instance"] = instance
+        self._kwargs.append(kwargs)
+
+    def finish_spans(self):
+        """Helper to finish all spans associated with this stream."""
+        formatted_completions = None
+        for span, kwargs in zip(self._dd_spans, self._kwargs):
+            if not formatted_completions:
+                formatted_completions = _process_finished_stream(
+                    self._dd_integration, span, kwargs, self._streamed_chunks, span.resource
+                )
+            elif self._dd_integration.is_pc_sampled_llmobs(span):
+                self._dd_integration.llmobs_set_tags(
+                    span, args=[], kwargs=kwargs, response=formatted_completions, operation=span.resource
+                )
path: ddtrace/contrib/internal/litellm/utils.py
user: login: Yun-Kim
body: Are we trying to just process the finished stream once? And attach them to both spans (if so, why?)
created_at: 2025-05-23T02:34:47Z
author_association: CONTRIBUTOR
diff_hunk: @@ -120,10 +142,12 @@ def _process_finished_stream(integration, span, kwargs, streamed_chunks, is_comp
             formatted_completions = [
                 openai_construct_message_from_streamed_chunks(choice) for choice in streamed_chunks
             ]
-        operation = "completion" if is_completion else "chat"
+        operation = operation if "router" in operation else "completion" if is_completion else "chat"
path: ddtrace/contrib/internal/litellm/utils.py
user: login: Yun-Kim
body: what is the actual operation (which contains "router")? I don't like modifying the value of a function arg in the function itself (let's do this outside `_process_finished_stream`)
created_at: 2025-05-23T02:36:54Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
path: ddtrace/llmobs/_integrations/litellm.py
user: login: Yun-Kim
body: Can we combine this instead of a nested conditional?
created_at: 2025-05-23T02:38:19Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
path: ddtrace/llmobs/_integrations/litellm.py
user: login: Yun-Kim
body: Why?
created_at: 2025-05-23T02:38:47Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
path: ddtrace/llmobs/_integrations/litellm.py
user: login: Yun-Kim
body: Also, the check is self-explanatory (no need for the comment)
created_at: 2025-05-23T02:39:09Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
path: ddtrace/llmobs/_integrations/litellm.py
user: login: Yun-Kim
body: Same here. Unnecessary comment
created_at: 2025-05-23T02:39:27Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
path: ddtrace/llmobs/_integrations/litellm.py
user: login: Yun-Kim
body: Why do we have to manually remove this key-value pair? Can we just avoid setting it in the first place?
created_at: 2025-05-23T02:40:20Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
+
+            if not kwargs.get("router_instance", None):
path: ddtrace/llmobs/_integrations/litellm.py
user: login: Yun-Kim
body: I prefer doing this once:
```
llm_router = kwargs.get("router_instance") # None is a redundant arg to .get()
if not llm_router:
    ... return
```
created_at: 2025-05-23T02:41:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
+
+            if not kwargs.get("router_instance", None):
+                span._set_ctx_items({METADATA: metadata})
+                return
+
+            routing_info = {}
+            llm_router = kwargs["router_instance"]
+            routing_info["router_general_settings"] = getattr(llm_router, "router_general_settings", None)
+            routing_info["routing_strategy"] = getattr(llm_router, "routing_strategy", None)
+            routing_info["routing_strategy_args"] = getattr(llm_router, "routing_strategy_args", None)
+            routing_info["provider_budget_config"] = getattr(llm_router, "provider_budget_config", None)
+            routing_info["retry_policy"] = getattr(llm_router, "retry_policy", None)
+            routing_info["enable_tag_filtering"] = getattr(llm_router, "enable_tag_filtering", None)
path: ddtrace/llmobs/_integrations/litellm.py
user: login: Yun-Kim
body: Can we just instantiate one dict instead of setting multiple values sequentially?
```
routing_info = {
    "router_general_settings": getattr(...), ...
}
```
created_at: 2025-05-23T02:42:51Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
+
+            if not kwargs.get("router_instance", None):
+                span._set_ctx_items({METADATA: metadata})
+                return
+
+            routing_info = {}
+            llm_router = kwargs["router_instance"]
+            routing_info["router_general_settings"] = getattr(llm_router, "router_general_settings", None)
+            routing_info["routing_strategy"] = getattr(llm_router, "routing_strategy", None)
+            routing_info["routing_strategy_args"] = getattr(llm_router, "routing_strategy_args", None)
+            routing_info["provider_budget_config"] = getattr(llm_router, "provider_budget_config", None)
+            routing_info["retry_policy"] = getattr(llm_router, "retry_policy", None)
+            routing_info["enable_tag_filtering"] = getattr(llm_router, "enable_tag_filtering", None)
+            model_list = llm_router.get_model_list() if hasattr(llm_router, "get_model_list") else []
+            routing_info["model_list"] = self._scrub_litellm_model_list(model_list)
+            metadata["router_settings"] = routing_info
+
+        span._set_ctx_items({METADATA: metadata})
+
+    def _scrub_litellm_model_list(self, model_list: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
path: ddtrace/llmobs/_integrations/litellm.py
user: login: Yun-Kim
body: Can you explain what we're doing here? What details are available in model_list? Can we just selectively take from the model_list instead of manually deleting kv pairs/attributes?
created_at: 2025-05-23T02:43:25Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
+
+            if not kwargs.get("router_instance", None):
+                span._set_ctx_items({METADATA: metadata})
+                return
+
+            routing_info = {}
+            llm_router = kwargs["router_instance"]
+            routing_info["router_general_settings"] = getattr(llm_router, "router_general_settings", None)
+            routing_info["routing_strategy"] = getattr(llm_router, "routing_strategy", None)
+            routing_info["routing_strategy_args"] = getattr(llm_router, "routing_strategy_args", None)
+            routing_info["provider_budget_config"] = getattr(llm_router, "provider_budget_config", None)
+            routing_info["retry_policy"] = getattr(llm_router, "retry_policy", None)
+            routing_info["enable_tag_filtering"] = getattr(llm_router, "enable_tag_filtering", None)
+            model_list = llm_router.get_model_list() if hasattr(llm_router, "get_model_list") else []
+            routing_info["model_list"] = self._scrub_litellm_model_list(model_list)
path: ddtrace/llmobs/_integrations/litellm.py
user: login: Yun-Kim
body: This reads better if we just do a conditional:
```
if hasattr(llm_router, "get_model_list"):
   model_list = llm_router.get_model_list() ...
```
created_at: 2025-05-23T02:44:53Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
+
+            if not kwargs.get("router_instance", None):
+                span._set_ctx_items({METADATA: metadata})
+                return
+
+            routing_info = {}
+            llm_router = kwargs["router_instance"]
+            routing_info["router_general_settings"] = getattr(llm_router, "router_general_settings", None)
+            routing_info["routing_strategy"] = getattr(llm_router, "routing_strategy", None)
+            routing_info["routing_strategy_args"] = getattr(llm_router, "routing_strategy_args", None)
+            routing_info["provider_budget_config"] = getattr(llm_router, "provider_budget_config", None)
+            routing_info["retry_policy"] = getattr(llm_router, "retry_policy", None)
+            routing_info["enable_tag_filtering"] = getattr(llm_router, "enable_tag_filtering", None)
+            model_list = llm_router.get_model_list() if hasattr(llm_router, "get_model_list") else []
+            routing_info["model_list"] = self._scrub_litellm_model_list(model_list)
+            metadata["router_settings"] = routing_info
+
+        span._set_ctx_items({METADATA: metadata})
+
+    def _scrub_litellm_model_list(self, model_list: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+        for model in model_list:
+            if "litellm_params" in model and "api_key" in model["litellm_params"]:
+                del model["litellm_params"]["api_key"]
+        return model_list
+
+    def _update_input_output_value(self, span: Span, span_kind: str = ""):
+        if span_kind == "llm":
+            return
+        input_messages = span._get_ctx_item(INPUT_MESSAGES)
+        output_messages = span._get_ctx_item(OUTPUT_MESSAGES)
+        if input_messages:
+            span._set_ctx_item(INPUT_VALUE, input_messages)
+        if output_messages:
+            span._set_ctx_item(OUTPUT_VALUE, output_messages)
+
+    def _skip_llm_span(self, kwargs: Dict[str, Any], model: Optional[str] = None) -> bool:
+        """
+        Determine whether an LLM span will be submitted for the given request from outside the LiteLLM integration.
+
+        Currently, this only applies to the OpenAI integration when the following conditions are met:
+            - request is not streamed
+            - model provider is OpenAI/AzureOpenAI
+            - the OpenAI integration is enabled
+        """
+        stream = kwargs.get("stream", False)
+        model_lower = model.lower() if model else ""
+        # model provider is unknown until request completes; therefore, this is a best effort attempt to check
+        # if model provider is Open AI or Azure
+        return (
+            any(prefix in model_lower for prefix in ("gpt", "openai", "azure"))
+            and not stream
+            and LLMObs._integration_is_enabled("openai")
         )
 
+    def _get_span_kind(
+        self, kwargs: Dict[str, Any], model: Optional[str] = None, operation: Optional[str] = None
+    ) -> str:
+        """
+        Workflow span should be submitted to LLMObs if:
+            - span represents a router operation
+            - base_url is set (indicates a request to the proxy) OR
+            - base_url is not set AND an LLM span will be submitted elsewhere
+        LLM spans should be submitted to LLMObs if:
+            - base_url is not set AND an LLM span will not be submitted elsewhere
+        """
+        router_operation = "router" in operation if operation else False
path: ddtrace/llmobs/_integrations/litellm.py
user: login: Yun-Kim
body: I'm seeing this pattern a lot. Can we extract this into a mini helper?
created_at: 2025-05-23T02:46:53Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
+
+            if not kwargs.get("router_instance", None):
+                span._set_ctx_items({METADATA: metadata})
+                return
+
+            routing_info = {}
+            llm_router = kwargs["router_instance"]
+            routing_info["router_general_settings"] = getattr(llm_router, "router_general_settings", None)
+            routing_info["routing_strategy"] = getattr(llm_router, "routing_strategy", None)
+            routing_info["routing_strategy_args"] = getattr(llm_router, "routing_strategy_args", None)
+            routing_info["provider_budget_config"] = getattr(llm_router, "provider_budget_config", None)
+            routing_info["retry_policy"] = getattr(llm_router, "retry_policy", None)
+            routing_info["enable_tag_filtering"] = getattr(llm_router, "enable_tag_filtering", None)
+            model_list = llm_router.get_model_list() if hasattr(llm_router, "get_model_list") else []
+            routing_info["model_list"] = self._scrub_litellm_model_list(model_list)
+            metadata["router_settings"] = routing_info
+
+        span._set_ctx_items({METADATA: metadata})
+
+    def _scrub_litellm_model_list(self, model_list: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+        for model in model_list:
+            if "litellm_params" in model and "api_key" in model["litellm_params"]:
+                del model["litellm_params"]["api_key"]
+        return model_list
+
+    def _update_input_output_value(self, span: Span, span_kind: str = ""):
+        if span_kind == "llm":
+            return
+        input_messages = span._get_ctx_item(INPUT_MESSAGES)
+        output_messages = span._get_ctx_item(OUTPUT_MESSAGES)
+        if input_messages:
+            span._set_ctx_item(INPUT_VALUE, input_messages)
+        if output_messages:
+            span._set_ctx_item(OUTPUT_VALUE, output_messages)
+
+    def _skip_llm_span(self, kwargs: Dict[str, Any], model: Optional[str] = None) -> bool:
+        """
+        Determine whether an LLM span will be submitted for the given request from outside the LiteLLM integration.
+
+        Currently, this only applies to the OpenAI integration when the following conditions are met:
+            - request is not streamed
+            - model provider is OpenAI/AzureOpenAI
+            - the OpenAI integration is enabled
+        """
+        stream = kwargs.get("stream", False)
+        model_lower = model.lower() if model else ""
+        # model provider is unknown until request completes; therefore, this is a best effort attempt to check
+        # if model provider is Open AI or Azure
+        return (
+            any(prefix in model_lower for prefix in ("gpt", "openai", "azure"))
+            and not stream
+            and LLMObs._integration_is_enabled("openai")
         )
 
+    def _get_span_kind(
+        self, kwargs: Dict[str, Any], model: Optional[str] = None, operation: Optional[str] = None
+    ) -> str:
+        """
+        Workflow span should be submitted to LLMObs if:
+            - span represents a router operation
+            - base_url is set (indicates a request to the proxy) OR
+            - base_url is not set AND an LLM span will be submitted elsewhere
+        LLM spans should be submitted to LLMObs if:
+            - base_url is not set AND an LLM span will not be submitted elsewhere
+        """
+        router_operation = "router" in operation if operation else False
+        base_url = kwargs.get("api_base", None)
path: ddtrace/llmobs/_integrations/litellm.py
user: login: Yun-Kim
body: the second arg to get() is None by default (built-in default value), redundant to specify here
created_at: 2025-05-23T02:47:26Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
+
+            if not kwargs.get("router_instance", None):
+                span._set_ctx_items({METADATA: metadata})
+                return
+
+            routing_info = {}
+            llm_router = kwargs["router_instance"]
+            routing_info["router_general_settings"] = getattr(llm_router, "router_general_settings", None)
+            routing_info["routing_strategy"] = getattr(llm_router, "routing_strategy", None)
+            routing_info["routing_strategy_args"] = getattr(llm_router, "routing_strategy_args", None)
+            routing_info["provider_budget_config"] = getattr(llm_router, "provider_budget_config", None)
+            routing_info["retry_policy"] = getattr(llm_router, "retry_policy", None)
+            routing_info["enable_tag_filtering"] = getattr(llm_router, "enable_tag_filtering", None)
+            model_list = llm_router.get_model_list() if hasattr(llm_router, "get_model_list") else []
+            routing_info["model_list"] = self._scrub_litellm_model_list(model_list)
+            metadata["router_settings"] = routing_info
+
+        span._set_ctx_items({METADATA: metadata})
+
+    def _scrub_litellm_model_list(self, model_list: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+        for model in model_list:
+            if "litellm_params" in model and "api_key" in model["litellm_params"]:
+                del model["litellm_params"]["api_key"]
+        return model_list
+
+    def _update_input_output_value(self, span: Span, span_kind: str = ""):
+        if span_kind == "llm":
+            return
+        input_messages = span._get_ctx_item(INPUT_MESSAGES)
+        output_messages = span._get_ctx_item(OUTPUT_MESSAGES)
+        if input_messages:
+            span._set_ctx_item(INPUT_VALUE, input_messages)
+        if output_messages:
+            span._set_ctx_item(OUTPUT_VALUE, output_messages)
+
+    def _skip_llm_span(self, kwargs: Dict[str, Any], model: Optional[str] = None) -> bool:
+        """
+        Determine whether an LLM span will be submitted for the given request from outside the LiteLLM integration.
+
+        Currently, this only applies to the OpenAI integration when the following conditions are met:
+            - request is not streamed
+            - model provider is OpenAI/AzureOpenAI
+            - the OpenAI integration is enabled
+        """
+        stream = kwargs.get("stream", False)
+        model_lower = model.lower() if model else ""
+        # model provider is unknown until request completes; therefore, this is a best effort attempt to check
+        # if model provider is Open AI or Azure
+        return (
+            any(prefix in model_lower for prefix in ("gpt", "openai", "azure"))
+            and not stream
+            and LLMObs._integration_is_enabled("openai")
         )
 
+    def _get_span_kind(
+        self, kwargs: Dict[str, Any], model: Optional[str] = None, operation: Optional[str] = None
+    ) -> str:
+        """
+        Workflow span should be submitted to LLMObs if:
+            - span represents a router operation
+            - base_url is set (indicates a request to the proxy) OR
+            - base_url is not set AND an LLM span will be submitted elsewhere
path: ddtrace/llmobs/_integrations/litellm.py
user: login: Yun-Kim
body: Can you elaborate why/what cases this would include? Not exactly following here
created_at: 2025-05-23T02:55:54Z
author_association: CONTRIBUTOR
diff_hunk: @@ -75,22 +173,11 @@ def _extract_llmobs_metrics(resp: Any) -> Dict[str, Any]:
 
     def should_submit_to_llmobs(self, kwargs: Dict[str, Any], model: Optional[str] = None) -> bool:
         """
-        Span should be NOT submitted to LLMObs if:
-            - base_url is not None: is a proxy request and we will capture the LLM request downstream
-            - non-streamed request and model provider is OpenAI/AzureOpenAI and the OpenAI integration
-                is enabled: this request will be captured in the OpenAI integration instead
+        LiteLLM spans will be submitted to LLMObs unless the following are true:
+            - base_url is not set AND
path: ddtrace/llmobs/_integrations/litellm.py
user: login: Yun-Kim
body: What does base_url not being set imply?
created_at: 2025-05-23T02:57:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
+
+            if not kwargs.get("router_instance", None):
+                span._set_ctx_items({METADATA: metadata})
+                return
+
+            routing_info = {}
+            llm_router = kwargs["router_instance"]
+            routing_info["router_general_settings"] = getattr(llm_router, "router_general_settings", None)
+            routing_info["routing_strategy"] = getattr(llm_router, "routing_strategy", None)
+            routing_info["routing_strategy_args"] = getattr(llm_router, "routing_strategy_args", None)
+            routing_info["provider_budget_config"] = getattr(llm_router, "provider_budget_config", None)
+            routing_info["retry_policy"] = getattr(llm_router, "retry_policy", None)
+            routing_info["enable_tag_filtering"] = getattr(llm_router, "enable_tag_filtering", None)
+            model_list = llm_router.get_model_list() if hasattr(llm_router, "get_model_list") else []
+            routing_info["model_list"] = self._scrub_litellm_model_list(model_list)
+            metadata["router_settings"] = routing_info
+
+        span._set_ctx_items({METADATA: metadata})
+
+    def _scrub_litellm_model_list(self, model_list: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+        for model in model_list:
+            if "litellm_params" in model and "api_key" in model["litellm_params"]:
+                del model["litellm_params"]["api_key"]
+        return model_list
+
+    def _update_input_output_value(self, span: Span, span_kind: str = ""):
+        if span_kind == "llm":
+            return
+        input_messages = span._get_ctx_item(INPUT_MESSAGES)
+        output_messages = span._get_ctx_item(OUTPUT_MESSAGES)
+        if input_messages:
+            span._set_ctx_item(INPUT_VALUE, input_messages)
+        if output_messages:
+            span._set_ctx_item(OUTPUT_VALUE, output_messages)
+
+    def _skip_llm_span(self, kwargs: Dict[str, Any], model: Optional[str] = None) -> bool:
path: ddtrace/llmobs/_integrations/litellm.py
user: login: Yun-Kim
body: Personally I find this method name confusing. Can we use a more descriptive name (ex: has_downstream_openai_span)?
created_at: 2025-05-23T02:58:45Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
+
+            if not kwargs.get("router_instance", None):
+                span._set_ctx_items({METADATA: metadata})
+                return
+
+            routing_info = {}
+            llm_router = kwargs["router_instance"]
+            routing_info["router_general_settings"] = getattr(llm_router, "router_general_settings", None)
+            routing_info["routing_strategy"] = getattr(llm_router, "routing_strategy", None)
+            routing_info["routing_strategy_args"] = getattr(llm_router, "routing_strategy_args", None)
+            routing_info["provider_budget_config"] = getattr(llm_router, "provider_budget_config", None)
+            routing_info["retry_policy"] = getattr(llm_router, "retry_policy", None)
+            routing_info["enable_tag_filtering"] = getattr(llm_router, "enable_tag_filtering", None)
+            model_list = llm_router.get_model_list() if hasattr(llm_router, "get_model_list") else []
+            routing_info["model_list"] = self._scrub_litellm_model_list(model_list)
+            metadata["router_settings"] = routing_info
+
+        span._set_ctx_items({METADATA: metadata})
+
+    def _scrub_litellm_model_list(self, model_list: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+        for model in model_list:
+            if "litellm_params" in model and "api_key" in model["litellm_params"]:
+                del model["litellm_params"]["api_key"]
+        return model_list
+
+    def _update_input_output_value(self, span: Span, span_kind: str = ""):
+        if span_kind == "llm":
+            return
+        input_messages = span._get_ctx_item(INPUT_MESSAGES)
+        output_messages = span._get_ctx_item(OUTPUT_MESSAGES)
+        if input_messages:
+            span._set_ctx_item(INPUT_VALUE, input_messages)
+        if output_messages:
+            span._set_ctx_item(OUTPUT_VALUE, output_messages)
+
+    def _skip_llm_span(self, kwargs: Dict[str, Any], model: Optional[str] = None) -> bool:
+        """
+        Determine whether an LLM span will be submitted for the given request from outside the LiteLLM integration.
+
+        Currently, this only applies to the OpenAI integration when the following conditions are met:
+            - request is not streamed
+            - model provider is OpenAI/AzureOpenAI
+            - the OpenAI integration is enabled
+        """
+        stream = kwargs.get("stream", False)
+        model_lower = model.lower() if model else ""
+        # model provider is unknown until request completes; therefore, this is a best effort attempt to check
+        # if model provider is Open AI or Azure
+        return (
+            any(prefix in model_lower for prefix in ("gpt", "openai", "azure"))
path: ddtrace/llmobs/_integrations/litellm.py
user: login: Yun-Kim
body: ```suggestion
        # best effort attempt to check if Open AI or Azure since model_provider is unknown until request completes
        is_openai_model = any(prefix in model_lower for prefix in ("gpt", "openai", "azure"))
        return (
            is_openai_model
```
created_at: 2025-05-23T02:59:57Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
+
+            if not kwargs.get("router_instance", None):
+                span._set_ctx_items({METADATA: metadata})
+                return
+
+            routing_info = {}
+            llm_router = kwargs["router_instance"]
+            routing_info["router_general_settings"] = getattr(llm_router, "router_general_settings", None)
+            routing_info["routing_strategy"] = getattr(llm_router, "routing_strategy", None)
+            routing_info["routing_strategy_args"] = getattr(llm_router, "routing_strategy_args", None)
+            routing_info["provider_budget_config"] = getattr(llm_router, "provider_budget_config", None)
+            routing_info["retry_policy"] = getattr(llm_router, "retry_policy", None)
+            routing_info["enable_tag_filtering"] = getattr(llm_router, "enable_tag_filtering", None)
+            model_list = llm_router.get_model_list() if hasattr(llm_router, "get_model_list") else []
+            routing_info["model_list"] = self._scrub_litellm_model_list(model_list)
+            metadata["router_settings"] = routing_info
+
+        span._set_ctx_items({METADATA: metadata})
+
+    def _scrub_litellm_model_list(self, model_list: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+        for model in model_list:
+            if "litellm_params" in model and "api_key" in model["litellm_params"]:
+                del model["litellm_params"]["api_key"]
+        return model_list
+
+    def _update_input_output_value(self, span: Span, span_kind: str = ""):
+        if span_kind == "llm":
+            return
+        input_messages = span._get_ctx_item(INPUT_MESSAGES)
+        output_messages = span._get_ctx_item(OUTPUT_MESSAGES)
+        if input_messages:
+            span._set_ctx_item(INPUT_VALUE, input_messages)
+        if output_messages:
+            span._set_ctx_item(OUTPUT_VALUE, output_messages)
+
+    def _skip_llm_span(self, kwargs: Dict[str, Any], model: Optional[str] = None) -> bool:
+        """
+        Determine whether an LLM span will be submitted for the given request from outside the LiteLLM integration.
+
+        Currently, this only applies to the OpenAI integration when the following conditions are met:
+            - request is not streamed
+            - model provider is OpenAI/AzureOpenAI
+            - the OpenAI integration is enabled
+        """
+        stream = kwargs.get("stream", False)
+        model_lower = model.lower() if model else ""
+        # model provider is unknown until request completes; therefore, this is a best effort attempt to check
+        # if model provider is Open AI or Azure
+        return (
+            any(prefix in model_lower for prefix in ("gpt", "openai", "azure"))
path: ddtrace/llmobs/_integrations/litellm.py
user: login: Yun-Kim
body: Can we also not just wait to check the model provider until after the request completes?
created_at: 2025-05-23T03:00:13Z
author_association: CONTRIBUTOR
diff_hunk: @@ -298,7 +298,9 @@ def openai_set_meta_tags_from_completion(span: Span, kwargs: Dict[str, Any], com
     if isinstance(prompt, str):
         prompt = [prompt]
     parameters = {
-        k: v for k, v in kwargs.items() if k not in ("model", "prompt", "api_key", "user_api_key", "user_api_key_hash")
+        k: v
+        for k, v in kwargs.items()
+        if k not in ("model", "prompt", "api_key", "user_api_key", "user_api_key_hash", "router_instance")
path: ddtrace/llmobs/_integrations/utils.py
user: login: Yun-Kim
body: Let's move these into a constant variable instead.
created_at: 2025-05-23T03:00:45Z
author_association: CONTRIBUTOR
diff_hunk: @@ -324,7 +326,17 @@ def openai_set_meta_tags_from_chat(span: Span, kwargs: Dict[str, Any], messages:
     parameters = {
         k: v
         for k, v in kwargs.items()
-        if k not in ("model", "messages", "tools", "functions", "api_key", "user_api_key", "user_api_key_hash")
+        if k
+        not in (
path: ddtrace/llmobs/_integrations/utils.py
user: login: Yun-Kim
body: Same here
created_at: 2025-05-23T03:00:51Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,4 @@
+---
+features:
+  - |
+    litellm: Adds APM and LLM Observability tracing support for LiteLLM's router methods.
path: releasenotes/notes/litellm-router-tracing-04d28e5b3fc73702.yaml
user: login: Yun-Kim
body: can we clarify which router methods?
created_at: 2025-05-23T03:02:19Z
author_association: CONTRIBUTOR
diff_hunk: @@ -200,43 +203,199 @@ async def test_atext_completion(self, litellm, request_vcr, llmobs_events, mock_
             tags={"ml_app": "<ml-app-name>", "service": "tests.contrib.litellm"},
         )
 
-    @flaky(until=1748750400, reason="Patching Open AI to be used within the LiteLLM library appears to be flaky")
-    def test_completion_openai_enabled(self, litellm, request_vcr, llmobs_events, mock_tracer, stream, n):
+    def test_completion_proxy(self, litellm, request_vcr_include_localhost, llmobs_events, mock_tracer, stream, n):
+        with request_vcr_include_localhost.use_cassette(get_cassette_name(stream, n, proxy=True)):
+            messages = [{"content": "Hey, what is up?", "role": "user"}]
+            resp = litellm.completion(
+                model="gpt-3.5-turbo",
+                messages=messages,
+                stream=stream,
+                n=n,
+                stream_options={"include_usage": True},
+                api_base="http://0.0.0.0:4000",
+            )
+            if stream:
+                output_messages, _ = consume_stream(resp, n)
+            else:
+                output_messages, _ = parse_response(resp)
+
+        span = mock_tracer.pop_traces()[0][0]
+        assert len(llmobs_events) == 1
+        assert llmobs_events[0] == _expected_llmobs_non_llm_span_event(
+            span,
+            span_kind="workflow",
+            input_value=safe_json(messages, ensure_ascii=False),
+            output_value=safe_json(output_messages, ensure_ascii=False),
+            metadata={
+                "stream": stream,
+                "n": n,
+                "stream_options": {"include_usage": True},
+                "api_base": "http://0.0.0.0:4000",
+                "model": "gpt-3.5-turbo",
+            },
+            tags={"ml_app": "<ml-app-name>", "service": "tests.contrib.litellm"},
+        )
+
+    def test_router_completion(self, litellm, request_vcr, llmobs_events, mock_tracer, router, stream, n):
         with request_vcr.use_cassette(get_cassette_name(stream, n)):
-            patch(openai=True)
-            import openai
+            messages = [{"content": "Hey, what is up?", "role": "user"}]
+            resp = router.completion(
+                model="gpt-3.5-turbo",
+                messages=messages,
+                stream=stream,
+                n=n,
+                stream_options={"include_usage": True},
+            )
+            if stream:
+                output_messages, _ = consume_stream(resp, n)
+            else:
+                output_messages, _ = parse_response(resp)
 
-            pin = Pin.get_from(openai)
-            pin._override(openai, tracer=mock_tracer)
+        trace = mock_tracer.pop_traces()[0]
+        assert len(trace) == 2
+        router_span = trace[0]
+
+        assert len(llmobs_events) == 2
+        router_event = llmobs_events[1]
+        assert router_event == _expected_llmobs_non_llm_span_event(
+            router_span,
+            span_kind="workflow",
+            input_value=safe_json(messages, ensure_ascii=False),
+            output_value=safe_json(output_messages, ensure_ascii=False),
+            metadata={
+                "stream": stream,
+                "n": n,
+                "stream_options": {"include_usage": True},
+                "router_settings": expected_router_settings,
+            },
+            tags={"ml_app": "<ml-app-name>", "service": "tests.contrib.litellm"},
+        )
 
+    async def test_router_acompletion(self, litellm, request_vcr, llmobs_events, mock_tracer, router, stream, n):
+        with request_vcr.use_cassette(get_cassette_name(stream, n)):
             messages = [{"content": "Hey, what is up?", "role": "user"}]
-            resp = litellm.completion(
+            resp = await router.acompletion(
                 model="gpt-3.5-turbo",
                 messages=messages,
                 stream=stream,
                 n=n,
                 stream_options={"include_usage": True},
             )
             if stream:
-                for _ in resp:
-                    pass
+                output_messages, _ = await async_consume_stream(resp, n)
+            else:
+                output_messages, _ = parse_response(resp)
 
-        assert len(llmobs_events) == 1
-        assert llmobs_events[0]["name"] == "OpenAI.createChatCompletion" if not stream else "litellm.request"
+        trace = mock_tracer.pop_traces()[0]
+        assert len(trace) == 2
+        router_span = trace[0]
+
+        assert len(llmobs_events) == 2
+        router_event = llmobs_events[1]
path: tests/contrib/litellm/test_litellm_llmobs.py
user: login: Yun-Kim
body: Can we also include the assertion on the first span as well?
created_at: 2025-05-23T03:04:14Z
author_association: CONTRIBUTOR
diff_hunk: @@ -138,6 +139,23 @@ def parse_response(resp, is_completion=False):
     return output_messages, token_metrics
 
 
+model_list = [
+    {
+        "model_name": "gpt-3.5-turbo",
+        "litellm_params": {
+            "model": "gpt-3.5-turbo",
+            "api_key": os.getenv("<not-a-real-key>"),
path: tests/contrib/litellm/utils.py
user: login: Yun-Kim
body: probably don't need the os.getenv here
created_at: 2025-05-23T03:04:43Z
author_association: CONTRIBUTOR
diff_hunk: @@ -23,30 +24,10 @@ def get_version() -> str:
 
 @with_traced_module
 def traced_completion(litellm, pin, func, instance, args, kwargs):
-    return _traced_completion(litellm, pin, func, instance, args, kwargs, False)
-
-
-@with_traced_module
-async def traced_acompletion(litellm, pin, func, instance, args, kwargs):
-    return await _traced_acompletion(litellm, pin, func, instance, args, kwargs, False)
-
-
-@with_traced_module
-def traced_text_completion(litellm, pin, func, instance, args, kwargs):
-    return _traced_completion(litellm, pin, func, instance, args, kwargs, True)
-
-
-@with_traced_module
-async def traced_atext_completion(litellm, pin, func, instance, args, kwargs):
-    return await _traced_acompletion(litellm, pin, func, instance, args, kwargs, True)
-
-
-def _traced_completion(litellm, pin, func, instance, args, kwargs, is_completion):
+    is_completion = "text" in func.__name__
path: ddtrace/contrib/internal/litellm/patch.py
user: login: ncybul
body: Yep! `completion` and `acompletion` are chat completions while `text_completion` and `atext_completion` are regular completions. `is_completion` is used when setting llmobs tags in the finally block, no?
created_at: 2025-05-23T14:28:18Z
author_association: CONTRIBUTOR
diff_hunk: @@ -107,6 +88,68 @@ async def _traced_acompletion(litellm, pin, func, instance, args, kwargs, is_com
             span.finish()
 
 
+@with_traced_module
+def traced_router_completion(litellm, pin, func, instance, args, kwargs):
+    operation = f"router.{func.__name__}"
+    integration = litellm._datadog_integration
+    model = get_argument_value(args, kwargs, 0, "model", None)
+    host = extract_host_tag(kwargs)
+    span = integration.trace(
+        pin,
+        operation,
+        model=model,
+        host=host,
+        submit_to_llmobs=True,
+    )
+    stream = kwargs.get("stream", False)
+    resp = None
+    try:
+        resp = func(*args, **kwargs)
+        if stream:
+            resp.add_router_span_info(span, kwargs, instance)
+        return resp
+    except Exception:
+        span.set_exc_info(*sys.exc_info())
+        raise
+    finally:
+        if not stream:
+            if integration.is_pc_sampled_llmobs(span):
+                kwargs["router_instance"] = instance
path: ddtrace/contrib/internal/litellm/patch.py
user: login: ncybul
body: Hmmm, what do you mean? I am adding `router_instance` myself because I need access to it in the LiteLLM integration (so using kwargs to pass it through).
created_at: 2025-05-23T14:52:24Z
author_association: CONTRIBUTOR
diff_hunk: @@ -35,13 +63,11 @@ def __iter__(self):
                 yield chunk
                 _loop_handler(chunk, self._streamed_chunks)
         except Exception:
-            self._dd_span.set_exc_info(*sys.exc_info())
+            for span in self._dd_spans:
+                span.set_exc_info(*sys.exc_info())
path: ddtrace/contrib/internal/litellm/utils.py
user: login: ncybul
body: I guess we do not? I figured it might be helpful to bubble up the error but I guess we could also just set it on the LLM span (first span). Is your concern that it's redundant to set the error on both spans?
created_at: 2025-05-23T14:53:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -10,15 +10,43 @@
 log = get_logger(__name__)
 
 
+def extract_host_tag(kwargs):
+    if "host" in kwargs.get("metadata", {}).get("headers", {}):
+        return kwargs["metadata"]["headers"]["host"]
+    return None
+
+
 class BaseTracedLiteLLMStream(wrapt.ObjectProxy):
-    def __init__(self, wrapped, integration, span, kwargs, is_completion=False):
+    def __init__(self, wrapped, integration, span, kwargs):
         super().__init__(wrapped)
         n = kwargs.get("n", 1) or 1
         self._dd_integration = integration
-        self._dd_span = span
-        self._kwargs = kwargs
+        self._dd_spans = [span]
+        self._kwargs = [kwargs]  # each span is associated with a different kwargs
         self._streamed_chunks = [[] for _ in range(n)]
-        self._is_completion = is_completion
+
+    def add_router_span_info(self, span, kwargs, instance):
+        """Handler to add router span to this streaming object.
+
+        Helps to ensure that all spans associated with a single stream are finished and have the correct tags.
+        """
+        self._dd_spans.append(span)
+        kwargs["router_instance"] = instance
+        self._kwargs.append(kwargs)
+
+    def finish_spans(self):
+        """Helper to finish all spans associated with this stream."""
+        formatted_completions = None
+        for span, kwargs in zip(self._dd_spans, self._kwargs):
+            if not formatted_completions:
+                formatted_completions = _process_finished_stream(
+                    self._dd_integration, span, kwargs, self._streamed_chunks, span.resource
+                )
+            elif self._dd_integration.is_pc_sampled_llmobs(span):
+                self._dd_integration.llmobs_set_tags(
+                    span, args=[], kwargs=kwargs, response=formatted_completions, operation=span.resource
+                )
path: ddtrace/contrib/internal/litellm/utils.py
user: login: ncybul
body: If we do not do this, then the router workflow span will not have access to the output value. This is kind of a bad experience if the router span is the root span as the trace in the traces page will also have a missing output. It's not strictly necessary, but I think it's a better user experience to also tag the output value if possible.
created_at: 2025-05-23T14:56:53Z
author_association: CONTRIBUTOR
diff_hunk: @@ -120,10 +142,12 @@ def _process_finished_stream(integration, span, kwargs, streamed_chunks, is_comp
             formatted_completions = [
                 openai_construct_message_from_streamed_chunks(choice) for choice in streamed_chunks
             ]
-        operation = "completion" if is_completion else "chat"
+        operation = operation if "router" in operation else "completion" if is_completion else "chat"
path: ddtrace/contrib/internal/litellm/utils.py
user: login: ncybul
body: Possible router operations include `router.completion`, `router.acompletion`, `router.text_completion`, and `router.atext_completion`. The reason I want to keep the original name for router operations is because that is how I determine if the span should be a workflow or LLM span in the LiteLLM integration. For the regular `completion`, `acompletion`, `text_completion`, and `atext_completion`, I coerce them into either "completion" or "chat" which I guess isn't necessary since I can just do this within the LiteLLM integration as well. I will update that to simplify this logic 😄 
created_at: 2025-05-23T15:09:04Z
author_association: CONTRIBUTOR
diff_hunk: @@ -120,10 +142,12 @@ def _process_finished_stream(integration, span, kwargs, streamed_chunks, is_comp
             formatted_completions = [
                 openai_construct_message_from_streamed_chunks(choice) for choice in streamed_chunks
             ]
-        operation = "completion" if is_completion else "chat"
+        operation = operation if "router" in operation else "completion" if is_completion else "chat"
path: ddtrace/contrib/internal/litellm/utils.py
user: login: ncybul
body: I modified this to maintain consistency such that I always pass int he operation name (span.resource) to the LiteLLM integration and decipher whether the operation is a chat or completion operation in the integration itself.
created_at: 2025-05-23T15:29:50Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +53,111 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if self.is_completion_operation(operation):
             openai_set_meta_tags_from_completion(span, kwargs, response)
         else:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
         )
 
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
+
+            if not kwargs.get("router_instance", None):
+                span._set_ctx_items({METADATA: metadata})
+                return
+
+            routing_info = {}
+            llm_router = kwargs["router_instance"]
+            routing_info["router_general_settings"] = getattr(llm_router, "router_general_settings", None)
+            routing_info["routing_strategy"] = getattr(llm_router, "routing_strategy", None)
+            routing_info["routing_strategy_args"] = getattr(llm_router, "routing_strategy_args", None)
+            routing_info["provider_budget_config"] = getattr(llm_router, "provider_budget_config", None)
+            routing_info["retry_policy"] = getattr(llm_router, "retry_policy", None)
+            routing_info["enable_tag_filtering"] = getattr(llm_router, "enable_tag_filtering", None)
+            model_list = llm_router.get_model_list() if hasattr(llm_router, "get_model_list") else []
+            routing_info["model_list"] = self._scrub_litellm_model_list(model_list)
+            metadata["router_settings"] = routing_info
+
+        span._set_ctx_items({METADATA: metadata})
+
+    def _scrub_litellm_model_list(self, model_list: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+        for model in model_list:
+            if "litellm_params" in model and "api_key" in model["litellm_params"]:
+                del model["litellm_params"]["api_key"]
+        return model_list
+
+    def _update_input_output_value(self, span: Span, span_kind: str = ""):
+        if span_kind == "llm":
+            return
+        input_messages = span._get_ctx_item(INPUT_MESSAGES)
+        output_messages = span._get_ctx_item(OUTPUT_MESSAGES)
+        if input_messages:
+            span._set_ctx_item(INPUT_VALUE, input_messages)
+        if output_messages:
+            span._set_ctx_item(OUTPUT_VALUE, output_messages)
+
+    def _skip_llm_span(self, kwargs: Dict[str, Any], model: Optional[str] = None) -> bool:
+        """
+        Determine whether an LLM span will be submitted for the given request from outside the LiteLLM integration.
+
+        Currently, this only applies to the OpenAI integration when the following conditions are met:
+            - request is not streamed
+            - model provider is OpenAI/AzureOpenAI
+            - the OpenAI integration is enabled
+        """
+        stream = kwargs.get("stream", False)
+        model_lower = model.lower() if model else ""
+        # model provider is unknown until request completes; therefore, this is a best effort attempt to check
+        # if model provider is Open AI or Azure
+        return (
+            any(prefix in model_lower for prefix in ("gpt", "openai", "azure"))
+            and not stream
+            and LLMObs._integration_is_enabled("openai")
+        )
+
+    def _get_span_kind(
+        self, kwargs: Dict[str, Any], model: Optional[str] = None, operation: Optional[str] = None
+    ) -> str:
+        """
+        Workflow span should be submitted to LLMObs if:
+            - span represents a router operation
+            - base_url is set (indicates a request to the proxy) OR
+            - base_url is not set AND an LLM span will be submitted elsewhere
+        LLM spans should be submitted to LLMObs if:
+            - base_url is not set AND an LLM span will not be submitted elsewhere
+        """
+        router_operation = "router" in operation if operation else False
+        base_url = kwargs.get("api_base", None)
+        if router_operation or base_url or self._skip_llm_span(kwargs, model):
+            return "workflow"
+        return "llm"
+
+    def is_completion_operation(self, operation: str) -> bool:
+        return operation in TEXT_COMPLETION_OPERATIONS
+    
     @staticmethod
-    def _extract_llmobs_metrics(resp: Any) -> Dict[str, Any]:
-        if not resp:
+    def _extract_llmobs_metrics(resp: Any, span_kind: str) -> Dict[str, Any]:
path: ddtrace/llmobs/_integrations/litellm.py
user: login: datadog-datadog-prod-us1[bot]
body: <!-- python-best-practices/any-type-disallow -->

<h3><span title="Warning"> :orange_circle: </span>Code Quality Violation</h3>



<details>
  <summary>
    do not use Any, use a concrete type <i>(...read more)</i>
  </summary>

<br>

Use the `Any` type very carefully. Most of the time, the `Any` type is used because we do not know exactly what type is being used. If you want to specify that a value can be of any type, use `object` instead of `Any`.


#### Learn More

 - [Python documentation: the `Any` type](https://docs.python.org/3/library/typing.html#the-any-type)
</details>

<p align="right"><a href="https://app.datadoghq.com/ci/code-analysis/github.com%2Fdatadog%2Fdd-trace-py/nicole-cybul%2Finstrument-router-methods/7f9bd41fb6240ab786d4faf06fdd546c9e6e093d/code-quality?index=cicodescan&query=%40git.repository.id%3A%22github.com%2Fdatadog%2Fdd-trace-py%22+%40git.branch%3A%22nicole-cybul%2Finstrument-router-methods%22+%40git.commit.sha%3A%227f9bd41fb6240ab786d4faf06fdd546c9e6e093d%22+-%40static_analysis.result.category%3Asecurity+-%40static_analysis.result.item_type%3Asecret&staticAnalysisId=AgAAAZb95jkpEG_k2wAAAAAAAAAYAAAAAEFaYjk1aXRtQUFDQl9lRWpOT251QUFBQQAAACQAAAAAMDE5NmZkZTYtM2Y4Mi00ZDFiLWJiNDItOWE0OTZiNjlhYmJh&trace=AgAAAZb95jkpEG_k2wAAAAAAAAAYAAAAAEFaYjk1aXRtQUFDQl9lRWpOT251QUFBQQAAACQAAAAAMDE5NmZkZTYtM2Y4Mi00ZDFiLWJiNDItOWE0OTZiNjlhYmJh" title="View in Datadog"><img src="https://app.datadoghq.com/api/ui/asset/images/pr_comments/software_delivery/datadog_icon.svg" alt="View in Datadog" align="center" /></a>&nbsp;&nbsp;<a href="https://dd.enterprise.slack.com/archives/C03NGS0N3GA" title="Leave us feedback"><picture><source media="(prefers-color-scheme: dark)" srcset="https://app.datadoghq.com/api/ui/asset/images/pr_comments/software_delivery/comment_dark.svg"/><img src="https://app.datadoghq.com/api/ui/asset/images/pr_comments/software_delivery/comment_light.svg" alt="Leave us feedback" align="center" /></picture></a>&nbsp;&nbsp;<a href="https://docs.datadoghq.com/static_analysis/rules/python-best-practices/any-type-disallow" title="Documentation"><picture><source media="(prefers-color-scheme: dark)" srcset="https://app.datadoghq.com/api/ui/asset/images/pr_comments/software_delivery/books_dark.svg"/><img src="https://app.datadoghq.com/api/ui/asset/images/pr_comments/software_delivery/books_light.svg" alt="Documentation" align="center" /></picture></a></p>

created_at: 2025-05-23T16:08:36Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
path: ddtrace/llmobs/_integrations/litellm.py
user: login: ncybul
body: `model` is one of the kwargs for a client side request which will get filtered out in `openai_set_meta_tags_from_chat` and `openai_set_meta_tags_from_completion`. This is essentially just adding it back in. However, for server-side spans,`model` still shows up because it is a nested attribute of the metadata which is why this is only necessary for client-side requests. Example: 
![image](https://github.com/user-attachments/assets/624e5da3-395f-4667-aeb1-2f6ee9ff13b8)

created_at: 2025-05-23T16:13:37Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
path: ddtrace/llmobs/_integrations/litellm.py
user: login: ncybul
body: yep!
created_at: 2025-05-23T16:14:54Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
path: ddtrace/llmobs/_integrations/litellm.py
user: login: ncybul
body: Hmmm, yeah its a bit tricky because its a nested field and we only filter out kwargs from metadata based on the top level field. Otherwise, I would need to add more logic to `openai_set_meta_tags_from_chat` and `openai_set_meta_tags_from_completion` to check for this key which wouldn't make much of a difference from how I am handling it now. Curious if you had something else in mind though?
created_at: 2025-05-23T16:21:05Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
+
+            if not kwargs.get("router_instance", None):
path: ddtrace/llmobs/_integrations/litellm.py
user: login: ncybul
body: Oh yeah agreed, much cleaner. thanks!
created_at: 2025-05-23T16:22:48Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
+
+            if not kwargs.get("router_instance", None):
+                span._set_ctx_items({METADATA: metadata})
+                return
+
+            routing_info = {}
+            llm_router = kwargs["router_instance"]
+            routing_info["router_general_settings"] = getattr(llm_router, "router_general_settings", None)
+            routing_info["routing_strategy"] = getattr(llm_router, "routing_strategy", None)
+            routing_info["routing_strategy_args"] = getattr(llm_router, "routing_strategy_args", None)
+            routing_info["provider_budget_config"] = getattr(llm_router, "provider_budget_config", None)
+            routing_info["retry_policy"] = getattr(llm_router, "retry_policy", None)
+            routing_info["enable_tag_filtering"] = getattr(llm_router, "enable_tag_filtering", None)
path: ddtrace/llmobs/_integrations/litellm.py
user: login: ncybul
body: yep for sure!
created_at: 2025-05-23T16:25:04Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
+
+            if not kwargs.get("router_instance", None):
+                span._set_ctx_items({METADATA: metadata})
+                return
+
+            routing_info = {}
+            llm_router = kwargs["router_instance"]
+            routing_info["router_general_settings"] = getattr(llm_router, "router_general_settings", None)
+            routing_info["routing_strategy"] = getattr(llm_router, "routing_strategy", None)
+            routing_info["routing_strategy_args"] = getattr(llm_router, "routing_strategy_args", None)
+            routing_info["provider_budget_config"] = getattr(llm_router, "provider_budget_config", None)
+            routing_info["retry_policy"] = getattr(llm_router, "retry_policy", None)
+            routing_info["enable_tag_filtering"] = getattr(llm_router, "enable_tag_filtering", None)
+            model_list = llm_router.get_model_list() if hasattr(llm_router, "get_model_list") else []
+            routing_info["model_list"] = self._scrub_litellm_model_list(model_list)
+            metadata["router_settings"] = routing_info
+
+        span._set_ctx_items({METADATA: metadata})
+
+    def _scrub_litellm_model_list(self, model_list: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
path: ddtrace/llmobs/_integrations/litellm.py
user: login: ncybul
body: this is one example entry in `model_list`:

![image](https://github.com/user-attachments/assets/3361170d-e34a-4644-a67c-30b11f5700c8)

Although the exact fields that are present depend on how the model configuration is set up (there are seemingly countless settings [here](https://docs.litellm.ai/docs/proxy/configs#llm-configs-model_list)). Unless we want just the model names, I would have to look into what are all the possible fields and then select from those to get the ones we think are relevant (but that feels more subjective and cumbersome to maintain as more features could be added).
created_at: 2025-05-23T16:33:13Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
+
+            if not kwargs.get("router_instance", None):
+                span._set_ctx_items({METADATA: metadata})
+                return
+
+            routing_info = {}
+            llm_router = kwargs["router_instance"]
+            routing_info["router_general_settings"] = getattr(llm_router, "router_general_settings", None)
+            routing_info["routing_strategy"] = getattr(llm_router, "routing_strategy", None)
+            routing_info["routing_strategy_args"] = getattr(llm_router, "routing_strategy_args", None)
+            routing_info["provider_budget_config"] = getattr(llm_router, "provider_budget_config", None)
+            routing_info["retry_policy"] = getattr(llm_router, "retry_policy", None)
+            routing_info["enable_tag_filtering"] = getattr(llm_router, "enable_tag_filtering", None)
+            model_list = llm_router.get_model_list() if hasattr(llm_router, "get_model_list") else []
+            routing_info["model_list"] = self._scrub_litellm_model_list(model_list)
path: ddtrace/llmobs/_integrations/litellm.py
user: login: ncybul
body: Sounds good, I had to rework this function because I hit the nested conditional limit, but I updated this per your suggestion.
created_at: 2025-05-23T16:45:43Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
+
+            if not kwargs.get("router_instance", None):
+                span._set_ctx_items({METADATA: metadata})
+                return
+
+            routing_info = {}
+            llm_router = kwargs["router_instance"]
+            routing_info["router_general_settings"] = getattr(llm_router, "router_general_settings", None)
+            routing_info["routing_strategy"] = getattr(llm_router, "routing_strategy", None)
+            routing_info["routing_strategy_args"] = getattr(llm_router, "routing_strategy_args", None)
+            routing_info["provider_budget_config"] = getattr(llm_router, "provider_budget_config", None)
+            routing_info["retry_policy"] = getattr(llm_router, "retry_policy", None)
+            routing_info["enable_tag_filtering"] = getattr(llm_router, "enable_tag_filtering", None)
+            model_list = llm_router.get_model_list() if hasattr(llm_router, "get_model_list") else []
+            routing_info["model_list"] = self._scrub_litellm_model_list(model_list)
+            metadata["router_settings"] = routing_info
+
+        span._set_ctx_items({METADATA: metadata})
+
+    def _scrub_litellm_model_list(self, model_list: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+        for model in model_list:
+            if "litellm_params" in model and "api_key" in model["litellm_params"]:
+                del model["litellm_params"]["api_key"]
+        return model_list
+
+    def _update_input_output_value(self, span: Span, span_kind: str = ""):
+        if span_kind == "llm":
+            return
+        input_messages = span._get_ctx_item(INPUT_MESSAGES)
+        output_messages = span._get_ctx_item(OUTPUT_MESSAGES)
+        if input_messages:
+            span._set_ctx_item(INPUT_VALUE, input_messages)
+        if output_messages:
+            span._set_ctx_item(OUTPUT_VALUE, output_messages)
+
+    def _skip_llm_span(self, kwargs: Dict[str, Any], model: Optional[str] = None) -> bool:
+        """
+        Determine whether an LLM span will be submitted for the given request from outside the LiteLLM integration.
+
+        Currently, this only applies to the OpenAI integration when the following conditions are met:
+            - request is not streamed
+            - model provider is OpenAI/AzureOpenAI
+            - the OpenAI integration is enabled
+        """
+        stream = kwargs.get("stream", False)
+        model_lower = model.lower() if model else ""
+        # model provider is unknown until request completes; therefore, this is a best effort attempt to check
+        # if model provider is Open AI or Azure
+        return (
+            any(prefix in model_lower for prefix in ("gpt", "openai", "azure"))
+            and not stream
+            and LLMObs._integration_is_enabled("openai")
         )
 
+    def _get_span_kind(
+        self, kwargs: Dict[str, Any], model: Optional[str] = None, operation: Optional[str] = None
+    ) -> str:
+        """
+        Workflow span should be submitted to LLMObs if:
+            - span represents a router operation
+            - base_url is set (indicates a request to the proxy) OR
+            - base_url is not set AND an LLM span will be submitted elsewhere
+        LLM spans should be submitted to LLMObs if:
+            - base_url is not set AND an LLM span will not be submitted elsewhere
+        """
+        router_operation = "router" in operation if operation else False
+        base_url = kwargs.get("api_base", None)
path: ddtrace/llmobs/_integrations/litellm.py
user: login: ncybul
body: Ahh thanks for catching. I fixed this one and few others.
created_at: 2025-05-23T16:49:52Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
+
+            if not kwargs.get("router_instance", None):
+                span._set_ctx_items({METADATA: metadata})
+                return
+
+            routing_info = {}
+            llm_router = kwargs["router_instance"]
+            routing_info["router_general_settings"] = getattr(llm_router, "router_general_settings", None)
+            routing_info["routing_strategy"] = getattr(llm_router, "routing_strategy", None)
+            routing_info["routing_strategy_args"] = getattr(llm_router, "routing_strategy_args", None)
+            routing_info["provider_budget_config"] = getattr(llm_router, "provider_budget_config", None)
+            routing_info["retry_policy"] = getattr(llm_router, "retry_policy", None)
+            routing_info["enable_tag_filtering"] = getattr(llm_router, "enable_tag_filtering", None)
+            model_list = llm_router.get_model_list() if hasattr(llm_router, "get_model_list") else []
+            routing_info["model_list"] = self._scrub_litellm_model_list(model_list)
+            metadata["router_settings"] = routing_info
+
+        span._set_ctx_items({METADATA: metadata})
+
+    def _scrub_litellm_model_list(self, model_list: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+        for model in model_list:
+            if "litellm_params" in model and "api_key" in model["litellm_params"]:
+                del model["litellm_params"]["api_key"]
+        return model_list
+
+    def _update_input_output_value(self, span: Span, span_kind: str = ""):
+        if span_kind == "llm":
+            return
+        input_messages = span._get_ctx_item(INPUT_MESSAGES)
+        output_messages = span._get_ctx_item(OUTPUT_MESSAGES)
+        if input_messages:
+            span._set_ctx_item(INPUT_VALUE, input_messages)
+        if output_messages:
+            span._set_ctx_item(OUTPUT_VALUE, output_messages)
+
+    def _skip_llm_span(self, kwargs: Dict[str, Any], model: Optional[str] = None) -> bool:
+        """
+        Determine whether an LLM span will be submitted for the given request from outside the LiteLLM integration.
+
+        Currently, this only applies to the OpenAI integration when the following conditions are met:
+            - request is not streamed
+            - model provider is OpenAI/AzureOpenAI
+            - the OpenAI integration is enabled
+        """
+        stream = kwargs.get("stream", False)
+        model_lower = model.lower() if model else ""
+        # model provider is unknown until request completes; therefore, this is a best effort attempt to check
+        # if model provider is Open AI or Azure
+        return (
+            any(prefix in model_lower for prefix in ("gpt", "openai", "azure"))
+            and not stream
+            and LLMObs._integration_is_enabled("openai")
         )
 
+    def _get_span_kind(
+        self, kwargs: Dict[str, Any], model: Optional[str] = None, operation: Optional[str] = None
+    ) -> str:
+        """
+        Workflow span should be submitted to LLMObs if:
+            - span represents a router operation
+            - base_url is set (indicates a request to the proxy) OR
+            - base_url is not set AND an LLM span will be submitted elsewhere
+        LLM spans should be submitted to LLMObs if:
+            - base_url is not set AND an LLM span will not be submitted elsewhere
+        """
+        router_operation = "router" in operation if operation else False
path: ddtrace/llmobs/_integrations/litellm.py
user: login: ncybul
body: Yep, good idea!
created_at: 2025-05-23T16:52:13Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
+
+            if not kwargs.get("router_instance", None):
+                span._set_ctx_items({METADATA: metadata})
+                return
+
+            routing_info = {}
+            llm_router = kwargs["router_instance"]
+            routing_info["router_general_settings"] = getattr(llm_router, "router_general_settings", None)
+            routing_info["routing_strategy"] = getattr(llm_router, "routing_strategy", None)
+            routing_info["routing_strategy_args"] = getattr(llm_router, "routing_strategy_args", None)
+            routing_info["provider_budget_config"] = getattr(llm_router, "provider_budget_config", None)
+            routing_info["retry_policy"] = getattr(llm_router, "retry_policy", None)
+            routing_info["enable_tag_filtering"] = getattr(llm_router, "enable_tag_filtering", None)
+            model_list = llm_router.get_model_list() if hasattr(llm_router, "get_model_list") else []
+            routing_info["model_list"] = self._scrub_litellm_model_list(model_list)
+            metadata["router_settings"] = routing_info
+
+        span._set_ctx_items({METADATA: metadata})
+
+    def _scrub_litellm_model_list(self, model_list: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+        for model in model_list:
+            if "litellm_params" in model and "api_key" in model["litellm_params"]:
+                del model["litellm_params"]["api_key"]
+        return model_list
+
+    def _update_input_output_value(self, span: Span, span_kind: str = ""):
+        if span_kind == "llm":
+            return
+        input_messages = span._get_ctx_item(INPUT_MESSAGES)
+        output_messages = span._get_ctx_item(OUTPUT_MESSAGES)
+        if input_messages:
+            span._set_ctx_item(INPUT_VALUE, input_messages)
+        if output_messages:
+            span._set_ctx_item(OUTPUT_VALUE, output_messages)
+
+    def _skip_llm_span(self, kwargs: Dict[str, Any], model: Optional[str] = None) -> bool:
+        """
+        Determine whether an LLM span will be submitted for the given request from outside the LiteLLM integration.
+
+        Currently, this only applies to the OpenAI integration when the following conditions are met:
+            - request is not streamed
+            - model provider is OpenAI/AzureOpenAI
+            - the OpenAI integration is enabled
+        """
+        stream = kwargs.get("stream", False)
+        model_lower = model.lower() if model else ""
+        # model provider is unknown until request completes; therefore, this is a best effort attempt to check
+        # if model provider is Open AI or Azure
+        return (
+            any(prefix in model_lower for prefix in ("gpt", "openai", "azure"))
+            and not stream
+            and LLMObs._integration_is_enabled("openai")
         )
 
+    def _get_span_kind(
+        self, kwargs: Dict[str, Any], model: Optional[str] = None, operation: Optional[str] = None
+    ) -> str:
+        """
+        Workflow span should be submitted to LLMObs if:
+            - span represents a router operation
+            - base_url is set (indicates a request to the proxy) OR
+            - base_url is not set AND an LLM span will be submitted elsewhere
path: ddtrace/llmobs/_integrations/litellm.py
user: login: ncybul
body: Yep no problem! It's a bit convoluted for sure but I can explain each component.

> span represents a router operation

For all router operations, we want to submit a workflow span because the router operation will always invoke some LiteLLM SDK method (e.g. `completion`, `acompletion`, `text_completion`, `atext_completion` defined here) which actually performs the LLM call. We will capture this downstream call as an LLM span, and the router operation is essentially just a wrapper around that.

> base_url is set (indicates a request to the proxy)

We have decided to submit all requests directed to a proxy server as workflow spans since again we want the actual downstream LLM call to be instrumented and submitted as an LLM span and we do not want to double submit LLM spans in this case. The `base_url` can be set directly on router methods as well as regular LiteLLM SDK methods.

> base_url is not set AND an LLM span will be submitted elsewhere

Actually while answering this question, I realized that this check is actually not necessary since we already avoid sending LLM Obs spans in this case (inside `should_submit_to_llmobs`). Here you can see an example trace where instead of having an additional `acompletion` span for example between `router.acompletion` and `OpenAI.createChatCompletion`, we avoid sending any span.

I will update the docstring and implementation to get rid of that unnecessary check from that last point.
created_at: 2025-05-23T17:12:26Z
author_association: CONTRIBUTOR
diff_hunk: @@ -75,22 +173,11 @@ def _extract_llmobs_metrics(resp: Any) -> Dict[str, Any]:
 
     def should_submit_to_llmobs(self, kwargs: Dict[str, Any], model: Optional[str] = None) -> bool:
         """
-        Span should be NOT submitted to LLMObs if:
-            - base_url is not None: is a proxy request and we will capture the LLM request downstream
-            - non-streamed request and model provider is OpenAI/AzureOpenAI and the OpenAI integration
-                is enabled: this request will be captured in the OpenAI integration instead
+        LiteLLM spans will be submitted to LLMObs unless the following are true:
+            - base_url is not set AND
path: ddtrace/llmobs/_integrations/litellm.py
user: login: ncybul
body: I am making the assumption that `base_url` being set indicates that the request is going to a LiteLLM proxy server. Therefore, if `base_url` is not set, this indicates a request directly to some model provider (like Open AI). The reason why this is relevant is because in the case that we are using the LiteLLM SDK to send a request directly to Open AI, we do not need to send a span from the LiteLLM integration... we can just rely on the instrumentation we have for Open AI.

For example, the following request uses LiteLLM's SDK to send a request to Open AI, resulting in this [trace](https://dd.datad0g.com/llm/traces?query=%40ml_app%3Anicole-test%20%40event_type%3Aspan%20%40parent_id%3Aundefined&agg_m=count&agg_m_source=base&agg_t=count&fromUser=false&llmPanels=%5B%7B%22t%22%3A%22sampleDetailPanel%22%2C%22rEID%22%3A%22AwAAAZb-Ha4qGZvOxwAAABhBWmItSGE0cUFBQnR2bnFCV2VoQkFBQUEAAAAkMDE5NmZlMWUtMmZhMC00OTgwLWIyODctMmM0YTM2Njc3MGYwAAABCg%22%7D%5D&sidepanelTab=trace&spanId=7579637784709610284&start=1748019799923&end=1748020699923&paused=false) (a single Open AI LLM span).
```
import litellm
messages = [{ "content": "What color is the sky?","role": "user"}]
response = litellm.completion(model="gpt-3.5-turbo", messages=messages, n=1, stream=False)
```
Even though we instrument `litellm.completion`, there is no need to send a span from the LiteLLM integration since LiteLLM uses Open AI's client under the hood which the Open AI integration instruments.
created_at: 2025-05-23T17:16:36Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
+
+            if not kwargs.get("router_instance", None):
+                span._set_ctx_items({METADATA: metadata})
+                return
+
+            routing_info = {}
+            llm_router = kwargs["router_instance"]
+            routing_info["router_general_settings"] = getattr(llm_router, "router_general_settings", None)
+            routing_info["routing_strategy"] = getattr(llm_router, "routing_strategy", None)
+            routing_info["routing_strategy_args"] = getattr(llm_router, "routing_strategy_args", None)
+            routing_info["provider_budget_config"] = getattr(llm_router, "provider_budget_config", None)
+            routing_info["retry_policy"] = getattr(llm_router, "retry_policy", None)
+            routing_info["enable_tag_filtering"] = getattr(llm_router, "enable_tag_filtering", None)
+            model_list = llm_router.get_model_list() if hasattr(llm_router, "get_model_list") else []
+            routing_info["model_list"] = self._scrub_litellm_model_list(model_list)
+            metadata["router_settings"] = routing_info
+
+        span._set_ctx_items({METADATA: metadata})
+
+    def _scrub_litellm_model_list(self, model_list: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+        for model in model_list:
+            if "litellm_params" in model and "api_key" in model["litellm_params"]:
+                del model["litellm_params"]["api_key"]
+        return model_list
+
+    def _update_input_output_value(self, span: Span, span_kind: str = ""):
+        if span_kind == "llm":
+            return
+        input_messages = span._get_ctx_item(INPUT_MESSAGES)
+        output_messages = span._get_ctx_item(OUTPUT_MESSAGES)
+        if input_messages:
+            span._set_ctx_item(INPUT_VALUE, input_messages)
+        if output_messages:
+            span._set_ctx_item(OUTPUT_VALUE, output_messages)
+
+    def _skip_llm_span(self, kwargs: Dict[str, Any], model: Optional[str] = None) -> bool:
path: ddtrace/llmobs/_integrations/litellm.py
user: login: ncybul
body: yeah lol, I honestly was not sure what to call this but that sgtm!

created_at: 2025-05-23T17:23:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
+
+            if not kwargs.get("router_instance", None):
+                span._set_ctx_items({METADATA: metadata})
+                return
+
+            routing_info = {}
+            llm_router = kwargs["router_instance"]
+            routing_info["router_general_settings"] = getattr(llm_router, "router_general_settings", None)
+            routing_info["routing_strategy"] = getattr(llm_router, "routing_strategy", None)
+            routing_info["routing_strategy_args"] = getattr(llm_router, "routing_strategy_args", None)
+            routing_info["provider_budget_config"] = getattr(llm_router, "provider_budget_config", None)
+            routing_info["retry_policy"] = getattr(llm_router, "retry_policy", None)
+            routing_info["enable_tag_filtering"] = getattr(llm_router, "enable_tag_filtering", None)
+            model_list = llm_router.get_model_list() if hasattr(llm_router, "get_model_list") else []
+            routing_info["model_list"] = self._scrub_litellm_model_list(model_list)
+            metadata["router_settings"] = routing_info
+
+        span._set_ctx_items({METADATA: metadata})
+
+    def _scrub_litellm_model_list(self, model_list: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+        for model in model_list:
+            if "litellm_params" in model and "api_key" in model["litellm_params"]:
+                del model["litellm_params"]["api_key"]
+        return model_list
+
+    def _update_input_output_value(self, span: Span, span_kind: str = ""):
+        if span_kind == "llm":
+            return
+        input_messages = span._get_ctx_item(INPUT_MESSAGES)
+        output_messages = span._get_ctx_item(OUTPUT_MESSAGES)
+        if input_messages:
+            span._set_ctx_item(INPUT_VALUE, input_messages)
+        if output_messages:
+            span._set_ctx_item(OUTPUT_VALUE, output_messages)
+
+    def _skip_llm_span(self, kwargs: Dict[str, Any], model: Optional[str] = None) -> bool:
+        """
+        Determine whether an LLM span will be submitted for the given request from outside the LiteLLM integration.
+
+        Currently, this only applies to the OpenAI integration when the following conditions are met:
+            - request is not streamed
+            - model provider is OpenAI/AzureOpenAI
+            - the OpenAI integration is enabled
+        """
+        stream = kwargs.get("stream", False)
+        model_lower = model.lower() if model else ""
+        # model provider is unknown until request completes; therefore, this is a best effort attempt to check
+        # if model provider is Open AI or Azure
+        return (
+            any(prefix in model_lower for prefix in ("gpt", "openai", "azure"))
path: ddtrace/llmobs/_integrations/litellm.py
user: login: ncybul
body: We need to determine whether to submit the span to LLM Obs at the time that the APM span is created (i.e. at the beginning of the request), no? 
created_at: 2025-05-23T17:27:38Z
author_association: CONTRIBUTOR
diff_hunk: @@ -138,6 +139,23 @@ def parse_response(resp, is_completion=False):
     return output_messages, token_metrics
 
 
+model_list = [
+    {
+        "model_name": "gpt-3.5-turbo",
+        "litellm_params": {
+            "model": "gpt-3.5-turbo",
+            "api_key": os.getenv("<not-a-real-key>"),
path: tests/contrib/litellm/utils.py
user: login: ncybul
body: yep, good point
created_at: 2025-05-23T17:57:53Z
author_association: CONTRIBUTOR
diff_hunk: @@ -107,6 +88,68 @@ async def _traced_acompletion(litellm, pin, func, instance, args, kwargs, is_com
             span.finish()
 
 
+@with_traced_module
+def traced_router_completion(litellm, pin, func, instance, args, kwargs):
+    operation = f"router.{func.__name__}"
+    integration = litellm._datadog_integration
+    model = get_argument_value(args, kwargs, 0, "model", None)
+    host = extract_host_tag(kwargs)
+    span = integration.trace(
+        pin,
+        operation,
+        model=model,
+        host=host,
+        submit_to_llmobs=True,
+    )
+    stream = kwargs.get("stream", False)
+    resp = None
+    try:
+        resp = func(*args, **kwargs)
+        if stream:
+            resp.add_router_span_info(span, kwargs, instance)
+        return resp
+    except Exception:
+        span.set_exc_info(*sys.exc_info())
+        raise
+    finally:
+        if not stream:
path: ddtrace/contrib/internal/litellm/patch.py
user: login: ncybul
body: The spans should still be finished within the `TracedLiteLLMStream`.

Here is an example [trace](https://dd.datad0g.com/llm/traces?query=%40ml_app%3Anicole-test%20%40event_type%3Aspan%20%40parent_id%3Aundefined&agg_m=count&agg_m_source=base&agg_t=count&fromUser=true&llmPanels=%5B%7B%22t%22%3A%22sampleDetailPanel%22%2C%22rEID%22%3A%22AwAAAZb-kVMgLD3OxwAAABhBWmIta1ZNZ0FBQ25GMzNCQnF2TUFBQUEAAAAkMDE5NmZlOTEtODVhOC00ZmE1LWIyOTYtMmZiNzZkZWE1MDVkAAAACg%22%7D%5D&sidepanelTab=trace&spanId=13832161768023803796&start=1748026782941&end=1748027682941&paused=false).
created_at: 2025-05-23T19:15:20Z
author_association: CONTRIBUTOR
diff_hunk: @@ -200,43 +203,199 @@ async def test_atext_completion(self, litellm, request_vcr, llmobs_events, mock_
             tags={"ml_app": "<ml-app-name>", "service": "tests.contrib.litellm"},
         )
 
-    @flaky(until=1748750400, reason="Patching Open AI to be used within the LiteLLM library appears to be flaky")
-    def test_completion_openai_enabled(self, litellm, request_vcr, llmobs_events, mock_tracer, stream, n):
+    def test_completion_proxy(self, litellm, request_vcr_include_localhost, llmobs_events, mock_tracer, stream, n):
+        with request_vcr_include_localhost.use_cassette(get_cassette_name(stream, n, proxy=True)):
+            messages = [{"content": "Hey, what is up?", "role": "user"}]
+            resp = litellm.completion(
+                model="gpt-3.5-turbo",
+                messages=messages,
+                stream=stream,
+                n=n,
+                stream_options={"include_usage": True},
+                api_base="http://0.0.0.0:4000",
+            )
+            if stream:
+                output_messages, _ = consume_stream(resp, n)
+            else:
+                output_messages, _ = parse_response(resp)
+
+        span = mock_tracer.pop_traces()[0][0]
+        assert len(llmobs_events) == 1
+        assert llmobs_events[0] == _expected_llmobs_non_llm_span_event(
+            span,
+            span_kind="workflow",
+            input_value=safe_json(messages, ensure_ascii=False),
+            output_value=safe_json(output_messages, ensure_ascii=False),
+            metadata={
+                "stream": stream,
+                "n": n,
+                "stream_options": {"include_usage": True},
+                "api_base": "http://0.0.0.0:4000",
+                "model": "gpt-3.5-turbo",
+            },
+            tags={"ml_app": "<ml-app-name>", "service": "tests.contrib.litellm"},
+        )
+
+    def test_router_completion(self, litellm, request_vcr, llmobs_events, mock_tracer, router, stream, n):
         with request_vcr.use_cassette(get_cassette_name(stream, n)):
-            patch(openai=True)
-            import openai
+            messages = [{"content": "Hey, what is up?", "role": "user"}]
+            resp = router.completion(
+                model="gpt-3.5-turbo",
+                messages=messages,
+                stream=stream,
+                n=n,
+                stream_options={"include_usage": True},
+            )
+            if stream:
+                output_messages, _ = consume_stream(resp, n)
+            else:
+                output_messages, _ = parse_response(resp)
 
-            pin = Pin.get_from(openai)
-            pin._override(openai, tracer=mock_tracer)
+        trace = mock_tracer.pop_traces()[0]
+        assert len(trace) == 2
+        router_span = trace[0]
+
+        assert len(llmobs_events) == 2
+        router_event = llmobs_events[1]
+        assert router_event == _expected_llmobs_non_llm_span_event(
+            router_span,
+            span_kind="workflow",
+            input_value=safe_json(messages, ensure_ascii=False),
+            output_value=safe_json(output_messages, ensure_ascii=False),
+            metadata={
+                "stream": stream,
+                "n": n,
+                "stream_options": {"include_usage": True},
+                "router_settings": expected_router_settings,
+            },
+            tags={"ml_app": "<ml-app-name>", "service": "tests.contrib.litellm"},
+        )
 
+    async def test_router_acompletion(self, litellm, request_vcr, llmobs_events, mock_tracer, router, stream, n):
+        with request_vcr.use_cassette(get_cassette_name(stream, n)):
             messages = [{"content": "Hey, what is up?", "role": "user"}]
-            resp = litellm.completion(
+            resp = await router.acompletion(
                 model="gpt-3.5-turbo",
                 messages=messages,
                 stream=stream,
                 n=n,
                 stream_options={"include_usage": True},
             )
             if stream:
-                for _ in resp:
-                    pass
+                output_messages, _ = await async_consume_stream(resp, n)
+            else:
+                output_messages, _ = parse_response(resp)
 
-        assert len(llmobs_events) == 1
-        assert llmobs_events[0]["name"] == "OpenAI.createChatCompletion" if not stream else "litellm.request"
+        trace = mock_tracer.pop_traces()[0]
+        assert len(trace) == 2
+        router_span = trace[0]
+
+        assert len(llmobs_events) == 2
+        router_event = llmobs_events[1]
path: tests/contrib/litellm/test_litellm_llmobs.py
user: login: ncybul
body: Yep, should be updated now
created_at: 2025-05-27T13:42:10Z
author_association: CONTRIBUTOR
diff_hunk: @@ -10,15 +10,43 @@
 log = get_logger(__name__)
 
 
+def extract_host_tag(kwargs):
+    if "host" in kwargs.get("metadata", {}).get("headers", {}):
+        return kwargs["metadata"]["headers"]["host"]
+    return None
+
+
 class BaseTracedLiteLLMStream(wrapt.ObjectProxy):
-    def __init__(self, wrapped, integration, span, kwargs, is_completion=False):
+    def __init__(self, wrapped, integration, span, kwargs):
         super().__init__(wrapped)
         n = kwargs.get("n", 1) or 1
         self._dd_integration = integration
-        self._dd_span = span
-        self._kwargs = kwargs
+        self._dd_spans = [span]
+        self._kwargs = [kwargs]  # each span is associated with a different kwargs
         self._streamed_chunks = [[] for _ in range(n)]
-        self._is_completion = is_completion
+
+    def add_router_span_info(self, span, kwargs, instance):
+        """Handler to add router span to this streaming object.
+
+        Helps to ensure that all spans associated with a single stream are finished and have the correct tags.
+        """
+        self._dd_spans.append(span)
+        kwargs["router_instance"] = instance
+        self._kwargs.append(kwargs)
+
+    def finish_spans(self):
+        """Helper to finish all spans associated with this stream."""
+        formatted_completions = None
+        for span, kwargs in zip(self._dd_spans, self._kwargs):
+            if not formatted_completions:
+                formatted_completions = _process_finished_stream(
+                    self._dd_integration, span, kwargs, self._streamed_chunks, span.resource
+                )
+            elif self._dd_integration.is_pc_sampled_llmobs(span):
+                self._dd_integration.llmobs_set_tags(
+                    span, args=[], kwargs=kwargs, response=formatted_completions, operation=span.resource
+                )
path: ddtrace/contrib/internal/litellm/utils.py
user: login: ncybul
body: Discussed offline with @Yun-Kim, and we're going to proceed with this strategy as it keeps things consistent for the streaming and non streaming cases (i.e. output values are attached to the workflow span).
created_at: 2025-05-27T13:43:41Z
author_association: CONTRIBUTOR
diff_hunk: @@ -23,30 +24,10 @@ def get_version() -> str:
 
 @with_traced_module
 def traced_completion(litellm, pin, func, instance, args, kwargs):
-    return _traced_completion(litellm, pin, func, instance, args, kwargs, False)
-
-
-@with_traced_module
-async def traced_acompletion(litellm, pin, func, instance, args, kwargs):
-    return await _traced_acompletion(litellm, pin, func, instance, args, kwargs, False)
-
-
-@with_traced_module
-def traced_text_completion(litellm, pin, func, instance, args, kwargs):
-    return _traced_completion(litellm, pin, func, instance, args, kwargs, True)
-
-
-@with_traced_module
-async def traced_atext_completion(litellm, pin, func, instance, args, kwargs):
-    return await _traced_acompletion(litellm, pin, func, instance, args, kwargs, True)
-
-
-def _traced_completion(litellm, pin, func, instance, args, kwargs, is_completion):
+    is_completion = "text" in func.__name__
path: ddtrace/contrib/internal/litellm/patch.py
user: login: ncybul
body: FYI: I ended up removing `is_completion` since this can be inferred from the operation name anyway.
created_at: 2025-05-27T13:45:04Z
author_association: CONTRIBUTOR
diff_hunk: @@ -35,13 +63,11 @@ def __iter__(self):
                 yield chunk
                 _loop_handler(chunk, self._streamed_chunks)
         except Exception:
-            self._dd_span.set_exc_info(*sys.exc_info())
+            for span in self._dd_spans:
+                span.set_exc_info(*sys.exc_info())
path: ddtrace/contrib/internal/litellm/utils.py
user: login: ncybul
body: Discussed with @Yun-Kim offline and agreed that it makes sense to only set the error on the LLM span. I made the appropriate changes to reflect this.
created_at: 2025-05-27T13:56:14Z
author_association: CONTRIBUTOR
diff_hunk: @@ -10,15 +10,43 @@
 log = get_logger(__name__)
 
 
+def extract_host_tag(kwargs):
+    if "host" in kwargs.get("metadata", {}).get("headers", {}):
+        return kwargs["metadata"]["headers"]["host"]
+    return None
+
+
 class BaseTracedLiteLLMStream(wrapt.ObjectProxy):
-    def __init__(self, wrapped, integration, span, kwargs, is_completion=False):
+    def __init__(self, wrapped, integration, span, kwargs):
         super().__init__(wrapped)
         n = kwargs.get("n", 1) or 1
         self._dd_integration = integration
-        self._dd_span = span
-        self._kwargs = kwargs
+        self._dd_spans = [span]
+        self._kwargs = [kwargs]  # each span is associated with a different kwargs
         self._streamed_chunks = [[] for _ in range(n)]
-        self._is_completion = is_completion
+
+    def add_router_span_info(self, span, kwargs, instance):
path: ddtrace/contrib/internal/litellm/utils.py
user: login: Kyle-Verhoog
body: ```suggestion
    def _add_router_span_info(self, span, kwargs, instance):
```

since we're adding these methods to the object proxy we'll want to make sure to indicate they're not for use by the user
created_at: 2025-05-28T18:00:01Z
author_association: MEMBER
diff_hunk: @@ -10,15 +10,43 @@
 log = get_logger(__name__)
 
 
+def extract_host_tag(kwargs):
+    if "host" in kwargs.get("metadata", {}).get("headers", {}):
+        return kwargs["metadata"]["headers"]["host"]
+    return None
+
+
 class BaseTracedLiteLLMStream(wrapt.ObjectProxy):
-    def __init__(self, wrapped, integration, span, kwargs, is_completion=False):
+    def __init__(self, wrapped, integration, span, kwargs):
         super().__init__(wrapped)
         n = kwargs.get("n", 1) or 1
         self._dd_integration = integration
-        self._dd_span = span
-        self._kwargs = kwargs
+        self._dd_spans = [span]
path: ddtrace/contrib/internal/litellm/utils.py
user: login: Kyle-Verhoog
body: nit: we should probably combine _dd_spans and _kwargs into a `List[Tuple[Span, Dict[str, Any]]` to ensure they remain in sync due to the `zip` call below which will raise if the lists are not the same length.
created_at: 2025-05-28T19:39:19Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,45 @@
+[[
+  {
+    "name": "litellm.request",
+    "service": "tests.contrib.litellm",
+    "resource": "router.completion",
+    "trace_id": 0,
+    "span_id": 1,
+    "parent_id": 0,
+    "type": "",
+    "error": 0,
+    "meta": {
+      "_dd.p.dm": "-0",
+      "_dd.p.tid": "682c9a2b00000000",
+      "language": "python",
+      "litellm.request.model": "gpt-3.5-turbo",
path: tests/snapshots/tests.contrib.litellm.test_litellm.test_litellm_router_completion.json
user: login: Kyle-Verhoog
body: wonder if we should standardize/generalize this tag for future proxy integrations. Then in the UI we can specialize how we present the requested model vs the resolved one (in case they differ)

wdyt @ncybul?

```suggestion
      "proxy.request.model": "gpt-3.5-turbo",
```
created_at: 2025-05-28T19:54:03Z
author_association: MEMBER
diff_hunk: @@ -75,22 +173,11 @@ def _extract_llmobs_metrics(resp: Any) -> Dict[str, Any]:
 
     def should_submit_to_llmobs(self, kwargs: Dict[str, Any], model: Optional[str] = None) -> bool:
         """
-        Span should be NOT submitted to LLMObs if:
-            - base_url is not None: is a proxy request and we will capture the LLM request downstream
-            - non-streamed request and model provider is OpenAI/AzureOpenAI and the OpenAI integration
-                is enabled: this request will be captured in the OpenAI integration instead
+        LiteLLM spans will be submitted to LLMObs unless the following are true:
+            - base_url is not set AND
path: ddtrace/llmobs/_integrations/litellm.py
user: login: Kyle-Verhoog
body: i wonder if we should make this configurable in case for whatever reason there is no openai integration enabled (its broken/not supported/user prefers the litellm interaction). This might be a premature optimization tho and we can actually add support for that if needed when the request comes.
created_at: 2025-05-28T19:58:09Z
author_association: MEMBER
diff_hunk: @@ -75,22 +178,11 @@ def _extract_llmobs_metrics(resp: Any) -> Dict[str, Any]:
 
     def should_submit_to_llmobs(self, kwargs: Dict[str, Any], model: Optional[str] = None) -> bool:
         """
-        Span should be NOT submitted to LLMObs if:
-            - base_url is not None: is a proxy request and we will capture the LLM request downstream
-            - non-streamed request and model provider is OpenAI/AzureOpenAI and the OpenAI integration
-                is enabled: this request will be captured in the OpenAI integration instead
+        LiteLLM spans will be submitted to LLMObs unless the following are true:
+            - base_url is not set AND
+            - the LLM request will be submitted elsewhere (e.g. OpenAI integration)
         """
-        base_url = kwargs.get("api_base", None)
-        if base_url is not None:
-            return False
-        stream = kwargs.get("stream", False)
-        model_lower = model.lower() if model else ""
-        # model provider is unknown until request completes; therefore, this is a best effort attempt to check
-        # if model provider is Open AI or Azure
-        if (
-            any(prefix in model_lower for prefix in ("gpt", "openai", "azure"))
-            and not stream
-            and LLMObs._integration_is_enabled("openai")
-        ):
+        base_url = kwargs.get("api_base")
+        if not base_url and self._has_downstream_openai_span(kwargs, model):
path: ddtrace/llmobs/_integrations/litellm.py
user: login: Kyle-Verhoog
body: should we also be checking for anthropic/aws/google since we have integrations for those as well?
created_at: 2025-05-28T19:58:32Z
author_association: MEMBER
diff_hunk: @@ -75,22 +173,11 @@ def _extract_llmobs_metrics(resp: Any) -> Dict[str, Any]:
 
     def should_submit_to_llmobs(self, kwargs: Dict[str, Any], model: Optional[str] = None) -> bool:
         """
-        Span should be NOT submitted to LLMObs if:
-            - base_url is not None: is a proxy request and we will capture the LLM request downstream
-            - non-streamed request and model provider is OpenAI/AzureOpenAI and the OpenAI integration
-                is enabled: this request will be captured in the OpenAI integration instead
+        LiteLLM spans will be submitted to LLMObs unless the following are true:
+            - base_url is not set AND
path: ddtrace/llmobs/_integrations/litellm.py
user: login: Kyle-Verhoog
body: oh actually i see that you check that the integration is enabled in the helper, nice!
created_at: 2025-05-28T19:59:44Z
author_association: MEMBER
diff_hunk: @@ -10,15 +10,43 @@
 log = get_logger(__name__)
 
 
+def extract_host_tag(kwargs):
+    if "host" in kwargs.get("metadata", {}).get("headers", {}):
+        return kwargs["metadata"]["headers"]["host"]
+    return None
+
+
 class BaseTracedLiteLLMStream(wrapt.ObjectProxy):
-    def __init__(self, wrapped, integration, span, kwargs, is_completion=False):
+    def __init__(self, wrapped, integration, span, kwargs):
         super().__init__(wrapped)
         n = kwargs.get("n", 1) or 1
         self._dd_integration = integration
-        self._dd_span = span
-        self._kwargs = kwargs
+        self._dd_spans = [span]
+        self._kwargs = [kwargs]  # each span is associated with a different kwargs
         self._streamed_chunks = [[] for _ in range(n)]
-        self._is_completion = is_completion
+
+    def add_router_span_info(self, span, kwargs, instance):
path: ddtrace/contrib/internal/litellm/utils.py
user: login: ncybul
body: ah yep, makes sense!
created_at: 2025-05-28T20:32:03Z
author_association: CONTRIBUTOR
diff_hunk: @@ -0,0 +1,45 @@
+[[
+  {
+    "name": "litellm.request",
+    "service": "tests.contrib.litellm",
+    "resource": "router.completion",
+    "trace_id": 0,
+    "span_id": 1,
+    "parent_id": 0,
+    "type": "",
+    "error": 0,
+    "meta": {
+      "_dd.p.dm": "-0",
+      "_dd.p.tid": "682c9a2b00000000",
+      "language": "python",
+      "litellm.request.model": "gpt-3.5-turbo",
path: tests/snapshots/tests.contrib.litellm.test_litellm.test_litellm_router_completion.json
user: login: ncybul
body: This tag is on the APM span and for LLM Obs spans that represent proxy requests, we do not tag the model separately (since these will be workflow spans and we do not have a model field for workflow spans). Are you suggesting that I add a `proxy.request.model` tag explicitly on the LLM Obs spans?
created_at: 2025-05-28T20:39:04Z
author_association: CONTRIBUTOR
diff_hunk: @@ -75,22 +178,11 @@ def _extract_llmobs_metrics(resp: Any) -> Dict[str, Any]:
 
     def should_submit_to_llmobs(self, kwargs: Dict[str, Any], model: Optional[str] = None) -> bool:
         """
-        Span should be NOT submitted to LLMObs if:
-            - base_url is not None: is a proxy request and we will capture the LLM request downstream
-            - non-streamed request and model provider is OpenAI/AzureOpenAI and the OpenAI integration
-                is enabled: this request will be captured in the OpenAI integration instead
+        LiteLLM spans will be submitted to LLMObs unless the following are true:
+            - base_url is not set AND
+            - the LLM request will be submitted elsewhere (e.g. OpenAI integration)
         """
-        base_url = kwargs.get("api_base", None)
-        if base_url is not None:
-            return False
-        stream = kwargs.get("stream", False)
-        model_lower = model.lower() if model else ""
-        # model provider is unknown until request completes; therefore, this is a best effort attempt to check
-        # if model provider is Open AI or Azure
-        if (
-            any(prefix in model_lower for prefix in ("gpt", "openai", "azure"))
-            and not stream
-            and LLMObs._integration_is_enabled("openai")
-        ):
+        base_url = kwargs.get("api_base")
+        if not base_url and self._has_downstream_openai_span(kwargs, model):
path: ddtrace/llmobs/_integrations/litellm.py
user: login: ncybul
body: We won't actually trace anything for Anthropic / AWS / Google since LiteLLM does not use the methods that we have instrumented for these integrations under the hood like it does for Open AI.
created_at: 2025-05-28T20:40:56Z
author_association: CONTRIBUTOR
diff_hunk: @@ -10,15 +10,43 @@
 log = get_logger(__name__)
 
 
+def extract_host_tag(kwargs):
+    if "host" in kwargs.get("metadata", {}).get("headers", {}):
+        return kwargs["metadata"]["headers"]["host"]
+    return None
+
+
 class BaseTracedLiteLLMStream(wrapt.ObjectProxy):
-    def __init__(self, wrapped, integration, span, kwargs, is_completion=False):
+    def __init__(self, wrapped, integration, span, kwargs):
         super().__init__(wrapped)
         n = kwargs.get("n", 1) or 1
         self._dd_integration = integration
-        self._dd_span = span
-        self._kwargs = kwargs
+        self._dd_spans = [span]
path: ddtrace/contrib/internal/litellm/utils.py
user: login: ncybul
body: Gotcha, that makes sense!
created_at: 2025-05-28T20:52:45Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
+
+            if not kwargs.get("router_instance", None):
+                span._set_ctx_items({METADATA: metadata})
+                return
+
+            routing_info = {}
+            llm_router = kwargs["router_instance"]
+            routing_info["router_general_settings"] = getattr(llm_router, "router_general_settings", None)
+            routing_info["routing_strategy"] = getattr(llm_router, "routing_strategy", None)
+            routing_info["routing_strategy_args"] = getattr(llm_router, "routing_strategy_args", None)
+            routing_info["provider_budget_config"] = getattr(llm_router, "provider_budget_config", None)
+            routing_info["retry_policy"] = getattr(llm_router, "retry_policy", None)
+            routing_info["enable_tag_filtering"] = getattr(llm_router, "enable_tag_filtering", None)
+            model_list = llm_router.get_model_list() if hasattr(llm_router, "get_model_list") else []
+            routing_info["model_list"] = self._scrub_litellm_model_list(model_list)
+            metadata["router_settings"] = routing_info
+
+        span._set_ctx_items({METADATA: metadata})
+
+    def _scrub_litellm_model_list(self, model_list: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
path: ddtrace/llmobs/_integrations/litellm.py
user: login: Kyle-Verhoog
body: yeah let's be explicit about including the fields from model_params
created_at: 2025-05-29T18:53:31Z
author_association: MEMBER
diff_hunk: @@ -23,87 +24,125 @@ def get_version() -> str:
 
 @with_traced_module
 def traced_completion(litellm, pin, func, instance, args, kwargs):
-    return _traced_completion(litellm, pin, func, instance, args, kwargs, False)
+    operation = func.__name__
+    integration = litellm._datadog_integration
+    model = get_argument_value(args, kwargs, 0, "model", None)
+    host = extract_host_tag(kwargs)
+    span = integration.trace(
+        pin,
+        operation,
+        model=model,
+        host=host,
+        submit_to_llmobs=integration.should_submit_to_llmobs(kwargs, model),
+    )
+    stream = kwargs.get("stream", False)
+    resp = None
+    try:
+        resp = func(*args, **kwargs)
+        if stream:
+            return TracedLiteLLMStream(resp, integration, span, kwargs)
+        return resp
+    except Exception:
+        span.set_exc_info(*sys.exc_info())
+        raise
+    finally:
+        # streamed spans will be finished separately once the stream generator is exhausted
+        if not stream:
+            if integration.is_pc_sampled_llmobs(span):
+                integration.llmobs_set_tags(span, args=args, kwargs=kwargs, response=resp, operation=operation)
+            span.finish()
 
 
 @with_traced_module
 async def traced_acompletion(litellm, pin, func, instance, args, kwargs):
-    return await _traced_acompletion(litellm, pin, func, instance, args, kwargs, False)
-
-
-@with_traced_module
-def traced_text_completion(litellm, pin, func, instance, args, kwargs):
-    return _traced_completion(litellm, pin, func, instance, args, kwargs, True)
+    operation = func.__name__
+    integration = litellm._datadog_integration
+    model = get_argument_value(args, kwargs, 0, "model", None)
+    host = extract_host_tag(kwargs)
+    span = integration.trace(
+        pin,
+        operation,
+        model=model,
+        host=host,
+        submit_to_llmobs=integration.should_submit_to_llmobs(kwargs, model),
+    )
+    stream = kwargs.get("stream", False)
+    resp = None
+    try:
+        resp = await func(*args, **kwargs)
+        if stream:
+            return TracedLiteLLMAsyncStream(resp, integration, span, kwargs)
+        return resp
+    except Exception:
+        span.set_exc_info(*sys.exc_info())
+        raise
+    finally:
+        # streamed spans will be finished separately once the stream generator is exhausted
+        if not stream:
+            if integration.is_pc_sampled_llmobs(span):
+                integration.llmobs_set_tags(span, args=args, kwargs=kwargs, response=resp, operation=operation)
+            span.finish()
 
 
 @with_traced_module
-async def traced_atext_completion(litellm, pin, func, instance, args, kwargs):
-    return await _traced_acompletion(litellm, pin, func, instance, args, kwargs, True)
-
-
-def _traced_completion(litellm, pin, func, instance, args, kwargs, is_completion):
+def traced_router_completion(litellm, pin, func, instance, args, kwargs):
+    operation = f"router.{func.__name__}"
     integration = litellm._datadog_integration
     model = get_argument_value(args, kwargs, 0, "model", None)
-    host = None
-    if "host" in kwargs.get("metadata", {}).get("headers", {}):
-        host = kwargs["metadata"]["headers"]["host"]
+    host = extract_host_tag(kwargs)
     span = integration.trace(
         pin,
-        func.__name__,
+        operation,
         model=model,
         host=host,
-        submit_to_llmobs=integration.should_submit_to_llmobs(kwargs, model),
+        submit_to_llmobs=True,
     )
     stream = kwargs.get("stream", False)
     resp = None
     try:
         resp = func(*args, **kwargs)
         if stream:
-            return TracedLiteLLMStream(resp, integration, span, kwargs, is_completion)
+            resp._add_router_span_info(span, kwargs, instance)
         return resp
     except Exception:
         span.set_exc_info(*sys.exc_info())
         raise
     finally:
-        # streamed spans will be finished separately once the stream generator is exhausted
         if not stream:
             if integration.is_pc_sampled_llmobs(span):
-                integration.llmobs_set_tags(
-                    span, args=args, kwargs=kwargs, response=resp, operation="completion" if is_completion else "chat"
-                )
+                kwargs["router_instance"] = instance
path: ddtrace/contrib/internal/litellm/patch.py
user: login: Kyle-Verhoog
body: ```suggestion
                kwargs["_dd_router_instance"] = instance
```
created_at: 2025-05-29T19:03:38Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,45 @@
+[[
+  {
+    "name": "litellm.request",
+    "service": "tests.contrib.litellm",
+    "resource": "router.completion",
+    "trace_id": 0,
+    "span_id": 1,
+    "parent_id": 0,
+    "type": "",
+    "error": 0,
+    "meta": {
+      "_dd.p.dm": "-0",
+      "_dd.p.tid": "682c9a2b00000000",
+      "language": "python",
+      "litellm.request.model": "gpt-3.5-turbo",
path: tests/snapshots/tests.contrib.litellm.test_litellm.test_litellm_router_completion.json
user: login: ncybul
body: Discussed with @Kyle-Verhoog offline, I think we will leave this tag as-is for now since this information isn't as relevant for APM spans and we don't currently support showing any model information on workflow spans.
created_at: 2025-05-29T20:03:42Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
+
+            if not kwargs.get("router_instance", None):
+                span._set_ctx_items({METADATA: metadata})
+                return
+
+            routing_info = {}
+            llm_router = kwargs["router_instance"]
+            routing_info["router_general_settings"] = getattr(llm_router, "router_general_settings", None)
+            routing_info["routing_strategy"] = getattr(llm_router, "routing_strategy", None)
+            routing_info["routing_strategy_args"] = getattr(llm_router, "routing_strategy_args", None)
+            routing_info["provider_budget_config"] = getattr(llm_router, "provider_budget_config", None)
+            routing_info["retry_policy"] = getattr(llm_router, "retry_policy", None)
+            routing_info["enable_tag_filtering"] = getattr(llm_router, "enable_tag_filtering", None)
+            model_list = llm_router.get_model_list() if hasattr(llm_router, "get_model_list") else []
+            routing_info["model_list"] = self._scrub_litellm_model_list(model_list)
+            metadata["router_settings"] = routing_info
+
+        span._set_ctx_items({METADATA: metadata})
+
+    def _scrub_litellm_model_list(self, model_list: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
path: ddtrace/llmobs/_integrations/litellm.py
user: login: ncybul
body: Sounds good, just updated it!
created_at: 2025-05-29T20:35:55Z
author_association: CONTRIBUTOR
diff_hunk: @@ -23,87 +24,125 @@ def get_version() -> str:
 
 @with_traced_module
 def traced_completion(litellm, pin, func, instance, args, kwargs):
-    return _traced_completion(litellm, pin, func, instance, args, kwargs, False)
+    operation = func.__name__
+    integration = litellm._datadog_integration
+    model = get_argument_value(args, kwargs, 0, "model", None)
+    host = extract_host_tag(kwargs)
+    span = integration.trace(
+        pin,
+        operation,
+        model=model,
+        host=host,
+        submit_to_llmobs=integration.should_submit_to_llmobs(kwargs, model),
+    )
+    stream = kwargs.get("stream", False)
+    resp = None
+    try:
+        resp = func(*args, **kwargs)
+        if stream:
+            return TracedLiteLLMStream(resp, integration, span, kwargs)
+        return resp
+    except Exception:
+        span.set_exc_info(*sys.exc_info())
+        raise
+    finally:
+        # streamed spans will be finished separately once the stream generator is exhausted
+        if not stream:
+            if integration.is_pc_sampled_llmobs(span):
+                integration.llmobs_set_tags(span, args=args, kwargs=kwargs, response=resp, operation=operation)
+            span.finish()
 
 
 @with_traced_module
 async def traced_acompletion(litellm, pin, func, instance, args, kwargs):
-    return await _traced_acompletion(litellm, pin, func, instance, args, kwargs, False)
-
-
-@with_traced_module
-def traced_text_completion(litellm, pin, func, instance, args, kwargs):
-    return _traced_completion(litellm, pin, func, instance, args, kwargs, True)
+    operation = func.__name__
+    integration = litellm._datadog_integration
+    model = get_argument_value(args, kwargs, 0, "model", None)
+    host = extract_host_tag(kwargs)
+    span = integration.trace(
+        pin,
+        operation,
+        model=model,
+        host=host,
+        submit_to_llmobs=integration.should_submit_to_llmobs(kwargs, model),
+    )
+    stream = kwargs.get("stream", False)
+    resp = None
+    try:
+        resp = await func(*args, **kwargs)
+        if stream:
+            return TracedLiteLLMAsyncStream(resp, integration, span, kwargs)
+        return resp
+    except Exception:
+        span.set_exc_info(*sys.exc_info())
+        raise
+    finally:
+        # streamed spans will be finished separately once the stream generator is exhausted
+        if not stream:
+            if integration.is_pc_sampled_llmobs(span):
+                integration.llmobs_set_tags(span, args=args, kwargs=kwargs, response=resp, operation=operation)
+            span.finish()
 
 
 @with_traced_module
-async def traced_atext_completion(litellm, pin, func, instance, args, kwargs):
-    return await _traced_acompletion(litellm, pin, func, instance, args, kwargs, True)
-
-
-def _traced_completion(litellm, pin, func, instance, args, kwargs, is_completion):
+def traced_router_completion(litellm, pin, func, instance, args, kwargs):
+    operation = f"router.{func.__name__}"
     integration = litellm._datadog_integration
     model = get_argument_value(args, kwargs, 0, "model", None)
-    host = None
-    if "host" in kwargs.get("metadata", {}).get("headers", {}):
-        host = kwargs["metadata"]["headers"]["host"]
+    host = extract_host_tag(kwargs)
     span = integration.trace(
         pin,
-        func.__name__,
+        operation,
         model=model,
         host=host,
-        submit_to_llmobs=integration.should_submit_to_llmobs(kwargs, model),
+        submit_to_llmobs=True,
     )
     stream = kwargs.get("stream", False)
     resp = None
     try:
         resp = func(*args, **kwargs)
         if stream:
-            return TracedLiteLLMStream(resp, integration, span, kwargs, is_completion)
+            resp._add_router_span_info(span, kwargs, instance)
         return resp
     except Exception:
         span.set_exc_info(*sys.exc_info())
         raise
     finally:
-        # streamed spans will be finished separately once the stream generator is exhausted
         if not stream:
             if integration.is_pc_sampled_llmobs(span):
-                integration.llmobs_set_tags(
-                    span, args=args, kwargs=kwargs, response=resp, operation="completion" if is_completion else "chat"
-                )
+                kwargs["router_instance"] = instance
path: ddtrace/contrib/internal/litellm/patch.py
user: login: ncybul
body: Renamed this key and added it to the constants file for better maintainability.
created_at: 2025-05-29T20:51:28Z
author_association: CONTRIBUTOR
diff_hunk: @@ -45,19 +54,108 @@ def _llmobs_set_tags(
         model_name, model_provider = self._model_map.get(model_name, (model_name, ""))
 
         # use Open AI helpers since response format will match Open AI
-        if operation == "completion":
+        if operation in TEXT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_completion(span, kwargs, response)
-        else:
+        elif operation in CHAT_COMPLETION_OPERATIONS:
             openai_set_meta_tags_from_chat(span, kwargs, response)
 
-        metrics = self._extract_llmobs_metrics(response)
+        # custom logic for updating metadata on litellm spans
+        self._update_litellm_metadata(span, kwargs, operation)
+
+        # update input and output value for non-LLM spans
+        span_kind = self._get_span_kind(kwargs, model_name, operation)
+        self._update_input_output_value(span, span_kind)
+
+        metrics = self._extract_llmobs_metrics(response, span_kind)
         span._set_ctx_items(
-            {SPAN_KIND: "llm", MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+            {SPAN_KIND: span_kind, MODEL_NAME: model_name or "", MODEL_PROVIDER: model_provider, METRICS: metrics}
+        )
+
+    def _update_litellm_metadata(self, span: Span, kwargs: Dict[str, Any], operation: str):
+        metadata = span._get_ctx_item(METADATA) or {}
+        base_url = kwargs.get("api_base", None)
+        # only add model to metadata if it's a litellm client request
+        if base_url:
+            if "model" in kwargs:
+                metadata["model"] = kwargs["model"]
+        # add router information to metadata if it's a span from the router
+        elif "router" in operation:
+            if "metadata" in metadata and "user_api_key" in metadata["metadata"]:
+                del metadata["metadata"]["user_api_key"]
path: ddtrace/llmobs/_integrations/litellm.py
user: login: ncybul
body: I decided to reformat the metadata within the litellm integration when I call `self._update_litellm_metadata`. I hardcoded the fields that I want to keep and update the metadata field to relect that.
created_at: 2025-05-29T21:18:49Z
author_association: CONTRIBUTOR

PR comments:
user: login: github-actions[bot]
created_at: 2025-05-16T18:13:51Z
body: `CODEOWNERS` have been resolved as:

```
releasenotes/notes/litellm-router-tracing-04d28e5b3fc73702.yaml         @DataDog/apm-python
tests/snapshots/tests.contrib.litellm.test_litellm.test_litellm_router_completion.json  @DataDog/apm-python
ddtrace/contrib/_litellm.py                                             @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/litellm/patch.py                               @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/litellm/utils.py                               @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/llmobs/_constants.py                                            @DataDog/ml-observability
ddtrace/llmobs/_integrations/litellm.py                                 @DataDog/ml-observability
ddtrace/llmobs/_integrations/utils.py                                   @DataDog/ml-observability
ddtrace/llmobs/_utils.py                                                @DataDog/ml-observability
docs/integrations.rst                                                   @DataDog/python-guild
docs/spelling_wordlist.txt                                              @DataDog/python-guild
tests/contrib/litellm/conftest.py                                       @DataDog/apm-core-python @DataDog/apm-idm-python
tests/contrib/litellm/test_litellm.py                                   @DataDog/apm-core-python @DataDog/apm-idm-python
tests/contrib/litellm/test_litellm_llmobs.py                            @DataDog/apm-core-python @DataDog/apm-idm-python
tests/contrib/litellm/utils.py                                          @DataDog/apm-core-python @DataDog/apm-idm-python
```
<!-- thollander/actions-comment-pull-request "codeowners_resolved" -->
author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-05-16T18:34:14Z
body: ## Bootstrap import analysis

Comparison of import times between this PR and base.

### Summary

The average import time from this PR is: 238 ± 4 ms.

The average import time from base is: 248 ± 8 ms.

The import time difference between this PR and base is: -10.5 ± 0.3 ms.


### Import time breakdown

The following import paths have **shrunk**:
<details style="padding-left: 1em"><summary><progress value="2757.2617382617377" max="237916.7982017982" style="width: 2em;"></progress>
            <code>ddtrace.auto</code>
            <span style="color: #888; font-size:small; font-family: arial;">2.757 ms</span>
            <span style="color: #888; font-size:small; font-family: arial;">(1.16%)</span>
        </summary><details style="padding-left: 1em"><summary><progress value="2043.1828171828165" max="237916.7982017982" style="width: 2em;"></progress>
            <code>ddtrace.bootstrap.sitecustomize</code>
            <span style="color: #888; font-size:small; font-family: arial;">2.043 ms</span>
            <span style="color: #888; font-size:small; font-family: arial;">(0.86%)</span>
        </summary><details style="padding-left: 1em"><summary><progress value="2043.1828171828165" max="237916.7982017982" style="width: 2em;"></progress>
            <code>ddtrace.bootstrap.preload</code>
            <span style="color: #888; font-size:small; font-family: arial;">2.043 ms</span>
            <span style="color: #888; font-size:small; font-family: arial;">(0.86%)</span>
        </summary><div style="padding-left: 2em"><progress value="816.1378621378626" max="237916.7982017982" style="width: 2em;"></progress>
            <code>ddtrace.internal.remoteconfig.client</code>
            <span style="color: #888; font-size:small; font-family: arial;">0.816 ms</span>
            <span style="color: #888; font-size:small; font-family: arial;">(0.34%)</span>
        </div></details>
</details>
<details style="padding-left: 1em"><summary><progress value="714.0789210789211" max="237916.7982017982" style="width: 2em;"></progress>
            <code>ddtrace</code>
            <span style="color: #888; font-size:small; font-family: arial;">0.714 ms</span>
            <span style="color: #888; font-size:small; font-family: arial;">(0.30%)</span>
        </summary><div style="padding-left: 2em"><progress value="36.1388611388611" max="237916.7982017982" style="width: 2em;"></progress>
            <code>ddtrace.internal._unpatched</code>
            <span style="color: #888; font-size:small; font-family: arial;">0.036 ms</span>
            <span style="color: #888; font-size:small; font-family: arial;">(0.02%)</span>
        </div></details>
</details>



<!-- thollander/actions-comment-pull-request "import_analysis" -->
author_association: CONTRIBUTOR
user: login: pr-commenter[bot]
created_at: 2025-05-16T18:59:05Z
body: ## Benchmarks

Benchmark execution time: 2025-05-30 15:05:06

Comparing candidate commit 7c920dd3d33b0f0ee5a519e4c071d6c4b0986789 in PR branch `nicole-cybul/instrument-router-methods` with baseline commit 7fbdc9f97e94d052a9eab609631e693f9b19e560 in branch `main`.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 480 metrics, 4 unstable metrics.


author_association: NONE