Repository: truera/trulens
Issue: [BUG] Using AzureOpenAI provider does not show token usage/price
Root cause: Validation Gap

**Bug Description**
Cost/Token Usage display does not work with AzureOpenai Provider

**To Reproduce**
Try to evaluate a RAG application using the AzureOpenai provider for any metric like groundedness or answer relevance

**Expected behavior**
It should display the Token usage/price

**Relevant Logs/Tracebacks**
I implemented both a custom retriever and one using Trullama. In both cases, I used AzureOpenAI as LLM.:

from trulens_eval.feedback.provider.openai import AzureOpenAI as fAzureOpenai
openai_provider = fAzureOpenai(deployment_name="chat_test")

then 
f_qa_relevance = (
    Feedback(
        openai_provider.relevance_with_cot_reasons,
        name = "Answer Relevance",
    )
    .on(Select.RecordCalls.retrieve.args.query)
    .on_output()
)

and so on. The usage is always 0.

**Environment:**
 - OS: Windows 11 and Archlinux both tested
 - python 3.10.13
 - Trulens 0.28.2

**Additional context**
none


<!-- Answer -->
The issue with the cost/token usage display not working for the AzureOpenAI provider when evaluating a RAG application for metrics like groundedness or answer relevance is likely due to incomplete or incorrect cost tracking implementation for Azure OpenAI models.

### Steps to Resolve:

1. **Verify Environment Variables**: Ensure that the following environment variables are correctly set:
   - `OPENAI_API_VERSION`
   - `OPENAI_API_TYPE`
   - `AZURE_OPENAI_DEPLOYMENT_NAME`
   - `AZURE_OPENAI_ENDPOINT`

2. **Check Cost Tracking Implementation**: Review the `AzureOpenAI` class to ensure that cost tracking is correctly implemented. Verify that the cost tracking logic is integrated and that API responses are parsed to extract cost information.

3. **Debugging**: Add logging or debugging statements in the `AzureOpenAI` class to trace the cost tracking logic and identify any failures.

4. **Update Trulens**: Make sure you are using the latest version of Trulens, as updates or bug fixes might address this issue.

If the problem persists, further investigation into the `AzureOpenAI` class and the `_test_llm_provider_endpoint` method may be necessary to identify any missing implementations related to cost tracking.


<!-- Dosu Comment Footer -->
<details>
<summary>

#### Details
</summary>

**Sources**
- [trulens_eval/tests/e2e/test_endpoints.py](https://github.com/truera/trulens/blob/main/trulens_eval/tests/e2e/test_endpoints.py)
- [trulens_eval/examples/expositional/models/azure_openai_llama_index.ipynb](https://github.com/truera/trulens/blob/main/trulens_eval/examples/expositional/models/azure_openai_llama_index.ipynb)

<br/>

**About Dosu**
This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.
</details>

*To continue the conversation, mention @dosu.*


@sfc-gh-dhuang can you take a look at this?

This is a duplicate of https://github.com/truera/trulens/issues/1342 so closing in favor of that which I saw first (though came later haha).