Repository: truera/trulens
Issue: [BUG] Azure OpenAI error with evaluation using CustomApp
Root cause: Configuration Error

**Bug Description**

When looking for information on this I found another issue that this seems similar to: https://github.com/truera/trulens/issues/1071 (which was never really resolved?).  Though in my case I am using TruCustomApp rather than Llamaindex.

The issue is that when running with a recorder and TruCustomApp I get these warnings:
`WARNING:trulens_eval.feedback.provider.endpoint.base:Callback class OpenAICallback is registered for handling create but there are no endpoints waiting to receive the result.`
and then eventually the error:
`openai.NotFoundError: Error code: 404 - {'error': {'code': 'DeploymentNotFound', 'message': 'The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.'}}`

_However_ before you say it's an issue with my provider, I've actually run some tests with invoking the provider directly and that works fine.  That is to say I can run things like `provider.relevance(prompt=question, response=response)` where I manually insert the appropriate text and there are zero issues.  But when I use the same provider (not just same settings, literally the same instance) inside of an app with recording suddenly it claims there's no endpoint.

**To Reproduce**
Below is a sample of my code, and yes I know it's kind of odd.  It's using LangChain but I could note utilize TruChain because I am working with an existing code base that was written in a bit of a non-standard way.  doc_search here is a LangChain tool and it's essentially functioning as a retriever but it doesn't actually inherent from BaseRetriever.  And as such I could not figure out how to get context when using TruChain.  That's the entire reason I went with TruCustomApp so I could specify it manually.  Though I then call retrieve_chunks from respond_to_query even though the result isn't used there just to try and log things.  That means it actually gets called twice each iteration because it's used again by agent_chain.  It doesn't seem like a great way of handling things but I'm not sure what a better way would be without rewriting the code I'm trying to evaluate here.

Anyway, I don't see how any of that would be related to the provider errors I'm getting.

```
provider = AzureOpenAI(
    deployment_name=MODEL,
    api_key=get_apim_key(MODEL),
    azure_endpoint=os.environ["APIM_ENDPOINT"] + MODEL,
    api_version=os.environ["AZURE_OPENAI_API_VERSION"]
)

class CustomApp:
    def __init__(self, doc_search, agent_chain):
        self.doc_search = doc_search
        self.agent_chain = agent_chain

    @instrument
    def retrieve_chunks(self, query):
        context = self.doc_search.run(query)
        return [x.page_content for x in context]
    
    @instrument
    def respond_to_query(self, input):
        context = self.retrieve_chunks(input)

        response = self.agent_chain.invoke({"input": input})
        return response['output']
    
f_groundedness = Feedback(provider.groundedness_measure_with_cot_reasons).on(Select.RecordCalls.retrieve_chunks.rets).on(Select.RecordCalls.respond_to_query.rets)
f_answer_relevance = Feedback(provider.relevance).on(Select.RecordCalls.respond_to_query.args.input).on(Select.RecordCalls.respond_to_query.rets)
f_context_relevance = Feedback(provider.context_relevance_with_cot_reasons).on(Select.RecordCalls.retrieve_chunks.args.query).on(Select.RecordCalls.retrieve_chunks.rets.collect()).aggregate(np.mean)

ca = CustomApp(doc_search, agent_chain)
tru_recorder = TruCustomApp(ca, app_id="Test", feedbacks=[f_context_relevance, f_groundedness, f_answer_relevance])

with open("test-queries.csv") as csvfile:
    reader = csv.reader(csvfile)

    with tru_recorder as recording:
        for num, row in enumerate(reader):
            question = row[0]

            ca.respond_to_query(question)
```

**Expected behavior**
I would expect to run the app with recording without errors.

**Relevant Logs/Tracebacks**
```
WARNING:trulens_eval.feedback.provider.endpoint.base:Callback class OpenAICallback is registered for handling create but there are no endpoints waiting to receive the result.
WARNING:trulens_eval.feedback.provider.endpoint.base:Callback class OpenAICallback is registered for handling create but there are no endpoints waiting to receive the result.
WARNING:trulens_eval.feedback.provider.endpoint.base:Callback class OpenAICallback is registered for handling create but there are no endpoints waiting to receive the result.
ERROR:trulens_eval.instruments:Error calling wrapped function respond_to_query.
ERROR:trulens_eval.instruments:Traceback (most recent call last):
  File "c:\Users\DKettle1\code\functionapp\azure-func-qtm-kb-search\.venv\Lib\site-packages\trulens_eval\instruments.py", line 511, in tru_wrapper
    rets, cost = mod_endpoint.Endpoint.track_all_costs_tally(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\DKettle1\code\functionapp\azure-func-qtm-kb-search\.venv\Lib\site-packages\trulens_eval\feedback\provider\endpoint\base.py", line 496, in track_all_costs_tally
    result, cbs = Endpoint.track_all_costs(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\DKettle1\code\functionapp\azure-func-qtm-kb-search\.venv\Lib\site-packages\trulens_eval\feedback\provider\endpoint\base.py", line 477, in track_all_costs
    return Endpoint._track_costs(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\DKettle1\code\functionapp\azure-func-qtm-kb-search\.venv\Lib\site-packages\trulens_eval\feedback\provider\endpoint\base.py", line 574, in _track_costs
    result: T = __func(*args, **kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\DKettle1\AppData\Local\Temp\ipykernel_23040\2912314505.py", line 16, in respond_to_query
    response = self.agent_chain.invoke({"input": input})
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\DKettle1\code\functionapp\azure-func-qtm-kb-search\.venv\Lib\site-packages\langchain\chains\base.py", line 166, in invoke
    raise e
  File "c:\Users\DKettle1\code\functionapp\azure-func-qtm-kb-search\.venv\Lib\site-packages\langchain\chains\base.py", line 156, in invoke
    self._call(inputs, run_manager=run_manager)
  File "c:\Users\DKettle1\code\functionapp\azure-func-qtm-kb-search\.venv\Lib\site-packages\langchain\agents\agent.py", line 1433, in _call
...
    raise self._make_status_error_from_response(err.response) from None
openai.NotFoundError: Error code: 404 - {'error': {'code': 'DeploymentNotFound', 'message': 'The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.'}}
```

**Environment:**
 - OS: Windows
 - Python 3.9
 - TruLens 0.32.0


Hey @dtkettler - thanks for your patience. This may be solved with TruLens 1.0, can you try upgrading and let me know if the issue persists?

The code you have above would remain the same, you would just need slightly different imports and a few additional packages:

`! pip install trulens-core trulens-providers-openai trulens-dashboard`

`from trulens_eval import AzureOpenAI` => `from trulens.providers.openai.provider import AzureOpenAI`
`from trulens_eval import TruCustomApp` => `from trulens.apps.custom import TruCustomApp`


Hi @dtkettler, will close this out for now but feel free to reopen if you find the issue is not resolved with TruLens>=1.0.0