Repository: truera/trulens
Issue: Feedback functions returning FAILED
Root cause: Configuration Error

**Bug Description**
The Feedback functions I'm trying to use to evaluate are returning FAILED.

**To Reproduce**
My code:

```
import numpy as np
from pinecone import Pinecone
from trulens_eval import Tru, Select
from trulens_eval.feedback import Groundedness
from trulens_eval.feedback.feedback import Feedback
from trulens_eval.feedback.provider.openai import AzureOpenAI as fAzure
from trulens_eval.tru_virtual import VirtualRecord, TruVirtual, VirtualApp

#This part is inside the function that does the retrieval
virtual_app = dict(llm=dict(model='gpt-35-turbo-16k', api_version='2024-04-25-preview'))
    virtual_app = VirtualApp(virtual_app)
    retriever = Select.RecordCalls.retriever
    virtual_app[retriever] = "retriever"

    tru_recorder = TruVirtual(
                    app_id="pinecone-retrieval", 
                    app=virtual_app, 
                    feedbacks=[f_groundedness, f_context_relevance],
                    feedback_mode="with_app_thread"
                    ) 
    rec = VirtualRecord(
        main_input=[query],
        main_output=[output],
        calls=
            {
                Select.RecordCalls.retriever.get_context: dict(
                    args=[query],
                    rets=[context]
                )
            }
        )
    tru_recorder.add_record(rec)

Tru().reset_database()
provider = fAzure(
                deployment_name='openai-******-**-******-*',
                endpoint='https://openai-******-**-******-*.openai.azure.com/',
                api_version= '2024-04-25-preview'
            )
grounded = Groundedness(groundedness_provider=provider)
f_groundedness = (
    Feedback(grounded.groundedness_measure_with_cot_reasons, name = "Groundedness")
    .on(Select.RecordCalls.retriever.get_context.rets[:].collect())  
    .on_output()
    .aggregate(grounded.grounded_statements_aggregator)
)
f_context_relevance = (
    Feedback(provider.qs_relevance_with_cot_reasons, name = "Context Relevance")
    .on_input()
    .on(Select.RecordCalls.retriever.get_context.rets[:])
    .aggregate(np.mean)
)

Tru().run_dashboard()
Tru().start_evaluator()
Tru().stop_evaluator()
```


**Relevant Logs/Tracebacks**
```
Starting dashboard ...
Config file already exists. Skipping writing process.
Credentials file already exists. Skipping writing process.


  You can now view your Streamlit app in your browser.



Dashboard started at http://*:8501 .
  Network URL: http://*:8501

  External URL: http://*:8501
Will keep max of 32 feedback(s) running.



Tasks are spread among max of 128 thread(s).
Will rerun running feedbacks after a minute.
Will rerun failed feedbacks after 5 minutes.
Feedback Status:   0%|                  Evaluator stopped.                                  | 0/24 [00:00<?, ?feedbacks/s, FAILED=24] 
Feedback Status:   0%|                                                                      | 0/24 [00:00<?, ?feedbacks/s, FAILED=24] 
Done Runs: 0runs [00:00, ?runs/s]runs/s]
Waiting for Runs: 0runs [00:00, ?runs/s]
```

**Environment:**
- OS: Windows 10
- Python - 3.10
- TruLens - 0.28.2



<!-- Answer -->
To address the issue with your feedback functions returning "FAILED", consider the following steps:

1. **Check Provider Initialization**: Ensure you're using the correct feedback provider and it's properly initialized. For LLM-based feedback, you should use an instance of `LLMProvider` or its subclasses tailored to your model.

2. **Endpoint Configuration**: Verify the endpoint configuration for your provider. It's crucial for the provider to communicate with the model backend correctly.

3. **Prompt Formatting**: Make sure your prompts are correctly formatted for the feedback functions you're using. Incorrect formatting can lead to evaluation failures.

4. **Review Logs for Errors**: Inspect your application's logs for any warnings or errors that could indicate what's causing the failures.

5. **Validate Input Data**: Ensure the input data you're passing to the feedback functions is correctly formatted and valid.

6. **Update Libraries**: Confirm that `trulens_eval`, `numpy`, and other dependencies are up to date to avoid compatibility issues.

7. **Adjust Parameters**: If your feedback functions allow specifying parameters like `temperature` and `normalize`, make sure they are set appropriately for your use case.

By following these steps, you should be able to identify and resolve the issues causing your feedback functions to fail. For more detailed guidance, refer to the relevant sections in the `trulens_eval` documentation and source code, particularly those related to feedback providers and the specific feedback functions you're using.


<!-- Dosu Comment Footer -->
<details>
<summary>

#### Details
</summary>

**Sources**
- [trulens_eval/trulens_eval/tests/context_relevance_benchmark.ipynb](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/tests/context_relevance_benchmark.ipynb)
- [trulens_eval/trulens_eval/feedback/groundedness.py](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/groundedness.py)
- [trulens_eval/trulens_eval/feedback/provider/base.py](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/base.py)
- [trulens_eval/trulens_eval/feedback/feedback.py](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/feedback.py)
- [trulens_eval/trulens_eval/tru_virtual.py](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/tru_virtual.py)
- [trulens_eval/examples/experimental/virtual_example.ipynb](https://github.com/truera/trulens/blob/main/trulens_eval/examples/experimental/virtual_example.ipynb)
- [trulens_eval/trulens_eval/feedback/v2/feedback.py](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/v2/feedback.py)
- [trulens_eval/trulens_eval/tru.py](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/tru.py)
- [trulens_eval/trulens_eval/feedback/provider/openai.py](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/openai.py)
- [trulens_eval/trulens_eval/database/sqlalchemy.py](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/database/sqlalchemy.py)

<br/>

**About Dosu**
This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.
</details>

*To continue the conversation, mention @dosu.*


Hi @DEBTHOMAZ - can you test the feedback directly by running the following and sharing the output/error?

```python
f_context_relevance("what is the capital of France?","France is a unitary, semi-presidential republic with its capital in Paris, it's largest city and main cultural and commercial center.")
```

Hello @joshreini1! Now returned this error:

```
Traceback (most recent call last):
  File "C:\Users\*\*.py", line 80, in *
    f_context_relevance("what is the capital of France?","France is a unitary, semi-presidential republic with its capital in Paris, it's largest city and main cultural and commercial center.")
  File "C:\Users\*\*-env\lib\site-packages\trulens_eval\feedback\feedback.py", line 432, in __call__
    return self.imp(*args, **kwargs)
  File "C:\Users\*\*-env\lib\site-packages\trulens_eval\feedback\provider\base.py", line 415, in qs_relevance_with_cot_reasons
    return self.context_relevance_with_cot_reasons(question, context)
  File "C:\Users\*\*-env\lib\site-packages\trulens_eval\feedback\provider\base.py", line 401, in context_relevance_with_cot_reasons
    return self.generate_score_and_reasons(system_prompt=system_prompt, user_prompt=user_prompt, temperature=temperature)
  File "C:\Users\*\*-env\lib\site-packages\trulens_eval\feedback\provider\base.py", line 251, in generate_score_and_reasons
    response = self.endpoint.run_in_pace(
  File "C:\Users\*\*-env\lib\site-packages\trulens_eval\feedback\provider\endpoint\base.py", line 306, in run_in_pace
    raise RuntimeError(
RuntimeError: Endpoint openai request failed 4 time(s):
        Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
        Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
        Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
        Error code: 404 - {'error': {'code': '404', 'message': 'Resource not found'}}
```

Thanks @DEBTHOMAZ - the root cause here is your Azure provider. Can you make sure the deployment name is correct and that the following environment variables are set?

- AZURE_OPENAI_ENDPOINT
- AZURE_OPENAI_API_KEY
- OPENAI_API_VERSION

Hello @joshreini1! 
Yes, they are set and correct, still not working..

Can you try in an isolated file in case there is some namespace conflict or other issue the below?

```
import os
os.environ["AZURE_OPENAI_API_KEY"] = "..." # azure
os.environ["AZURE_OPENAI_ENDPOINT"] = "https://<your endpoint here>.openai.azure.com/" # azure
os.environ["OPENAI_API_VERSION"] = "2023-07-01-preview" # may need updating
os.environ["OPENAI_API_TYPE"] = "azure"

from trulens_eval.feedback.provider.openai import AzureOpenAI
provider = AzureOpenAI(deployment_name="...")

provider.relevance(
    prompt="Where is Germany?",
    response="Poland is in Europe."
)
```

Hello @joshreini1! 
I got the endpoint failing again with 404 code, so now I'm convinced that the problem is the endpoint.

Can you tell me if there is any configuration in the endpoint that could be causing this error?

The OpenAI model is 'gpt-35-turbo-16k', I don't know if it could be a problem..

OBS: There is one thing tho, I did the test with another endpoint that I know is working, and the same error happened. 

@joshreini1 Hello! Any updates on this matter?

No updates - seems to be a problem with your endpoint, but unable to reproduce (since it's your endpoint). Others have mentioned that fiddling with environmental variables can fix this issue.