Repository: truera/trulens
Issue: Use OpenAI provider with base_url
Root cause: Configuration Error

```python
from trulens_eval.feedback.provider import OpenAI
from trulens_eval import Feedback
import numpy as np

provider = OpenAI()

from trulens_eval.app import App
context = App.select_context(rag_chain)

f_groundedness = (
    Feedback(provider.groundedness_measure_with_cot_reasons, name = "Groundedness")
    .on(context.collect()) # collect context chunks into a list
    .on_output()
)

f_answer_relevance = (
    Feedback(provider.relevance_with_cot_reasons, name = "Answer Relevance")
    .on_input_output()
)

f_context_relevance = (
    Feedback(provider.context_relevance_with_cot_reasons, name = "Context Relevance")
    .on_input()
    .on(context)
    .aggregate(np.mean)
)
``` 
![image](https://github.com/user-attachments/assets/d21a4a22-253d-42a3-8f2f-85ac0380e314)

The above feedback funtion cannot be correctly called for, I suppose the reason is that the OpenAI's key is not correct. I wonder know can we define the base_url by ourselves?

<!-- Greeting -->
@SetonLiang 您好! 我是一个可以帮助您解决错误、回答问题并成为贡献者的机器人。我将在等待人类维护者期间为您提供帮助。

<!-- Answer -->
Yes, you can define the `base_url` for the OpenAI provider manually. Here is an example of how to do it:

```python
from trulens_eval.feedback.provider.openai import OpenAI
from trulens_eval.feedback.provider.endpoint import OpenAIEndpoint

# Define your custom base_url
custom_base_url = "https://api.custom-openai.com/v1"

# Create an OpenAIEndpoint with the custom base_url
custom_endpoint = OpenAIEndpoint(base_url=custom_base_url)

# Create an OpenAI provider with the custom endpoint
openai_provider = OpenAI(endpoint=custom_endpoint)

# Now you can use openai_provider with the custom base_url
```

In your case, you can replace the `provider` initialization with the custom endpoint:

```python
provider = OpenAI(endpoint=custom_endpoint)
```

This should help ensure that the feedback functions are correctly called with the specified `base_url` <sup>[[1]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/openai.py)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


It seems that it's still empty with regard to those three metrics.
```python
answer_relevance_result = f_answer_relevance("What is Task Decomposition?", rag_chain.invoke("What is Task Decomposition?"))
```  
Can I use the above statement to test whether the api key can be used or not? It may cause a bug about the wrong call of api key when I use now.

<!-- Answer -->
To test whether the API key is valid and can be used, you can indeed use a feedback function call. However, ensure that the feedback function is correctly set up and that the API key is properly configured in your environment variables.

Here's a simplified example to test the `f_answer_relevance` feedback function:

```python
import os
from trulens_eval.feedback.provider import OpenAI
from trulens_eval import Feedback

# Set your OpenAI API key
os.environ["OPENAI_API_KEY"] = "your_openai_api_key"

# Initialize the OpenAI provider
provider = OpenAI()

# Define the feedback function for answer relevance
f_answer_relevance = Feedback(provider.relevance_with_cot_reasons, name="Answer Relevance").on_input_output()

# Test the feedback function
question = "What is Task Decomposition?"
answer = "Task decomposition is the process of breaking down a complex task into smaller, more manageable components."

# Call the feedback function
answer_relevance_result = f_answer_relevance(question, answer)

# Print the result
print(answer_relevance_result)
```

Make sure to replace `"your_openai_api_key"` with your actual OpenAI API key. This code will help you verify if the API key is valid and if the feedback function is working correctly.

If the result is still empty, double-check the following:
1. Ensure the API key is correctly set and valid.
2. Verify that the `relevance_with_cot_reasons` function is correctly implemented and not deprecated.
3. Check for any warnings or errors in the console output that might indicate issues with the API call.

If you encounter any specific errors or warnings, please share them for further assistance.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


![image](https://github.com/user-attachments/assets/d68c5ff3-c894-410b-a3c8-e7a14ad50273)
Even though I define the base_url as the way you provided above, it may cause an incorrect api key error.

<!-- Answer -->
To resolve the issue of the OpenAI API key being incorrectly recognized and used, resulting in a 401 error code, you should ensure that the API key is correctly set and passed to the OpenAI client. Here is a step-by-step approach:

1. **Check Environment Variables**: Ensure that the API key is set in the environment variables. The OpenAI library typically looks for the `OPENAI_API_KEY` environment variable.

2. **Pass API Key Directly**: If you are passing the API key directly in the code, ensure it is correctly passed to the `OpenAIClient`.

Here is an example of how to initialize the `OpenAIClient` with the API key:

```python
import openai as oai
from trulens_eval.feedback.provider.endpoint.openai import OpenAIClient

# Ensure the API key is set in the environment variable
import os
os.environ["OPENAI_API_KEY"] = "your-api-key"

# Alternatively, pass the API key directly
client = oai.OpenAI(api_key="your-api-key")

# Initialize the OpenAIClient with the OpenAI client
openai_client = OpenAIClient(client=client)
```

Ensure that the `api_key` is correctly set and passed to the `OpenAIClient`. If the API key is not correctly set, the OpenAI API will return a 401 error code indicating unauthorized access.

Additionally, you can refer to the `trulens_eval` library's quickstart guide for setting up API keys and initializing the OpenAI client:

```python
import os

os.environ["OPENAI_API_KEY"] = "sk-..."

from trulens_eval import Tru
from trulens_eval.feedback.provider import OpenAI

# Initialize provider class
provider = OpenAI()

# select context to be used in feedback. the location of context is app specific.
from trulens_eval.app import App

context = App.select_context(rag_chain)

# Define a groundedness feedback function
f_groundedness = (
    Feedback(provider.groundedness_measure_with_cot_reasons
            ).on(context.collect())  # collect context chunks into a list
    .on_output()
)

# Question/answer relevance between overall question and answer.
f_answer_relevance = (Feedback(provider.relevance).on_input_output())
# Question/statement relevance between question and each context chunk.
f_context_relevance = (
    Feedback(provider.context_relevance_with_cot_reasons
            ).on_input().on(context).aggregate(np.mean)
)
```

To further ensure that the API keys are correctly set, you can use the `check_keys` or `check_or_set_keys` functions from the `trulens_eval.keys` module:

```python
from trulens_eval.keys import check_keys, check_or_set_keys

# Check if the keys are set
check_keys("OPENAI_API_KEY")

# Alternatively, set the keys if they are not already set
check_or_set_keys(OPENAI_API_KEY="your-api-key")
```

By following these steps and ensuring the API key is correctly set, you should be able to resolve the 401 error code issue <sup>[[1]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/endpoint/openai.py)</sup><sup>[[2]](https://github.com/truera/trulens/blob/main/trulens_eval/generated_files/all_tools.py)</sup><sup>[[3]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/keys.py)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


It's OK when examing the api key by this method.However, it still cause the incorrect error while invoking the f_answers_relevance to get the scores.How should I do, because I define the base_url by myself.

<!-- Answer -->
To resolve the issue of encountering an incorrect API key error when invoking the `f_answer_relevance` function, despite defining the `base_url`, ensure that the environment variable for the API key is correctly set. Here is the relevant code snippet:

```python
import os
os.environ["OPENAI_API_KEY"] = "your_actual_openai_api_key"
```

If the API key is correctly set and you are still encountering the error, double-check that the key is valid and has the necessary permissions.

Additionally, you can use the `check_keys` or `check_or_set_keys` functions from the `trulens_eval.keys` module to ensure that the keys are set correctly:

```python
from trulens_eval.keys import check_keys, check_or_set_keys

# To check if the keys are set
check_keys("OPENAI_API_KEY")

# Alternatively, to set the keys if they are not already set
check_or_set_keys(OPENAI_API_KEY="your_actual_openai_api_key")
```

These functions will help verify that the keys are set correctly and provide informative errors if they are not <sup>[[1]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/tests/answer_relevance_benchmark_small.ipynb)</sup><sup>[[2]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/keys.py)</sup>.

If the issue persists, ensure that the custom `base_url` is correctly passed to the `OpenAIEndpoint` class. Here is an example:

```python
from trulens_eval.feedback.provider.openai import OpenAI
from trulens_eval.feedback.provider.endpoint import OpenAIEndpoint

# Define your custom base_url
custom_base_url = "https://api.custom-openai.com/v1"

# Create an OpenAIEndpoint with the custom base_url
custom_endpoint = OpenAIEndpoint(base_url=custom_base_url)

# Create an OpenAI provider with the custom endpoint
provider = OpenAI(endpoint=custom_endpoint)
```

Make sure that the `base_url` is included in the `**kwargs` and passed correctly to the `OpenAIEndpoint` class <sup>[[3]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/openai.py)</sup><sup>[[4]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/endpoint/openai.py)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


Could the setting of base_url not be https://api.custom-openai.com/v1? For example, change the domain name

<!-- Answer -->
To correctly set a custom `base_url` for the OpenAI API and avoid receiving a 401 error indicating an incorrect API key, you can set the `OPENAI_API_BASE` environment variable. Here is how you can do it:

```python
import os

# Set the custom base URL for the OpenAI API
os.environ["OPENAI_API_BASE"] = "https://your-custom-base-url.com/v1"

# Set the OpenAI API key
os.environ["OPENAI_API_KEY"] = "sk-..."

# Initialize the provider class with the custom base URL
from trulens_eval.feedback.provider_apis import OpenAIEndpoint
openai_endpoint = OpenAIEndpoint(api_base=os.environ["OPENAI_API_BASE"], api_key=os.environ["OPENAI_API_KEY"])

# Define the feedback function
from trulens_eval import Feedback

f_answer_relevance = Feedback(openai_endpoint.relevance).on_input_output()
```

If you are using Azure OpenAI, you need to set additional environment variables and use the `AzureOpenAI` class instead:

```python
import os

# Set the Azure OpenAI environment variables
os.environ["AZURE_OPENAI_API_KEY"] = "..." # azure
os.environ["AZURE_OPENAI_ENDPOINT"] = "https://<your endpoint here>.openai.azure.com/" # azure
os.environ["OPENAI_API_VERSION"] = "2023-07-01-preview" # may need updating
os.environ["OPENAI_API_TYPE"] = "azure"

# Initialize the AzureOpenAI provider
from trulens_eval.feedback.provider import AzureOpenAI
azopenai = AzureOpenAI(
    deployment_name="your-deployment-name",
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
    api_version=os.environ["OPENAI_API_VERSION"]
)

# Define the feedback function
from trulens_eval import Feedback

f_answer_relevance = Feedback(azopenai.relevance).on_input_output()
```

This code sets the necessary environment variables and initializes the `OpenAIEndpoint` or `AzureOpenAI` provider with the custom base URL and API key. This should help avoid the 401 error related to the incorrect API key <sup>[[1]](https://github.com/truera/trulens/blob/main/trulens_eval/generated_files/all_tools.py)</sup><sup>[[2]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/keys.py)</sup><sup>[[3]](https://github.com/truera/trulens/blob/main/trulens_eval/examples/expositional/models/azure_openai_llama_index.ipynb)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


TypeError: OpenAI.__init__() got an unexpected keyword argument 'api_base'

<!-- Answer -->
To correctly set a custom `base_url` for the OpenAI API without encountering a `TypeError` when initializing the `OpenAI` class, you need to pass the `base_url` parameter in the `client_kwargs` dictionary when creating an instance of the `OpenAIClient`. Here is an example of how to do this:

```python
from trulens_eval.feedback.provider.endpoint.openai import OpenAIClient

# Define your custom base URL
custom_base_url = "https://api.custom-openai.com/v1"

# Create the OpenAIClient with the custom base URL
client_kwargs = {
    "base_url": custom_base_url,
    # Add other necessary parameters here
}

openai_client = OpenAIClient(client_kwargs=client_kwargs)
```

This ensures that the `base_url` is correctly set without causing a `TypeError`. The `OpenAIClient` class is designed to accept additional arguments through `client_kwargs`, which are then passed to the underlying OpenAI client <sup>[[1]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/endpoint/openai.py)</sup><sup>[[2]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/openai.py)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


ValueError: `client_kwargs` and `client_cls` are both needed to deserialize an openai.`OpenAI` client.

<!-- Answer -->
To correctly set a custom `base_url` for the OpenAI API without encountering the `ValueError` that states "`client_kwargs` and `client_cls` are both needed to deserialize an openai.`OpenAI` client", you need to ensure that both `client_cls` and `client_kwargs` are provided when initializing the `OpenAIClient`.

Here is an example of how you can set a custom `base_url`:

```python
from trulens_eval.feedback.provider.endpoint.openai import OpenAIClient
from trulens_eval.utils.pyschema import Class
import openai as oai

# Define the custom base URL
custom_base_url = "https://api.custom-openai.com/v1"

# Define the client class and client kwargs
client_cls = Class.of_class(oai.OpenAI)
client_kwargs = {
    "api_key": "your-api-key",
    "base_url": custom_base_url
}

# Initialize the OpenAIClient with the custom base URL
client = OpenAIClient(client_cls=client_cls, client_kwargs=client_kwargs)
```

This ensures that both `client_cls` and `client_kwargs` are provided, thus avoiding the `ValueError` <sup>[[1]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/endpoint/openai.py)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


```python
openai_client = OpenAIClient(client_cls=client_cls, client_kwargs=client_kwargs)
print(openai_client)
# Create an OpenAIEndpoint with the custom base_url
custom_endpoint = OpenAIEndpoint(openai_client)
print(custom_endpoint.client)
provider = OpenAI(endpoint=custom_endpoint)
print(provider)
``` 
The openai_client's base_url and api_key have changed successfully,but it cannot convey to the custom_endpoint, so as the provider

<!-- Answer -->
To ensure that the custom `base_url` and `api_key` are correctly propagated through the `OpenAIClient`, `OpenAIEndpoint`, and `OpenAI` provider, you need to make sure that these parameters are passed correctly at each step. Here is an example of how to do this:

```python
from trulens_eval.feedback.provider.endpoint.openai import OpenAIClient, OpenAIEndpoint
from trulens_eval.feedback.provider.openai import OpenAI
from trulens_eval.utils.pyschema import Class
import openai as oai

# Define the custom base URL and API key
custom_base_url = "https://api.custom-openai.com/v1"
api_key = "your-api-key"

# Define the client class and client kwargs
client_cls = Class.of_class(oai.OpenAI)
client_kwargs = {
    "api_key": api_key,
    "base_url": custom_base_url
}

# Initialize the OpenAIClient with the custom base URL and API key
openai_client = OpenAIClient(client_cls=client_cls, client_kwargs=client_kwargs)
print(openai_client)

# Create an OpenAIEndpoint with the custom OpenAIClient
custom_endpoint = OpenAIEndpoint(client=openai_client)
print(custom_endpoint.client)

# Create an OpenAI provider with the custom endpoint
provider = OpenAI(endpoint=custom_endpoint)
print(provider)
```

This code ensures that the `base_url` and `api_key` are correctly passed through the `OpenAIClient`, `OpenAIEndpoint`, and `OpenAI` provider. The `OpenAIClient` is initialized with the custom `base_url` and `api_key`, and then it is passed to the `OpenAIEndpoint`. Finally, the `OpenAIEndpoint` is used to initialize the `OpenAI` provider.

Make sure to replace `"your-api-key"` with your actual API key. This setup should ensure that the custom `base_url` and `api_key` are correctly propagated through all the components <sup>[[1]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/endpoint/openai.py)</sup><sup>[[2]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/openai.py)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


However, the custom_endpoint.client is 
client=<openai.OpenAI object at 0x0000020545D60710> client_cls=openai.OpenAI client_kwargs={'organization': None, 'project': None, 'base_url': URL('https://api.openai.com/v1/'), where the base_url does not change.

<!-- Answer -->
To ensure that the custom `base_url` is correctly propagated through the `OpenAIClient`, `OpenAIEndpoint`, and `OpenAI` provider, you need to pass the `base_url` parameter through the `client_kwargs` when initializing the `OpenAIClient` and `OpenAIEndpoint`.

Here's how you can do it:

1. **Initialize `OpenAIClient` with `base_url`:**

   ```python
   from trulens_eval.feedback.provider.endpoint.openai import OpenAIClient
   from trulens_eval.utils.pyschema import Class
   import openai as oai

   # Define the custom base URL and API key
   custom_base_url = "https://your.custom.base.url"
   api_key = "your-api-key"

   # Define the client class and client kwargs
   client_cls = Class.of_class(oai.OpenAI)
   client_kwargs = {
       "api_key": api_key,
       "base_url": custom_base_url
   }

   # Initialize the OpenAIClient with the custom base URL and API key
   openai_client = OpenAIClient(client_cls=client_cls, client_kwargs=client_kwargs)
   ```

2. **Initialize `OpenAIEndpoint` with `OpenAIClient`:**

   ```python
   from trulens_eval.feedback.provider.endpoint.openai import OpenAIEndpoint

   # Create an OpenAIEndpoint with the custom OpenAIClient
   custom_endpoint = OpenAIEndpoint(client=openai_client)
   ```

3. **Alternatively, initialize `OpenAIEndpoint` directly with `base_url`:**

   ```python
   custom_endpoint = OpenAIEndpoint(
       api_key=api_key,
       base_url=custom_base_url
   )
   ```

4. **Create an `OpenAI` provider with the custom endpoint:**

   ```python
   from trulens_eval.feedback.provider.openai import OpenAI

   provider = OpenAI(endpoint=custom_endpoint)
   ```

By following these steps, you ensure that the custom `base_url` is correctly propagated through the `OpenAIClient`, `OpenAIEndpoint`, and `OpenAI` provider <sup>[[1]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/endpoint/openai.py)</sup><sup>[[2]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/openai.py)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


The above two methods could not work well, the custom base_url is failed tp propagate through the OpenAIEndpoint

<!-- Answer -->
To ensure that the custom `base_url` is correctly propagated through the `OpenAIClient`, `OpenAIEndpoint`, and `OpenAI` provider, you need to pass the `base_url` parameter through the `client_kwargs` when initializing the `OpenAIClient` and `OpenAIEndpoint`. Here is how you can do it:

1. **Initialize `OpenAIClient` with `base_url`:**

   ```python
   from trulens_eval.feedback.provider.endpoint.openai import OpenAIClient
   from trulens_eval.utils.pyschema import Class
   import openai as oai

   # Define the custom base URL and API key
   custom_base_url = "https://your.custom.base.url"
   api_key = "your-api-key"

   # Define the client class and client kwargs
   client_cls = Class.of_class(oai.OpenAI)
   client_kwargs = {
       "api_key": api_key,
       "base_url": custom_base_url
   }

   # Initialize the OpenAIClient with the custom base URL and API key
   openai_client = OpenAIClient(client_cls=client_cls, client_kwargs=client_kwargs)
   ```

2. **Initialize `OpenAIEndpoint` with `OpenAIClient`:**

   ```python
   from trulens_eval.feedback.provider.endpoint.openai import OpenAIEndpoint

   # Create an OpenAIEndpoint with the custom OpenAIClient
   custom_endpoint = OpenAIEndpoint(client=openai_client)
   ```

3. **Alternatively, initialize `OpenAIEndpoint` directly with `base_url`:**

   ```python
   custom_endpoint = OpenAIEndpoint(
       api_key=api_key,
       base_url=custom_base_url
   )
   ```

4. **Create an `OpenAI` provider with the custom endpoint:**

   ```python
   from trulens_eval.feedback.provider.openai import OpenAI

   provider = OpenAI(endpoint=custom_endpoint)
   ```

By following these steps, you ensure that the custom `base_url` is correctly propagated through the `OpenAIClient`, `OpenAIEndpoint`, and `OpenAI` provider <sup>[[1]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/endpoint/openai.py)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


```python
custom_endpoint = OpenAIEndpoint(openai_client)
``` 
The above statement is no use, also the following
```python
custom_endpoint = OpenAIEndpoint(
    api_key=api_key,
    base_url=custom_base_url
)
``` 
The custom_endpoint.client is also client=<openai.OpenAI object at 0x0000020545D60710> client_cls=openai.OpenAI client_kwargs={'organization': None, 'project': None, 'base_url': URL('https://api.openai.com/v1/'),

Hey - @dosu is a bit off track here. The [OpenAI ](https://www.trulens.org/trulens_eval/api/provider/openai/#trulens_eval.feedback.provider.openai.OpenAI) feedback provider allows you to pass through arbitrary keyword arguments, such as base_url.

```python
from trulens_eval.feedback.provider import OpenAI

provider = OpenAI(base_url=<your url here>)
```

Let me know if this works for you.

<!-- Answer -->
To correctly set and propagate the custom `base_url` through the `OpenAI` provider, you can pass the `base_url` as a keyword argument when initializing the `OpenAI` provider. Here is how you can do it:

```python
from trulens_eval.feedback.provider import OpenAI

# Initialize the OpenAI provider with a custom base_url
provider = OpenAI(base_url="https://your-custom-base-url.com")

# Now you can use the provider with the custom base_url
```

This approach ensures that the `base_url` is correctly passed and used by the `OpenAI` provider <sup>[[1]](https://github.com/truera/trulens/blob/main/trulens_eval/trulens_eval/feedback/provider/openai.py)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*


```python
# Define the feedback function for answer relevance
f_answer_relevance = Feedback(provider.relevance_with_cot_reasons, name="Answer Relevance").on_input_output()

# Test the feedback function
question = "What is Task Decomposition?"
answer = "Task decomposition is the process of breaking down a complex task into smaller, more manageable components."

# Call the feedback function
answer_relevance_result = f_answer_relevance(question, answer)

# Print the result
print(answer_relevance_result)
``` 
After changing the base_url,  I test the code above, but it still casue some error
![image](https://github.com/user-attachments/assets/4a1f63ce-2054-4366-9b97-007577fc17e2)


@SetonLiang by any chance are you defining a `from trulens_eval.feedback.provider.OpenAI` elsewhere before in the code?