Repository: explodinggradients/ragas
Issue: Exception raised in Job APIConnectionError(Connection error.)
Root cause: Architectural Constraint

Hello, I use local data, including reference and response data, use Azure, and then use ragas to obtain accuracy indicators, but errors are reported:
`
Evaluating:   0%|          | 1/301 [00:04<21:46,  4.36s/it]Exception raised in Job[7]: APIConnectionError(Connection error.)
`
 I checked the ragas and here and couldn't find an answer to my question.

**Your Question**
2 questions,
One is, I want to know what this error is and how to solve it. In the following source code, I have tried to set all the places where llm can be set to my configuration, but the problem is still reported, and there is no other prompt. (The code is changed from the example of the official website.)
The other is that if you want to get the value of Semantic Similarity, you should use answer_similarity or Semantic Similarity. I think both of them work, but I don't know which one is right and the difference between them.

**Code Examples**
`
os.environ['AZURE_OPENAI_API_KEY'] = 'xxxxxxx'.
os.environ['HTTP_PROXY'] = 'http://xxxxx'
os.environ['HTTPS_PROXY'] = 'http://xxxxx'

azure_config = {
    "base_url": "https://yaotalai.openai.azure.com/",  # your endpoint
    "model_deployment": "gpt-35-turbo-16k",  # your model deployment name
    "model_name": "gpt35",  # your model name
    "embedding_deployment": "text-embedding-ada-002",  # your embedding deployment name
    "embedding_name": "embedding",  # your embedding name
}

evaluator_llm = LangchainLLMWrapper(AzureChatOpenAI(
    openai_api_version="2023-05-15",
    azure_endpoint=azure_config["base_url"],
    azure_deployment=azure_config["model_deployment"],
    model=azure_config["model_name"],
    validate_base_url=False,
))

# init the embeddings for answer_relevancy, answer_correctness and answer_similarity
evaluator_embeddings = LangchainEmbeddingsWrapper(AzureOpenAIEmbeddings(
    openai_api_version="2023-05-15",
    azure_endpoint=azure_config["base_url"],
    azure_deployment=azure_config["embedding_deployment"],
    model=azure_config["embedding_name"],
))

file_path = 'ts.xlsx'
data = pd.read_excel(file_path)
selected_columns = data[['reference', 'response']]
dataset = Dataset.from_pandas(selected_columns)

# Define the metrics you want to evaluate
# answer_similarity.llm = evaluator_llm
# answer_similarity.embeddings = evaluator_embeddings
metrics = [
    # answer_similarity,
    FactualCorrectness(llm=evaluator_llm),
    SemanticSimilarity(llm=evaluator_llm, embeddings=evaluator_embeddings),
]

# Evaluate the dataset using the selected metrics
results = evaluate(
    dataset = dataset,
    llm = evaluator_llm,
    embeddings = evaluator_embeddings,
    metrics = metrics)
print(results)
`


Sorry, the above code format is incorrect, the correct source code is here.
```
os.environ['AZURE_OPENAI_API_KEY'] = 'xxxxxxx'.
os.environ['HTTP_PROXY'] = 'http://xxxxx/'
os.environ['HTTPS_PROXY'] = 'http://xxxxx/'

azure_config = {
"base_url": "https://yaotalai.openai.azure.com/", # your endpoint
"model_deployment": "gpt-35-turbo-16k", # your model deployment name
"model_name": "gpt35", # your model name
"embedding_deployment": "text-embedding-ada-002", # your embedding deployment name
"embedding_name": "embedding", # your embedding name
}

evaluator_llm = LangchainLLMWrapper(AzureChatOpenAI(
openai_api_version="2023-05-15",
azure_endpoint=azure_config["base_url"],
azure_deployment=azure_config["model_deployment"],
model=azure_config["model_name"],
validate_base_url=False,
))


evaluator_embeddings = LangchainEmbeddingsWrapper(AzureOpenAIEmbeddings(
openai_api_version="2023-05-15",
azure_endpoint=azure_config["base_url"],
azure_deployment=azure_config["embedding_deployment"],
model=azure_config["embedding_name"],
))

file_path = 'ts.xlsx'
data = pd.read_excel(file_path)
selected_columns = data[['reference', 'response']]
dataset = Dataset.from_pandas(selected_columns)


answer_similarity.llm = evaluator_llm
answer_similarity.embeddings = evaluator_embeddings
metrics = [
answer_similarity,
FactualCorrectness(llm=evaluator_llm),
SemanticSimilarity(llm=evaluator_llm, embeddings=evaluator_embeddings),
]


results = evaluate(
dataset = dataset,
llm = evaluator_llm,
embeddings = evaluator_embeddings,
metrics = metrics)
print(results)
```

Hi everybody,

I believe I am facing the same issue and I have done a lot of digging to discover where things are going wrong.

# My situation

- I am sitting behind a corporate proxy with ssl certificate authentication
- By creating my Langchain AzureChatOpenAI models with an http_proxy (httpx.Client) everything works fine on synchronous requests, using directly this model
- When using the ragas library, after configuring an LLM based metric with my Langchain wrapped model, I get the connection error mentioned by OP

# My research results

I believe (one of) the culprit(s) to be the function `single_turn_score` in SingleTurnMetric, as this is a synchronous function that relies on the implementation of the asynchronous function `_single_turn_ascore`:

![Image](https://github.com/user-attachments/assets/15a8f1d1-99a7-4b92-83a5-b8d42f1eb0fa)

![Image](https://github.com/user-attachments/assets/9b077af4-caec-44b7-819f-500e07642aee)

From there, a massive call hierarchy of asynchronous functions opens up, eventually issueing asynchronous requests to the Azure OpenAI client. These requests are not initialized with my proxy settings and thus fail.

Attempting to pass the proxy settings via environment variables failed as well, because by default, asyncio sessions are initialized with `trust_env = False` (https://docs.aiohttp.org/en/stable/client_reference.html#aiohttp.ClientSession) and as they are deeply nested in the ragas code, changing this initialization is neither feasible not elegant.

Unfortunately, there are no inheritance-based quick fixes possible either, because the call hierarchy is so deeply nested and for many of the asynchronous functions there are no synchronous counterparts that one could rely on, when overwriting the function `single_turn_score` in a custom subclass. One would have to inherit and overwrite so many functions to remain fully synchronous that this is not feasible.

I also tried to initialize a global asyncio session, just to have control over proxy and ssl settings, I even tried monkey patching the ClientSession class from aiohttp, but no luck.

I know this is a niche case and it's understandable that one asynchronous implementation might be reused by synchronizing the last step on the call stack, thus, saving on implementation time and complexity, but this makes it not viable to use ragas for such corporate and secured situations.

That being said, if I (or OP) missed something, and it is in fact possible to configure an http proxy with ssl certificate so that the llms wrapped in `LangchainLLMWrapper` have access to it on asynchronous requests, please let us know!