Repository: Marker-Inc-Korea/AutoRAG
Issue: [Feature Request] Can you add azure open ai implementation for autoRAG?
Root cause: Documentation Deficiency


I want to add azure open ai credentials for embedding and llm model for performing AutoRAG.

Current embedding only support openai credentials.

and if possible add documentation for Azure Open AI, quick start


@wanjeakshay 
Hello you can use azure openai and embedding with llama_index in AutoRAG.

Here is the documentation how to implement it.
https://docs.auto-rag.com/local_model.html

If you are in trouble to do it, please ask me again (mention please)

Hi @vkehfdl1, thank you for the doc link, I updated my code to include azure AI embedding like this

from llama_index.llms.azure_openai import AzureOpenAI
from llama_index.embeddings.azure_openai import AzureOpenAIEmbedding
import autorag
from autorag import LazyInit

# You need to deploy your own embedding model as well as your own chat completion model
embed_model = AzureOpenAIEmbedding(
    model=my_embed_model,
    deployment_name=deployment_name,
    api_key=api_key,
    azure_endpoint=azure_endpoint,
    api_version=api_version,
)

autorag.embedding_models['azureAI'] = LazyInit(AzureOpenAIEmbedding, model_name="text-embedding-ada-002")

also updated my config file
node_lines:
- node_line_name: retrieve_node_line
  nodes:
    - node_type: retrieval
      strategy:
        metrics: [retrieval_f1, retrieval_recall, retrieval_precision]
      top_k: 3
      modules:
        - module_type: vectordb
          embedding_model: azureAI
        - module_type: bm25
        - module_type: hybrid_rrf
          target_modules: ('bm25', 'vectordb')
          rrf_k: [5, 10, 30]
          
          
while executing following block of code
from langchain_community.callbacks import get_openai_callback

with get_openai_callback() as cb:
  evaluator.start_trial('/content/config.yaml')

print(f"Total Tokens: {cb.total_tokens}")
print(f"Completion Tokens: {cb.completion_tokens}")
print(f"Total Cost (USD): ${cb.total_cost}")

getting error:
WARNING:llama_index.embeddings.openai.utils:Retrying llama_index.embeddings.openai.base.get_embeddings in 0.14871869129193727 seconds as it raised APIConnectionError: Connection error..
WARNING:llama_index.embeddings.openai.utils:Retrying llama_index.embeddings.openai.base.get_embeddings in 0.8552518927105286 seconds as it raised APIConnectionError: Connection error..
WARNING:llama_index.embeddings.openai.utils:Retrying llama_index.embeddings.openai.base.get_embeddings in 2.653576097114074 seconds as it raised APIConnectionError: Connection error..

all my configurations are correct I have cross checked


also how can I add multiple models in it for testing which model is suitable, can we customely give chunk size and all??


@wanjeakshay 
Hello, it looks like you missed the api key and azure_endpoint at the embedding model. 

```python
import os
import autorag

autorag.embedding_models['azureAI'] = LazyInit(AzureOpenAIEmbedding, model_name="text-embedding-ada-002”,
api_key=os.getenv(“AZURE_API_KEY”), azure_endpoint=os.getenv(“AZURE_OPENAI_ENDPOINT"))
```

And you need to set the `AZURE_API_KEY` and `AZURE_OPENAI_ENDPOINT` env variable properly.

Or, it looks like you can set `OPENAI_API_KEY` env variable as api key and `AZURE_OPENAI_ENDPOINT` env varaible as azure endpoint in the llama index. You can try this also.

Since it looks you managed, and works well in the AutoRAG, closing this issue.

> [@wanjeakshay](https://github.com/wanjeakshay) Hello you can use azure openai and embedding with llama_index in AutoRAG.
> 
> Here is the documentation how to implement it. https://docs.auto-rag.com/local_model.html
> 
> If you are in trouble to do it, please ask me again (mention please)

Hi @vkehfdl1 I read the document which folder files I need change the configuration to support Azure OPENAI

Because there is too much python file。。

Thanks

@hellangleZ 
You don’t need to change AutoRAG python file for azure OpenAI
All you need to do is add one line for adding generator_model, and run it as python code, not in cli.
Please read the instructions again and follow it.

> [@hellangleZ](https://github.com/hellangleZ) You don’t need to change AutoRAG python file for azure OpenAI All you need to do is add one line for adding generator_model, and run it as python code, not in cli. Please read the instructions again and follow it.

Hi @vkehfdl1 

This is my configure.yaml

 node_lines:
- node_line_name: retrieve_node_line  # Arbitrary node line name
  nodes:
    - node_type: retrieval
      strategy:
        metrics: [retrieval_f1, retrieval_recall, retrieval_precision]
      top_k: 3
      modules:
        - module_type: vectordb
          vectordb: default
          embedding_model: azureAI1
- node_line_name: post_retrieve_node_line  # Arbitrary node line name
  nodes:
    - node_type: prompt_maker
      strategy:
        metrics: [bleu, meteor, rouge]
      modules:
        - module_type: fstring
          prompt: "Read the passages and answer the given question. \n Question: {query} \n Passage: {retrieved_contents} \n Answer : "
    - node_type: generator
      strategy:
        metrics: [bleu, rouge]
      modules:
        - module_type: llama_index_llm
          llm: azureAI2


This is my code:


import os
from llama_index.llms.azure_openai import AzureOpenAI
from llama_index.embeddings.azure_openai import AzureOpenAIEmbedding
import autorag
from autorag import LazyInit
from autorag.evaluator import Evaluator


os.environ["AZURE_OPENAI_API_KEY"] = "*****************************"
os.environ["AZURE_OPENAI_ENDPOINT"] = "https://a-wxf.openai.azure.com"


autorag.embedding_models['azureAI1'] = LazyInit(
    AzureOpenAIEmbedding,
    model_name="text-embedding-3-small",
    api_key=os.getenv("AZURE_OPENAI_API_KEY"), 
    azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
    deployment_name="ada3",
    api_version="2023-05-15",
)

autorag.generator_models['azureAI2'] = LazyInit(
    AzureOpenAI,
    model_name="gpt-4o",
    api_key=os.getenv("AZURE_OPENAI_API_KEY"), 
    azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
    deployment_name="gpt-4o",
    api_version="2024-08-01-preview",
)


evaluator = Evaluator(qa_data_path='/aml/project_auto_rag/qa_train.parquet', 
                      corpus_data_path='/aml/project_auto_rag/corpus.parquet',
                      project_dir='/aml/project_auto_rag')
evaluator.start_trial('/aml/project_auto_rag/config.yaml')

But it  gets the error:

After check maybe the main issue is                LocalProtocolError: Illegal header value b'Bearer '      

Could you help to check where is the issue, and how to fix?

[10/31/24 14:27:59] INFO     [_base_client.py:1661] >> Retrying request to _base_client.py:1661
                             /embeddings in 0.903491 seconds                                   
[10/31/24 14:28:00] INFO     [_base_client.py:1661] >> Retrying request to _base_client.py:1661
                             /embeddings in 1.889825 seconds                                   
[10/31/24 14:28:02] INFO     [_base_client.py:1661] >> Retrying request to _base_client.py:1661
                             /embeddings in 3.178619 seconds                                   
[10/31/24 14:28:05] INFO     [_base_client.py:1661] >> Retrying request to _base_client.py:1661
                             /embeddings in 6.003244 seconds                                   
[10/31/24 14:28:11] INFO     [_base_client.py:1661] >> Retrying request to _base_client.py:1661
                             /embeddings in 7.638996 seconds                                   
[10/31/24 14:28:19] INFO     [_base_client.py:1661] >> Retrying request to _base_client.py:1661
                             /embeddings in 7.206635 seconds                                   
[10/31/24 14:28:26] INFO     [_base_client.py:1661] >> Retrying request to _base_client.py:1661
                             /embeddings in 7.582277 seconds                                   
[10/31/24 14:28:34] INFO     [_base_client.py:1661] >> Retrying request to _base_client.py:1661
                             /embeddings in 7.394449 seconds                                   
[10/31/24 14:28:41] INFO     [_base_client.py:1661] >> Retrying request to _base_client.py:1661
                             /embeddings in 6.714776 seconds                                   
[10/31/24 14:28:48] ERROR    [__init__.py:60] >> Unexpected exception            __init__.py:60
                             ╭─────── Traceback (most recent call last) ───────╮               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/httpx/_transports/default.py:72 in       │               
                             │ map_httpcore_exceptions                         │               
                             │                                                 │               
                             │    69 @contextlib.contextmanager                │               
                             │    70 def map_httpcore_exceptions() -> typing.I │               
                             │    71 │   try:                                  │               
                             │ ❱  72 │   │   yield                             │               
                             │    73 │   except Exception as exc:              │               
                             │    74 │   │   mapped_exc = None                 │               
                             │    75                                           │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/httpx/_transports/default.py:377 in      │               
                             │ handle_async_request                            │               
                             │                                                 │               
                             │   374 │   │   │   extensions=request.extensions │               
                             │   375 │   │   )                                 │               
                             │   376 │   │   with map_httpcore_exceptions():   │               
                             │ ❱ 377 │   │   │   resp = await self._pool.handl │               
                             │   378 │   │                                     │               
                             │   379 │   │   assert isinstance(resp.stream, ty │               
                             │   380                                           │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/httpcore/_async/connection_pool.py:216   │               
                             │ in handle_async_request                         │               
                             │                                                 │               
                             │   213 │   │   │   │   closing = self._assign_re │               
                             │   214 │   │   │                                 │               
                             │   215 │   │   │   await self._close_connections │               
                             │ ❱ 216 │   │   │   raise exc from None           │               
                             │   217 │   │                                     │               
                             │   218 │   │   # Return the response. Note that  │               
                             │   219 │   │   # the point at which the response │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/httpcore/_async/connection_pool.py:196   │               
                             │ in handle_async_request                         │               
                             │                                                 │               
                             │   193 │   │   │   │                             │               
                             │   194 │   │   │   │   try:                      │               
                             │   195 │   │   │   │   │   # Send the request on │               
                             │ ❱ 196 │   │   │   │   │   response = await conn │               
                             │   197 │   │   │   │   │   │   pool_request.requ │               
                             │   198 │   │   │   │   │   )                     │               
                             │   199 │   │   │   │   except ConnectionNotAvail │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/httpcore/_async/connection.py:101 in     │               
                             │ handle_async_request                            │               
                             │                                                 │               
                             │    98 │   │   │   self._connect_failed = True   │               
                             │    99 │   │   │   raise exc                     │               
                             │   100 │   │                                     │               
                             │ ❱ 101 │   │   return await self._connection.han │               
                             │   102 │                                         │               
                             │   103 │   async def _connect(self, request: Req │               
                             │   104 │   │   timeouts = request.extensions.get │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/httpcore/_async/http11.py:143 in         │               
                             │ handle_async_request                            │               
                             │                                                 │               
                             │   140 │   │   │   with AsyncShieldCancellation( │               
                             │   141 │   │   │   │   async with Trace("respons │               
                             │   142 │   │   │   │   │   await self._response_ │               
                             │ ❱ 143 │   │   │   raise exc                     │               
                             │   144 │                                         │               
                             │   145 │   # Sending the request...              │               
                             │   146                                           │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/httpcore/_async/http11.py:93 in          │               
                             │ handle_async_request                            │               
                             │                                                 │               
                             │    90 │   │   │   │   async with Trace(         │               
                             │    91 │   │   │   │   │   "send_request_headers │               
                             │    92 │   │   │   │   ) as trace:               │               
                             │ ❱  93 │   │   │   │   │   await self._send_requ │               
                             │    94 │   │   │   │   async with Trace("send_re │               
                             │    95 │   │   │   │   │   await self._send_requ │               
                             │    96 │   │   │   except WriteError:            │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/httpcore/_async/http11.py:151 in         │               
                             │ _send_request_headers                           │               
                             │                                                 │               
                             │   148 │   │   timeouts = request.extensions.get │               
                             │   149 │   │   timeout = timeouts.get("write", N │               
                             │   150 │   │                                     │               
                             │ ❱ 151 │   │   with map_exceptions({h11.LocalPro │               
                             │   152 │   │   │   event = h11.Request(          │               
                             │   153 │   │   │   │   method=request.method,    │               
                             │   154 │   │   │   │   target=request.url.target │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/context │               
                             │ lib.py:158 in __exit__                          │               
                             │                                                 │               
                             │   155 │   │   │   │   # tell if we get the same │               
                             │   156 │   │   │   │   value = typ()             │               
                             │   157 │   │   │   try:                          │               
                             │ ❱ 158 │   │   │   │   self.gen.throw(value)     │               
                             │   159 │   │   │   except StopIteration as exc:  │               
                             │   160 │   │   │   │   # Suppress StopIteration  │               
                             │   161 │   │   │   │   # was passed to throw().  │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/httpcore/_exceptions.py:14 in            │               
                             │ map_exceptions                                  │               
                             │                                                 │               
                             │   11 │   except Exception as exc:  # noqa: PIE7 │               
                             │   12 │   │   for from_exc, to_exc in map.items( │               
                             │   13 │   │   │   if isinstance(exc, from_exc):  │               
                             │ ❱ 14 │   │   │   │   raise to_exc(exc) from exc │               
                             │   15 │   │   raise  # pragma: nocover           │               
                             │   16                                            │               
                             │   17                                            │               
                             ╰─────────────────────────────────────────────────╯               
                             LocalProtocolError: Illegal header value b'Bearer '               
                                                                                               
                             The above exception was the direct cause of the                   
                             following exception:                                              
                                                                                               
                             ╭─────── Traceback (most recent call last) ───────╮               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1571 in _request  │               
                             │                                                 │               
                             │   1568 │   │   │   kwargs["auth"] = self.custom │               
                             │   1569 │   │                                    │               
                             │   1570 │   │   try:                             │               
                             │ ❱ 1571 │   │   │   response = await self._clien │               
                             │   1572 │   │   │   │   request,                 │               
                             │   1573 │   │   │   │   stream=stream or self._s │               
                             │   1574 │   │   │   │   **kwargs,                │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/httpx/_client.py:1674 in send            │               
                             │                                                 │               
                             │   1671 │   │                                    │               
                             │   1672 │   │   auth = self._build_request_auth( │               
                             │   1673 │   │                                    │               
                             │ ❱ 1674 │   │   response = await self._send_hand │               
                             │   1675 │   │   │   request,                     │               
                             │   1676 │   │   │   auth=auth,                   │               
                             │   1677 │   │   │   follow_redirects=follow_redi │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/httpx/_client.py:1702 in                 │               
                             │ _send_handling_auth                             │               
                             │                                                 │               
                             │   1699 │   │   │   request = await auth_flow.__ │               
                             │   1700 │   │   │                                │               
                             │   1701 │   │   │   while True:                  │               
                             │ ❱ 1702 │   │   │   │   response = await self._s │               
                             │   1703 │   │   │   │   │   request,             │               
                             │   1704 │   │   │   │   │   follow_redirects=fol │               
                             │   1705 │   │   │   │   │   history=history,     │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/httpx/_client.py:1739 in                 │               
                             │ _send_handling_redirects                        │               
                             │                                                 │               
                             │   1736 │   │   │   for hook in self._event_hook │               
                             │   1737 │   │   │   │   await hook(request)      │               
                             │   1738 │   │   │                                │               
                             │ ❱ 1739 │   │   │   response = await self._send_ │               
                             │   1740 │   │   │   try:                         │               
                             │   1741 │   │   │   │   for hook in self._event_ │               
                             │   1742 │   │   │   │   │   await hook(response) │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/httpx/_client.py:1776 in                 │               
                             │ _send_single_request                            │               
                             │                                                 │               
                             │   1773 │   │   │   )                            │               
                             │   1774 │   │                                    │               
                             │   1775 │   │   with request_context(request=req │               
                             │ ❱ 1776 │   │   │   response = await transport.h │               
                             │   1777 │   │                                    │               
                             │   1778 │   │   assert isinstance(response.strea │               
                             │   1779 │   │   response.request = request       │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/httpx/_transports/default.py:376 in      │               
                             │ handle_async_request                            │               
                             │                                                 │               
                             │   373 │   │   │   content=request.stream,       │               
                             │   374 │   │   │   extensions=request.extensions │               
                             │   375 │   │   )                                 │               
                             │ ❱ 376 │   │   with map_httpcore_exceptions():   │               
                             │   377 │   │   │   resp = await self._pool.handl │               
                             │   378 │   │                                     │               
                             │   379 │   │   assert isinstance(resp.stream, ty │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/context │               
                             │ lib.py:158 in __exit__                          │               
                             │                                                 │               
                             │   155 │   │   │   │   # tell if we get the same │               
                             │   156 │   │   │   │   value = typ()             │               
                             │   157 │   │   │   try:                          │               
                             │ ❱ 158 │   │   │   │   self.gen.throw(value)     │               
                             │   159 │   │   │   except StopIteration as exc:  │               
                             │   160 │   │   │   │   # Suppress StopIteration  │               
                             │   161 │   │   │   │   # was passed to throw().  │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/httpx/_transports/default.py:89 in       │               
                             │ map_httpcore_exceptions                         │               
                             │                                                 │               
                             │    86 │   │   │   raise                         │               
                             │    87 │   │                                     │               
                             │    88 │   │   message = str(exc)                │               
                             │ ❱  89 │   │   raise mapped_exc(message) from ex │               
                             │    90                                           │               
                             │    91                                           │               
                             │    92 HTTPCORE_EXC_MAP = {                      │               
                             ╰─────────────────────────────────────────────────╯               
                             LocalProtocolError: Illegal header value b'Bearer '               
                                                                                               
                             The above exception was the direct cause of the                   
                             following exception:                                              
                                                                                               
                             ╭─────── Traceback (most recent call last) ───────╮               
                             │ /aml/project_auto_rag/test.py:36 in <module>    │               
                             │                                                 │               
                             │   33 evaluator = Evaluator(qa_data_path='/aml/p │               
                             │   34 │   │   │   │   │     corpus_data_path='/a │               
                             │   35 │   │   │   │   │     project_dir='/aml/pr │               
                             │ ❱ 36 evaluator.start_trial('/aml/project_auto_r │               
                             │   37                                            │               
                             │   38                                            │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/autorag/evaluator.py:137 in start_trial  │               
                             │                                                 │               
                             │   134 │   │   │   validator = Validator(        │               
                             │   135 │   │   │   │   qa_data_path=self.qa_data │               
                             │   136 │   │   │   )                             │               
                             │ ❱ 137 │   │   │   validator.validate(yaml_path) │               
                             │   138 │   │                                     │               
                             │   139 │   │   os.environ["PROJECT_DIR"] = self. │               
                             │   140                                           │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/autorag/validator.py:80 in validate      │               
                             │                                                 │               
                             │   77 │   │   │   │   corpus_data_path=corpus_pa │               
                             │   78 │   │   │   │   project_dir=temp_project_d │               
                             │   79 │   │   │   )                              │               
                             │ ❱ 80 │   │   │   evaluator.start_trial(yaml_pat │               
                             │   81 │   │   │   qa_path.close()                │               
                             │   82 │   │   │   corpus_path.close()            │               
                             │   83 │   │   │   os.unlink(qa_path.name)        │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/autorag/evaluator.py:170 in start_trial  │               
                             │                                                 │               
                             │   167 │   │   │   )                             │               
                             │   168 │   │   ):                                │               
                             │   169 │   │   │   loop = get_event_loop()       │               
                             │ ❱ 170 │   │   │   loop.run_until_complete(self. │               
                             │   171 │   │                                     │               
                             │   172 │   │   trial_summary_df = pd.DataFrame(  │               
                             │   173 │   │   │   columns=[                     │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/asyncio │               
                             │ /base_events.py:687 in run_until_complete       │               
                             │                                                 │               
                             │    684 │   │   if not future.done():            │               
                             │    685 │   │   │   raise RuntimeError('Event lo │               
                             │    686 │   │                                    │               
                             │ ❱  687 │   │   return future.result()           │               
                             │    688 │                                        │               
                             │    689 │   def stop(self):                      │               
                             │    690 │   │   """Stop running the event loop.  │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/autorag/evaluator.py:533 in              │               
                             │ __ingest_vectordb                               │               
                             │                                                 │               
                             │   530 │   │   │   # get the target ingest corpu │               
                             │   531 │   │   │   for vectordb in vectordb_list │               
                             │   532 │   │   │   │   target_corpus = await fil │               
                             │ ❱ 533 │   │   │   │   await vectordb_ingest(vec │               
                             │   534 │   │   else:                             │               
                             │   535 │   │   │   # get the target ingest corpu │               
                             │   536 │   │   │   for vectordb in vectordb_list │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/autorag/nodes/retrieval/vectordb.py:256  │               
                             │ in vectordb_ingest                              │               
                             │                                                 │               
                             │   253 │   │   content_batches = make_batch(new_ │               
                             │   254 │   │   id_batches = make_batch(new_ids,  │               
                             │   255 │   │   for content_batch, id_batch in zi │               
                             │ ❱ 256 │   │   │   await vectordb.add(ids=id_bat │               
                             │   257                                           │               
                             │   258                                           │               
                             │   259 def run_query_embedding_batch(            │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/autorag/vectordb/chroma.py:69 in add     │               
                             │                                                 │               
                             │    66 │                                         │               
                             │    67 │   async def add(self, ids: List[str], t │               
                             │    68 │   │   texts = self.truncated_inputs(tex │               
                             │ ❱  69 │   │   text_embeddings = await self.embe │               
                             │    70 │   │   if isinstance(self.collection, As │               
                             │    71 │   │   │   await self.collection.add(ids │               
                             │    72 │   │   else:                             │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/llama_index/core/instrumentation/dispatc │               
                             │ her.py:357 in async_wrapper                     │               
                             │                                                 │               
                             │   354 │   │   │   │   tags=tags,                │               
                             │   355 │   │   │   )                             │               
                             │   356 │   │   │   try:                          │               
                             │ ❱ 357 │   │   │   │   result = await func(*args │               
                             │   358 │   │   │   except BaseException as e:    │               
                             │   359 │   │   │   │   self.event(SpanDropEvent( │               
                             │   360 │   │   │   │   self.span_drop(id_=id_, b │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/llama_index/core/base/embeddings/base.py │               
                             │ :407 in aget_text_embedding_batch               │               
                             │                                                 │               
                             │   404 │   │   │   │   except ImportError:       │               
                             │   405 │   │   │   │   │   nested_embeddings = a │               
                             │   406 │   │   │   else:                         │               
                             │ ❱ 407 │   │   │   │   nested_embeddings = await │               
                             │   408 │   │                                     │               
                             │   409 │   │   result_embeddings = [             │               
                             │   410 │   │   │   embedding for embeddings in n │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/llama_index/embeddings/openai/base.py:44 │               
                             │ 2 in _aget_text_embeddings                      │               
                             │                                                 │               
                             │   439 │   async def _aget_text_embeddings(self, │               
                             │   440 │   │   """Asynchronously get text embedd │               
                             │   441 │   │   aclient = self._get_aclient()     │               
                             │ ❱ 442 │   │   return await aget_embeddings(     │               
                             │   443 │   │   │   aclient,                      │               
                             │   444 │   │   │   texts,                        │               
                             │   445 │   │   │   engine=self._text_engine,     │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/tenacity/asyncio/__init__.py:189 in      │               
                             │ async_wrapped                                   │               
                             │                                                 │               
                             │   186 │   │   │   # calling the same wrapped fu │               
                             │   187 │   │   │   copy = self.copy()            │               
                             │   188 │   │   │   async_wrapped.statistics = co │               
                             │ ❱ 189 │   │   │   return await copy(fn, *args,  │               
                             │   190 │   │                                     │               
                             │   191 │   │   # Preserve attributes             │               
                             │   192 │   │   async_wrapped.retry = self  # typ │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/tenacity/asyncio/__init__.py:111 in      │               
                             │ __call__                                        │               
                             │                                                 │               
                             │   108 │   │                                     │               
                             │   109 │   │   retry_state = RetryCallState(retr │               
                             │   110 │   │   while True:                       │               
                             │ ❱ 111 │   │   │   do = await self.iter(retry_st │               
                             │   112 │   │   │   if isinstance(do, DoAttempt): │               
                             │   113 │   │   │   │   try:                      │               
                             │   114 │   │   │   │   │   result = await fn(*ar │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/tenacity/asyncio/__init__.py:153 in iter │               
                             │                                                 │               
                             │   150 │   │   self._begin_iter(retry_state)     │               
                             │   151 │   │   result = None                     │               
                             │   152 │   │   for action in self.iter_state.act │               
                             │ ❱ 153 │   │   │   result = await action(retry_s │               
                             │   154 │   │   return result                     │               
                             │   155 │                                         │               
                             │   156 │   def __iter__(self) -> t.Generator[Att │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/tenacity/_utils.py:99 in inner           │               
                             │                                                 │               
                             │    96 │   │   return call                       │               
                             │    97 │                                         │               
                             │    98 │   async def inner(*args: typing.Any, ** │               
                             │ ❱  99 │   │   return call(*args, **kwargs)      │               
                             │   100 │                                         │               
                             │   101 │   return inner                          │               
                             │   102                                           │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/tenacity/__init__.py:418 in exc_check    │               
                             │                                                 │               
                             │   415 │   │   │   │   fut = t.cast(Future, rs.o │               
                             │   416 │   │   │   │   retry_exc = self.retry_er │               
                             │   417 │   │   │   │   if self.reraise:          │               
                             │ ❱ 418 │   │   │   │   │   raise retry_exc.rerai │               
                             │   419 │   │   │   │   raise retry_exc from fut. │               
                             │   420 │   │   │                                 │               
                             │   421 │   │   │   self._add_action_func(exc_che │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/tenacity/__init__.py:185 in reraise      │               
                             │                                                 │               
                             │   182 │                                         │               
                             │   183 │   def reraise(self) -> t.NoReturn:      │               
                             │   184 │   │   if self.last_attempt.failed:      │               
                             │ ❱ 185 │   │   │   raise self.last_attempt.resul │               
                             │   186 │   │   raise self                        │               
                             │   187 │                                         │               
                             │   188 │   def __str__(self) -> str:             │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/concurr │               
                             │ ent/futures/_base.py:449 in result              │               
                             │                                                 │               
                             │   446 │   │   │   │   if self._state in [CANCEL │               
                             │   447 │   │   │   │   │   raise CancelledError( │               
                             │   448 │   │   │   │   elif self._state == FINIS │               
                             │ ❱ 449 │   │   │   │   │   return self.__get_res │               
                             │   450 │   │   │   │                             │               
                             │   451 │   │   │   │   self._condition.wait(time │               
                             │   452                                           │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/concurr │               
                             │ ent/futures/_base.py:401 in __get_result        │               
                             │                                                 │               
                             │   398 │   def __get_result(self):               │               
                             │   399 │   │   if self._exception:               │               
                             │   400 │   │   │   try:                          │               
                             │ ❱ 401 │   │   │   │   raise self._exception     │               
                             │   402 │   │   │   finally:                      │               
                             │   403 │   │   │   │   # Break a reference cycle │               
                             │   404 │   │   │   │   self = None               │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/tenacity/asyncio/__init__.py:114 in      │               
                             │ __call__                                        │               
                             │                                                 │               
                             │   111 │   │   │   do = await self.iter(retry_st │               
                             │   112 │   │   │   if isinstance(do, DoAttempt): │               
                             │   113 │   │   │   │   try:                      │               
                             │ ❱ 114 │   │   │   │   │   result = await fn(*ar │               
                             │   115 │   │   │   │   except BaseException:  #  │               
                             │   116 │   │   │   │   │   retry_state.set_excep │               
                             │   117 │   │   │   │   else:                     │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/llama_index/embeddings/openai/base.py:20 │               
                             │ 5 in aget_embeddings                            │               
                             │                                                 │               
                             │   202 │   list_of_text = [text.replace("\n", "  │               
                             │   203 │                                         │               
                             │   204 │   data = (                              │               
                             │ ❱ 205 │   │   await aclient.embeddings.create(i │               
                             │   206 │   ).data                                │               
                             │   207 │   return [d.embedding for d in data]    │               
                             │   208                                           │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/resources/embeddings.py:236 in    │               
                             │ create                                          │               
                             │                                                 │               
                             │   233 │   │   │                                 │               
                             │   234 │   │   │   return obj                    │               
                             │   235 │   │                                     │               
                             │ ❱ 236 │   │   return await self._post(          │               
                             │   237 │   │   │   "/embeddings",                │               
                             │   238 │   │   │   body=maybe_transform(params,  │               
                             │   239 │   │   │   options=make_request_options( │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1838 in post      │               
                             │                                                 │               
                             │   1835 │   │   opts = FinalRequestOptions.const │               
                             │   1836 │   │   │   method="post", url=path, jso │               
                             │        async_to_httpx_files(files), **options   │               
                             │   1837 │   │   )                                │               
                             │ ❱ 1838 │   │   return await self.request(cast_t │               
                             │   1839 │                                        │               
                             │   1840 │   async def patch(                     │               
                             │   1841 │   │   self,                            │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1532 in request   │               
                             │                                                 │               
                             │   1529 │   │   else:                            │               
                             │   1530 │   │   │   retries_taken = 0            │               
                             │   1531 │   │                                    │               
                             │ ❱ 1532 │   │   return await self._request(      │               
                             │   1533 │   │   │   cast_to=cast_to,             │               
                             │   1534 │   │   │   options=options,             │               
                             │   1535 │   │   │   stream=stream,               │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1595 in _request  │               
                             │                                                 │               
                             │   1592 │   │   │   log.debug("Encountered Excep │               
                             │   1593 │   │   │                                │               
                             │   1594 │   │   │   if remaining_retries > 0:    │               
                             │ ❱ 1595 │   │   │   │   return await self._retry │               
                             │   1596 │   │   │   │   │   input_options,       │               
                             │   1597 │   │   │   │   │   cast_to,             │               
                             │   1598 │   │   │   │   │   retries_taken=retrie │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1665 in           │               
                             │ _retry_request                                  │               
                             │                                                 │               
                             │   1662 │   │                                    │               
                             │   1663 │   │   await anyio.sleep(timeout)       │               
                             │   1664 │   │                                    │               
                             │ ❱ 1665 │   │   return await self._request(      │               
                             │   1666 │   │   │   options=options,             │               
                             │   1667 │   │   │   cast_to=cast_to,             │               
                             │   1668 │   │   │   retries_taken=retries_taken  │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1595 in _request  │               
                             │                                                 │               
                             │   1592 │   │   │   log.debug("Encountered Excep │               
                             │   1593 │   │   │                                │               
                             │   1594 │   │   │   if remaining_retries > 0:    │               
                             │ ❱ 1595 │   │   │   │   return await self._retry │               
                             │   1596 │   │   │   │   │   input_options,       │               
                             │   1597 │   │   │   │   │   cast_to,             │               
                             │   1598 │   │   │   │   │   retries_taken=retrie │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1665 in           │               
                             │ _retry_request                                  │               
                             │                                                 │               
                             │   1662 │   │                                    │               
                             │   1663 │   │   await anyio.sleep(timeout)       │               
                             │   1664 │   │                                    │               
                             │ ❱ 1665 │   │   return await self._request(      │               
                             │   1666 │   │   │   options=options,             │               
                             │   1667 │   │   │   cast_to=cast_to,             │               
                             │   1668 │   │   │   retries_taken=retries_taken  │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1595 in _request  │               
                             │                                                 │               
                             │   1592 │   │   │   log.debug("Encountered Excep │               
                             │   1593 │   │   │                                │               
                             │   1594 │   │   │   if remaining_retries > 0:    │               
                             │ ❱ 1595 │   │   │   │   return await self._retry │               
                             │   1596 │   │   │   │   │   input_options,       │               
                             │   1597 │   │   │   │   │   cast_to,             │               
                             │   1598 │   │   │   │   │   retries_taken=retrie │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1665 in           │               
                             │ _retry_request                                  │               
                             │                                                 │               
                             │   1662 │   │                                    │               
                             │   1663 │   │   await anyio.sleep(timeout)       │               
                             │   1664 │   │                                    │               
                             │ ❱ 1665 │   │   return await self._request(      │               
                             │   1666 │   │   │   options=options,             │               
                             │   1667 │   │   │   cast_to=cast_to,             │               
                             │   1668 │   │   │   retries_taken=retries_taken  │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1595 in _request  │               
                             │                                                 │               
                             │   1592 │   │   │   log.debug("Encountered Excep │               
                             │   1593 │   │   │                                │               
                             │   1594 │   │   │   if remaining_retries > 0:    │               
                             │ ❱ 1595 │   │   │   │   return await self._retry │               
                             │   1596 │   │   │   │   │   input_options,       │               
                             │   1597 │   │   │   │   │   cast_to,             │               
                             │   1598 │   │   │   │   │   retries_taken=retrie │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1665 in           │               
                             │ _retry_request                                  │               
                             │                                                 │               
                             │   1662 │   │                                    │               
                             │   1663 │   │   await anyio.sleep(timeout)       │               
                             │   1664 │   │                                    │               
                             │ ❱ 1665 │   │   return await self._request(      │               
                             │   1666 │   │   │   options=options,             │               
                             │   1667 │   │   │   cast_to=cast_to,             │               
                             │   1668 │   │   │   retries_taken=retries_taken  │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1595 in _request  │               
                             │                                                 │               
                             │   1592 │   │   │   log.debug("Encountered Excep │               
                             │   1593 │   │   │                                │               
                             │   1594 │   │   │   if remaining_retries > 0:    │               
                             │ ❱ 1595 │   │   │   │   return await self._retry │               
                             │   1596 │   │   │   │   │   input_options,       │               
                             │   1597 │   │   │   │   │   cast_to,             │               
                             │   1598 │   │   │   │   │   retries_taken=retrie │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1665 in           │               
                             │ _retry_request                                  │               
                             │                                                 │               
                             │   1662 │   │                                    │               
                             │   1663 │   │   await anyio.sleep(timeout)       │               
                             │   1664 │   │                                    │               
                             │ ❱ 1665 │   │   return await self._request(      │               
                             │   1666 │   │   │   options=options,             │               
                             │   1667 │   │   │   cast_to=cast_to,             │               
                             │   1668 │   │   │   retries_taken=retries_taken  │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1595 in _request  │               
                             │                                                 │               
                             │   1592 │   │   │   log.debug("Encountered Excep │               
                             │   1593 │   │   │                                │               
                             │   1594 │   │   │   if remaining_retries > 0:    │               
                             │ ❱ 1595 │   │   │   │   return await self._retry │               
                             │   1596 │   │   │   │   │   input_options,       │               
                             │   1597 │   │   │   │   │   cast_to,             │               
                             │   1598 │   │   │   │   │   retries_taken=retrie │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1665 in           │               
                             │ _retry_request                                  │               
                             │                                                 │               
                             │   1662 │   │                                    │               
                             │   1663 │   │   await anyio.sleep(timeout)       │               
                             │   1664 │   │                                    │               
                             │ ❱ 1665 │   │   return await self._request(      │               
                             │   1666 │   │   │   options=options,             │               
                             │   1667 │   │   │   cast_to=cast_to,             │               
                             │   1668 │   │   │   retries_taken=retries_taken  │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1595 in _request  │               
                             │                                                 │               
                             │   1592 │   │   │   log.debug("Encountered Excep │               
                             │   1593 │   │   │                                │               
                             │   1594 │   │   │   if remaining_retries > 0:    │               
                             │ ❱ 1595 │   │   │   │   return await self._retry │               
                             │   1596 │   │   │   │   │   input_options,       │               
                             │   1597 │   │   │   │   │   cast_to,             │               
                             │   1598 │   │   │   │   │   retries_taken=retrie │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1665 in           │               
                             │ _retry_request                                  │               
                             │                                                 │               
                             │   1662 │   │                                    │               
                             │   1663 │   │   await anyio.sleep(timeout)       │               
                             │   1664 │   │                                    │               
                             │ ❱ 1665 │   │   return await self._request(      │               
                             │   1666 │   │   │   options=options,             │               
                             │   1667 │   │   │   cast_to=cast_to,             │               
                             │   1668 │   │   │   retries_taken=retries_taken  │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1595 in _request  │               
                             │                                                 │               
                             │   1592 │   │   │   log.debug("Encountered Excep │               
                             │   1593 │   │   │                                │               
                             │   1594 │   │   │   if remaining_retries > 0:    │               
                             │ ❱ 1595 │   │   │   │   return await self._retry │               
                             │   1596 │   │   │   │   │   input_options,       │               
                             │   1597 │   │   │   │   │   cast_to,             │               
                             │   1598 │   │   │   │   │   retries_taken=retrie │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1665 in           │               
                             │ _retry_request                                  │               
                             │                                                 │               
                             │   1662 │   │                                    │               
                             │   1663 │   │   await anyio.sleep(timeout)       │               
                             │   1664 │   │                                    │               
                             │ ❱ 1665 │   │   return await self._request(      │               
                             │   1666 │   │   │   options=options,             │               
                             │   1667 │   │   │   cast_to=cast_to,             │               
                             │   1668 │   │   │   retries_taken=retries_taken  │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1595 in _request  │               
                             │                                                 │               
                             │   1592 │   │   │   log.debug("Encountered Excep │               
                             │   1593 │   │   │                                │               
                             │   1594 │   │   │   if remaining_retries > 0:    │               
                             │ ❱ 1595 │   │   │   │   return await self._retry │               
                             │   1596 │   │   │   │   │   input_options,       │               
                             │   1597 │   │   │   │   │   cast_to,             │               
                             │   1598 │   │   │   │   │   retries_taken=retrie │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1665 in           │               
                             │ _retry_request                                  │               
                             │                                                 │               
                             │   1662 │   │                                    │               
                             │   1663 │   │   await anyio.sleep(timeout)       │               
                             │   1664 │   │                                    │               
                             │ ❱ 1665 │   │   return await self._request(      │               
                             │   1666 │   │   │   options=options,             │               
                             │   1667 │   │   │   cast_to=cast_to,             │               
                             │   1668 │   │   │   retries_taken=retries_taken  │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1595 in _request  │               
                             │                                                 │               
                             │   1592 │   │   │   log.debug("Encountered Excep │               
                             │   1593 │   │   │                                │               
                             │   1594 │   │   │   if remaining_retries > 0:    │               
                             │ ❱ 1595 │   │   │   │   return await self._retry │               
                             │   1596 │   │   │   │   │   input_options,       │               
                             │   1597 │   │   │   │   │   cast_to,             │               
                             │   1598 │   │   │   │   │   retries_taken=retrie │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1665 in           │               
                             │ _retry_request                                  │               
                             │                                                 │               
                             │   1662 │   │                                    │               
                             │   1663 │   │   await anyio.sleep(timeout)       │               
                             │   1664 │   │                                    │               
                             │ ❱ 1665 │   │   return await self._request(      │               
                             │   1666 │   │   │   options=options,             │               
                             │   1667 │   │   │   cast_to=cast_to,             │               
                             │   1668 │   │   │   retries_taken=retries_taken  │               
                             │                                                 │               
                             │ /home/root123/miniconda3/lib/python3.12/site-pa │               
                             │ ckages/openai/_base_client.py:1605 in _request  │               
                             │                                                 │               
                             │   1602 │   │   │   │   )                        │               
                             │   1603 │   │   │                                │               
                             │   1604 │   │   │   log.debug("Raising connectio │               
                             │ ❱ 1605 │   │   │   raise APIConnectionError(req │               
                             │   1606 │   │                                    │               
                             │   1607 │   │   log.debug(                       │               
                             │   1608 │   │   │   'HTTP Request: %s %s "%i %s" │               
                             │        response.status_code, response.reason_ph │               
                             ╰─────────────────────────────────────────────────╯               
                             APIConnectionError: Connection error.



@hellangleZ Can you check your api key or endpoint? It looks like the api key has some error.

> [@hellangleZ](https://github.com/hellangleZ) Can you check your api key or endpoint? It looks like the api key has some error.

Hi  @vkehfdl1  I check my api key is could work, is there something wrong in my code ?

Could you please help to check?

Thanks

This is my configure.yaml

node_lines:

node_line_name: retrieve_node_line # Arbitrary node line name
nodes:
node_type: retrieval
strategy:
metrics: [retrieval_f1, retrieval_recall, retrieval_precision]
top_k: 3
modules:
module_type: vectordb
vectordb: default
embedding_model: azureAI1
node_line_name: post_retrieve_node_line # Arbitrary node line name
nodes:
node_type: prompt_maker
strategy:
metrics: [bleu, meteor, rouge]
modules:
module_type: fstring
prompt: "Read the passages and answer the given question. \n Question: {query} \n Passage: {retrieved_contents} \n Answer : "
node_type: generator
strategy:
metrics: [bleu, rouge]
modules:
module_type: llama_index_llm
llm: azureAI2
This is my code:

import os
from llama_index.llms.azure_openai import AzureOpenAI
from llama_index.embeddings.azure_openai import AzureOpenAIEmbedding
import autorag
from autorag import LazyInit
from autorag.evaluator import Evaluator

os.environ["AZURE_OPENAI_API_KEY"] = "*****************************"
os.environ["AZURE_OPENAI_ENDPOINT"] = "https://a-wxf.openai.azure.com/"

autorag.embedding_models['azureAI1'] = LazyInit(
AzureOpenAIEmbedding,
model_name="text-embedding-3-small",
api_key=os.getenv("AZURE_OPENAI_API_KEY"),
azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
deployment_name="ada3",
api_version="2023-05-15",
)

autorag.generator_models['azureAI2'] = LazyInit(
AzureOpenAI,
model_name="gpt-4o",
api_key=os.getenv("AZURE_OPENAI_API_KEY"),
azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
deployment_name="gpt-4o",
api_version="2024-08-01-preview",
)

evaluator = Evaluator(qa_data_path='/aml/project_auto_rag/qa_train.parquet',
corpus_data_path='/aml/project_auto_rag/corpus.parquet',
project_dir='/aml/project_auto_rag')
evaluator.start_trial('/aml/project_auto_rag/config.yaml')



Hi  @wanjeakshay  Is you resolve the problem or not, if you do , could you please share your code ?

Thanks

![Image](https://github.com/user-attachments/assets/0d401045-4345-4ab5-8962-d1d996c98a32)
 I  confirm it is not a api key issue, after I check with llama_index common rag， it works good by the api

same question