Repository: Marker-Inc-Korea/AutoRAG
Issue: Can I use certain tags from the ollama model in the generator node? (generator node에서 ollama 모델의 특정 태그를 사용 할 수 있나요?)
Root cause: Configuration Error

I want to use a specific ```model:tag``` provided by ollama.

Ref
- https://ollama.com/library/qwen2:72b

However, if i use the ollama specific tag as shown below, i got an error.
```
      modules:
        - module_type: llama_index_llm
          llm: ollama
          model: [llama3, qwen2:72b]
          temperature: 0.7
          batch: 1
```

<details>
<summary>error message</summary>

```
[08/22/24 12:35:30] ERROR    [__init__.py:73] >> Unexpected exception                                                         __init__.py:73
                             ╭───────────────────────────── Traceback (most recent call last) 

.... 
                             ReadTimeout                                                                                                    
                                                                                                                                            
                             The above exception was the direct cause of the following exception:                                           
                                                                                                                                            
                             ╭───────────────────────────── Traceback (most recent call last) ──────────────────────────────╮               
                             │ /root/workspace/corporate-llm/autorag/main.py:27 in <module>                                 │               
                             │                                                                                              │               
                             │   24                                                                                         │               
                             │   25                                                                                         │               
                             │   26 if __name__ == '__main__':                                                              │               
                             │ ❱ 27 │   main()                                                                              │               
                             │   28                                                                                         │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/click/core.py:1157 in __call__         │               
                             │                                                                                              │               
                             │   1154 │                                                                                     │               
                             │   1155 │   def __call__(self, *args: t.Any, **kwargs: t.Any) -> t.Any:                       │               
                             │   1156 │   │   """Alias for :meth:`main`."""                                                 │               
                             │ ❱ 1157 │   │   return self.main(*args, **kwargs)                                             │               
                             │   1158                                                                                       │               
                             │   1159                                                                                       │               
                             │   1160 class Command(BaseCommand):                                                           │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/click/core.py:1078 in main             │               
                             │                                                                                              │               
                             │   1075 │   │   try:                                                                          │               
                             │   1076 │   │   │   try:                                                                      │               
                             │   1077 │   │   │   │   with self.make_context(prog_name, args, **extra) as ctx:              │               
                             │ ❱ 1078 │   │   │   │   │   rv = self.invoke(ctx)                                             │               
                             │   1079 │   │   │   │   │   if not standalone_mode:                                           │               
                             │   1080 │   │   │   │   │   │   return rv                                                     │               
                             │   1081 │   │   │   │   │   # it's not safe to `ctx.exit(rv)` here!                           │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/click/core.py:1434 in invoke           │               
                             │                                                                                              │               
                             │   1431 │   │   │   echo(style(message, fg="red"), err=True)                                  │               
                             │   1432 │   │                                                                                 │               
                             │   1433 │   │   if self.callback is not None:                                                 │               
                             │ ❱ 1434 │   │   │   return ctx.invoke(self.callback, **ctx.params)                            │               
                             │   1435 │                                                                                     │               
                             │   1436 │   def shell_complete(self, ctx: Context, incomplete: str) -> t.List["CompletionItem │               
                             │   1437 │   │   """Return a list of completions for the incomplete value. Looks               │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/click/core.py:783 in invoke            │               
                             │                                                                                              │               
                             │    780 │   │                                                                                 │               
                             │    781 │   │   with augment_usage_errors(__self):                                            │               
                             │    782 │   │   │   with ctx:                                                                 │               
                             │ ❱  783 │   │   │   │   return __callback(*args, **kwargs)                                    │               
                             │    784 │                                                                                     │               
                             │    785 │   def forward(                                                                      │               
                             │    786 │   │   __self, __cmd: "Command", *args: t.Any, **kwargs: t.Any  # noqa: B902         │               
                             │                                                                                              │               
                             │ /root/workspace/corporate-llm/autorag/main.py:23 in main                                     │               
                             │                                                                                              │               
                             │   20 │   if not os.path.exists(project_dir):                                                 │               
                             │   21 │   │   os.makedirs(project_dir)                                                        │               
                             │   22 │   evaluator = Evaluator(qa_data_path, corpus_data_path, project_dir=project_dir)      │               
                             │ ❱ 23 │   evaluator.start_trial(config)                                                       │               
                             │   24                                                                                         │               
                             │   25                                                                                         │               
                             │   26 if __name__ == '__main__':                                                              │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/autorag/evaluator.py:98 in start_trial │               
                             │                                                                                              │               
                             │    95 │   │   │   if i == 0:                                                                 │               
                             │    96 │   │   │   │   previous_result = self.qa_data                                         │               
                             │    97 │   │   │   logger.info(f'Running node line {node_line_name}...')                      │               
                             │ ❱  98 │   │   │   previous_result = run_node_line(node_line, node_line_dir, previous_result) │               
                             │    99 │   │   │                                                                              │               
                             │   100 │   │   │   trial_summary_df = self._append_node_line_summary(node_line_name,          │               
                             │       node_line_dir, trial_summary_df)                                                       │               
                             │   101                                                                                        │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/autorag/node_line.py:45 in             │               
                             │ run_node_line                                                                                │               
                             │                                                                                              │               
                             │   42 │                                                                                       │               
                             │   43 │   summary_lst = []                                                                    │               
                             │   44 │   for node in nodes:                                                                  │               
                             │ ❱ 45 │   │   previous_result = node.run(previous_result, node_line_dir)                      │               
                             │   46 │   │   node_summary_df = load_summary_file(os.path.join(node_line_dir, node.node_type, │               
                             │      'summary.csv'))                                                                         │               
                             │   47 │   │   best_node_row = node_summary_df.loc[node_summary_df['is_best']]                 │               
                             │   48 │   │   summary_lst.append({                                                            │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/autorag/schema/node.py:57 in run       │               
                             │                                                                                              │               
                             │    54 │   def run(self, previous_result: pd.DataFrame, node_line_dir: str) -> pd.DataFrame:  │               
                             │    55 │   │   logger.info(f'Running node {self.node_type}...')                               │               
                             │    56 │   │   input_modules, input_params = self.get_param_combinations()                    │               
                             │ ❱  57 │   │   return self.run_node(modules=input_modules,                                    │               
                             │    58 │   │   │   │   │   │   │    module_params=input_params,                               │               
                             │    59 │   │   │   │   │   │   │    previous_result=previous_result,                          │               
                             │    60 │   │   │   │   │   │   │    node_line_dir=node_line_dir,                              │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/autorag/nodes/generator/run.py:43 in   │               
                             │ run_generator_node                                                                           │               
                             │                                                                                              │               
                             │   40 │   │   raise ValueError("You must have 'generation_gt' column in qa.parquet.")         │               
                             │   41 │   generation_gt = list(map(lambda x: x.tolist(), qa_data['generation_gt'].tolist()))  │               
                             │   42 │                                                                                       │               
                             │ ❱ 43 │   results, execution_times = zip(*map(lambda x: measure_speed(                        │               
                             │   44 │   │   x[0], project_dir=project_dir, previous_result=previous_result, **x[1]),        │               
                             │   45 │   │   │   │   │   │   │   │   │   │   zip(modules, module_params)))                   │               
                             │   46 │   average_times = list(map(lambda x: x / len(results[0]), execution_times))           │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/autorag/nodes/generator/run.py:43 in   │               
                             │ <lambda>                                                                                     │               
                             │                                                                                              │               
                             │   40 │   │   raise ValueError("You must have 'generation_gt' column in qa.parquet.")         │               
                             │   41 │   generation_gt = list(map(lambda x: x.tolist(), qa_data['generation_gt'].tolist()))  │               
                             │   42 │                                                                                       │               
                             │ ❱ 43 │   results, execution_times = zip(*map(lambda x: measure_speed(                        │               
                             │   44 │   │   x[0], project_dir=project_dir, previous_result=previous_result, **x[1]),        │               
                             │   45 │   │   │   │   │   │   │   │   │   │   zip(modules, module_params)))                   │               
                             │   46 │   average_times = list(map(lambda x: x / len(results[0]), execution_times))           │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/autorag/strategy.py:14 in              │               
                             │ measure_speed                                                                                │               
                             │                                                                                              │               
                             │    11 │   Method for measuring execution speed of the function.                              │               
                             │    12 │   """                                                                                │               
                             │    13 │   start_time = time.time()                                                           │               
                             │ ❱  14 │   result = func(*args, **kwargs)                                                     │               
                             │    15 │   end_time = time.time()                                                             │               
                             │    16 │   return result, end_time - start_time                                               │               
                             │    17                                                                                        │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/autorag/utils/util.py:56 in wrapper    │               
                             │                                                                                              │               
                             │    53 │   def decorator_result_to_dataframe(func: Callable):                                 │               
                             │    54 │   │   @functools.wraps(func)                                                         │               
                             │    55 │   │   def wrapper(*args, **kwargs) -> pd.DataFrame:                                  │               
                             │ ❱  56 │   │   │   results = func(*args, **kwargs)                                            │               
                             │    57 │   │   │   if len(column_names) == 1:                                                 │               
                             │    58 │   │   │   │   df_input = {column_names[0]: results}                                  │               
                             │    59 │   │   │   else:                                                                      │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/autorag/nodes/generator/base.py:43 in  │               
                             │ wrapper                                                                                      │               
                             │                                                                                              │               
                             │   40 │   │   │   │   │   │   │   │    "You can check valid llm names from                    │               
                             │      autorag.generator_models.")                                                             │               
                             │   41 │   │   │   batch = kwargs.pop('batch', 16)                                             │               
                             │   42 │   │   │   llm_instance = generator_models[llm](**kwargs)                              │               
                             │ ❱ 43 │   │   │   result = func(prompts=prompts, llm=llm_instance, batch=batch)               │               
                             │   44 │   │   │   del llm_instance                                                            │               
                             │   45 │   │   │   return result                                                               │               
                             │   46 │   │   else:                                                                           │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/autorag/nodes/generator/llama_index_ll │               
                             │ m.py:30 in llama_index_llm                                                                   │               
                             │                                                                                              │               
                             │   27 │   """                                                                                 │               
                             │   28 │   tasks = [llm.acomplete(prompt) for prompt in prompts]                               │               
                             │   29 │   loop = asyncio.get_event_loop()                                                     │               
                             │ ❱ 30 │   results = loop.run_until_complete(process_batch(tasks, batch_size=batch))           │               
                             │   31 │                                                                                       │               
                             │   32 │   generated_texts = list(map(lambda x: x.text, results))                              │               
                             │   33 │   tokenizer = AutoTokenizer.from_pretrained("gpt2", use_fast=False)                   │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/asyncio/base_events.py:649 in run_until_complete     │               
                             │                                                                                              │               
                             │    646 │   │   if not future.done():                                                         │               
                             │    647 │   │   │   raise RuntimeError('Event loop stopped before Future completed.')         │               
                             │    648 │   │                                                                                 │               
                             │ ❱  649 │   │   return future.result()                                                        │               
                             │    650 │                                                                                     │               
                             │    651 │   def stop(self):                                                                   │               
                             │    652 │   │   """Stop running the event loop.                                               │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/autorag/utils/util.py:273 in           │               
                             │ process_batch                                                                                │               
                             │                                                                                              │               
                             │   270 │                                                                                      │               
                             │   271 │   for i in range(0, len(tasks), batch_size):                                         │               
                             │   272 │   │   batch = tasks[i:i + batch_size]                                                │               
                             │ ❱ 273 │   │   batch_results = await asyncio.gather(*batch)                                   │               
                             │   274 │   │   results.extend(batch_results)                                                  │               
                             │   275 │                                                                                      │               
                             │   276 │   return results                                                                     │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/llama_index/core/instrumentation/dispa │               
                             │ tcher.py:290 in async_wrapper                                                                │               
                             │                                                                                              │               
                             │   287 │   │   │   │   tags=tags,                                                             │               
                             │   288 │   │   │   )                                                                          │               
                             │   289 │   │   │   try:                                                                       │               
                             │ ❱ 290 │   │   │   │   result = await func(*args, **kwargs)                                   │               
                             │   291 │   │   │   except BaseException as e:                                                 │               
                             │   292 │   │   │   │   self.event(SpanDropEvent(span_id=id_, err_str=str(e)))                 │               
                             │   293 │   │   │   │   self.span_drop(id_=id_, bound_args=bound_args, instance=instance, err= │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/llama_index/core/llms/callbacks.py:334 │               
                             │ in wrapped_async_llm_predict                                                                 │               
                             │                                                                                              │               
                             │   331 │   │   │   │   )                                                                      │               
                             │   332 │   │   │   │                                                                          │               
                             │   333 │   │   │   │   try:                                                                   │               
                             │ ❱ 334 │   │   │   │   │   f_return_val = await f(_self, *args, **kwargs)                     │               
                             │   335 │   │   │   │   except BaseException as e:                                             │               
                             │   336 │   │   │   │   │   callback_manager.on_event_end(                                     │               
                             │   337 │   │   │   │   │   │   CBEventType.LLM,                                               │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/llama_index/llms/ollama/base.py:399 in │               
                             │ acomplete                                                                                    │               
                             │                                                                                              │               
                             │   396 │   async def acomplete(                                                               │               
                             │   397 │   │   self, prompt: str, formatted: bool = False, **kwargs: Any                      │               
                             │   398 │   ) -> CompletionResponse:                                                           │               
                             │ ❱ 399 │   │   return await achat_to_completion_decorator(self.achat)(prompt, **kwargs)       │               
                             │   400 │                                                                                      │               
                             │   401 │   @llm_completion_callback()                                                         │               
                             │   402 │   def stream_complete(                                                               │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/llama_index/core/base/llms/generic_uti │               
                             │ ls.py:221 in wrapper                                                                         │               
                             │                                                                                              │               
                             │   218 │   async def wrapper(prompt: str, **kwargs: Any) -> CompletionResponse:               │               
                             │   219 │   │   # normalize input                                                              │               
                             │   220 │   │   messages = prompt_to_messages(prompt)                                          │               
                             │ ❱ 221 │   │   chat_response = await func(messages, **kwargs)                                 │               
                             │   222 │   │   # normalize output                                                             │               
                             │   223 │   │   return chat_response_to_completion_response(chat_response)                     │               
                             │   224                                                                                        │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/llama_index/core/instrumentation/dispa │               
                             │ tcher.py:290 in async_wrapper                                                                │               
                             │                                                                                              │               
                             │   287 │   │   │   │   tags=tags,                                                             │               
                             │   288 │   │   │   )                                                                          │               
                             │   289 │   │   │   try:                                                                       │               
                             │ ❱ 290 │   │   │   │   result = await func(*args, **kwargs)                                   │               
                             │   291 │   │   │   except BaseException as e:                                                 │               
                             │   292 │   │   │   │   self.event(SpanDropEvent(span_id=id_, err_str=str(e)))                 │               
                             │   293 │   │   │   │   self.span_drop(id_=id_, bound_args=bound_args, instance=instance, err= │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/llama_index/core/llms/callbacks.py:76  │               
                             │ in wrapped_async_llm_chat                                                                    │               
                             │                                                                                              │               
                             │    73 │   │   │   │   │   },                                                                 │               
                             │    74 │   │   │   │   )                                                                      │               
                             │    75 │   │   │   │   try:                                                                   │               
                             │ ❱  76 │   │   │   │   │   f_return_val = await f(_self, messages, **kwargs)                  │               
                             │    77 │   │   │   │   except BaseException as e:                                             │               
                             │    78 │   │   │   │   │   callback_manager.on_event_end(                                     │               
                             │    79 │   │   │   │   │   │   CBEventType.LLM,                                               │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/llama_index/llms/ollama/base.py:369 in │               
                             │ achat                                                                                        │               
                             │                                                                                              │               
                             │   366 │   │                                                                                  │               
                             │   367 │   │   tools = kwargs.pop("tools", None)                                              │               
                             │   368 │   │                                                                                  │               
                             │ ❱ 369 │   │   response = await self.async_client.chat(                                       │               
                             │   370 │   │   │   model=self.model,                                                          │               
                             │   371 │   │   │   messages=ollama_messages,                                                  │               
                             │   372 │   │   │   stream=False,                                                              │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/ollama/_client.py:653 in chat          │               
                             │                                                                                              │               
                             │    650 │     if images := message.get('images'):                                             │               
                             │    651 │   │   message['images'] = [_encode_image(image) for image in images]                │               
                             │    652 │                                                                                     │               
                             │ ❱  653 │   return await self._request_stream(                                                │               
                             │    654 │     'POST',                                                                         │               
                             │    655 │     '/api/chat',                                                                    │               
                             │    656 │     json={                                                                          │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/ollama/_client.py:517 in               │               
                             │ _request_stream                                                                              │               
                             │                                                                                              │               
                             │    514 │   if stream:                                                                        │               
                             │    515 │     return await self._stream(*args, **kwargs)                                      │               
                             │    516 │                                                                                     │               
                             │ ❱  517 │   response = await self._request(*args, **kwargs)                                   │               
                             │    518 │   return response.json()                                                            │               
                             │    519                                                                                       │               
                             │    520   @overload                                                                           │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/ollama/_client.py:482 in _request      │               
                             │                                                                                              │               
                             │    479 │   super().__init__(httpx.AsyncClient, host, **kwargs)                               │               
                             │    480                                                                                       │               
                             │    481   async def _request(self, method: str, url: str, **kwargs) -> httpx.Response:        │               
                             │ ❱  482 │   response = await self._client.request(method, url, **kwargs)                      │               
                             │    483 │                                                                                     │               
                             │    484 │   try:                                                                              │               
                             │    485 │     response.raise_for_status()                                                     │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/httpx/_client.py:1574 in request       │               
                             │                                                                                              │               
                             │   1571 │   │   │   timeout=timeout,                                                          │               
                             │   1572 │   │   │   extensions=extensions,                                                    │               
                             │   1573 │   │   )                                                                             │               
                             │ ❱ 1574 │   │   return await self.send(request, auth=auth, follow_redirects=follow_redirects) │               
                             │   1575 │                                                                                     │               
                             │   1576 │   @asynccontextmanager                                                              │               
                             │   1577 │   async def stream(                                                                 │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/httpx/_client.py:1661 in send          │               
                             │                                                                                              │               
                             │   1658 │   │                                                                                 │               
                             │   1659 │   │   auth = self._build_request_auth(request, auth)                                │               
                             │   1660 │   │                                                                                 │               
                             │ ❱ 1661 │   │   response = await self._send_handling_auth(                                    │               
                             │   1662 │   │   │   request,                                                                  │               
                             │   1663 │   │   │   auth=auth,                                                                │               
                             │   1664 │   │   │   follow_redirects=follow_redirects,                                        │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/httpx/_client.py:1689 in               │               
                             │ _send_handling_auth                                                                          │               
                             │                                                                                              │               
                             │   1686 │   │   │   request = await auth_flow.__anext__()                                     │               
                             │   1687 │   │   │                                                                             │               
                             │   1688 │   │   │   while True:                                                               │               
                             │ ❱ 1689 │   │   │   │   response = await self._send_handling_redirects(                       │               
                             │   1690 │   │   │   │   │   request,                                                          │               
                             │   1691 │   │   │   │   │   follow_redirects=follow_redirects,                                │               
                             │   1692 │   │   │   │   │   history=history,                                                  │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/httpx/_client.py:1726 in               │               
                             │ _send_handling_redirects                                                                     │               
                             │                                                                                              │               
                             │   1723 │   │   │   for hook in self._event_hooks["request"]:                                 │               
                             │   1724 │   │   │   │   await hook(request)                                                   │               
                             │   1725 │   │   │                                                                             │               
                             │ ❱ 1726 │   │   │   response = await self._send_single_request(request)                       │               
                             │   1727 │   │   │   try:                                                                      │               
                             │   1728 │   │   │   │   for hook in self._event_hooks["response"]:                            │               
                             │   1729 │   │   │   │   │   await hook(response)                                              │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/httpx/_client.py:1763 in               │               
                             │ _send_single_request                                                                         │               
                             │                                                                                              │               
                             │   1760 │   │   │   )                                                                         │               
                             │   1761 │   │                                                                                 │               
                             │   1762 │   │   with request_context(request=request):                                        │               
                             │ ❱ 1763 │   │   │   response = await transport.handle_async_request(request)                  │               
                             │   1764 │   │                                                                                 │               
                             │   1765 │   │   assert isinstance(response.stream, AsyncByteStream)                           │               
                             │   1766 │   │   response.request = request                                                    │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/httpx/_transports/default.py:372 in    │               
                             │ handle_async_request                                                                         │               
                             │                                                                                              │               
                             │   369 │   │   │   content=request.stream,                                                    │               
                             │   370 │   │   │   extensions=request.extensions,                                             │               
                             │   371 │   │   )                                                                              │               
                             │ ❱ 372 │   │   with map_httpcore_exceptions():                                                │               
                             │   373 │   │   │   resp = await self._pool.handle_async_request(req)                          │               
                             │   374 │   │                                                                                  │               
                             │   375 │   │   assert isinstance(resp.stream, typing.AsyncIterable)                           │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/contextlib.py:153 in __exit__                        │               
                             │                                                                                              │               
                             │   150 │   │   │   │   # tell if we get the same exception back                               │               
                             │   151 │   │   │   │   value = typ()                                                          │               
                             │   152 │   │   │   try:                                                                       │               
                             │ ❱ 153 │   │   │   │   self.gen.throw(typ, value, traceback)                                  │               
                             │   154 │   │   │   except StopIteration as exc:                                               │               
                             │   155 │   │   │   │   # Suppress StopIteration *unless* it's the same exception that         │               
                             │   156 │   │   │   │   # was passed to throw().  This prevents a StopIteration                │               
                             │                                                                                              │               
                             │ /opt/conda/envs/llm_hugg/lib/python3.10/site-packages/httpx/_transports/default.py:86 in     │               
                             │ map_httpcore_exceptions                                                                      │               
                             │                                                                                              │               
                             │    83 │   │   │   raise                                                                      │               
                             │    84 │   │                                                                                  │               
                             │    85 │   │   message = str(exc)                                                             │               
                             │ ❱  86 │   │   raise mapped_exc(message) from exc                                             │               
                             │    87                                                                                        │               
                             │    88                                                                                        │               
                             │    89 HTTPCORE_EXC_MAP = {                                                                   │               
                             ╰──────────────────────────────────────────────────────────────────────────────────────────────╯               
                             ReadTimeout                                                                                                    
sys:1: RuntimeWarning: coroutine 'Dispatcher.span.<locals>.async_wrapper' was never awaited
```

</details>

autorag works when i use untagged model
```
      modules:
        - module_type: llama_index_llm
          llm: ollama
          model: llama3
          temperature: 0.7
          batch: 1
```


@khlee369 Thanks for the bug report. We will investigate why this isn't working

It seems that the issue is related to the default setting of ```DEFAULT_REQUEST_TIMEOUT = 30.0``` in ```llama_index.llms.ollama```. When loading a model with [ollama](https://ollama.com/) and performing inference, if the process takes longer than 30 seconds, the coroutine fails to await and the request is immediately terminated.

To temporarily resolve this issue, you can either modify ```DEFAULT_REQUEST_TIMEOUT = 30.0``` in ```llama_index.llms.ollama.base``` (e.g., change it to 3000.0), or set the request_timeout in the config.yaml file under the generator node.

```
      modules:
        - module_type: llama_index_llm
          llm: ollama
          model: qwen2:72b
          temperature: 0.7
          request_timeout: 3000
          batch: 1
```
----
Ref
- https://github.com/Marker-Inc-Korea/AutoRAG/pull/638#issuecomment-2308750039

Close this issue since this is not the code problem.
You have to set request_timeout more higher value.

Can you write the codes that I used to run the library with Ulama, because I have been trying to run it for a while and it is not working?



https://github.com/Marker-Inc-Korea/AutoRAG/issues/1115

autorag evaluate ^
  --qa_data_path ./qa_train.parquet ^
  --corpus_data_path ./corpus.parquet ^
  --project_dir AutoRAG ^
  --config ./config.yaml

ImportError: cannot import name 'IncludeEnum' from 'chromadb.api.types'
                             (C:\ProgramData\anaconda3\envs\2\Lib\site-packages\chromadb\api\types.py)

win10

 conda

                             (C:\ProgramData\anaconda3\envs\2\Lib\site-packages\chromadb\api\types.py)

(2) C:\Users\TARGET STORE\Desktop\7\AutoRAG>pip list
Package                                  Version
---------------------------------------- ---------------
absl-py                                  2.3.0
accelerate                               0.34.2
aiofiles                                 24.1.0
aiohappyeyeballs                         2.6.1
aiohttp                                  3.12.11
aiolimiter                               1.2.1
aiosignal                                1.3.2
aiosqlite                                0.21.0
altair                                   5.5.0
annotated-types                          0.7.0
anthropic                                0.28.1
anyio                                    4.9.0
asgiref                                  3.8.1
asttokens                                3.0.0
attrs                                    25.3.0
Authlib                                  1.3.1
AutoRAG                                  0.3.13
backoff                                  2.2.1
banks                                    2.1.2
bcrypt                                   4.3.0
beautifulsoup4                           4.13.4
bleach                                   6.2.0
blinker                                  1.9.0
bm25s                                    0.2.13
bokeh                                    3.7.3
boto3                                    1.38.32
botocore                                 1.38.32
build                                    1.2.2.post1
cachetools                               5.5.2
certifi                                  2025.4.26
cffi                                     1.17.1
charset-normalizer                       3.4.2
chroma-hnswlib                           0.7.3
chromadb                                 1.0.12
click                                    8.2.1
clickhouse-connect                       0.8.17
cohere                                   5.15.0
colorama                                 0.4.6
coloredlogs                              15.0.1
comm                                     0.2.2
contourpy                                1.3.2
couchbase                                4.4.0
cramjam                                  2.10.0
cryptography                             45.0.3
cuda-python                              12.6.0
cycler                                   0.12.1
dataclasses-json                         0.6.7
datasets                                 3.6.0
debugpy                                  1.8.14
decorator                                5.2.1
Deprecated                               1.2.18
deprecation                              2.1.0
dill                                     0.3.8
dirtyjson                                1.0.8
distro                                   1.9.0
duckdb                                   1.3.0
durationpy                               0.10
emoji                                    2.14.1
eval_type_backport                       0.2.2
evaluate                                 0.4.3
executing                                2.2.0
fastapi                                  0.115.9
fastavro                                 1.11.1
fastparquet                              2024.11.0
ffmpy                                    0.6.0
filelock                                 3.18.0
filetype                                 1.2.0
Flask                                    3.1.1
flatbuffers                              25.2.10
fonttools                                4.58.2
frozenlist                               1.6.2
fsspec                                   2025.3.0
gitdb                                    4.0.12
GitPython                                3.1.44
google-auth                              2.40.3
googleapis-common-protos                 1.70.0
gradio                                   5.33.0
gradio_client                            1.10.2
greenlet                                 3.2.3
griffe                                   1.7.3
groovy                                   0.1.2
grpcio                                   1.67.1
grpcio-health-checking                   1.67.1
grpcio-tools                             1.67.1
h11                                      0.16.0
h2                                       4.2.0
hnswlib                                  0.8.0
hpack                                    4.1.0
httpcore                                 1.0.9
httptools                                0.6.4
httpx                                    0.27.2
httpx-sse                                0.4.0
huggingface-hub                          0.32.4
humanfriendly                            10.0
Hypercorn                                0.17.3
hyperframe                               6.1.0
idna                                     3.10
importlib_metadata                       8.7.0
importlib_resources                      6.5.2
inquirerpy                               0.3.4
ipykernel                                6.29.5
ipython                                  9.3.0
ipython_pygments_lexers                  1.1.1
ipywidgets                               8.1.7
ipywidgets_bokeh                         1.6.0
itsdangerous                             2.2.0
jedi                                     0.19.2
Jinja2                                   3.1.6
jiter                                    0.10.0
jmespath                                 1.0.1
joblib                                   1.5.1
jsonpatch                                1.33
jsonpath-python                          1.0.6
jsonpointer                              3.0.0
jsonschema                               4.24.0
jsonschema-specifications                2025.4.1
jupyter_client                           8.6.3
jupyter_core                             5.8.1
jupyterlab_widgets                       3.0.15
kiwisolver                               1.4.8
kubernetes                               32.0.1
langchain                                0.3.25
langchain-community                      0.3.24
langchain-core                           0.3.64
langchain-openai                         0.3.21
langchain-text-splitters                 0.3.8
langchain-unstructured                   0.1.6
langchain-upstage                        0.6.0
langsmith                                0.3.45
linkify-it-py                            2.0.3
llama-cloud                              0.1.19
llama-cloud-services                     0.6.21
llama-index                              0.11.22
llama-index-agent-openai                 0.3.4
llama-index-cli                          0.3.1
llama-index-core                         0.11.23
llama-index-embeddings-huggingface       0.5.4
llama-index-embeddings-ollama            0.3.1
llama-index-embeddings-openai            0.2.5
llama-index-indices-managed-llama-cloud  0.6.0
llama-index-legacy                       0.9.48.post4
llama-index-llms-anthropic               0.2.1
llama-index-llms-bedrock                 0.2.6
llama-index-llms-ollama                  0.6.1
llama-index-llms-openai                  0.2.16
llama-index-llms-openai-like             0.2.0
llama-index-multi-modal-llms-openai      0.2.3
llama-index-program-openai               0.2.0
llama-index-question-gen-openai          0.2.0
llama-index-readers-file                 0.2.2
llama-index-readers-llama-parse          0.3.0
llama-index-retrievers-bm25              0.4.0
llama-index-vector-stores-chroma         0.4.1
llama-parse                              0.6.21
llamaindex-py-client                     0.1.19
lxml                                     5.4.0
lz4                                      4.4.4
Markdown                                 3.8
markdown-it-py                           3.0.0
MarkupSafe                               3.0.2
marshmallow                              3.26.1
matplotlib                               3.10.3
matplotlib-inline                        0.1.7
mdit-py-plugins                          0.4.2
mdurl                                    0.1.2
mixedbread-ai                            2.2.6
mmh3                                     5.1.0
mpmath                                   1.3.0
multidict                                6.4.4
multiprocess                             0.70.16
mypy_extensions                          1.1.0
narwhals                                 1.41.1
nest-asyncio                             1.6.0
networkx                                 3.5
nltk                                     3.9.1
numpy                                    1.26.4
oauthlib                                 3.2.2
ollama                                   0.3.3
ollama-rag                               0.4.1
onnxruntime                              1.19.2
openai                                   1.84.0
opentelemetry-api                        1.34.0
opentelemetry-exporter-otlp-proto-common 1.34.0
opentelemetry-exporter-otlp-proto-grpc   1.34.0
opentelemetry-instrumentation            0.55b0
opentelemetry-instrumentation-asgi       0.55b0
opentelemetry-instrumentation-fastapi    0.55b0
opentelemetry-proto                      1.34.0
opentelemetry-sdk                        1.34.0
opentelemetry-semantic-conventions       0.55b0
opentelemetry-util-http                  0.55b0
orjson                                   3.10.18
overrides                                7.7.0
packaging                                24.2
pandas                                   2.3.0
panel                                    1.7.1
param                                    2.2.0
parso                                    0.8.4
pfzy                                     0.3.4
pillow                                   11.2.1
pinecone                                 7.0.2
pinecone-plugin-assistant                1.6.1
pinecone-plugin-interface                0.0.7
pip                                      25.1
platformdirs                             4.3.8
portalocker                              2.10.1
posthog                                  4.4.0
priority                                 2.0.0
prompt_toolkit                           3.0.51
propcache                                0.3.1
protobuf                                 5.29.5
protoc-gen-openapiv2                     0.0.1
psutil                                   7.0.0
pulsar-client                            3.7.0
pure_eval                                0.2.3
pyarrow                                  20.0.0
pyasn1                                   0.6.1
pyasn1_modules                           0.4.2
pycparser                                2.22
pydantic                                 2.9.2
pydantic_core                            2.23.4
pydantic-settings                        2.9.1
pydeck                                   0.9.1
pydub                                    0.25.1
Pygments                                 2.19.1
pymilvus                                 2.5.10
pyngrok                                  7.2.11
pyparsing                                3.2.3
pypdf                                    4.3.1
PyPika                                   0.48.9
pyproject_hooks                          1.2.0
pyreadline3                              3.5.4
PyStemmer                                2.2.0.3
python-dateutil                          2.9.0.post0
python-dotenv                            1.1.0
python-multipart                         0.0.20
pytz                                     2025.2
pyviz_comms                              3.0.4
pywin32                                  310
PyYAML                                   6.0.2
pyzmq                                    26.4.0
qdrant-client                            1.14.2
Quart                                    0.20.0
rank-bm25                                0.2.2
referencing                              0.36.2
regex                                    2024.11.6
requests                                 2.32.3
requests-oauthlib                        2.0.0
requests-toolbelt                        1.0.0
rich                                     14.0.0
rouge_score                              0.1.2
rpds-py                                  0.25.1
rsa                                      4.9.1
ruff                                     0.11.13
s3transfer                               0.13.0
sacrebleu                                2.5.1
safehttpx                                0.1.6
safetensors                              0.5.3
scikit-learn                             1.7.0
scipy                                    1.15.3
seaborn                                  0.13.2
semantic-version                         2.10.0
sentence-transformers                    4.1.0
setuptools                               80.9.0
shellingham                              1.5.4
six                                      1.17.0
smmap                                    5.0.2
sniffio                                  1.3.1
soupsieve                                2.7
SQLAlchemy                               2.0.41
stack-data                               0.6.3
starlette                                0.45.3
streamlit                                1.45.1
striprtf                                 0.0.26
sympy                                    1.14.0
tabulate                                 0.9.0
tenacity                                 8.5.0
threadpoolctl                            3.6.0
tiktoken                                 0.9.0
tokenizers                               0.19.1
tokenlog                                 0.0.2
toml                                     0.10.2
tomlkit                                  0.13.3
torch                                    2.7.0
tornado                                  6.5.1
tqdm                                     4.67.1
traitlets                                5.14.3
transformers                             4.44.2
typer                                    0.16.0
types-requests                           2.32.0.20250602
typing_extensions                        4.14.0
typing-inspect                           0.9.0
typing-inspection                        0.4.1
tzdata                                   2025.2
uc-micro-py                              1.0.3
ujson                                    5.10.0
unstructured-client                      0.28.1
urllib3                                  2.4.0
uvicorn                                  0.34.3
validators                               0.34.0
voyageai                                 0.3.2
watchdog                                 6.0.0
watchfiles                               1.0.5
wcwidth                                  0.2.13
weaviate-client                          4.15.0
webencodings                             0.5.1
websocket-client                         1.8.0
websockets                               15.0.1
Werkzeug                                 3.1.3
wheel                                    0.45.1
widgetsnbextension                       4.0.14
wrapt                                    1.17.2
wsproto                                  1.2.0
xxhash                                   3.5.0
xyzservices                              2025.4.0
yarl                                     1.20.0
zipp                                     3.23.0
zstandard                                0.23.0

(2) C:\Users\TARGET STORE\Desktop\7\AutoRAG>

python 3.11

????

Has support for Olama been stopped or what?Can someone who has run the library with Olama share with us his method from the beginning, because I tried and searched, but all attempts failed?