Repository: Marker-Inc-Korea/AutoRAG
Issue: [HotFix] Error at tart.py
GitHub labels: bug

```
[04/23/24 04:19:32] ERROR    [__init__.py:71] >> Unexpected exception                                                                                                         __init__.py:71
                             ╭───────────────────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────────────────╮               
                             │ /usr/local/bin/autorag:8 in <module>                                                                                                         │               
                             │                                                                                                                                              │               
                             │   5 from autorag.cli import cli                                                                                                              │               
                             │   6 if __name__ == '__main__':                                                                                                               │               
                             │   7 │   sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])                                                                     │               
                             │ ❱ 8 │   sys.exit(cli())                                                                                                                      │               
                             │   9                                                                                                                                          │               
                             │                                                                                                                                              │               
                             │ /usr/local/lib/python3.10/dist-packages/click/core.py:1157 in __call__                                                                       │               
                             │                                                                                                                                              │               
                             │   1154 │                                                                                                                                     │               
                             │   1155 │   def __call__(self, *args: t.Any, **kwargs: t.Any) -> t.Any:                                                                       │               
                             │   1156 │   │   """Alias for :meth:`main`."""                                                                                                 │               
                             │ ❱ 1157 │   │   return self.main(*args, **kwargs)                                                                                             │               
                             │   1158                                                                                                                                       │               
                             │   1159                                                                                                                                       │               
                             │   1160 class Command(BaseCommand):                                                                                                           │               
                             │                                                                                                                                              │               
                             │ /usr/local/lib/python3.10/dist-packages/click/core.py:1078 in main                                                                           │               
                             │                                                                                                                                              │               
                             │   1075 │   │   try:                                                                                                                          │               
                             │   1076 │   │   │   try:                                                                                                                      │               
                             │   1077 │   │   │   │   with self.make_context(prog_name, args, **extra) as ctx:                                                              │               
                             │ ❱ 1078 │   │   │   │   │   rv = self.invoke(ctx)                                                                                             │               
                             │   1079 │   │   │   │   │   if not standalone_mode:                                                                                           │               
                             │   1080 │   │   │   │   │   │   return rv                                                                                                     │               
                             │   1081 │   │   │   │   │   # it's not safe to `ctx.exit(rv)` here!                                                                           │               
                             │                                                                                                                                              │               
                             │ /usr/local/lib/python3.10/dist-packages/click/core.py:1688 in invoke                                                                         │               
                             │                                                                                                                                              │               
                             │   1685 │   │   │   │   super().invoke(ctx)                                                                                                   │               
                             │   1686 │   │   │   │   sub_ctx = cmd.make_context(cmd_name, args, parent=ctx)                                                                │               
                             │   1687 │   │   │   │   with sub_ctx:                                                                                                         │               
                             │ ❱ 1688 │   │   │   │   │   return _process_result(sub_ctx.command.invoke(sub_ctx))                                                           │               
                             │   1689 │   │                                                                                                                                 │               
                             │   1690 │   │   # In chain mode we create the contexts step by step, but after the                                                            │               
                             │   1691 │   │   # base command has been invoked.  Because at that point we do not                                                             │               
                             │                                                                                                                                              │               
                             │ /usr/local/lib/python3.10/dist-packages/click/core.py:1434 in invoke                                                                         │               
                             │                                                                                                                                              │               
                             │   1431 │   │   │   echo(style(message, fg="red"), err=True)                                                                                  │               
                             │   1432 │   │                                                                                                                                 │               
                             │   1433 │   │   if self.callback is not None:                                                                                                 │               
                             │ ❱ 1434 │   │   │   return ctx.invoke(self.callback, **ctx.params)                                                                            │               
                             │   1435 │                                                                                                                                     │               
                             │   1436 │   def shell_complete(self, ctx: Context, incomplete: str) -> t.List["CompletionItem"]:                                              │               
                             │   1437 │   │   """Return a list of completions for the incomplete value. Looks                                                               │               
                             │                                                                                                                                              │               
                             │ /usr/local/lib/python3.10/dist-packages/click/core.py:783 in invoke                                                                          │               
                             │                                                                                                                                              │               
                             │    780 │   │                                                                                                                                 │               
                             │    781 │   │   with augment_usage_errors(__self):                                                                                            │               
                             │    782 │   │   │   with ctx:                                                                                                                 │               
                             │ ❱  783 │   │   │   │   return __callback(*args, **kwargs)                                                                                    │               
                             │    784 │                                                                                                                                     │               
                             │    785 │   def forward(                                                                                                                      │               
                             │    786 │   │   __self, __cmd: "Command", *args: t.Any, **kwargs: t.Any  # noqa: B902                                                         │               
                             │                                                                                                                                              │               
                             │ /usr/local/lib/python3.10/dist-packages/autorag/cli.py:34 in evaluate                                                                        │               
                             │                                                                                                                                              │               
                             │    31 │   if not os.path.exists(config):                                                                                                     │               
                             │    32 │   │   raise ValueError(f"Config file {config} does not exist.")                                                                      │               
                             │    33 │   evaluator = Evaluator(qa_data_path, corpus_data_path, project_dir=project_dir)                                                     │               
                             │ ❱  34 │   evaluator.start_trial(config)                                                                                                      │               
                             │    35 │   logger.info('Evaluation complete.')                                                                                                │               
                             │    36                                                                                                                                        │               
                             │    37                                                                                                                                        │               
                             │                                                                                                                                              │               
                             │ /usr/local/lib/python3.10/dist-packages/autorag/evaluator.py:95 in start_trial                                                               │               
                             │                                                                                                                                              │               
                             │    92 │   │   │   if i == 0:                                                                                                                 │               
                             │    93 │   │   │   │   previous_result = self.qa_data                                                                                         │               
                             │    94 │   │   │   logger.info(f'Running node line {node_line_name}...')                                                                      │               
                             │ ❱  95 │   │   │   previous_result = run_node_line(node_line, node_line_dir, previous_result)                                                 │               
                             │    96 │   │   │                                                                                                                              │               
                             │    97 │   │   │   trial_summary_df = self._append_node_line_summary(node_line_name,                                                          │               
                             │       node_line_dir, trial_summary_df)                                                                                                       │               
                             │    98                                                                                                                                        │               
                             │                                                                                                                                              │               
                             │ /usr/local/lib/python3.10/dist-packages/autorag/node_line.py:45 in run_node_line                                                             │               
                             │                                                                                                                                              │               
                             │   42 │                                                                                                                                       │               
                             │   43 │   summary_lst = []                                                                                                                    │               
                             │   44 │   for node in nodes:                                                                                                                  │               
                             │ ❱ 45 │   │   previous_result = node.run(previous_result, node_line_dir)                                                                      │               
                             │   46 │   │   node_summary_df = load_summary_file(os.path.join(node_line_dir, node.node_type,                                                 │               
                             │      'summary.csv'))                                                                                                                         │               
                             │   47 │   │   best_node_row = node_summary_df.loc[node_summary_df['is_best']]                                                                 │               
                             │   48 │   │   summary_lst.append({                                                                                                            │               
                             │                                                                                                                                              │               
                             │ /usr/local/lib/python3.10/dist-packages/autorag/schema/node.py:57 in run                                                                     │               
                             │                                                                                                                                              │               
                             │    54 │   def run(self, previous_result: pd.DataFrame, node_line_dir: str) -> pd.DataFrame:                                                  │               
                             │    55 │   │   logger.info(f'Running node {self.node_type}...')                                                                               │               
                             │    56 │   │   input_modules, input_params = self.get_param_combinations()                                                                    │               
                             │ ❱  57 │   │   return self.run_node(modules=input_modules,                                                                                    │               
                             │    58 │   │   │   │   │   │   │    module_params=input_params,                                                                               │               
                             │    59 │   │   │   │   │   │   │    previous_result=previous_result,                                                                          │               
                             │    60 │   │   │   │   │   │   │    node_line_dir=node_line_dir,                                                                              │               
                             │                                                                                                                                              │               
                             │ /usr/local/lib/python3.10/dist-packages/autorag/nodes/passagereranker/run.py:39 in run_passage_reranker_node                                 │               
                             │                                                                                                                                              │               
                             │   36 │   project_dir = pathlib.PurePath(node_line_dir).parent.parent                                                                         │               
                             │   37 │   retrieval_gt = pd.read_parquet(os.path.join(project_dir, "data",                                                                    │               
                             │      "qa.parquet"))['retrieval_gt'].tolist()                                                                                                 │               
                             │   38 │                                                                                                                                       │               
                             │ ❱ 39 │   results, execution_times = zip(*map(lambda task: measure_speed(                                                                     │               
                             │   40 │   │   task[0], project_dir=project_dir, previous_result=previous_result, **task[1]),                                                  │               
                             │      zip(modules, module_params)))                                                                                                           │               
                             │   41 │   average_times = list(map(lambda x: x / len(results[0]), execution_times))                                                           │               
                             │   42                                                                                                                                         │               
                             │                                                                                                                                              │               
                             │ /usr/local/lib/python3.10/dist-packages/autorag/nodes/passagereranker/run.py:39 in <lambda>                                                  │               
                             │                                                                                                                                              │               
                             │   36 │   project_dir = pathlib.PurePath(node_line_dir).parent.parent                                                                         │               
                             │   37 │   retrieval_gt = pd.read_parquet(os.path.join(project_dir, "data",                                                                    │               
                             │      "qa.parquet"))['retrieval_gt'].tolist()                                                                                                 │               
                             │   38 │                                                                                                                                       │               
                             │ ❱ 39 │   results, execution_times = zip(*map(lambda task: measure_speed(                                                                     │               
                             │   40 │   │   task[0], project_dir=project_dir, previous_result=previous_result, **task[1]),                                                  │               
                             │      zip(modules, module_params)))                                                                                                           │               
                             │   41 │   average_times = list(map(lambda x: x / len(results[0]), execution_times))                                                           │               
                             │   42                                                                                                                                         │               
                             │                                                                                                                                              │               
                             │ /usr/local/lib/python3.10/dist-packages/autorag/strategy.py:13 in measure_speed                                                              │               
                             │                                                                                                                                              │               
                             │    10 │   Method for measuring execution speed of the function.                                                                              │               
                             │    11 │   """                                                                                                                                │               
                             │    12 │   start_time = time.time()                                                                                                           │               
                             │ ❱  13 │   result = func(*args, **kwargs)                                                                                                     │               
                             │    14 │   end_time = time.time()                                                                                                             │               
                             │    15 │   return result, end_time - start_time                                                                                               │               
                             │    16                                                                                                                                        │               
                             │                                                                                                                                              │               
                             │ /usr/local/lib/python3.10/dist-packages/autorag/utils/util.py:40 in wrapper                                                                  │               
                             │                                                                                                                                              │               
                             │    37 │   def decorator_result_to_dataframe(func: Callable):                                                                                 │               
                             │    38 │   │   @functools.wraps(func)                                                                                                         │               
                             │    39 │   │   def wrapper(*args, **kwargs) -> pd.DataFrame:                                                                                  │               
                             │ ❱  40 │   │   │   results = func(*args, **kwargs)                                                                                            │               
                             │    41 │   │   │   if len(column_names) == 1:                                                                                                 │               
                             │    42 │   │   │   │   df_input = {column_names[0]: results}                                                                                  │               
                             │    43 │   │   │   else:                                                                                                                      │               
                             │                                                                                                                                              │               
                             │ /usr/local/lib/python3.10/dist-packages/autorag/nodes/passagereranker/base.py:48 in wrapper                                                  │               
                             │                                                                                                                                              │               
                             │   45 │   │   │   │   = func(contents_list=contents, scores_list=scores, ids_list=ids,                                                        │               
                             │      time_list=times, *args, **kwargs)                                                                                                       │               
                             │   46 │   │   else:                                                                                                                           │               
                             │   47 │   │   │   reranked_contents, reranked_ids, reranked_scores \                                                                          │               
                             │ ❱ 48 │   │   │   │   = func(queries=queries, contents_list=contents, scores_list=scores,                                                     │               
                             │      ids_list=ids, *args, **kwargs)                                                                                                          │               
                             │   49 │   │                                                                                                                                   │               
                             │   50 │   │   return reranked_contents, reranked_ids, reranked_scores                                                                         │               
                             │   51                                                                                                                                         │               
                             │                                                                                                                                              │               
                             │ /usr/local/lib/python3.10/dist-packages/autorag/nodes/passagereranker/tart/tart.py:46 in tart                                                │               
                             │                                                                                                                                              │               
                             │   43 │   nested_list = [[['{} [SEP] {}'.format(instruction, query)] for _ in contents] for                                                   │               
                             │      query, contents in                                                                                                                      │               
                             │   44 │   │   │   │      zip(queries, contents_list)]                                                                                         │               
                             │   45 │                                                                                                                                       │               
                             │ ❱ 46 │   rerank_scores = flatten_apply(tart_run_model, nested_list, model=model,                                                             │               
                             │      batch_size=batch,                                                                                                                       │               
                             │   47 │   │   │   │   │   │   │   │     tokenizer=tokenizer, device=device,                                                                   │               
                             │      contents_list=contents_list)                                                                                                            │               
                             │   48 │                                                                                                                                       │               
                             │   49 │   df = pd.DataFrame({                                                                                                                 │               
                             │                                                                                                                                              │               
                             │ /usr/local/lib/python3.10/dist-packages/autorag/utils/util.py:300 in flatten_apply                                                           │               
                             │                                                                                                                                              │               
                             │   297 │   """                                                                                                                                │               
                             │   298 │   df = pd.DataFrame({'col1': nested_list})                                                                                           │               
                             │   299 │   df = df.explode('col1')                                                                                                            │               
                             │ ❱ 300 │   df['result'] = func(df['col1'].tolist(), **kwargs)                                                                                 │               
                             │   301 │   return df.groupby(level=0, sort=False)['result'].apply(list).tolist()                                                              │               
                             │   302                                                                                                                                        │               
                             │   303                                                                                                                                        │               
                             │                                                                                                                                              │               
                             │ /usr/local/lib/python3.10/dist-packages/autorag/nodes/passagereranker/tart/tart.py:72 in tart_run_model                                      │               
                             │                                                                                                                                              │               
                             │   69 │   for batch_texts, batch_contents in zip(batch_input_texts, batch_contents_list):                                                     │               
                             │   70 │   │   flattened_batch_texts = list(chain.from_iterable(batch_texts))                                                                  │               
                             │   71 │   │   flattened_batch_contents = list(chain.from_iterable(batch_contents))                                                            │               
                             │ ❱ 72 │   │   feature = tokenizer(flattened_batch_texts, flattened_batch_contents,                                                            │               
                             │      padding=True, truncation=True,                                                                                                          │               
                             │   73 │   │   │   │   │   │   │   return_tensors="pt").to(device)                                                                             │               
                             │   74 │   │   with torch.no_grad():                                                                                                           │               
                             │   75 │   │   │   pred_scores = model(**feature).logits                                                                                       │               
                             │                                                                                                                                              │               
                             │ /usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils_base.py:2872 in __call__                                             │               
                             │                                                                                                                                              │               
                             │   2869 │   │   │   # input mode in this case.                                                                                                │               
                             │   2870 │   │   │   if not self._in_target_context_manager:                                                                                   │               
                             │   2871 │   │   │   │   self._switch_to_input_mode()                                                                                          │               
                             │ ❱ 2872 │   │   │   encodings = self._call_one(text=text, text_pair=text_pair, **all_kwargs)                                                  │               
                             │   2873 │   │   if text_target is not None:                                                                                                   │               
                             │   2874 │   │   │   self._switch_to_target_mode()                                                                                             │               
                             │   2875 │   │   │   target_encodings = self._call_one(text=text_target,                                                                       │               
                             │        text_pair=text_pair_target, **all_kwargs)                                                                                             │               
                             │                                                                                                                                              │               
                             │ /usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils_base.py:2953 in _call_one                                            │               
                             │                                                                                                                                              │               
                             │   2950 │   │   │   │   │   " `text`."                                                                                                        │               
                             │   2951 │   │   │   │   )                                                                                                                     │               
                             │   2952 │   │   │   if text_pair is not None and len(text) != len(text_pair):                                                                 │               
                             │ ❱ 2953 │   │   │   │   raise ValueError(                                                                                                     │               
                             │   2954 │   │   │   │   │   f"batch length of `text`: {len(text)} does not match batch length of                                              │               
                             │        `text_pair`:"                                                                                                                         │               
                             │   2955 │   │   │   │   │   f" {len(text_pair)}."                                                                                             │               
                             │   2956 │   │   │   │   )                                                                                                                     │               
                             ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯               
                             ValueError: batch length of `text`: 64 does not match batch length of `text_pair`: 150. 
```