Repository: tatsu-lab/alpaca_eval
Issue: Pandas merge error
Root cause: Interface Contract Mismatch

Trying to test the eval script:

alpaca_eval --model_outputs 'example/outputs.json'

I tried with python3.10 and python3.11, and got the same error like this:

```
  File "/export/home/cxia/congyingxia-scratchpad/alpaca_eval/envs/bin/alpaca_eval", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/export/home/cxia/congyingxia-scratchpad/alpaca_eval/envs/lib/python3.11/site-packages/alpaca_eval/main.py", line 483
, in main
    fire.Fire(evaluate)
  File "/export/home/cxia/congyingxia-scratchpad/alpaca_eval/envs/lib/python3.11/site-packages/fire/core.py", line 141, in Fi
re
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/export/home/cxia/congyingxia-scratchpad/alpaca_eval/envs/lib/python3.11/site-packages/fire/core.py", line 475, in _F
ire
    component, remaining_args = _CallAndUpdateTrace(
                                ^^^^^^^^^^^^^^^^^^^^
  File "/export/home/cxia/congyingxia-scratchpad/alpaca_eval/envs/lib/python3.11/site-packages/fire/core.py", line 691, in _C
allAndUpdateTrace
    component = fn(*varargs, **kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^
  File "/export/home/cxia/congyingxia-scratchpad/alpaca_eval/envs/lib/python3.11/site-packages/alpaca_eval/main.py", line 126
, in evaluate
    annotations = annotator.annotate_head2head(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/export/home/cxia/congyingxia-scratchpad/alpaca_eval/envs/lib/python3.11/site-packages/alpaca_eval/annotators/pairwis
e_evaluator.py", line 316, in annotate_head2head
    out = self.annotate_pairs(df_to_annotate, **decoding_kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/export/home/cxia/congyingxia-scratchpad/alpaca_eval/envs/lib/python3.11/site-packages/alpaca_eval/annotators/pairwise_evaluator.py", line 344, in annotate_pairs
    df_to_annotate = self._preprocess(to_annotate)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/export/home/cxia/congyingxia-scratchpad/alpaca_eval/envs/lib/python3.11/site-packages/alpaca_eval/annotators/pairwise_evaluator.py", line 387, in _preprocess
    df_to_annotate = self._merge_annotations(df_to_annotate, self.df_annotations)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/export/home/cxia/congyingxia-scratchpad/alpaca_eval/envs/lib/python3.11/site-packages/alpaca_eval/annotators/pairwise_evaluator.py", line 533, in _merge_annotations
    df_to_annotate = df_to_annotate.merge(
                     ^^^^^^^^^^^^^^^^^^^^^
  File "/export/home/cxia/congyingxia-scratchpad/alpaca_eval/envs/lib/python3.11/site-packages/pandas/core/frame.py", line 9843, in merge
    return merge(
           ^^^^^^
  File "/export/home/cxia/congyingxia-scratchpad/alpaca_eval/envs/lib/python3.11/site-packages/pandas/core/reshape/merge.py", line 148, in merge
    op = _MergeOperation(
         ^^^^^^^^^^^^^^^^
  File "/export/home/cxia/congyingxia-scratchpad/alpaca_eval/envs/lib/python3.11/site-packages/pandas/core/reshape/merge.py", line 741, in __init__
    self._maybe_coerce_merge_keys()
  File "/export/home/cxia/congyingxia-scratchpad/alpaca_eval/envs/lib/python3.11/site-packages/pandas/core/reshape/merge.py", line 1401, in _maybe_coerce_merge_keys
    raise ValueError(msg)
ValueError: You are trying to merge on object and int64 columns. If you wish to proceed you should use pd.concat
```



Fixed by converting df dtype to object.
 
`df_partially_annotated = df_partially_annotated.astype(str)`

Adding this line to line 533 in alpaca_eval/annotators/pairwise_evaluator.py

thanks we added this!