Repository: tatsu-lab/alpaca_eval
Issue: TypeError When Using weighted_alpaca_eval_gpt4_turbo Annotator
Root cause: External Dependency Breakage

Hi,

AlpacaEval2.0 looks really cool! Thanks for making this.

Unfortunately, when I try an run ```alpaca_eval --model_outputs model_outputs.json --annotators_config 'weighted_alpaca_eval_gpt4_turbo'``` I get the following error:

```
Traceback (most recent call last):
  File "/data/evan/miniconda3/envs/alpaca/bin/alpaca_eval", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/data/evan/miniconda3/envs/alpaca/lib/python3.11/site-packages/alpaca_eval/main.py", line 586, in main
    fire.Fire(evaluate)
  File "/data/evan/miniconda3/envs/alpaca/lib/python3.11/site-packages/fire/core.py", line 141, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/evan/miniconda3/envs/alpaca/lib/python3.11/site-packages/fire/core.py", line 475, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
                                ^^^^^^^^^^^^^^^^^^^^
  File "/data/evan/miniconda3/envs/alpaca/lib/python3.11/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^
  File "/data/evan/miniconda3/envs/alpaca/lib/python3.11/site-packages/alpaca_eval/main.py", line 143, in evaluate
    annotations = annotator.annotate_head2head(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/evan/miniconda3/envs/alpaca/lib/python3.11/site-packages/alpaca_eval/annotators/pairwise_evaluator.py", line 237, in annotate_head2head
    out = self.__call__(df_to_annotate, **decoding_kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/evan/miniconda3/envs/alpaca/lib/python3.11/site-packages/alpaca_eval/annotators/base.py", line 183, in __call__
    df_annotated = self._annotate(curr_df_to_annotate, **decoding_kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/evan/miniconda3/envs/alpaca/lib/python3.11/site-packages/alpaca_eval/annotators/base.py", line 284, in _annotate
    curr_annotated = self.annotators[annotator](
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/evan/miniconda3/envs/alpaca/lib/python3.11/site-packages/alpaca_eval/annotators/base.py", line 636, in __call__
    completions = self.fn_completions(prompts=prompts, **self.completions_kwargs, **decoding_kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/evan/miniconda3/envs/alpaca/lib/python3.11/site-packages/alpaca_eval/decoders/openai.py", line 152, in openai_completions
    completions = list(
                  ^^^^^
  File "/data/evan/miniconda3/envs/alpaca/lib/python3.11/site-packages/tqdm/std.py", line 1182, in __iter__
    for obj in iterable:
  File "/data/evan/miniconda3/envs/alpaca/lib/python3.11/multiprocessing/pool.py", line 873, in next
    raise value
TypeError: Completions.create() got an unexpected keyword argument 'logprobs'
```

OpenAI is on version 1.3.3

Note everything works as expected when using ```alpaca_eval_gpt4_turbo_fn``` annotator instead.

Also note that I do not have an Azure Endpoint setup, and am just setting the api key via environment variable (the old way).

Thanks for the help!

you have to use `OpenAI>=1.5.0` let me update that!

fixed, lmk if you have other issues