Repository: EleutherAI/lm-evaluation-harness
Issue: Can't use OpenAI Chat Completion API call with simplest tasks
Root cause: Documentation Deficiency

I tried to run `lm_eval --model openai-chatcompletions --model_args model=gpt-4o-mini --tasks gsm8k` with the latest version of lm-eval-harness after setting the OPENAI_API_KEY, but it's throwing 400 errors. Would really appreciate any feedback on this. The full log is shown below:


```
Traceback (most recent call last):
  File "/weka/scratch/djiang21/miniconda/envs/lm_eval/bin/lm_eval", line 8, in <module>
    sys.exit(cli_evaluate())
  File "/weka/scratch/djiang21/lm-eval-original/lm-evaluation-harness/lm_eval/__main__.py", line 382, in cli_evaluate
    results = evaluator.simple_evaluate(
  File "/weka/scratch/djiang21/lm-eval-original/lm-evaluation-harness/lm_eval/utils.py", line 397, in _wrapper
    return fn(*args, **kwargs)
  File "/weka/scratch/djiang21/lm-eval-original/lm-evaluation-harness/lm_eval/evaluator.py", line 296, in simple_evaluate
    results = evaluate(
  File "/weka/scratch/djiang21/lm-eval-original/lm-evaluation-harness/lm_eval/utils.py", line 397, in _wrapper
    return fn(*args, **kwargs)
  File "/weka/scratch/djiang21/lm-eval-original/lm-evaluation-harness/lm_eval/evaluator.py", line 468, in evaluate
    resps = getattr(lm, reqtype)(cloned_reqs)
  File "/weka/scratch/djiang21/lm-eval-original/lm-evaluation-harness/lm_eval/models/api_models.py", line 548, in generate_until
    outputs = retry(
  File "/weka/scratch/djiang21/miniconda/envs/lm_eval/lib/python3.10/site-packages/tenacity/__init__.py", line 336, in wrapped_f
    return copy(f, *args, **kw)
  File "/weka/scratch/djiang21/miniconda/envs/lm_eval/lib/python3.10/site-packages/tenacity/__init__.py", line 475, in __call__
    do = self.iter(retry_state=retry_state)
  File "/weka/scratch/djiang21/miniconda/envs/lm_eval/lib/python3.10/site-packages/tenacity/__init__.py", line 376, in iter
    result = action(retry_state)
  File "/weka/scratch/djiang21/miniconda/envs/lm_eval/lib/python3.10/site-packages/tenacity/__init__.py", line 418, in exc_check
    raise retry_exc.reraise()
  File "/weka/scratch/djiang21/miniconda/envs/lm_eval/lib/python3.10/site-packages/tenacity/__init__.py", line 185, in reraise
    raise self.last_attempt.result()
  File "/weka/scratch/djiang21/miniconda/envs/lm_eval/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/weka/scratch/djiang21/miniconda/envs/lm_eval/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/weka/scratch/djiang21/miniconda/envs/lm_eval/lib/python3.10/site-packages/tenacity/__init__.py", line 478, in __call__
    result = fn(*args, **kwargs)
  File "/weka/scratch/djiang21/lm-eval-original/lm-evaluation-harness/lm_eval/models/api_models.py", line 331, in model_call
    response.raise_for_status()
  File "/weka/scratch/djiang21/miniconda/envs/lm_eval/lib/python3.10/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://api.openai.com/v1/chat/completions
Requesting API:   0%|                                                                      | 0/1319 [00:03<?, ?it/s]
```

Hi. You need to add `--apply_chat_template`. Bit unintuitive, I'll see about adding a warning.

Thanks for your reply! It's working now. However, it seems like those OpenAI calls are not parallelized, which takes too much time to run (e.g. 1 hour+ on GSM8K). Is there a way to parallelize the calling to OpenAI APIs?

Could someone please help with this question?

Have you tried using the `batch_size` or `num_concurrent` model kwargs (passable via adding to  `--model_args`) at all?