Repository: EleutherAI/lm-evaluation-harness
Issue: Error of `continuation_logprobs_dicts` is `None` when running with `vllm` on multi-choice tasks
Root cause: External Dependency Breakage

Error message:

```stderr
Traceback (most recent call last):
  File "/path/to/python/env/bin/lm_eval", line 8, in <module>
    sys.exit(cli_evaluate())
             ^^^^^^^^^^^^^^
  File "/data/user_data/yuxuanto/projects/pref-explore/repos/lm-evaluation-harness/lm_eval/__main__.py", line 382, in cli_evaluate
    results = evaluator.simple_evaluate(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/user_data/yuxuanto/projects/pref-explore/repos/lm-evaluation-harness/lm_eval/utils.py", line 397, in _wrapper
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/data/user_data/yuxuanto/projects/pref-explore/repos/lm-evaluation-harness/lm_eval/evaluator.py", line 296, in simple_evaluate
    results = evaluate(
              ^^^^^^^^^
  File "/data/user_data/yuxuanto/projects/pref-explore/repos/lm-evaluation-harness/lm_eval/utils.py", line 397, in _wrapper
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/data/user_data/yuxuanto/projects/pref-explore/repos/lm-evaluation-harness/lm_eval/evaluator.py", line 468, in evaluate
    resps = getattr(lm, reqtype)(cloned_reqs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/user_data/yuxuanto/projects/pref-explore/repos/lm-evaluation-harness/lm_eval/api/model.py", line 375, in loglikelihood
    return self._loglikelihood_tokens(new_reqs, disable_tqdm=disable_tqdm)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/user_data/yuxuanto/projects/pref-explore/repos/lm-evaluation-harness/lm_eval/models/vllm_causallms.py", line 448, in _loglikelihood_tokens
    answer = self._parse_logprobs(
             ^^^^^^^^^^^^^^^^^^^^^
  File "/data/user_data/yuxuanto/projects/pref-explore/repos/lm-evaluation-harness/lm_eval/models/vllm_causallms.py", line 493, in _parse_logprobs
    continuation_logprobs_dicts = [
                                  ^
TypeError: 'NoneType' object is not iterable
```

Minimal reproduction code

```shell
pip install -e ".[vllm]"
lm_eval --model vllm \
    --model_args "pretrained=meta-llama/Meta-Llama-3-8B-Instruct,tensor_parallel_size=1,dtype=auto,gpu_memory_utilization=0.9,data_parallel_size=1,max_model_len=4096" \
    --tasks "leaderboard_gpqa_diamond" \
    --batch_size "auto" \
    --output_path "output/meta-llama--Meta-Llama-3-8B-Instruct
```

Evironment information
```
$ pip show lm-eval-harness   
Name: lm_eval
Version: 0.4.3
Summary: A framework for evaluating language models
Home-page: 
Author: 
Author-email: EleutherAI <contact@eleuther.ai>
License: MIT
Location: /path/to/python/env/lib/python3.11/site-packages
Editable project location: /data/user_data/yuxuanto/projects/pref-explore/repos/lm-evaluation-harness
Requires: accelerate, datasets, dill, evaluate, evaluate, jsonlines, more-itertools, numexpr, peft, pybind11, pytablewriter, rouge-score, sacrebleu, scikit-learn, sqlitedict, torch, tqdm-multiprocess, transformers, word2number, zstandard
Required-by:
$ pip show vllm                                      
Name: vllm
Version: 0.4.2
Summary: A high-throughput and memory-efficient inference and serving engine for LLMs
Home-page: https://github.com/vllm-project/vllm
Author: vLLM Team
Author-email: 
License: Apache 2.0
Location: /path/to/python/env/lib/python3.11/site-packages
Requires: cmake, fastapi, filelock, lm-format-enforcer, ninja, numpy, nvidia-ml-py, openai, outlines, prometheus-client, prometheus-fastapi-instrumentator, psutil, py-cpuinfo, pydantic, ray, requests, sentencepiece, tiktoken, tokenizers, torch, transformers, typing-extensions, uvicorn, vllm-nccl-cu12, xformers
Required-by:
```




your vllm version is too old.