Repository: open-compass/opencompass
Issue: [Bug] LMDeploy TypeError: cannot unpack non-iterable EngineOutput object
Root cause: External Dependency Breakage

### 先决条件

- [X] 我已经搜索过 [问题](https://github.com/open-compass/opencompass/issues/) 和 [讨论](https://github.com/open-compass/opencompass/discussions) 但未得到预期的帮助。
- [x] 错误在 [最新版本](https://github.com/open-compass/opencompass) 中尚未被修复。

### 问题类型

我修改了代码（配置不视为代码），或者我正在处理我自己的任务/模型/数据集。

### 环境

```
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.5 LTS"
Python 3.10.14
OpenCompass 02e7eec91131cec2050a0bfee3eb07f37298491f Mar 28, 2024
```

### 重现问题 - 代码/配置示例

```
from mmengine.config import read_base

with read_base():
    from .datasets.subjective.multiround.mtbench_single_judge_diff_temp import subjective_datasets

from opencompass.models import HuggingFaceCausalLM, HuggingFace, HuggingFaceChatGLM3, OpenAI, SkyOpenAI, LmdeployPytorchModel
from opencompass.models.openai_api import OpenAIAllesAPIN
from opencompass.partitioners import NaivePartitioner, SizePartitioner
from opencompass.partitioners.sub_naive import SubjectiveNaivePartitioner
from opencompass.partitioners.sub_size import SubjectiveSizePartitioner
from opencompass.runners import LocalRunner
from opencompass.runners import SlurmSequentialRunner
from opencompass.tasks import OpenICLInferTask
from opencompass.tasks.subjective_eval import SubjectiveEvalTask
from opencompass.summarizers import MTBenchSummarizer
from opencompass.models.turbomind import TurboMindModel

# -------------Inference Stage ----------------------------------------
# For subjective evaluation, we often set do sample for models
from opencompass.models import VLLM


GPU_NUMS = 4

_meta_template = dict(
    round=[
        dict(role="HUMAN", begin='
user\n', end='
\n'),
        dict(role="BOT", begin="
assistant\n", end='
\n', generate=True),
    ],
    eos_token_id=151645,
)


models = [
    dict(
        type=VLLM,
        abbr='Qwen1.5-14B-Chat',
        path='/xxx/Qwen1.5-14B-Chat',
        model_kwargs=dict(tensor_parallel_size=GPU_NUMS),
        meta_template=_meta_template,
        max_out_len=100,
        max_seq_len=2048,
        batch_size=GPU_NUMS * 8,
        generation_kwargs=dict(temperature=0.1, top_p=0.9),
        # end_str='
',
        end_str='
',
        run_cfg=dict(num_gpus=GPU_NUMS, num_procs=1),
    )
]

datasets = [*subjective_datasets]

# -------------Evalation Stage ----------------------------------------

## ------------- JudgeLLM Configuration

judge_models = [
    dict(
        type=LmdeployPytorchModel,
        abbr='Qwen1.5-72B-Chat',
        path='/xxx/Qwen1.5-72B-Chat',
        engine_config=dict(session_len=2048,
                        max_batch_size=8,
                        tp=GPU_NUMS,
                        cache_max_entry_count=0.2),
        gen_config=dict(top_p=0.9,
                        temperature=0.1,
                        max_new_tokens=1024),
        max_out_len=1024,
        max_seq_len=2048,
        concurrency=8,
        meta_template=_meta_template,
        batch_size=8,
        end_str='
',
        run_cfg=dict(num_gpus=GPU_NUMS, num_procs=1),
    )
]

## single evaluation
eval = dict(
    partitioner=dict(type=SubjectiveSizePartitioner, strategy='split', max_task_size=10000, mode='singlescore', models=models, judge_models=judge_models),
    runner=dict(type=LocalRunner, max_num_workers=32, task=dict(type=SubjectiveEvalTask)),
)

summarizer = dict(type=MTBenchSummarizer, judge_type='single')

work_dir = 'outputs/mtbench/'

```

### 重现问题 - 命令或脚本

```
python run.py config/xxx.py --dump-eval-details --debug
```

### 重现问题 - 错误信息

```
2024-04-25 16:22:04,774 - lmdeploy - INFO - build CacheEngine with config:CacheConfig(block_size=64, num_cpu_blocks=102, num_gpu_blocks=178, window_size=-1, cache_max_entry_count=0.2, max_prefill_token_num=4096)
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
[2024-04-25 16:22:08,710] [opencompass.openicl.icl_inferencer.icl_gen_inferencer] [INFO] Starting inference process...
  0%|          | 0/5 [00:33<?, ?it/s]
Traceback (most recent call last):
  File "/maindata/data/shared/Security-SFT/cmz/opencompass/opencompass/tasks/subjective_eval.py", line 439, in <module>
    inferencer.run()
  File "/maindata/data/shared/Security-SFT/cmz/opencompass/opencompass/tasks/subjective_eval.py", line 94, in run
    self._score(model_cfg, dataset_cfg, eval_cfg, output_column,
  File "/maindata/data/shared/Security-SFT/cmz/opencompass/opencompass/tasks/subjective_eval.py", line 371, in _score
    result = icl_evaluator.score(predictions=model_preds,
  File "/maindata/data/shared/Security-SFT/cmz/opencompass/opencompass/openicl/icl_evaluator/lm_evaluator.py", line 229, in score
    self.inferencer.inference(retriever=retriever,
  File "/maindata/data/shared/Security-SFT/cmz/opencompass/opencompass/openicl/icl_inferencer/icl_gen_inferencer.py", line 146, in inference
    results = self.model.generate_from_template(
  File "/maindata/data/shared/Security-SFT/cmz/opencompass/opencompass/models/base.py", line 165, in generate_from_template
    return self.generate(inputs, max_out_len=max_out_len, **kwargs)
  File "/maindata/data/shared/Security-SFT/cmz/opencompass/opencompass/models/lmdeploy_pytorch.py", line 101, in generate
    _results = list(
  File "/maindata/data/shared/Security-SFT/common_tools/mambaforge/envs/opencompass/lib/python3.10/concurrent/futures/_base.py", line 621, in result_iterator
    yield _result_or_cancel(fs.pop())
  File "/maindata/data/shared/Security-SFT/common_tools/mambaforge/envs/opencompass/lib/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel
    return fut.result(timeout)
  File "/maindata/data/shared/Security-SFT/common_tools/mambaforge/envs/opencompass/lib/python3.10/concurrent/futures/_base.py", line 458, in result
    return self.__get_result()
  File "/maindata/data/shared/Security-SFT/common_tools/mambaforge/envs/opencompass/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/maindata/data/shared/Security-SFT/common_tools/mambaforge/envs/opencompass/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/maindata/data/shared/Security-SFT/cmz/opencompass/opencompass/models/lmdeploy_pytorch.py", line 148, in _generate
    _, output_ids, _ = generator.infer(session_id,
TypeError: cannot unpack non-iterable EngineOutput object
eflops29:82577:83526 [0] NCCL INFO [Service thread] Connection closed by localRank 0
eflops29:82577:82577 [0] NCCL INFO comm 0x16bd53f0 rank 0 nranks 4 cudaDev 0 busId 10000 - Abort COMPLETE
[2024-04-25 16:22:50,429] torch.distributed.elastic.multiprocessing.api: [ERROR] failed (exitcode: 1) local_rank: 0 (pid: 82577) of binary: /maindata/data/shared/Security-SFT/common_tools/mambaforge/envs/opencompass/bin/python
Traceback (most recent call last):
  File "/maindata/data/shared/Security-SFT/common_tools/mambaforge/envs/opencompass/bin/torchrun", line 8, in <module>
    sys.exit(main())
  File "/maindata/data/shared/Security-SFT/common_tools/mambaforge/envs/opencompass/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 346, in wrapper
    return f(*args, **kwargs)
  File "/maindata/data/shared/Security-SFT/common_tools/mambaforge/envs/opencompass/lib/python3.10/site-packages/torch/distributed/run.py", line 806, in main
    run(args)
  File "/maindata/data/shared/Security-SFT/common_tools/mambaforge/envs/opencompass/lib/python3.10/site-packages/torch/distributed/run.py", line 797, in run
    elastic_launch(
  File "/maindata/data/shared/Security-SFT/common_tools/mambaforge/envs/opencompass/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 134, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/maindata/data/shared/Security-SFT/common_tools/mambaforge/envs/opencompass/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 264, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
============================================================
/maindata/data/shared/Security-SFT/cmz/opencompass/opencompass/tasks/subjective_eval.py FAILED
------------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2024-04-25_16:22:50
  host      : eflops29
  rank      : 0 (local_rank: 0)
  exitcode  : 1 (pid: 82577)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================
```

### 其他信息

_No response_

Do you updated lmdeploy to 0.4.0? This may caused by the version diff with opencompass, you can try to reinstall lmdeploy==0.3.0 or change ` _, output_ids, _ = generator.infer(session_id,` to ` output_ids = generator.infer(session_id,`

The newest PR fix this problem