Repository: modelscope/evalscope
Issue: 测评llama2-7b  bbh出错
Root cause: Validation Gap

报错：
Traceback (most recent call last):
  File "/home/anaconda3/envs/evalscope/bin/evalscope", line 33, in <module>
    sys.exit(load_entry_point('evalscope', 'console_scripts', 'evalscope')())
  File "/home/evalscope/evalscope/cli/cli.py", line 23, in run_cmd
    cmd.execute()
  File "/home//evalscope/evalscope/cli/start_eval.py", line 31, in execute
    run_task(self.args)
  File "/home//evalscope/evalscope/run.py", line 36, in run_task
    return run_single_task(task_cfg, run_time)
  File "/home//evalscope/evalscope/run.py", line 51, in run_single_task
    return evaluate_model(task_cfg, outputs)
  File "/home//evalscope/evalscope/run.py", line 109, in evaluate_model
    res_dict = evaluator.eval(infer_cfg=task_cfg.generation_config, debug=task_cfg.debug, limit=task_cfg.limit)
  File "/home//evalscope/evalscope/evaluator/evaluator.py", line 387, in eval
    answers_list: list = self.get_answers(
  File "/home///evalscope/evalscope/evaluator/evaluator.py", line 188, in get_answers
    answer_d: dict = self._pred_answer(
  File "/home//evalscope/evalscope/evaluator/evaluator.py", line 95, in _pred_answer
    ans: dict = self.model_adapter.predict(inputs=input_d, infer_cfg=infer_cfg)
  File "/home/anaconda3/envs/evalscope/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/home//evalscope/evalscope/models/model_adapter.py", line 450, in predict
    response = self._model_generate(query, infer_cfg)
  File "/home//evalscope/evalscope/models/model_adapter.py", line 411, in _model_generate
    formatted_prompt = self.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
  File "/home//anaconda3/envs/evalscope/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1617, in apply_chat_template
    chat_template = self.get_chat_template(chat_template, tools)
  File "/home/anaconda3/envs/evalscope/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1785, in get_chat_template
    raise ValueError(
ValueError: Cannot use chat template functions because tokenizer.chat_template is not set and no template argument was passed! For information about writing templates and setting the tokenizer.chat_template attribute, please see the documentation at https://huggingface.co/docs/transformers/main/en/chat_templating


请使用 Llama-2-7b-chat 模型来评测；或者自定义提供template，例如
```bash
--chat-template "{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{% set system_message = messages[0]['content'] %}{% else %}{% set loop_messages = messages %}{% set system_message = false %}{% endif %}{% for message in loop_messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if loop.index0 == 0 and system_message != false %}{% set content = '
\\n' + system_message + '\\n
\\n\\n' + message['content'] %}{% else %}{% set content = message['content'] %}{% endif %}{% if message['role'] == 'user' %}{{ bos_token + '
 ' + content.strip() + ' 
' }}{% elif message['role'] == 'assistant' %}{{ ' '  + content.strip() + ' ' + eos_token }}{% endif %}{% endfor %}"
```

我们再修改一下这个部分，对于没有chat template的base model去掉apply_chat_template

EvalScope v0.9.0 已发布，可以看看这个问题解决了吗

bbh是否支持使用vllm推理加速呢，现在测评的时间超过2h

可以使用vllm拉起服务来评测，参考：https://evalscope.readthedocs.io/zh-cn/latest/get_started/basic_usage.html#api

> 由于长时间未收到活动，我们将关闭此问题。如果您有任何疑问，请随时重新打开它。如果EvalScope对您有所帮助，欢迎给我们点个STAR以示支持，谢谢！