Repository: open-compass/opencompass
Issue: 如何在needlebench 上使用llama3 [Feature] 
Root cause: Documentation Deficiency

### Describe the feature

我的sh文件如下：
export CUDA_VISIBLE_DEVICES=2
python run.py --dataset needlebench_single_8k --models hf_llama3_8b_instruct --model-kwargs device_map='auto' trust_remote_code=True  --summarizer needlebench/needlebench_8k_summarizer --slurm -p partition_name -q reserved --max-num-workers 32 --max-partition-size 8000 --num-gpus 1

### Will you implement it?

- [ ] I would like to implement this feature and create a PR!

报错为 
/bin/sh: srun: command not found
/bin/sh: srun: command not found
/bin/sh: srun: command not found

您好，看起来您不在`Slurm`集群环境下中进行测试，可以尝试去掉`Slurm`相关的参数。尝试下面的参数：
```bash
python run.py --dataset needlebench_single_8k --models hf_llama3_8b_instruct --model-kwargs device_map='auto' trust_remote_code=True --summarizer needlebench/needlebench_8k_summarizer  --num-gpus 1
```

> 您好，看起来您不在`Slurm`集群环境下中进行测试，可以尝试去掉`Slurm`相关的参数。尝试下面的参数：
> 
> ```shell
> python run.py --dataset needlebench_single_8k --models hf_llama3_8b_instruct --model-kwargs device_map='auto' trust_remote_code=True --summarizer needlebench/needlebench_8k_summarizer  --num-gpus 1
> ```

感谢，这个问题解决了，但是推理出来的全是空白，这是报错
00Depth89_origin_zh_8k,llama-3-8b-instruct-hf/Length8000Depth94_origin_zh_8k,llama-3-8b-instruct-hf/Length8000Depth100_origin_zh_8k] on GPU 2
100%|█████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:36<00:00, 36.29s/it]
04/30 18:00:02 - OpenCompass - INFO - Partitioned into 160 tasks.
launch OpenICLEval[llama-3-8b-instruct-hf/Length5000Depth0_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length5000Depth5_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length5000Depth10_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length5000Depth15_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length5000Depth21_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length5000Depth26_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length5000Depth31_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length5000Depth36_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length5000Depth42_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length5000Depth47_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length5000Depth52_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length5000Depth57_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length5000Depth63_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length5000Depth68_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length5000Depth73_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length5000Depth78_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length5000Depth84_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length5000Depth89_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length5000Depth94_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length5000Depth100_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length6000Depth0_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length6000Depth5_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length6000Depth10_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length6000Depth15_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length6000Depth21_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length6000Depth26_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length6000Depth31_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length6000Depth36_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length6000Depth42_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length6000Depth47_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length6000Depth52_origin_en_8k] on CPU
launch OpenICLEval[llama-3-8b-instruct-hf/Length6000Depth57_origin_en_8k] on CPU
  2%|█▌                                                                                 | 3/160 [00:23<08:59,  3.44s/it]04/30 18:00:34 - OpenCompass - ERROR - /home/wumao/opencompass-main/opencompass/runners/local.py - _launch - 206 - task OpenICLEval[llama-3-8b-instruct-hf/Length5000Depth21_origin_en_8k] fail, see

这个不是报错信息，看一下错误日志里是什么bug信息呢，在.out文件里


> 这个不是报错信息，看一下错误日志里是什么bug信息呢，在.out文件里

报错为：
Traceback (most recent call last):
  File "/home/wumao/opencompass-main/opencompass/tasks/openicl_eval.py", line 366, in <module>
    inferencer.run()
  File "/home/wumao/opencompass-main/opencompass/tasks/openicl_eval.py", line 112, in run
    self._score()
  File "/home/wumao/opencompass-main/opencompass/tasks/openicl_eval.py", line 158, in _score
    preds = {k: [pred.get(k) for pred in preds] for k in preds[0]}
IndexError: list index out of range

> > 这个不是报错信息，看一下错误日志里是什么bug信息呢，在.out文件里
> 
> 报错为： Traceback (most recent call last): File "/home/wumao/opencompass-main/opencompass/tasks/openicl_eval.py", line 366, in inferencer.run() File "/home/wumao/opencompass-main/opencompass/tasks/openicl_eval.py", line 112, in run self._score() File "/home/wumao/opencompass-main/opencompass/tasks/openicl_eval.py", line 158, in _score preds = {k: [pred.get(k) for pred in preds] for k in preds[0]} IndexError: list index out of range

解决了 之前数据放错位置了

好的，如果还有其他问题或需要帮助，随时告诉我。祝好运！

> > > 这个不是报错信息，看一下错误日志里是什么bug信息呢，在.out文件里
> > 
> > 
> > 报错为： Traceback (most recent call last): File "/home/wumao/opencompass-main/opencompass/tasks/openicl_eval.py", line 366, in inferencer.run() File "/home/wumao/opencompass-main/opencompass/tasks/openicl_eval.py", line 112, in run self._score() File "/home/wumao/opencompass-main/opencompass/tasks/openicl_eval.py", line 158, in _score preds = {k: [pred.get(k) for pred in preds] for k in preds[0]} IndexError: list index out of range
> 
> 解决了 之前数据放错位置了

我也遇到了相同的问题，请问这个报错是如何解决的？