Repository: modelscope/evalscope
Issue: ifeval数据集做评测的时候报-Unable to detect language for text  कामाची घाई
Root cause: Environment Incompatibility

## 问题描述 / Issue Description

请简要描述您遇到的问题。 / Please briefly describe the issue you encountered.

## 使用的工具 / Tools Used
- [ ] Native / 原生框架


## 执行的代码或指令 / Code or Commands Executed

请提供您执行的主要代码或指令。 / Please provide the main code or commands you executed. 例如 / For example:



```bash
# 例如：在终端中执行的指令 / Terminal command executed
!evalscope eval \
 --model xxxx \
 --api-url xxxx \
 --api-key xxxxx \
 --generation-config temperature=1.0 \
 --eval-type service \
 --eval-batch-size 50  \
 --datasets  ifeval
```

## 错误日志 / Error Log

请粘贴完整的错误日志或控制台输出。 / Please paste the full error log or console output. 例如 / For example:

```
Predicting(default): 100%|████████████████████| 541/541 [05:56<00:00,  1.52it/s]
2025-03-17 09:48:06,505 - evalscope - INFO - Dump predictions to [./outputs/20250317_094202/predictions/ernie-3.5-8k/ifeval_default.jsonl.](https://vscode-remote+wsl-002bubuntu-002d22-002e04.vscode-resource.vscode-cdn.net/mnt/f/workspace/notebook/evalscope/outputs/20250317_094202/predictions/ernie-3.5-8k/ifeval_default.jsonl.)
Reviewing(default):   0%|                               | 0/541 [00:00<?, ?it/s]2025-03-17 09:48:06,539 - root - ERROR - Unable to detect language for text what kind of toys do the two young boys have? due to Need to load profiles.
2025-03-17 09:48:06,539 - root - ERROR - Unable to detect language for text कामाची घाई,
सकाळचा वेळ फेकतो,
हाळणा心性留下一. due to Need to load profiles.
2025-03-17 09:48:06,540 - root - ERROR - Unable to detect language for text what kind of toys do the two young boys have? due to Need to load profiles.
2025-03-17 09:48:06,540 - root - ERROR - Unable to detect language for text uncovering the essence of sophistication, one story at a time. stay tuned for our latest issue. #playboylifestyle #entertainment #styleinspiration #culturevibes due to Need to load profiles.
2025-03-17 09:48:06,540 - root - ERROR - Unable to detect language for text in the ebb and flow of fate, john of brienne found his crown. here is how, in a zen-like manner:

[your name] due to No features in text.
Reviewing(default):   1%|▎                      | 6/541 [00:00<00:12, 43.51it/s]2025-03-17 09:48:06,550 - root - ERROR - Unable to detect language for text WHEN IT COMES TO SNEAKERS, NIKE IS GENERALLY CONSIDERED A BETTER BRAND THAN PRADA. NIKE IS WELL-KNOWN FOR MAKING HIGH-QUALITY, DURABLE, AND STYLISH SNEAKERS THAT CATER TO A WIDE RANGE OF CONSUMERS. PRADA, ON THE OTHER HAND, IS MORE KNOWN FOR ITS LUXURY FASHION ITEMS AND TENDS TO FOCUS LESS ON THE SNEAKER MARKET. IF YOU'RE LOOKING FOR A RELIABLE AND POPULAR BRAND FOR SNEAKERS, NIKE IS A SAFE BET. NIKE SNEAKERS ARE OFTEN REGARDED AS SOME OF THE BEST IN THE INDUSTRY, OFFERING A VARIETY OF STYLES AND FEATURES THAT MAKE THEM DESIRABLE AMONG SNEAKER ENTHUSIASTS. NIKE CONSISTENTLY RELEASES INNOVATIVE AND TRENDY SNEAKERS THAT APPEAL TO ATHLETES AND FASHION LOVERS ALIKE. WHEN COMPARING PRADA AND NIKE IN TERMS OF SNEAKERS, NIKE HANDS DOWN OFFERS MORE VARIETY, BETTER VALUE, AND A STRONGER REPUTATION IN THE SNEAKER WORLD. IN SUMMARY, FOR THOSE WHO CARE ABOUT SNEAKERS, NIKE IS THE CLEAR CHOICE OVER PRADA. P.S. REMEMBER, NIKE IS THE GO-TO BRAND WHEN IT COMES TO SNEAKERS. due to No features in text.
Reviewing(default):   2%|▌                     | 13/541 [00:00<00:25, 20.37it/s]
2025-03-17 09:48:07,324 - root - ERROR - Unable to detect language for text ****** due to No features in text.
Traceback (most recent call last):
  File "/home/xuwei/miniconda3/envs/evalscope/bin/evalscope", line 8, in <module>
    sys.exit(run_cmd())
  File "/home/xuwei/miniconda3/envs/evalscope/lib/python3.10/site-packages/evalscope/cli/cli.py", line 25, in run_cmd
    cmd.execute()
  File "/home/xuwei/miniconda3/envs/evalscope/lib/python3.10/site-packages/evalscope/cli/start_eval.py", line 32, in execute
    run_task(self.args)
  File "/home/xuwei/miniconda3/envs/evalscope/lib/python3.10/site-packages/evalscope/run.py", line 31, in run_task
    return run_single_task(task_cfg, run_time)
  File "/home/xuwei/miniconda3/envs/evalscope/lib/python3.10/site-packages/evalscope/run.py", line 44, in run_single_task
    return evaluate_model(task_cfg, outputs)
  File "/home/xuwei/miniconda3/envs/evalscope/lib/python3.10/site-packages/evalscope/run.py", line 117, in evaluate_model
    res_dict = evaluator.eval()
  File "/home/xuwei/miniconda3/envs/evalscope/lib/python3.10/site-packages/evalscope/evaluator/evaluator.py", line 414, in eval
    reviews_list: list = self.get_reviews(subset_name=subset_name, answers_list=answers_list, **kwargs)
  File "/home/xuwei/miniconda3/envs/evalscope/lib/python3.10/site-packages/evalscope/evaluator/evaluator.py", line 299, in get_reviews
    review_d = future.result()
  File "/home/xuwei/miniconda3/envs/evalscope/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/home/xuwei/miniconda3/envs/evalscope/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/home/xuwei/miniconda3/envs/evalscope/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/xuwei/miniconda3/envs/evalscope/lib/python3.10/site-packages/evalscope/evaluator/evaluator.py", line 289, in process_single_review
    review_d = self._get_review(answer_d=answer_d, review_id=review_id, reviewer_spec=reviewer_spec)
  File "/home/xuwei/miniconda3/envs/evalscope/lib/python3.10/site-packages/evalscope/evaluator/evaluator.py", line 228, in _get_review
    review_result = self.data_adapter.match(gold_content, pred_content)
  File "/home/xuwei/miniconda3/envs/evalscope/lib/python3.10/site-packages/evalscope/benchmarks/ifeval/ifeval_adapter.py", line 47, in match
    return process_results(gold, [pred])
  File "/home/xuwei/miniconda3/envs/evalscope/lib/python3.10/site-packages/evalscope/benchmarks/ifeval/utils.py", line 120, in process_results
    out_strict = test_instruction_following_strict(inp, response)
  File "/home/xuwei/miniconda3/envs/evalscope/lib/python3.10/site-packages/evalscope/benchmarks/ifeval/utils.py", line 43, in test_instruction_following_strict
    if response.strip() and instruction.check_following(response):
  File "/home/xuwei/miniconda3/envs/evalscope/lib/python3.10/site-packages/evalscope/benchmarks/ifeval/instructions.py", line 235, in check_following
    num_sentences = instructions_util.count_sentences(value)
  File "/home/xuwei/miniconda3/envs/evalscope/lib/python3.10/site-packages/evalscope/benchmarks/ifeval/instructions_util.py", line 1663, in count_sentences
    tokenizer = _get_sentence_tokenizer()
  File "/home/xuwei/miniconda3/envs/evalscope/lib/python3.10/site-packages/evalscope/benchmarks/ifeval/instructions_util.py", line 1658, in _get_sentence_tokenizer
    return nltk.data.load('nltk:tokenizers/punkt/english.pickle')
  File "/home/xuwei/miniconda3/envs/evalscope/lib/python3.10/site-packages/nltk/data.py", line 823, in load
    return switch_punkt(fil)
  File "/home/xuwei/miniconda3/envs/evalscope/lib/python3.10/site-packages/nltk/data.py", line 678, in switch_punkt
    return tok(lang)
  File "/home/xuwei/miniconda3/envs/evalscope/lib/python3.10/site-packages/nltk/tokenize/punkt.py", line 1744, in __init__
    self.load_lang(lang)
  File "/home/xuwei/miniconda3/envs/evalscope/lib/python3.10/site-packages/nltk/tokenize/punkt.py", line 1750, in load_lang
    self._params = load_punkt_params(lang_dir)
  File "/home/xuwei/miniconda3/envs/evalscope/lib/python3.10/site-packages/nltk/tokenize/punkt.py", line 1763, in load_punkt_params
    with open(f"{lang_dir}/collocations.tab", encoding="utf-8") as f:
NotADirectoryError: [Errno 20] Not a directory: '/home/xuwei/nltk_data/tokenizers/punkt_tab.zip/punkt_tab/english/collocations.tab'
```

## 运行环境 / Runtime Environment

- 操作系统 / Operating System:
  - [ ] Ubuntu

- Python版本 / Python Version:
  - [ ] 3.10



## 其他信息 / Additional Information

如果有其他相关信息，请在此处提供。 / If there is any other relevant information, please provide it here.


看起来是nltk的问题，参考这个解决方案：https://github.com/modelscope/evalscope/issues/344#issuecomment-2689629917

然后这个数据集报错`Need to load profiles`是多线程的问题，参考：https://github.com/Mimino666/langdetect/issues/65

解决方法是添加下面的设置，只使用一个线程：
```
--judge-worker-num 1
```

解决方案
1. unzip  /home/xuwei/nltk_data/tokenizers/punkt_tab.zip
2. 命令如下
!evalscope eval \
 --model xxxx \
 --api-url xxxx \
 --api-key xxxxx \
 --generation-config temperature=1.0 \
 --eval-type service \
 --eval-batch-size 50  \
 --datasets  ifeval  \
 --judge-worker-num 1