Repository: EleutherAI/lm-evaluation-harness
Issue: Issue with `truthfulqa_fr_mc1`, `truthfulqa_fr_mc2`, `lambada_mt_fr`, `mgsm_cot_native_fr`
Root cause: Documentation Deficiency

While trying to evaluate on the following tasks, I got the following errors:

* `truthful_qa_fr_mc1` and `truthful_qa_fr_mc2`

```
Traceback (most recent call last):
  File "/opt/conda/envs/fr-de-lb/bin/lm_eval", line 8, in <module>
    sys.exit(cli_evaluate())
  File "/home/ayushthakur/lm-eval/llm-leaderboard-fr-de/lm-evaluation-harness/lm_eval/__main__.py", line 288, in cli_evaluate
    results = evaluator.simple_evaluate(
  File "/home/ayushthakur/lm-eval/llm-leaderboard-fr-de/lm-evaluation-harness/lm_eval/utils.py", line 288, in _wrapper
    return fn(*args, **kwargs)
  File "/home/ayushthakur/lm-eval/llm-leaderboard-fr-de/lm-evaluation-harness/lm_eval/evaluator.py", line 154, in simple_evaluate
    task_dict = get_task_dict(tasks, task_manager)
  File "/home/ayushthakur/lm-eval/llm-leaderboard-fr-de/lm-evaluation-harness/lm_eval/tasks/__init__.py", line 412, in get_task_dict
    task_name_from_string_dict = task_manager.load_task_or_group(string_task_name_list)
  File "/home/ayushthakur/lm-eval/llm-leaderboard-fr-de/lm-evaluation-harness/lm_eval/tasks/__init__.py", line 253, in load_task_or_group
    collections.ChainMap(
  File "/home/ayushthakur/lm-eval/llm-leaderboard-fr-de/lm-evaluation-harness/lm_eval/tasks/__init__.py", line 164, in _load_individual_task_or_group
    return load_task(task_config, task=name_or_config, group=parent_name)
  File "/home/ayushthakur/lm-eval/llm-leaderboard-fr-de/lm-evaluation-harness/lm_eval/tasks/__init__.py", line 153, in load_task
    task_object = ConfigurableTask(config=config)
  File "/home/ayushthakur/lm-eval/llm-leaderboard-fr-de/lm-evaluation-harness/lm_eval/api/task.py", line 711, in __init__
    if self.fewshot_docs() is not None:
  File "/home/ayushthakur/lm-eval/llm-leaderboard-fr-de/lm-evaluation-harness/lm_eval/api/task.py", line 835, in fewshot_docs
    return super().fewshot_docs()
  File "/home/ayushthakur/lm-eval/llm-leaderboard-fr-de/lm-evaluation-harness/lm_eval/api/task.py", line 306, in fewshot_docs
    return self.validation_docs()
  File "/home/ayushthakur/lm-eval/llm-leaderboard-fr-de/lm-evaluation-harness/lm_eval/api/task.py", line 813, in validation_docs
    self.dataset[self.config.validation_split]
  File "/opt/conda/envs/fr-de-lb/lib/python3.10/site-packages/datasets/dataset_dict.py", line 74, in __getitem__
    return super().__getitem__(k)
KeyError: 'validation'
```

* `lambada_mt_fr`

```
Traceback (most recent call last):
  File "/opt/conda/envs/fr-de-lb/bin/lm_eval", line 8, in <module>
    sys.exit(cli_evaluate())
  File "/home/ayushthakur/lm-eval/llm-leaderboard-fr-de/lm-evaluation-harness/lm_eval/__main__.py", line 263, in cli_evaluate
    raise ValueError(
ValueError: Tasks not found: lambada_mt_fr. Try `lm-eval --tasks list` for list of available tasks, or '--verbosity DEBUG' to troubleshoot task registration issues.
```

* `mgsm_cot_native_fr`

```
Traceback (most recent call last):
  File "/opt/conda/envs/fr-de-lb/bin/lm_eval", line 8, in <module>
    sys.exit(cli_evaluate())
  File "/home/ayushthakur/lm-eval/llm-leaderboard-fr-de/lm-evaluation-harness/lm_eval/__main__.py", line 263, in cli_evaluate
    raise ValueError(
ValueError: Tasks not found: mgsm_cot_native_fr. Try `lm-eval --tasks list` for list of available tasks, or '--verbosity DEBUG' to troubleshoot task registration issues.
```

cc: @haileyschoelkopf 





Hey @ayulockin , thanks for reporting this! Let me check it out today and let you know what's going on.

For `truthfulqa_*`: pushed a fix here: #1488

For `lambada_mt_fr`: the correct task name is `lambada_openai_mt_fr`, sorry for confusion!




For `mgsm_cot_native_fr`: the correct task name is `mgsm_native_cot_fr`, also sorry for any confusion here!

Thanks for the updates @haileyschoelkopf 🌮