Repository: huggingface/lighteval
Issue: [BUG] Cannot list tasks if the user doesn't install multilingual extras
Root cause: Validation Gap

## Describe the bug
The following command will show the tasks supported by lighteval
```
lighteval tasks list
```

Because [multilingual] is an optional-dependencies in pyproject.toml, the users may expect the list command can work successfully without install with `pip install lighteval[multilingual]`. However, it will raise ModuleNotFoundError. After install langcodes, the problem still cannot be resolved. 

```
╭────────────────────────────────────────────── Traceback (most recent call last) ───────────────────────────────────────────────╮
│ /home/elichen/projects/lighteval_pr/lighteval-fork/src/lighteval/main_tasks.py:78 in list                                      │
│                                                                                                                                │
│   75 │   """List all tasks"""                                                                                                  │
│   76 │   from lighteval.tasks.registry import Registry                                                                         │
│   77 │                                                                                                                         │
│ ❱ 78 │   registry = Registry(custom_tasks=custom_tasks, load_community=True, load_extended=Tr                                  │
│   79 │   registry.print_all_tasks(suites=suites)                                                                               │
│   80                                                                                                                           │
│   81                                                                                                                           │
│                                                                                                                                │
│ /home/elichen/projects/lighteval_pr/lighteval-fork/src/lighteval/tasks/registry.py:172 in __init__                             │
│                                                                                                                                │
│   169 │   │   self._activate_loading_of_optional_suite()  # we dynamically set the loading par                                 │
│   170 │   │                                                                                                                    │
│   171 │   │   # We load all task to                                                                                            │
│ ❱ 172 │   │   self._task_registry = self._load_full_registry()                                                                 │
│   173 │   │                                                                                                                    │
│   174 │   │   self.task_to_configs = self._update_task_configs()                                                               │
│   175                                                                                                                          │
│                                                                                                                                │
│ /home/elichen/projects/lighteval_pr/lighteval-fork/src/lighteval/tasks/registry.py:265 in _load_full_registry                  │
│                                                                                                                                │
│   262 │   │                                                                                                                    │
│   263 │   │   # Need to load multilingual tasks                                                                                │
│   264 │   │   if self._load_multilingual:                                                                                      │
│ ❱ 265 │   │   │   import lighteval.tasks.multilingual.tasks as multilingual_tasks                                              │
│   266 │   │   │                                                                                                                │
│   267 │   │   │   custom_tasks_module.append(multilingual_tasks)                                                               │
│   268                                                                                                                          │
│                                                                                                                                │
│ /home/elichen/projects/lighteval_pr/lighteval-fork/src/lighteval/tasks/multilingual/tasks.py:26 in <module>                    │
│                                                                                                                                │
│     23 from functools import partial                                                                                           │
│     24 from itertools import permutations                                                                                      │
│     25                                                                                                                         │
│ ❱   26 from langcodes import Language as LangCodeLanguage                                                                      │
│     27 from langcodes import standardize_tag                                                                                   │
│     28                                                                                                                         │
│     29 from lighteval.metrics.dynamic_metrics import (                                                                         │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
ModuleNotFoundError: No module named 'langcodes'
``` 

## To Reproduce
Just install the lighteval with the following commmand with main branch
```
pip install -e lighteval[vllm,math]
```

And run the list tasks command
```
lighteval tasks list

or

lighteval tasks list --suites harness  # cannot work too.
```

## Expected behavior
I expect the list command should work without [multilingual] installation

For example,
```
lighteval tasks list --suites harness # This command should work
```

## Version info
Operating System
==============
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.5 LTS
Release:	22.04
Codename:	jammy

lighteval version
============
3a71f68869bb2b9da12e318ad0900a1a30dffde5 (current main)

I think this problem comes from `def _load_full_registry` in `lighteval-fork/src/lighteval/tasks/registry.py`

The workaround could be
```
        # Need to load multilingual tasks
        if self._load_multilingual:
            try:
                import lighteval.tasks.multilingual.tasks as multilingual_tasks  # <--- Error raised here.
                custom_tasks_module.append(multilingual_tasks)
            except ImportError:
                pass  # leave some message
```

i am also facing the same issue. what's the fix? do we really need to install vllm via lighteval only? in my env i already had vllm so it should have worked but it didn't. Then tried creating a new env and getting this error: 

python -m pip install -e lighteval[vllm,math]
ERROR: lighteval[vllm,math] is not a valid editable requirement. It should either be a path to a local project or a VCS URL (beginning with bzr+http, bzr+https, bzr+ssh, bzr+sftp, bzr+ftp, bzr+lp, bzr+file, git+http, git+https, git+ssh, git+git, git+file, hg+file, hg+http, hg+https, hg+ssh, hg+static-http, svn+ssh, svn+http, svn+https, svn+svn, svn+file).

@elichen3051 good catch! yes, we assume that to list all tasks, all deps are available - do you want to open a PR with your fix?

Hi, I believe that the list tasks is not showing properly due to the missing suite prefixes. To still print the tasks without the prefix, I created [this PR](https://github.com/huggingface/lighteval/pull/1087).

hey @bram-pramono ! thanks for the fix, i reviewed and we will merge shortly :)