Repository: huggingface/lighteval
Issue: [BUG] Compatibility issue with datasets>=3.5.0: trust_remote_code parameter not accepted
Root cause: External Dependency Breakage

# Bug Report: `trust_remote_code` compatibility issue with custom tasks

## Problem  
LightEval fails to load custom tasks with newer versions of the datasets library due to an incompatible parameter.  
  
## Reproduction  
1. Install LightEval with datasets>=3.5.0  
2. Try to load custom tasks using `--custom-tasks` parameter  
3. Error occurs during task loading  

## Expected behavior

When using `custom_tasks_directory` parameter in `PipelineParameters`, LightEval should successfully load custom task modules from Python files without requiring the `trust_remote_code=True` parameter that is incompatible with newer versions of the HuggingFace datasets library.

The expected flow should be:
1. User specifies `custom_tasks_directory="path/to/tasks.py"` in `PipelineParameters`
2. LightEval loads the custom tasks module using `importlib.import_module()` or similar
3. Custom tasks are registered and available for evaluation
4. Evaluation proceeds normally

## Root cause

The issue is in `src/lighteval/tasks/registry.py` in the `create_custom_tasks_module` function:

```python
def create_custom_tasks_module(custom_tasks: Union[str, Path, ModuleType]) -> ModuleType:
    # ... other code ...
    if isinstance(custom_tasks, (str, Path)) and os.path.exists(custom_tasks):
        dataset_module = dataset_module_factory(str(custom_tasks), trust_remote_code=True)  # ← This line causes the error
        return importlib.import_module(dataset_module.module_path)
```

The `trust_remote_code=True` parameter was added in commit [d9dced1d](https://github.com/huggingface/lighteval/commit/d9dced1d) by Clémentine Fourrier on 2024-10-17, but it's incompatible with newer versions of the datasets library.

## Version info

- **Operating System**: Windows 10 (10.0.26100)
- **LightEval version**: 0.10.0 (latest available)
- **HuggingFace datasets version**: 4.1.0 (latest available)
- **Transformers version**: 4.56.1
- **Python environment**: conda environment with Python 3.12

## Proposed Solution  
Modify the `create_custom_tasks_module` function to handle the parameter gracefully with a try/catch approach.  

Hi! If you want to use the latest datasets version, you should build lighteval from main, as it relies on `datasets>=4.0` (see [here](https://github.com/huggingface/lighteval/blob/main/pyproject.toml), and the PR adding the upgrade by @NathanHB [here](https://github.com/huggingface/lighteval/pull/924)) so we should have fixed this issue. (The code you are referring to also no longer exists as far as I can tell)

Can you try this and tell us if it works for you? If not, can you provide a MWE?

Thank you for the clarification! You're absolutely right - I should have checked the main branch first.
I've tested with the latest version from main (pip install git+https://github.com/huggingface/lighteval.git) and can confirm that the trust_remote_code issue has been fixed! 
Sorry for the troubles :)

No problem, thanks for using the lib! 

@NathanHB when do you wanna plan the next release? I feel we have enough fixes to publish stg

yeah planning it now