Repository: huggingface/lighteval
Issue: [BUG] ValueError: You can't create a model without either a list of model_args or a model_config_path.
Root cause: Validation Gap

## Describe the bug
Trying to run the evaluation using quantization model/ inference endpoint, because the GPU method causes cuda out of memory. However, I am unable to use the model_config_path in the argument, value error is returned.

## To Reproduce
1. Git clone the project
2. Pip install . in virtual environment
3. Modified yaml files:

- endpoint_model.yaml (this file is stored in examples/model_configs)
```
model:
  type: "endpoint" # can be base, tgi, or endpoint
  base_params:
    endpoint_name: "llama-3-storm" # needs to be lower case without special characters
    model: "akjindal53244/Llama-3.1-Storm-8B"
    revision: "main"
    dtype: "bfloat16" # can be any of "awq", "eetq", "gptq", "4bit' or "8bit" (will use bitsandbytes), "bfloat16" or "float16"
    reuse_existing: false # if true, ignore all params in instance, and don't delete the endpoint after evaluation
  instance:
    accelerator: "gpu"
    region: "eu-west-1"
    vendor: "aws"
    instance_size: "medium"
    instance_type: "g5.2xlarge"
    framework: "pytorch"
    endpoint_type: "protected"
    namespace: null # The namespace under which to launch the endopint. Defaults to the current user's namespace
    image_url: null # Optionally specify the docker image to use when launching the endpoint model. E.g., launching models with later releases of the TGI container with support for newer models.
    env_vars:
      null # Optional environment variables to include when launching the endpoint. e.g., `MAX_INPUT_LENGTH: 2048`
  generation:
    add_special_tokens: true
```
- quantized_model.yaml (this file is stored in the root folder of lighteval)
```
model_args:
  pretrained: akjindal53244/Llama-3.1-Storm-8B
  quantization:
    load_in_4bit: true
    bnb_4bit_compute_dtype: float16
    bnb_4bit_quant_type: nf4
    bnb_4bit_use_double_quant: true
```
4. Run commands 

- inference endpoint method:
```
lighteval accelerate \
    --model_config_path="C:\Users\CSOC\Documents\lighteval\lighteval\examples\model_configs\endpoint_model.yaml" \
    --tasks "bigbench|fact_checker|0|0" \
    --output_dir output_dir
```
- quantization method:
```
lighteval accelerate \
    --model_config_path="C:\Users\CSOC\Documents\lighteval\lighteval\quantized_model.yaml" \
    --tasks "bigbench|fact_checker|0|0" \
    --output_dir output_dir
```

Full log:
```
2024-09-10 17:14:50.679383: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-09-10 17:14:50.853234: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-09-10 17:14:50.918535: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-09-10 17:14:50.937413: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-09-10 17:14:51.061512: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI AVX512_BF16 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-09-10 17:14:51.917501: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
INFO:absl:Using default tokenizer.
INFO:absl:Using default tokenizer.
INFO:absl:Using default tokenizer.
INFO:absl:Using default tokenizer.
INFO:absl:Using default tokenizer.
WARNING:lighteval.logging.hierarchical_logger:main: (0, Namespace(subcommand='accelerate', model_config_path='C:\\Users\\CSOC\\Documents\\lighteval\\lighteval\\examples\\model_configs\\endpoint_model.yaml', model_args=None, max_samples=None, override_batch_size=-1, job_id='', output_dir='output_dir', save_details=False, push_to_hub=False, push_to_tensorboard=False, public_run=False, results_org=None, use_chat_template=False, system_prompt=None, dataset_loading_processes=1, custom_tasks=None, tasks='bigbench|fact_checker|0|0', cache_dir=None, num_fewshot_seeds=1)),  {
WARNING:lighteval.logging.hierarchical_logger:} [0:00:00.035451]
Traceback (most recent call last):
  File "/home/project/anaconda3/envs/lighteval/bin/lighteval", line 8, in <module>
    sys.exit(cli_evaluate())
  File "/home/project/anaconda3/envs/lighteval/lib/python3.10/site-packages/lighteval/__main__.py", line 58, in cli_evaluate
    main_accelerate(args)
  File "/home/project/anaconda3/envs/lighteval/lib/python3.10/site-packages/lighteval/logging/hierarchical_logger.py", line 175, in wrapper
    return fn(*args, **kwargs)
  File "/home/project/anaconda3/envs/lighteval/lib/python3.10/site-packages/lighteval/main_accelerate.py", line 70, in main
    model_config = create_model_config(
  File "/home/project/anaconda3/envs/lighteval/lib/python3.10/site-packages/lighteval/models/model_config.py", line 327, in create_model_config
    raise ValueError("You can't create a model without either a list of model_args or a model_config_path.")     
ValueError: You can't create a model without either a list of model_args or a model_config_path.
```
## Expected behavior
The evaluation will run on inference endpoints/ quantization method.

## Version info
OS: Ubuntu (WSL)
python: 3.10
Cuda: 12.2
Commit: [7261d80](https://github.com/huggingface/lighteval/commit/7261d80d5679cd91c5c20cf2a7823f092ff66251)

Hi ! Thanks for the report, this should be fixed in: #298 

The bug was solved, thank you!