OSError: HuggingFaceM4/idefics2-8b does not appear to have a file named config.json.
Author: zzxslp
Pull request: no

### System Info

I try to use idefics2 with the example code in its model card. I can load the **HuggingFaceM4/idefics2-8b-base** model without a problem. However, I get the following error when using **HuggingFaceM4/idefics2-8b**. What could be the issue?
Tried transformer version 4.40.0, 4.40.2, and 4.41.1 and encountered the same error.

The full error log is attached:

Traceback (most recent call last):
File "/export/home/xxx.py", line 19, in
processor = AutoProcessor.from_pretrained("HuggingFaceM4/idefics2-8b")
File "/root/miniconda3/envs/eval/lib/python3.10/site-packages/transformers/models/auto/processing_auto.py", line 287, in from_pretrained
config = AutoConfig.from_pretrained(
File "/root/miniconda3/envs/eval/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 934, in from_pretrained
config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
File "/root/miniconda3/envs/eval/lib/python3.10/site-packages/transformers/configuration_utils.py", line 632, in get_config_dict
config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
File "/root/miniconda3/envs/eval/lib/python3.10/site-packages/transformers/configuration_utils.py", line 689, in _get_config_dict
resolved_config_file = cached_file(
File "/root/miniconda3/envs/eval/lib/python3.10/site-packages/transformers/utils/hub.py", line 370, in cached_file
raise EnvironmentError(
OSError: HuggingFaceM4/idefics2-8b does not appear to have a file named config.json. Checkout 'https://huggingface.co/HuggingFaceM4/idefics2-8b/tree/None' for available files.

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
import requests
import torch
from PIL import Image
from io import BytesIO

from transformers import AutoProcessor, AutoModelForVision2Seq
from transformers.image_utils import load_image

DEVICE = "cuda:0"

# Note that passing the image urls (instead of the actual pil images) to the processor is also possible
image1 = load_image("https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg")
image2 = load_image("https://cdn.britannica.com/59/94459-050-DBA42467/Skyline-Chicago.jpg")
image3 = load_image("https://cdn.britannica.com/68/170868-050-8DDE8263/Golden-Gate-Bridge-San-Francisco.jpg")

processor = AutoProcessor.from_pretrained("HuggingFaceM4/idefics2-8b")
model = AutoModelForVision2Seq.from_pretrained(
    "HuggingFaceM4/idefics2-8b",
).to(DEVICE)

# Create inputs
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image"},
            {"type": "text", "text": "What do we see in this image?"},
        ]
    },
    {
        "role": "assistant",
        "content": [
            {"type": "text", "text": "In this image, we can see the city of New York, and more specifically the Statue of Liberty."},
        ]
    },
    {
        "role": "user",
        "content": [
            {"type": "image"},
            {"type": "text", "text": "And how about this image?"},
        ]
    },       
]
prompt = processor.apply_chat_template(messages, add_generation_prompt=True)
inputs = processor(text=prompt, images=[image1, image2], return_tensors="pt")
inputs = {k: v.to(DEVICE) for k, v in inputs.items()}


# Generate
generated_ids = model.generate(**inputs, max_new_tokens=500)
generated_texts = processor.batch_decode(generated_ids, skip_special_tokens=True)

print(generated_texts)
# ['User: What do we see in this image? \nAssistant: In this image, we can see the city of New York, and more specifically the Statue of Liberty. \nUser: And how about this image? \nAssistant: In this image we can see buildings, trees, lights, water and sky.']

```

### Expected behavior

I should be able to load the model with the example code in [HuggingFaceM4/idefics2-8b](https://huggingface.co/HuggingFaceM4/idefics2-8b)

Comments:
Fixed it on my side, a local empty folder was created by mistake
`./HuggingFaceM4/idefics2-8b`