bug in loading gpt 2 in a pipeline
Author: surya-narayanan
Pull request: no

### System Info


Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.32.0.dev0
- Platform: Linux-5.15.0-69-generic-x86_64-with-glibc2.35
- Python version: 3.10.10
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.1
- Accelerate version: 0.21.0
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.0.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Yes

```
Cell In[5], line 23
     20 query_model_pipelines = []
     22 for query_model_name in query_model_names:
---> 23     query_model_pipelines.append(pipeline(model=query_model_name, 
     24                                           trust_remote_code=True, 
     25                                           max_new_tokens= 500,
     26                                           device_map="auto", 
     27                                           torch_dtype=torch.bfloat16,
     28                                           ))

File [~/miniconda3/envs/vornoi/lib/python3.10/site-packages/transformers/pipelines/__init__.py:793](https://vscode-remote+ssh-002dremote-002bthomsonlab-002d2-002ejamesgornet-002ecom.vscode-resource.vscode-cdn.net/home/suryahari/Vornoi/QA/~/miniconda3/envs/vornoi/lib/python3.10/site-packages/transformers/pipelines/__init__.py:793), in pipeline(task, model, config, tokenizer, feature_extractor, image_processor, framework, revision, use_fast, token, device, device_map, torch_dtype, trust_remote_code, model_kwargs, pipeline_class, **kwargs)
    791 if isinstance(model, str) or framework is None:
    792     model_classes = {"tf": targeted_task["tf"], "pt": targeted_task["pt"]}
--> 793     framework, model = infer_framework_load_model(
    794         model,
    795         model_classes=model_classes,
    796         config=config,
    797         framework=framework,
    798         task=task,
    799         **hub_kwargs,
    800         **model_kwargs,
    801     )
    803 model_config = model.config
    804 hub_kwargs["_commit_hash"] = model.config._commit_hash

File [~/miniconda3/envs/vornoi/lib/python3.10/site-packages/transformers/pipelines/base.py:276](https://vscode-remote+ssh-002dremote-002bthomsonlab-002d2-002ejamesgornet-002ecom.vscode-resource.vscode-cdn.net/home/suryahari/Vornoi/QA/~/miniconda3/envs/vornoi/lib/python3.10/site-packages/transformers/pipelines/base.py:276), in infer_framework_load_model(model, config, model_classes, task, framework, **model_kwargs)
    273             continue
    275     if isinstance(model, str):
--> 276         raise ValueError(f"Could not load model {model} with any of the following classes: {class_tuple}.")
    278 if framework is None:
    279     framework = infer_framework(model.__class__)

ValueError: Could not load model gpt2 with any of the following classes: (, ).

```
### Who can help?

 @younesbelkada @ArthurZucker  @Narsil 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

can't reproduce :( 

https://colab.research.google.com/drive/19CmvYMZqQqBDfp-I1_oh4ImvcFynJygI#scrollTo=Ib94uj4w8zH7



### Expected behavior

should work as here: https://colab.research.google.com/drive/19CmvYMZqQqBDfp-I1_oh4ImvcFynJygI#scrollTo=Ib94uj4w8zH7

Comments:
i have a feeling that this error was created when an env was created, transformers was installed, (the code worked fine), diffusers was installed, (and the code broke). 

Both colabs are private so can't reproduce, but thanks for reporting! 

fwiw - anecdotal evidence suggests that this happens when disks are full - this behavior was likely never observed in colabs / testing environments. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.