ValueError: Could not load model EleutherAI/gpt-neo-2.7B with any of the following classes:
Author: hongyi-zhao
Pull request: no

### Feature request

Want to run "EleutherAI/gpt-neo-2.7B"

### Motivation

Want to run "EleutherAI/gpt-neo-2.7B"

### Your contribution

```python
(datasci) werner@X10DAi:~$ ipython
Python 3.11.1 (main, Dec 22 2022, 17:06:07) [GCC 12.2.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.7.0 -- An enhanced Interactive Python. Type '?' for help.

   ...: 
   ...:    # Load the ChatGPT-4 pipeline
   ...:    chatbot = pipeline("text2text-generation", model="EleutherAI/gpt-neo-
   ...: 2.7B")
   ...: 
   ...:    # Define a function to interact with the chatbot
   ...:    def chat():
   ...:        while True:
   ...:            # Get user input
   ...:            user_input = input("You: ")
   ...: 
   ...:            # Exit if user enters "exit"
   ...:            if user_input.lower() == "exit":
   ...:                break
   ...: 
   ...:            # Generate response from chatbot
   ...:            response = chatbot(user_input, max_length=50)[0]["generated_t
   ...: ext"]
   ...: 
   ...:            # Print response
   ...:            print("Chatbot:", response)
   ...: 
   ...:    # Call the chat function to start the chatbot
   ...:    chat()
2023-03-27 08:24:43.589129: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2023-03-27 08:24:43.636446: I tensorflow/tsl/cuda/cudart_stub.cc:28] Could not find cuda drivers on your machine, GPU will not be used.
2023-03-27 08:24:43.637021: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-03-27 08:24:44.519506: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[1], line 4
      1 from transformers import pipeline
      3 # Load the ChatGPT-4 pipeline
----> 4 chatbot = pipeline("text2text-generation", model="EleutherAI/gpt-neo-2.7B")
      6 # Define a function to interact with the chatbot
      7 def chat():

File ~/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/transformers/pipelines/__init__.py:776, in pipeline(task, model, config, tokenizer, feature_extractor, image_processor, framework, revision, use_fast, use_auth_token, device, device_map, torch_dtype, trust_remote_code, model_kwargs, pipeline_class, **kwargs)
    772 # Infer the framework from the model
    773 # Forced if framework already defined, inferred if it's None
    774 # Will load the correct model if possible
    775 model_classes = {"tf": targeted_task["tf"], "pt": targeted_task["pt"]}
--> 776 framework, model = infer_framework_load_model(
    777     model,
    778     model_classes=model_classes,
    779     config=config,
    780     framework=framework,
    781     task=task,
    782     **hub_kwargs,
    783     **model_kwargs,
    784 )
    786 model_config = model.config
    787 hub_kwargs["_commit_hash"] = model.config._commit_hash

File ~/.pyenv/versions/3.11.1/envs/datasci/lib/python3.11/site-packages/transformers/pipelines/base.py:271, in infer_framework_load_model(model, config, model_classes, task, framework, **model_kwargs)
    268             continue
    270     if isinstance(model, str):
--> 271         raise ValueError(f"Could not load model {model} with any of the following classes: {class_tuple}.")
    273 framework = "tf" if "keras.engine.training.Model" in str(inspect.getmro(model.__class__)) else "pt"
    274 return framework, model

ValueError: Could not load model EleutherAI/gpt-neo-2.7B with any of the following classes: (<class 'transformers.models.auto.modeling_tf_auto.TFAutoModelForSeq2SeqLM'>,).


```

Comments:
Hi @hongyi-zhao, 

The issue is arising because the checkpoint `"EleutherAI/gpt-neo-2.7B"` is for the [GPT Neo](https://huggingface.co/docs/transformers/v4.27.2/en/model_doc/gpt_neo), which has architectures for the text generation -- [GPTNeoForCausalLM](https://huggingface.co/docs/transformers/v4.27.2/en/model_doc/gpt_neo#transformers.GPTNeoForCausalLM) -- and sequence classification -- [GPTNeoForSequenceClassification](https://huggingface.co/docs/transformers/v4.27.2/en/model_doc/gpt_neo#transformers.GPTNeoForSequenceClassification) -- tasks. The pipeline in the shared snippet is `"text2text-generation"` for which this model doesn't have a compatible class. 

I am very confused about the names of these models and the matching relationships between them, so I get straight to the point where I am most concerned: will this project help me to use the latest GPT-4 or their other future newest models?

There are many cutting-edge models available and that continue to be added to transformers library. Unfortunately GPT-4 isn't one of them, as OpenAI hasn't open sourced the weights. 

The models can be explored on [the hub](https://huggingface.co/). For example [here are the models](https://huggingface.co/models?pipeline_tag=text2text-generation&sort=downloads) for the selected `text2text-generation` pipeline in the example above. There's more information about the [Text2TextGenerationPipeline in the docs](https://huggingface.co/docs/transformers/v4.27.2/en/main_classes/pipelines#transformers.Text2TextGenerationPipeline). 

Thank you very much for your comments and explanations.

for anyone facing this issue again:
I had this error when  environment had the new PyTorch v2 . 
Uninstalling torch `v2.0` and installing torch `v1.11` solved the issue.