Unable to use fill-mask pipeline on gpt-neo model
Author: pidugusundeep
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.5.0
- Platform: googleColab
- Python version:3.7

Models: `GPT neo`

Code : 
```
#Import Hugging Face's Transformers
from transformers import pipeline
generator = pipeline('fill-mask', model='EleutherAI/gpt-neo-1.3B')
```
Error: 
![Screenshot 2021-05-19 at 10 55 30 AM](https://user-images.githubusercontent.com/10946649/118760614-bbc54080-b890-11eb-9076-5fe010f249b7.png)

Can someone help me know what could the reason be for not able to use the fill-mask on `gpt-neo` model?


Comments:
Fill-mask is for encoder-only models like BERT and RoBERTa. The GPT-neo model is a decoder-only model that is capable of doing text generation. There's a `TextGenerationPipeline` available, so you might try that out. The documentation can be found [here](https://huggingface.co/transformers/main_classes/pipelines.html#transformers.TextGenerationPipeline).

I read through articles that this model can be used to do grammar checking? Please share relevant documentation for the same.

Yes it can do grammar checking similar to GPT-3, in a zero-shot manner. So you can for example try the following prompt:

```
Original: She no went to the market.
Standard American English:
```

Normally, if GPT-neo is smart enough, it will then generate `She didn't go to the market.`

These big generation models like GPT-3 and GPT-neo can learn in a zero-shot manner, just by giving a few examples, and then ask the model what comes next. So in this case, I didn't even give one example, I asked the model directly for an answer. You can also first provide several examples ("Original" and "Standard American English" pairs) to the model, and then ask it to predict what comes next. 

Great, Can you please share some sample implementation on google collab notebook? 

I just copied the code sample from the [model card](https://huggingface.co/EleutherAI/gpt-neo-1.3B):

```
from transformers import pipeline
generator = pipeline('text-generation', model='EleutherAI/gpt-neo-1.3B')
generator("Original: She no went to the market. Standard American English: She didn't go to the market. Original: I loving eating pizza. Standard American English:", do_sample=True, min_length=50)
```

Isn't the text generation specific to generating new text with a given prompt?
I tried using the same format as what you have provided and this was the response 
Input: `Original: She no went to the market. Standard American English:` 
Output: `Original: She no went to the market. Standard American English: No, I didn’t go to the market yesterday.` 
This completely changed the 3rd person to 1st person? is the format `Original:xxx Standard American English:` important and is this how it does the grammar correction?

Isn't the text generation specific to generating new text with a given prompt? => well, normally it is meant to generate new text given a prompt indeed. But as models like GPT-3 and GPT-neo are so powerful and are trained on a lot of data, they are capable of performing what the authors of GPT-3 call "in-context learning": this means that the model knows what to do just based on a given prompt. See the [GPT-3 paper](https://arxiv.org/abs/2005.14165) for more info. 

I've just tried it with GPT-3 and it works. However, GPT-neo doesn't seem as powerful. This is logical since GPT-3 has 175 billion parameters, whereas GPT-neo only has 1.3 billion (there's also a 2.7 billion variant available). 

Maybe you can try by giving more examples in the prompt. Sometimes it seems to work:

![image](https://user-images.githubusercontent.com/48327001/118779543-718d9080-b88b-11eb-8cd4-d556fb5f89a4.png)


@NielsRogge What other [task-specific pipelines](https://huggingface.co/transformers/main_classes/pipelines.html) can I use the gpt-neo model with?

I think the GPT-neo models only support the `TextGenerationPipeline`. But do not that they can be used for summarization, you can just provide a text followed by "TLDR:", and then the model will generate a summary.  

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.