TextGenerationPipeline giving FutureWarning about AutoModelWithLMHead
Author: pranavpsv
Pull request: no
Labels: name: wontfix

@TevenLeScao
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.0.2
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6
- PyTorch version (GPU?): 1.6.0 (False)
- Tensorflow version (GPU?): 2.3.0 (True)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


## Information

Model I am using (Bert, XLNet ...): GPT2LMHeadModel

The TextGenerationPipeline object causes FutureWarning about class `AutoModelWithLMHead` being deprecated
```
from transformers import pipeline
text_generator = pipeline("text-generation", model="pranavpsv/gpt2-genre-story-generator")

```

## To reproduce

Steps to reproduce the behavior:

Run the above script. This is the warning:

```
/usr/local/lib/python3.6/dist-packages/transformers/modeling_auto.py:798: FutureWarning: The class `AutoModelWithLMHead` is deprecated and will be removed in a future version. Please use `AutoModelForCausalLM` for causal language models, `AutoModelForMaskedLM` for masked language models and `AutoModelForSeq2SeqLM` for encoder-decoder models.
  FutureWarning,

```
## Expected behavior
I expected no warning since I thought the TextGenerationPipeline would use AutoModelForCausalLM object or GPT2LMHeadModel object for the model.


Comments:
Hey @pranavpsv, 

Yes, I think we can switch the `TextGenerationPipeline` to `AutoModelForCausalLM` and `AutoModelForSeq2SeqLM`. So check if the model is in one of the above and then use it, instead of using `AutoModelWithLMHead`. Also once we have a `ConditionalTextGenerationPipeline` we can remove the `AutoModelForSeq2SeqLM` dependency. Feel free to open a PR about this :-) 

@patrickvonplaten  Got it, thank you for the info :)

My model is just a GPT2 model, so I believe it should use AutoModelForCausalLM.

I just checked on the transformers master branch in the pipelines.py source code [file](https://github.com/huggingface/transformers/blob/d0e42a7bed3de9271ae39c575d7eeb54cf985921/src/transformers/pipelines.py#L2432). 
It shows that the pipelines.py file from master branch doesn't use AutoModelWithLMHead. 

However, for some reason, when pip installing the latest version of transformers, the Pipeline object (with task as text-generation) still gives the AutoModelWithLMHead deprecated warning (indicating that it might be importing AutoModelWithLMHead). 

 To confirm, I found that the installed transformers (3.0.2) pipelines.py file imports AutoModelWithLMHead (which is what could be causing this warning):

```
# The pipelines.py file
if is_torch_available():
    import torch
    from .modeling_auto import (
        AutoModel,
        AutoModelForSequenceClassification,
        AutoModelForQuestionAnswering,
        AutoModelForTokenClassification,
        AutoModelWithLMHead,
        AutoModelForSeq2SeqLM,
    )

```
There seems to be a discrepancy between master branch and the 3.0.2 release pipelines object.


For now, I'm doing the following to avoid the warning.


```
model = GPT2LMHeadModel.from_pretrained(checkpoint)
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
text_generator = TextGenerationPipeline(model=model, tokenizer=tokenizer)

```

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.