Loading from AutoModel gives ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds
Author: patrickvonplaten
Pull request: no

Executing the following code snippet states that ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds.

```py
from transformers import AutoModel, AutoTokenizer 
model_name = "castorini/t5-base-canard" 

model = AutoModel.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

context = '''
    Frank Zappa ||| Disbandment ||| What group disbanded ||| Zappa and the Mothers of Invention ||| When did they disband?
'''

encoded_input = tokenizer(
    context,
    padding='max_length',
    max_length=512,
    truncation=True,
    return_tensors="pt",
)
decoder_input = tokenizer(
    context,
    padding='max_length',
    max_length=512,
    truncation=True,
    return_tensors="pt",
)

encoder_output = model.generate(input_ids=encoded_input["input_ids"], decoder_input_ids=decoder_input["input_ids"])
output = tokenizer.decode(
    encoder_output[0],
    skip_special_tokens=True
)
output
```

yields this error:

```
Some weights of the model checkpoint at castorini/t5-base-canard were not used when initializing T5Model: ['lm_head.weight']
- This IS expected if you are initializing T5Model from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing T5Model from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Input length of decoder_input_ids is 512, but ``max_length`` is set to 20. This can lead to unexpected behavior. You should consider increasing ``config.max_length`` or ``max_length``.
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-11-b9fe12b71812>](https://localhost:8080/#) in <module>()
     24 )
     25 
---> 26 encoder_output = model.generate(input_ids=encoded_input["input_ids"], decoder_input_ids=decoder_input["input_ids"])
     27 output = tokenizer.decode(
     28     encoder_output[0],

6 frames
[/usr/local/lib/python3.7/dist-packages/transformers/models/t5/modeling_t5.py](https://localhost:8080/#) in forward(self, input_ids, attention_mask, encoder_hidden_states, encoder_attention_mask, inputs_embeds, head_mask, cross_attn_head_mask, past_key_values, use_cache, output_attentions, output_hidden_states, return_dict)
    925         else:
    926             err_msg_prefix = "decoder_" if self.is_decoder else ""
--> 927             raise ValueError(f"You have to specify either {err_msg_prefix}input_ids or {err_msg_prefix}inputs_embeds")
    928 
    929         if inputs_embeds is None:

ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds
```

Comments:
The problem here is that `AutoModel` does not load the full T5 model wiht a language head. One should use `AutoModelForSeq2SeqLM` inseatd:

```py
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer 
model_name = "castorini/t5-base-canard" 

model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

context = '''
    Frank Zappa ||| Disbandment ||| What group disbanded ||| Zappa and the Mothers of Invention ||| When did they disband?
'''

encoded_input = tokenizer(
    context,
    padding='max_length',
    max_length=512,
    truncation=True,
    return_tensors="pt",
)
decoder_input = tokenizer(
    context,
    padding='max_length',
    max_length=512,
    truncation=True,
    return_tensors="pt",
)

encoder_output = model.generate(input_ids=encoded_input["input_ids"], decoder_input_ids=decoder_input["input_ids"])
output = tokenizer.decode(
    encoder_output[0],
    skip_special_tokens=True
)
output
```

This should work just fine @dxlong2000

> The problem here is that `AutoModel` does not load the full T5 model wiht a language head. One should use `AutoModelForSeq2SeqLM` inseatd:
> 
> ```python
> from transformers import AutoModelForSeq2SeqLM, AutoTokenizer 
> model_name = "castorini/t5-base-canard" 
> 
> model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
> tokenizer = AutoTokenizer.from_pretrained(model_name)
> 
> context = '''
>     Frank Zappa ||| Disbandment ||| What group disbanded ||| Zappa and the Mothers of Invention ||| When did they disband?
> '''
> 
> encoded_input = tokenizer(
>     context,
>     padding='max_length',
>     max_length=512,
>     truncation=True,
>     return_tensors="pt",
> )
> decoder_input = tokenizer(
>     context,
>     padding='max_length',
>     max_length=512,
>     truncation=True,
>     return_tensors="pt",
> )
> 
> encoder_output = model.generate(input_ids=encoded_input["input_ids"], decoder_input_ids=decoder_input["input_ids"])
> output = tokenizer.decode(
>     encoder_output[0],
>     skip_special_tokens=True
> )
> output
> ```
> 
> This should work just fine @dxlong2000

Sure thanks @patrickvonplaten !

> ```python
> model.generate(input_ids=encoded_input["input_ids"], decoder_input_ids=decoder_input["input_ids"])
> ```

Dear Patrick, thank you very much for all your helpful answers here on github. They've helped me so many times!

If I train a T5 model from `AutoModelForSeq2SeqLM` with my custom dataset with `Trainer`, what type of decoder input is the Trainer using?
It seems I do not have to specify, and I've tried deducing it from the source and docs but I couldn't find an answer to that.
I'd super appreciate some insights!