Minor inconsistency in "Transformers-based Encoder-Decoder Models" blog post
Author: sgrigory
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.21.3
- Platform: Darwin-20.6.0-x86_64-i386-64bit
- Python version: 3.7.6
- Huggingface_hub version: 0.9.1
- PyTorch version (GPU?): 1.9.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help?

@patrickvonplaten

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Thanks for the nice [Transformers-based Encoder-Decoder Models ](https://huggingface.co/blog/encoder-decoder) blog post. When going through it, I saw the following snippet:
```python
from transformers import MarianMTModel, MarianTokenizer
import torch

tokenizer = MarianTokenizer.from_pretrained("Helsinki-NLP/opus-mt-en-de")
model = MarianMTModel.from_pretrained("Helsinki-NLP/opus-mt-en-de")
embeddings = model.get_input_embeddings()

# get encoded input vectors
input_ids = tokenizer("I want to buy a car", return_tensors="pt").input_ids

# create ids of encoded input vectors
decoder_input_ids = tokenizer("<pad> Ich will ein", return_tensors="pt", add_special_tokens=False).input_ids

# pass decoder input_ids and encoded input vectors to decoder
decoder_output_vectors = model.base_model.decoder(decoder_input_ids).last_hidden_state
```
(see https://github.com/huggingface/blog/blob/79821a50374d16ff7954cecea45fe174443b892b/encoder-decoder.md?plain=1#L1097-L1112)

Contrary to what the comment says, the encoded input vectors are not passed to the decoder. This is confusing. In addition, if a reader would try to turn `decoder_output_vectors` computed this way into logits and then decoded tokens, they would get gibberish ('RLmaligtemütig' instead of 'Ich will will ein Auto')

 I see that this was introduced in https://github.com/huggingface/blog/commit/5913cce7a1e45ec6a3a4a45f9604e82c5d7c6f88 and that [the notebook](https://github.com/huggingface/blog/blob/main/notebooks/05_encoder_decoder.ipynb) still has the old, consistent, version

### Expected behavior

- The comments are consistent with the code
- Markdown is consistent with the notebook

For example:
```python
# get encoded input vectors
input_ids = tokenizer("I want to buy a car", return_tensors="pt").input_ids

# compute encoder output
encoded_output_vectors = model.base_model.encoder(input_ids, return_dict=True).last_hidden_state

# create ids of encoded input vectors
decoder_input_ids = tokenizer("<pad> Ich will ein", return_tensors="pt", add_special_tokens=False).input_ids

# pass decoder input_ids and encoded input vectors to decoder
decoder_output_vectors = model.base_model.decoder(decoder_input_ids, encoder_hidden_states=encoded_output_vectors).last_hidden_state
```

Let me know what you think!

Comments:
Hey @sgrigory,

This sounds very reasonable - would you like to open a PR to fix it? :-)

> Hey @sgrigory,
> 
> This sounds very reasonable - would you like to open a PR to fix it? :-)

@patrickvonplaten Sure, let me open a PR for this