Cross Attention of MarianMT translation model, inconsistent with paper!
Author: 42694647426
Pull request: no

### System Info

When I'm inspecting the cross-attention layers from the pretrained transformer translation model (MarianMT model), It is very strange that the cross attention from layer 0 and 1 provide best alignment between input and output. I used bertviz to visualize all heads from all 6 layers, and tried different language, english to german and english to chinese, it all gives the same results, which does not make sense because the last layers should be more accurate according to the paper _Jointly Learning to Align and Translate with Transformer Models_ [https://arxiv.org/pdf/1909.02074.pdf](url)
![image](https://user-images.githubusercontent.com/44487593/229558767-deeb4fe1-8e62-41aa-9116-cf4e55ccfac6.png)

But when I'm looking at the cross attention of model _Helsinki-NLP/opus-mt-en-de_ and _Helsinki-NLP/opus-mt-en-zh_ , the layer 1 gives the best alignment. the code is below:
```python
from transformers import AutoTokenizer, AutoModel
import os
os.environ['TRANSFORMERS_CACHE'] = '/data2/hanyings/.cache'

tokenizer = AutoTokenizer.from_pretrained("Helsinki-NLP/opus-mt-en-de")
model = AutoModel.from_pretrained("Helsinki-NLP/opus-mt-en-de", output_attentions=True)

encoder_input_ids = tokenizer("She sees the small elephant.", return_tensors="pt", add_special_tokens=True).input_ids
with tokenizer.as_target_tokenizer():
    decoder_input_ids = tokenizer("Sie sieht den kleinen Elefanten.", return_tensors="pt", add_special_tokens=True).input_ids

outputs = model(input_ids=encoder_input_ids, decoder_input_ids=decoder_input_ids)

encoder_text = tokenizer.convert_ids_to_tokens(encoder_input_ids[0])
decoder_text = tokenizer.convert_ids_to_tokens(decoder_input_ids[0])

from bertviz import model_view
model_view(
    encoder_attention=outputs.encoder_attentions,
    decoder_attention=outputs.decoder_attentions,
    cross_attention=outputs.cross_attentions,
    encoder_tokens= encoder_text,
    decoder_tokens = decoder_text
)
```
And the results are:
![image](https://user-images.githubusercontent.com/44487593/229560299-f6792ad1-5984-4a29-80fb-79403855b43a.png)
![image](https://user-images.githubusercontent.com/44487593/229561124-f84d41d0-ceed-49ac-98b6-91ce47f14424.png)
From the above pictures, I observed that the first 2 layers give the best alignment whereas the last layers do not align the input and output tokens properly. Can you please help me to explain why this happens? and If the alignment of the last layer is not accurate, how does the model provide correct predictions? @ArthurZucker @younesbelkada @gante Please! It is very important for my research project!


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
from transformers import AutoTokenizer, AutoModel
import os
os.environ['TRANSFORMERS_CACHE'] = '/data2/hanyings/.cache'

tokenizer = AutoTokenizer.from_pretrained("Helsinki-NLP/opus-mt-en-de")
model = AutoModel.from_pretrained("Helsinki-NLP/opus-mt-en-de", output_attentions=True)

encoder_input_ids = tokenizer("She sees the small elephant.", return_tensors="pt", add_special_tokens=True).input_ids
with tokenizer.as_target_tokenizer():
    decoder_input_ids = tokenizer("Sie sieht den kleinen Elefanten.", return_tensors="pt", add_special_tokens=True).input_ids

outputs = model(input_ids=encoder_input_ids, decoder_input_ids=decoder_input_ids)

encoder_text = tokenizer.convert_ids_to_tokens(encoder_input_ids[0])
decoder_text = tokenizer.convert_ids_to_tokens(decoder_input_ids[0])

from bertviz import model_view
model_view(
    encoder_attention=outputs.encoder_attentions,
    decoder_attention=outputs.decoder_attentions,
    cross_attention=outputs.cross_attentions,
    encoder_tokens= encoder_text,
    decoder_tokens = decoder_text
)
```

### Expected behavior

The bottom layers give better alignment (layer0 and 1)

Comments:
Hey @42694647426 👋 

As per our [issues guidelines](https://github.com/huggingface/transformers/blob/main/ISSUES.md), we reserve GitHub issues for bugs in the repository and/or feature requests. For any other matters, we'd like to invite you to use our [forum](https://discuss.huggingface.co/) 🤗 

In this case, I'd also advise attempting to reach out to the original authors of the paper you linked, as well as the creators of the Marian models in question (Helsinki-NLP)!

> Hey @42694647426 👋
> 
> As per our [issues guidelines](https://github.com/huggingface/transformers/blob/main/ISSUES.md), we reserve GitHub issues for bugs in the repository and/or feature requests. For any other matters, we'd like to invite you to use our [forum](https://discuss.huggingface.co/) 🤗
> 
> In this case, I'd also advise attempting to reach out to the original authors of the paper you linked, as well as the creators of the Marian models in question (Helsinki-NLP)!

Hi @gante, thank you for the quick reply! Since a few papers have proven that the last two encoder-decoder layers should give the best alignment (the second last layer is actually the best from the paper mentioned above) and it also makes sense that the last layer should have gained the most information to generate output. 

Is there any possibility that the cross_attention in the output sequence is ordered reversely(layer 0 is actually the last layer i.e. the layer closest to the output)? 

Thank you for your help.


I don't think the cross_attentions is outputted in reverse. Looking at line 1075 for [marian](https://github.com/huggingface/transformers/blob/main/src/transformers/models/marian/modeling_marian.py) we see that the cross attentions is added one layer at a time with the correct order:
```
for idx, decoder_layer in enumerate(self.layers):
    ...
    if encoder_hidden_states is not None:
        all_cross_attentions += (layer_outputs[2],)
```

Looking at the generation utils with beam search, I don't see any rearranging of the cross attentions happening either.

But now that you mention it, it is kind of weird how the alignment in the early layers give the best results - but it's not a bug from the looks of it.

Is it possible the model is still providing correct outputs because the positional info is being propagated to the successive layers? [P-Transformer](https://arxiv.org/pdf/2212.05830.pdf) claims this isn't generally the case, but that's document translation.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.