EncoderDecoderModel forward decoder_attention_mask can't execute the default behavior mentioned in the document
Author: efsotr
Pull request: no

@ArthurZucker @younesbelkada

EncoderDecoderModel forward decoder_attention_mask can't execute the default behavior mentioned in the document.

For example, 
EncoderDecoderModel (BertModel, BertLMHeadModel)

In EncoderDecoderModel forward:
decoder_attention_mask  will be directly passed in self.decoder as attention_mask. [code link](https://github.com/huggingface/transformers/blob/a3975f94f3a090a54ed4ec78ab736ce6aaee6742/src/transformers/models/encoder_decoder/modeling_encoder_decoder.py#LL616C9-L629C10)
~~~python
        # Decode
        decoder_outputs = self.decoder(
            input_ids=decoder_input_ids,
            attention_mask=decoder_attention_mask,
            encoder_hidden_states=encoder_hidden_states,
            encoder_attention_mask=attention_mask,
            inputs_embeds=decoder_inputs_embeds,
            output_attentions=output_attentions,
            output_hidden_states=output_hidden_states,
            use_cache=use_cache,
            past_key_values=past_key_values,
            return_dict=return_dict,
            **kwargs_decoder,
        )
~~~

In BertLMHeadModel forward:
attention_mask  will be directly passed in self.bert as attention_mask. [code link](https://github.com/huggingface/transformers/blob/7f8b909189547944617741d8d3c6c84504701693/src/transformers/models/bert/modeling_bert.py#L1234)

~~~python
outputs = self.bert(
          input_ids,
          attention_mask=attention_mask,
          token_type_ids=token_type_ids,
          position_ids=position_ids,
          head_mask=head_mask,
          inputs_embeds=inputs_embeds,
          encoder_hidden_states=encoder_hidden_states,
          encoder_attention_mask=encoder_attention_mask,
          past_key_values=past_key_values,
          use_cache=use_cache,
          output_attentions=output_attentions,
          output_hidden_states=output_hidden_states,
          return_dict=return_dict,
      )
~~~

In BertModel forward:

attention_mask will be filled by number one if it is None. [code link](https://github.com/huggingface/transformers/blob/7f8b909189547944617741d8d3c6c84504701693/src/transformers/models/bert/modeling_bert.py#LL980C9-L981C108)
~~~python
        if attention_mask is None:
            attention_mask = torch.ones(((batch_size, seq_length + past_key_values_length)), device=device)
~~~

if decoder_attention_mask is None in EncoderDecoderModel forward, then it will be filled by number one.
but in [code link](https://github.com/huggingface/transformers/blob/a3975f94f3a090a54ed4ec78ab736ce6aaee6742/src/transformers/models/encoder_decoder/modeling_encoder_decoder.py#LL106C9-L108C32)
~~~
        decoder_attention_mask (`torch.BoolTensor` of shape `(batch_size, target_sequence_length)`, *optional*):
            Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also
            be used by default.
~~~

Comments:
Hi @efsotr 
Thanks for the issue, indeed there seems to be a typo, one could replace the docstring with the correct behavior (the default mask will be created by the decoder)
Do you want to open a Pull Request to address these changes?
Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.