Enable `decoder_attention_mask` in `generate` function
Author: samuelpullely
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

The documentation for `model_kwargs` in the generate function states that model specific keyword arguments will be forwarded to the `forward` function of the model.

https://github.com/huggingface/transformers/blob/4cf38148dc98b3df1df6eb2f06e4f02448026b19/src/transformers/generation/utils.py#L1216-L1219

However, this is currently not the case for the model specific keyword argument `decoder_attention_mask` when it is passed to the `generate` function.

This PR makes the necessary adjustments such that `decoder_attention_mask` can be passed to `generate` and will be used as an optional input in the `forward` function. More precisely, it is now possible to specify `decoder_input_ids` and `decoder_attention_mask` in the `generate` function for `BartForConditionalGeneration` such that some tokens in `decoder_input_ids` are masked.

To illustrate the change, we can use the (slightly modified) mask filling example from https://huggingface.co/docs/transformers/model_doc/bart#mask-filling

```python
import torch
from transformers import BartForConditionalGeneration, BartTokenizer

model = BartForConditionalGeneration.from_pretrained("facebook/bart-large", forced_bos_token_id=0)
tokenizer = BartTokenizer.from_pretrained("facebook/bart-large")
sentence = "UN Chief Says There Is No <mask> in Syria"
batch = tokenizer(sentence, return_tensors="pt")
padding_size = 3
decoder_input_ids = torch.tensor(
    [[model.config.decoder_start_token_id] + padding_size * [model.config.pad_token_id] + [model.config.bos_token_id]], 
    dtype=torch.long
)
decoder_attention_mask = torch.where(decoder_input_ids==model.config.pad_token_id, 0, 1)
generated_ids = model.generate(input_ids=batch["input_ids"], use_cache=False, max_new_tokens=20, 
                               decoder_input_ids=decoder_input_ids, decoder_attention_mask=decoder_attention_mask)
decoded = tokenizer.batch_decode(generated_ids)
for seq in decoded:
    print(seq)
```

Note that `use_cache=False` is required when `decoder_input_ids` is manually specified.

Output without PR:

```

<pad><pad><pad>

,

```
Clearly, padding tokens are not ignored because `decoder_attention_mask` is not forwarded to the `forward` function. Hence, the strange output.

Output with PR:

```

<pad><pad><pad>
UN Chief Says There Is No Plan B for Peace in Syria

```


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.

https://discuss.huggingface.co/t/is-t5-expected-to-ignore-padding-tokens-in-decoder-input-ids-when-decoder-attention-mask-is-not-provided/10271

- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker and @younesbelkada

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts and @NielsRogge
- speech models: @sanchit-gandhi

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger and @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hey! Make sure to run `make fixup` to pass all the quality tests 😉 

Thank you both for the quick replies. I’m having difficulties setting up the development environment on my M1 Mac, similar to what is mentioned in #18355. I managed to run `make fixup` but there was still an error regarding TensorFlow. However, one style correction has been made. Please let me know if you have any other suggestions.

Style is okay now, but it seems like the `repo-consistency` is not. Running `make repo-consistency` should solve this 

Thanks for the guidance. A small change was required for `repo-consistency`. I think it should be fine now.

To add some context, I needed this feature when implementing the _self-debiasing_ method from [this paper](https://arxiv.org/abs/2103.00453) with `BartForConditionalGeneration`. 

This method uses a prefix in order to change the probability distribution of the generated tokens towards a specific bias. Given this biased probability distribution, it is possible to adjust the probability distribution of the text without the prefix using `LogitsProcessor` such that the generated text is less biased. The following images are from the original paper:

<p float="left">
  <img src="https://user-images.githubusercontent.com/51292066/207309163-73dd40a1-99ad-449d-8220-1c2fcb5863ec.png" width="400" />
  <img src="https://user-images.githubusercontent.com/51292066/207308497-3f93fe02-a162-4f27-b7a1-6fee0a5861db.png" width="400" /> 
</p>

When using this method with `BartForConditionalGeneration` we need to make sure that the generated tokens start at the same position for the text with and without the prefix because they are both processed in the same batch. To achieve this, it is necessary to manually specify `decoder_input_ids` where padding is applied to the left and padding tokens are ignored with `decoder_attention_mask`. 

To illustrate this with an example, let's take the text `This guy is a jerk because he never listens` and mask the rude word jerk. As encoder input for `BartForConditionalGeneration`, we use 

```
['This guy is a<mask> because he never listens', 'The following text contains rude, disrespectful, or unreasonable language:\nThis guy is a<mask> because he never listens']
```

As decoder input, we use

```
['', ''The following text contains rude, disrespectful, or unreasonable language:\n']
```

Padding is applied to the decoder input such that the decoder starts generating new tokens at the same position for both inputs. For `decoder_input_ids`, we get

```
[[2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0], [2, 0, 133, 511, 2788, 6308, 21820, 6, 26401, 6, 50, 24941, 2777, 35, 50118]]
```

and ignore padding by using the following `decoder_attention_mask`

```
[[1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]
```


Hi @samuelpullely 👋 

Thank you for the PR and the use-case example with self-debiasing! The PR looks good to me, but it's missing one small detail: tests 🤗  If possible, I'd like to ask you to add an integration test, perhaps in the [Bart integration tests section](https://github.com/huggingface/transformers/blob/76d02feadbc99bbccd86e67b02728338a2469f22/tests/models/bart/test_modeling_bart.py#L841). It can be a copy of your example above -- it would go a long way ensuring we don't regress. Let me know if you need a hand.

Hi @gante, thanks for your feedback! I’ve added an integration test. Please let me know if I should adjust anything.

@sgugger can we merge this PR? :)