Data collator ignores the input attention_mask
Author: hacobe
Pull request: no

## Environment info

- `transformers` version: 4.11.3
- Platform: Linux-5.4.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.7.12
- PyTorch version (GPU?): 1.9.0+cu102 (False)
- Tensorflow version (GPU?): 2.6.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help

@sgugger
@patrickvonplaten 

## Information

The DataCollators do not respect the attention_mask passed to them.

Here's the context:

I was trying to train a GPT2 model where the input_ids are:
```
<PROMPT TOKEN IDS> + <COMPLETION TOKEN IDS>
```

I want to mask out the completion tokens during training, so I set the attention_mask to:

```
[1] * (no. of prompt tokens) + [0] * (no. of completion tokens)
```

The labels are the same as the input_ids, except that the prompt token IDs are replaced with -100 in order to ignore the predictions for prompt tokens in the loss and only penalize the model on predicting completion tokens.

Here is a toy example of the data that would be passed to the data collator following that format:

```python
features = [
  {"input_ids": [1,2,3,4,5,6], "attention_mask": [1,1,1,1,1,0], "labels": [-100,-100,-100,-100,-100,6]},
  {"input_ids": [1,2,3], "attention_mask": [1,1,0], "labels": [-100,-100,4]},
]
```

1-5 in the first example are the prompt token IDs and 6 is the label token ID.
1 and 2 in the second example are the prompt token IDs and 3 is the label token ID. 

I would expect that DataCollatorForSeq2Seq(features) would return the following attention_mask:

```
[[1,1,1,1,1,0],
 [1,1,0,0,0,0]]
```

i.e., padding the second example to make it the same length as the first example.

Instead it returns:

```
[[1,1,1,1,1,0],
 [1,1,1,0,0,0]]
```

Notice that the attention_mask[1,2] is 1 instead of 0 (3 ones instead of 2 ones at the start of the 2nd example), which unmasks the completion token for the 2nd example.

## To reproduce

See this colab: https://colab.research.google.com/drive/1DcvH01SyfIPBwlzqiUCPaSjWKvkAoD2E#scrollTo=wHAfa4t5sZHg

I've also copied and passed the relevant code below:

```python
import transformers

tokenizer = transformers.GPT2Tokenizer.from_pretrained('distilgpt2')
tokenizer.pad_token = tokenizer.unk_token

features = [
  {"input_ids": [1,2,3,4,5,6], "attention_mask": [1,1,1,1,1,0], "labels": [-100,-100,-100,-100,-100,6]},
  {"input_ids": [1,2,3], "attention_mask": [1,1,0], "labels": [-100,-100,4]},
]
output = transformers.DataCollatorForSeq2Seq(tokenizer)(features)

x = output['attention_mask'][0, :].tolist()
print(x) # [1, 1, 1, 0, 0, 0]
assert x == [1,1,0] + [0] * 3
```


Comments:
This is an unexpected behavior that takes its root in the `tokenizer.pad` method (you can do `tokenizer.pad` on your features and see it re-creates the attention mask). This is because all models expect the attention mask to indicate where padding has been applied.

In your case, you are training a model with unusual attentions masks (in the sense that it does not only indicate the pad tokens) so you should redefine your own data collators and avoid using the `tokenizer.pad` method.

Thanks so much for the quick reply! I've written a custom data collator that solves my problem, but I want to ask one more related question before closing the issue. Is there any way to fine-tune a GPT-style model on a conditional generation task that does not involve defining the attention_mask as I have above? If there's some more conventional way of doing that, then I can switch to that approach and use the pre-defined DataCollators. Otherwise, it seems strange to call this way of defining the attention mask "unusual" (though I understand you mean that in a particular sense), because in this case it would be the "usual" way to fine-tune a GPT-style model on a conditional generation task.

I'll also note that the Hugging Face glossary (https://huggingface.co/transformers/glossary.html#attention-mask) defines an attention_mask as indicating "which tokens should be attended to, and which should not" rather than indicating which tokens are padding and which are not.

Thanks in advance for your help with this issue!

What you say make sense, and I think we should we fix the problem at its souce, e.g. the `pad` method of the `tokenizer`, to honor the `attention_mask` passed. I will look into this tomorrow.

After some more thought, I don't think I need to mask out the completion during training. The GPT model just predicts the next word based on the previous ones. So I could drop the attention_mask from my example above, the default DataCollator would work fine and that would be the standard way of fine-tuning a GPT model on a conditional text generation task. **Is this correct?**

I couldn't find an example of fine-tuning using an autoregressive model in Hugging Face's [collection](https://github.com/huggingface/transformers/blob/master/examples/pytorch) of examples. I found an example of conditional text generation using an already fine-tuned autoregressive model [here](https://github.com/huggingface/transformers/blob/master/examples/pytorch/text-generation/run_generation.py) and an example of fine-tuning using sequence to sequence models [here](https://github.com/huggingface/transformers/blob/master/examples/pytorch/summarization/run_summarization.py).

All that being said, I still think honoring the attention mask is a good idea, but I wanted to update this thread, because if the above is correct, then the attention_mask I provided in my first comment is indeed unusual.

@sgugger Could you answer my last question in the thread just to tie up loose ends? Thanks so much for resolving the issue with the attention_mask!

I think you can drop the custom attention mask in this case, yes.