label_attention_mask in Bart for conditional sequence generation and other seq2seq models.
Author: JulesGM
Pull request: no

I find it very counterintuitive that the Bart class for conditional generation doesn't take label_attention_masks as an input, and that people are left to manually convert their padding tokens to values of -100. 

-> I think many new users are likely getting bitten by this, and will also be in the future.
-> I think it's a pain to recode everytime identically when it could just be part of the function.
-> It breaks symmetry with the inputs. Why do you need it for the inputs but not the outputs? Strange.

The behavior could be altered without breaking current code. Indeed, labels could test if its argument is a dict, behave how it normally does if it's not, and look for `input_ids` and `attention_mask` if it is, without breaking, as giving a `dict` as an argument would just break the code currently.

Comments:
Hi @JulesGM ! 
There is already an argument for that. Every seq2seq model has the `decoder_attention_mask` argument with which you can specify the attention mask for `decode_input_ids` which are fed to the decoder.

Thanks.
"For translation and summarization training, decoder_input_ids should be
provided. If no decoder_input_ids is provided, the model will create this
tensor by shifting the input_ids to the right for denoising pre-training
following the paper."
Is that how it works even if labels are provided?  The labels and the
(shifted) decoder inputs are the same 99.99% of the time.
Do I still need to feed a label value, and does it need to have -100 masks,
even if I feed a decoder inputs and a decoder attention mask?
Do I need to shift the decoder input values myself?

On Wed., Mar. 2, 2022, 8:03 a.m. Suraj Patil, ***@***.***>
wrote:

> Hi @JulesGM <https://github.com/JulesGM> !
> There is already an argument for that. Every seq2seq model has the
> decoder_attention_mask argument with which you can specify the attention
> mask for decode_input_ids which are fed to the decoder.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/15878#issuecomment-1056906142>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAYU34KQODAMMGPIJDL7DR3U55RIHANCNFSM5PVDIUTQ>
> .
> Triage notifications on the go with GitHub Mobile for iOS
> <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
> or Android
> <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
>
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>


Hi @JulesGM !

The docstring is bit misleading, it should not shift `input_ids` even for denoising pre-training. I will fix that.

> Is that how it works even if labels are provided?  Th
No, if `labels` are passed then `decoder_input_ids` are created by shifting the `labels`.

> The labels and the(shifted) decoder inputs are the same 99.99% of the time.

They should be different, i.e if `labels` are like this:
`
 this is a sent 
`

then `decoder_inputIds` will look like this after shifting
`
 
 this is a sent`

> Do I still need to feed a label value, and does it need to have -100 masks,
even if I feed a decoder inputs and a decoder attention mask?

For training `labels` are required, `decoder_input_ids` are created using `labels` as I explained above. 
And no  adding -100 is not strictly necessary, passing `decoder_attention_mask` should also work.

Also, for general questions like these, I would recommend using the [forum](https://discuss.huggingface.co/) as we issues for bug reports and feature requests. Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.