BartForConditionalGeneration decoder_input_ids problem
Author: Alexander-Jin
Pull request: no
Labels: name: Good First Documentation Issue

Here
https://huggingface.co/transformers/model_doc/bart.html

It says that for both BartForConditionalGeneration and BartModel, decoder_input_ids will be generated by shifting input_ids if not given. 

However, from the source codes, forward function of BartForConditionalGeneration has these lines:

if labels is not None:
            if decoder_input_ids is None and decoder_inputs_embeds is None:
                decoder_input_ids = shift_tokens_right(
                    labels, self.config.pad_token_id, self.config.decoder_start_token_id
                )

which suggests it is shifted from labels, instead of input_ids.

Can someone confirm what is the decoder_input_ids and labels for text summarization for

1. training

2. generation

Thanks.


Comments:
Indeed, it should be fixed in the docs: the `decoder_input_ids` are created by shifting the `labels`, not the `input_ids`.

It's best viewed as follows:


```

                     hello    world    
 => labels

                       DECODER

decoder_start_token_id   
     hello    world => decoder input ids
```
As you can see, during training, the `decoder_input_ids` are equal to the `labels`, but shifted one position to the right, prepended by the `decoder_start_token_id`.

At inference time, the decoder_input_ids are generated autoregressively by the model: i.e. we start by just setting decoder_input_ids = [decoder_start_token_id], then let it generate the token that it thinks will follow it, let's say token with id 120. Next, we set decoder_input_ids = [decoder_start_token_id, 120], and provide this to the model, and we take its prediction for the next token. And so on.

Could you open a PR for fixing the docs issue?

Thanks!

> Indeed, it should be fixed in the docs: the `decoder_input_ids` are created by shifting the `labels`, not the `input_ids`.
> 
> It's best viewed as follows:
> 
> ```
> 
                     hello    world    
 => labels
> 
>                        DECODER
> 
> decoder_start_token_id   
     hello    world => decoder input ids
> ```
> 
> As you can see, during training, the `decoder_input_ids` are equal to the `labels`, but shifted one position to the right, prepended by the `decoder_start_token_id`.
> 
> At inference time, the decoder_input_ids are generated autoregressively by the model: i.e. we start by just setting decoder_input_ids = [decoder_start_token_id], then let it generate the token that it thinks will follow it, let's say token with id 120. Next, we set decoder_input_ids = [decoder_start_token_id, 120], and provide this to the model, and we take its prediction for the next token. And so on.
> 
> Could you open a PR for fixing the docs issue?
> 
> Thanks!

I would like to if I can, but I cannot figure out how to update doc pages. 

Hi,

The docstrings of the BART model can be found [here](https://github.com/huggingface/transformers/blob/master/src/transformers/models/bart/modeling_bart.py). 

Each model's documentation is written as a .rst file (which is similar to Markdown), which can be found [here](https://github.com/huggingface/transformers/blob/master/docs/source/model_doc/bart.rst). The docstrings defined in `modeling_bart.py` will end up together with the classes defined there.