Inquiry about the difference between two Approaches to Mask Infilling using BART Model in the official document
Author: Hyfred
Pull request: no

I came across two different methods of using the BART model for mask infilling at this URL: https://huggingface.co/docs/transformers/main/en/model_doc/bart#bart

The first method involves taking the logit corresponding to the mask word in the decoding last hidden representation and using softmax to directly predict which word in the vocabulary it belongs to.
```python 
from transformers import AutoTokenizer, BartForConditionalGeneration

tokenizer = AutoTokenizer.from_pretrained("facebook/bart-base")
model = BartForConditionalGeneration.from_pretrained("facebook/bart-base")

TXT = "My friends are <mask> but they eat too many carbs."
input_ids = tokenizer([TXT], return_tensors="pt")["input_ids"]
logits = model(input_ids).logits

masked_index = (input_ids[0] == tokenizer.mask_token_id).nonzero().item()
probs = logits[0, masked_index].softmax(dim=0)
values, predictions = probs.topk(5)

tokenizer.decode(predictions).split()
```
The second method involves using the .generate function to restore the masked word in a sentence.
```python
from transformers import BartForConditionalGeneration, BartTokenizer

model = BartForConditionalGeneration.from_pretrained("facebook/bart-large", forced_bos_token_id=0)
tok = BartTokenizer.from_pretrained("facebook/bart-large")
example_english_phrase = "UN Chief Says There Is No <mask> in Syria"
batch = tok(example_english_phrase, return_tensors="pt")
generated_ids = model.generate(batch["input_ids"])
assert tok.batch_decode(generated_ids, skip_special_tokens=True) == [
    "UN Chief Says There Is No Plan to Stop Chemical Weapons in Syria"
]
```

I'm keen on understanding the differences between these two approaches. Unfortunately, I haven't found any relevant explanations. Additionally, I'd like to ascertain whether both methods rely on context to predict the masked word rather than solely considering the words before the mask.

Any insights or documentation clarifying these would be greatly appreciated!!

Comments:
Hey! Sorry I didn't really have time to dig and must say that I didn't know the generate pipeline could fill up the hole for you! 
The `beam-search` algorithm is used for that, which for me would mean that you explore more with this technique. Otherwise it should be similar 😉 

By default the generate method here also uses beam search, so don't think there is any big difference, but it's more convenient to use generate rather than having to compute the mask position (more so if you have 2 mask or more!). 🤗 

> By default the generate method here also uses beam search, so don't think there is any big difference, but it's more convenient to use generate rather than having to compute the mask position (more so if you have 2 mask or more!). 🤗

Yes, I think you're right. when I use the 'bart_base' version, there is no difference between those two decoding strategies. Because although the bart is an encoder-decoder framework, the pre-train task 'infilling the blank' hasn't utilized the generation ability. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.