fill_mask helper
Author: julien-c
Pull request: yes


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2576?src=pr&el=h1) Report
> Merging [#2576](https://codecov.io/gh/huggingface/transformers/pull/2576?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/9d87eafd118739a4c121d69d7cff425264f01e1c?src=pr&el=desc) will **decrease** coverage by `29.56%`.
> The diff coverage is `6.25%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/2576/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/2576?src=pr&el=tree)

```diff
@@             Coverage Diff             @@
##           master    #2576       +/-   ##
===========================================
- Coverage   74.51%   44.94%   -29.57%     
===========================================
  Files          87       87               
  Lines       14920    14951       +31     
===========================================
- Hits        11117     6720     -4397     
- Misses       3803     8231     +4428
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/2576?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/\_\_init\_\_.py](https://codecov.io/gh/huggingface/transformers/pull/2576/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9fX2luaXRfXy5weQ==) | `77.1% <0%> (-21.69%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/2576/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `0% <0%> (-61.1%)` | :arrow_down: |
| [src/transformers/pipelines.py](https://codecov.io/gh/huggingface/transformers/pull/2576/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9waXBlbGluZXMucHk=) | `61.51% <11.76%> (-6.95%)` | :arrow_down: |
| [src/transformers/modeling\_xlm\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/2576/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ194bG1fcm9iZXJ0YS5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [src/transformers/modeling\_camembert.py](https://codecov.io/gh/huggingface/transformers/pull/2576/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19jYW1lbWJlcnQucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [src/transformers/optimization.py](https://codecov.io/gh/huggingface/transformers/pull/2576/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9vcHRpbWl6YXRpb24ucHk=) | `0% <0%> (-96%)` | :arrow_down: |
| [src/transformers/modeling\_distilbert.py](https://codecov.io/gh/huggingface/transformers/pull/2576/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19kaXN0aWxiZXJ0LnB5) | `0% <0%> (-95.86%)` | :arrow_down: |
| [src/transformers/modeling\_ctrl.py](https://codecov.io/gh/huggingface/transformers/pull/2576/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19jdHJsLnB5) | `0% <0%> (-94.28%)` | :arrow_down: |
| [src/transformers/modeling\_bert.py](https://codecov.io/gh/huggingface/transformers/pull/2576/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19iZXJ0LnB5) | `0% <0%> (-87.91%)` | :arrow_down: |
| [src/transformers/modeling\_xlm.py](https://codecov.io/gh/huggingface/transformers/pull/2576/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ194bG0ucHk=) | `0% <0%> (-86.42%)` | :arrow_down: |
| ... and [20 more](https://codecov.io/gh/huggingface/transformers/pull/2576/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/2576?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2576?src=pr&el=footer). Last update [9d87eaf...55069c7](https://codecov.io/gh/huggingface/transformers/pull/2576?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


This feels like a very similar method to [generate](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_utils.py#L567). If we want to add it to the library's internals, don't you think it would make sense to add it directly to the models' internals like `generate`? I feel it would be consistent to mirror it but for mask filling.

This way it could be called like this:

```py
model.fill_mask(x)
```

Furthermore, I think that handling strings is nice, but handling lists of tokens would be better, like `generate` does -> more coherent with `generate`, only handle model data (tokens) and no need to pass a tokenizer to a `modeling_utils` internal method.

Ok I'll take a deeper look next week @LysandreJik 

Ok, I see what you mean. 

Our desired use case is that we would be able to do this in one line, on a string. It's a usage sample we've been seeing more and more (initially from RoBERTa), as it lets one check that the LM works well, in one line.

Examples:
- https://github.com/musixmatchresearch/umberto#fairseq-1
- https://camembert-model.fr/

I see two options:
- define it in its own utils-like file, e.g. `fill_mask.py` or `hub_utils.py` or whatever
- create a `FillMaskPipeline`, as the pipeline already has a tokenizer and a model

Wdyt?

The second option would look something like the last commit and would be used like:

```python
masked_line = "Le camembert est <mask> :)"

model = CamembertForMaskedLM.from_pretrained("camembert-base")
tokenizer = CamembertTokenizer.from_pretrained("camembert-base")


fill_mask = FillMaskPipeline(model, tokenizer)

print(fill_mask(masked_line))
```

I'm hyped by the pipeline option. I believe filling a mask, being the main use-case of an MLM trained model would be a very nice pipeline to have, alongside sequence classification, question answering and named entity recognition.

I think you are very right and I wholeheartedly agree with you. <3 <3

Looks good to me, you can merge

@mfuntowicz As the pipeline exposes its tokenizer, I'm guessing you can already do something like (untested):

```python
fill_mask = pipeline("fill-mask")
fill_mask(f"My name is {fill_mask.tokenizer.mask_token}.")
```

Nice one @julien-c!