KeyError: 'logits'
Author: ralgond
Pull request: no

Hi, 

I am pretraining a model from scratch, when test, I encounted this error.

test file:
==============================

from transformers import pipeline, AutoModel, AutoTokenizer

model = AutoModel.from_pretrained('./wentian/output/checkpoint-200000/')
tokenizer = AutoTokenizer.from_pretrained('./wentian/output/checkpoint-200000/')

pipe = pipeline(task='fill-mask', model=model, tokenizer=tokenizer)
mask = pipe.tokenizer.mask_token
print(pipe.tokenizer.encode("高庄洗车水枪高压自来水[MASK]托多功能强力冼车超强增压水抢家用喷头"))

print (mask) # is [MASK]
pipe("高庄洗车水枪高压自来水[MASK]托多功能强力冼车超强增压水抢家用喷头", top_k=10)

==============================


output:
==============================

[2, 987, 1, 555, 875, 539, 502, 987, 201, 781, 494, 539, 4, 404, 278, 175, 768, 377, 173, 1, 875, 867, 377, 267, 201, 539, 1, 317, 634, 241, 284, 3]
[MASK]
Traceback (most recent call last):
  File "wentian_test.py", line 17, in <module>
    pipe("高庄洗车水枪高压自来水[MASK]托多功能强力冼车超强增压水抢家用喷头", top_k=10)
  File "/mnt/d/ml/tianchi/wentian/wentian2/lib/python3.8/site-packages/transformers-4.18.0.dev0-py3.8.egg/transformers/pipelines/fill_mask.py", line 225, in __call__
    outputs = super().__call__(inputs, **kwargs)
  File "/mnt/d/ml/tianchi/wentian/wentian2/lib/python3.8/site-packages/transformers-4.18.0.dev0-py3.8.egg/transformers/pipelines/base.py", line 1026, in __call__
    return self.run_single(inputs, preprocess_params, forward_params, postprocess_params)
  File "/mnt/d/ml/tianchi/wentian/wentian2/lib/python3.8/site-packages/transformers-4.18.0.dev0-py3.8.egg/transformers/pipelines/base.py", line 1034, in run_single
    outputs = self.postprocess(model_outputs, **postprocess_params)
  File "/mnt/d/ml/tianchi/wentian/wentian2/lib/python3.8/site-packages/transformers-4.18.0.dev0-py3.8.egg/transformers/pipelines/fill_mask.py", line 96, in postprocess
    outputs = model_outputs["logits"]
  File "/mnt/d/ml/tianchi/wentian/wentian2/lib/python3.8/site-packages/transformers-4.18.0.dev0-py3.8.egg/transformers/utils/generic.py", line 218, in __getitem__
    return inner_dict[k]
KeyError: 'logits'

==============================

Comments:
the environment:

transformers ==4.18.0.dev0 or 4.17.0
tokenizers==0.12.0

Hey @ralgond, what is the checkpoint you're using? Can you reproduce with a checkpoint on the hub?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hi, I received the same error here, using the code from this link: https://huggingface.co/neuralmind/bert-base-portuguese-cased

```
from transformers import pipeline

pipe = pipeline('fill-mask', model=model, tokenizer=tokenizer)

pipe('Tinha uma [MASK] no meio do caminho.')
```

output:
```
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
[<ipython-input-140-f83d9db26e10>](https://localhost:8080/#) in <module>()
      1 pipe = pipeline('fill-mask', model=model, tokenizer=tokenizer)
      2 
----> 3 res = pipe('Tinha uma [MASK] no meio do caminho.')
      4 res

4 frames
[/usr/local/lib/python3.7/dist-packages/transformers/utils/generic.py](https://localhost:8080/#) in __getitem__(self, k)
    218         if isinstance(k, str):
    219             inner_dict = {k: v for (k, v) in self.items()}
--> 220             return inner_dict[k]
    221         else:
    222             return self.to_tuple()[k]

KeyError: 'logits'
```

I received the same problem when I pre-trained bert with MLM.

cc @narsil, have you seen this error before?

@camila-cg @Sette ,

This seems to work

```python
from transformers import pipeline

pipe = pipeline("fill-mask", model="neuralmind/bert-base-portuguese-cased")

print(pipe("Tinha uma [MASK] no meio do caminho."))
```

In the example you shared you're not showing what is `model`. My guess is that you loaded with `AutoModel.from_pretrained(..)` instead of using the correct `AutoModelForMaskedLM`.

If I do this:

```python
from transformers import pipeline, AutoModel, AutoTokenizer

pipe = pipeline(
    "fill-mask",
    model=AutoModel.from_pretrained("neuralmind/bert-base-portuguese-cased"),
    tokenizer=AutoTokenizer.from_pretrained("neuralmind/bert-base-portuguese-cased"),
)

print(pipe("Tinha uma [MASK] no meio do caminho."))
```

then I get that error `KeyError: 'logits'`. That's because the model is missing the correct head and is not sending the `logits` so the pipeline crashes.

Is that it ?

In my case, I was passing the model object bert as an argument, the right thing would be to pass the model directory. It worked for me.

The code below works fine for me. The only difference is that use AutoModelForMaskedLM instead of AutoModel

```
from transformers import pipeline, AutoModelForMaskedLM, AutoTokenizer

model_path = '/roberta-base/'
model = AutoModelForMaskedLM.from_pretrained(model_path)
tokenizer = AutoTokenizer.from_pretrained(model_path)
fill_mask = pipeline('fill-mask', model=model, tokenizer=tokenizer)
res = fill_mask("Spend time online and odds are you'll have to type a username and password to check your e-mail, access your bank <mask> or read a newspaper story.    Enter Microsoft Corp...")
```

> @camila-cg @Sette ,
> 
> This seems to work
> 
> ```python
> from transformers import pipeline
> 
> pipe = pipeline("fill-mask", model="neuralmind/bert-base-portuguese-cased")
> 
> print(pipe("Tinha uma [MASK] no meio do caminho."))
> ```
> 
> In the example you shared you're not showing what is `model`. My guess is that you loaded with `AutoModel.from_pretrained(..)` instead of using the correct `AutoModelForMaskedLM`.
> 
> If I do this:
> 
> ```python
> from transformers import pipeline, AutoModel, AutoTokenizer
> 
> pipe = pipeline(
>     "fill-mask",
>     model=AutoModel.from_pretrained("neuralmind/bert-base-portuguese-cased"),
>     tokenizer=AutoTokenizer.from_pretrained("neuralmind/bert-base-portuguese-cased"),
> )
> 
> print(pipe("Tinha uma [MASK] no meio do caminho."))
> ```
> 
> then I get that error `KeyError: 'logits'`. That's because the model is missing the correct head and is not sending the `logits` so the pipeline crashes.
> 
> Is that it ?

I think it is the bert implementation causes this error. @zzk0 's roberta works fine