KeyError in Camembert in QuestionAnsweringPipeline
Author: tuanardouin
Pull request: no
Labels: name: wontfix

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):
`Camembert ("illuin/camembert-large-fquad")`

Language I am using the model on (English, Chinese ...):
`French`

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

Maybe related to this issue :

https://github.com/huggingface/transformers/issues/4674

## To reproduce

Context file : 
[context_mono.txt](https://github.com/huggingface/transformers/files/4752416/context_mono.txt)



```
import torch
from transformers import pipeline

def analayse():
    if torch.cuda.is_available() == True:
        print('GPU is available')
        device = 0
    else:
        print('GPU is not available')
        device = -1

    nlp_camembert_gpu_f = pipeline("question-answering", model='illuin/camembert-large-fquad', tokenizer='illuin/camembert-large-fquad', device=device)

    context = ''
    with open('context_mono.txt') as file:
        context_lines = [line for line in file]
        for line in context_lines:
            context += line

    answer_C = nlp_camembert_gpu_f(question='Le loyer est-il révisé annuellement ou triennalemment ?', context=context)

def main_file():
    analayse()

if __name__ == '__main__':
    main_file()

```


## Error trace

```
Traceback (most recent call last):
  File "qa_bug.py", line 26, in <module>
    main_file()
  File "qa_bug.py", line 23, in main_file
    analayse()
  File "qa_bug.py", line 20, in analayse
    answer_C = nlp_camembert_gpu_f(question='Le loyer est-il révisé annuellement ou triennalemment ?', context=context)
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/transformers/pipelines.py", line 1229, in __call__
    for s, e, score in zip(starts, ends, scores)
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/transformers/pipelines.py", line 1229, in <listcomp>
    for s, e, score in zip(starts, ends, scores)
KeyError: 377
```

## Expected behavior

Getting an answer.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 2.11.0
- Platform:  Kernel: 5.3.0-1019-aws x86_64 Distro: Ubuntu 18.04.4 LTS
- Python version: 3.7.7
- PyTorch version (GPU?): 1.5.0 (True)
- Tensorflow version (GPU?): (False)
- Using GPU in script?: Yes (same problem on CPU)
- Using distributed or parallel set-up in script?: No


Comments:
Thank you for your fast answer.

Your patch seems to fix some KeyError but not all of them.

Here is an example of a context and a question that raise it :

[context2.txt](https://github.com/huggingface/transformers/files/4758844/context2.txt)

Question : 
Quel est l'étage se situe les locaux ?

Indeed, this PR was not the correct fix so I closed it. Will open a new one soon.

Just for your information, your patch also return an empty response string, but with the right location in the context.

Example :
[context_empty_response.txt](https://github.com/huggingface/transformers/files/4764895/context_empty_response.txt)


Question :
```
Quel est la taille en mètres carrés des locaux ?
```

Thanks for your help

@LysandreJik Do you have an update on this ?
Can I help you in any way ? 

After some research it appears that my problem came from the fact that I was using a model trained with a `max_seq_length` set to 512 but was using the pipeline with this variable set to the default : 384.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.