KeyError Issue in Question answering
Author: siddBanPsu
Pull request: no

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): Distillbert

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* my own modified scripts: (give details below)


## To reproduce

Steps to reproduce the behavior:

1. Just run this code.

```
from transformers import pipeline

MODEL_LOC = "distilbert-base-uncased-distilled-squad"
TOKENIZER_LOC = "distilbert-base-uncased-distilled-squad"

qa = pipeline(
    "question-answering",
    model=MODEL_LOC,
    tokenizer=TOKENIZER_LOC
)
context = " ".join(["The incubation period is around 5 days (range: 4-7 days) with a maximum of 12-13 day"]*10)
qa({"question": "incubation period?", "context": context})
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
It should just provide a json output. If I change *10 to *5, it works. 
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.0.2
- Platform: OS X
- Python version: 3.6
- PyTorch version (GPU?): 1.5
- Tensorflow version (GPU?):
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: no


Comments:
I believe this issue has been recurring in other versions due to tokenization related indexing issue. 

I did the following to get the span from the answer. 

```
MODEL_LOC = "distilbert-base-uncased-distilled-squad"
TOKENIZER_LOC = "distilbert-base-uncased-distilled-squad"
model = AutoModelForQuestionAnswering.from_pretrained(MODEL_LOC)
tokenizer = AutoTokenizer.from_pretrained(TOKENIZER_LOC)

def get_answer(question, contexts: List[str]):
    q_c_pairs = [(question, c) for c in contexts]
    encoding = tokenizer.batch_encode_plus(q_c_pairs, 
                                           max_length=256,
                                           pad_to_max_length=True,
                                           truncation=True,
                                           )
    q_encoding = tokenizer.encode_plus(question, max_length=256, truncation=True,)
    q_len = len(q_encoding["input_ids"])
    answer_encoding = torch.tensor(encoding["input_ids"])[:, q_len:]
    model_input_names = tokenizer.model_input_names + ["input_ids"]
    fw_args = {k: torch.tensor(encoding[k]) for k in model_input_names}
    with torch.no_grad():
        start_scores, end_scores = model(**fw_args)
    start_scores_prob = F.softmax(start_scores, dim=1)
    start_scores_prob = start_scores_prob[:, q_len:]
    start_max, start_max_index = torch.max(start_scores_prob, dim=1)
    end_scores_prob = F.softmax(end_scores, dim=1)
    end_scores_prob = end_scores_prob[:, q_len:]
    # Making sure only indices beyond start_index are considered. Forcibly make ones before start_idx as 0
    end_scores_dummy = torch.ones_like(end_scores_prob)
    end_scores_prob = (torch.arange(end_scores_dummy.size(1)) > start_max_index.unsqueeze(1)) * 1.0 * end_scores_prob
    end_max, end_max_index = torch.max(end_scores_prob, dim=1)
    probs = (start_max * end_max)
    for i, (s, e, p) in enumerate(zip(start_max_index, end_max_index, probs)):
        token_ids = answer_encoding[i][s:e + 1]
        tokens = tokenizer.convert_ids_to_tokens(token_ids, skip_special_tokens=True)
        ans_string = tokenizer.convert_tokens_to_string(tokens)
        print(s, e, ans_string, p.cpu().numpy())

question = "What is the incubation period"
context = " ".join(["The incubation period is around 5 days with a maximum of 12-13 days"] * 20)

get_answer(question, [context])
```
I get the desired output and seems to work fine for many cases I tried and with different lengths.


Hello! This seems to have been patched on `master`, it fails for me on `v3.0.2` but not on `master`.

The fix will be in the next version, in the meantime you can install from source:

```py
pip install git+https://github.com/huggingface/transformers
```

Works fine now. Thanks.