trainer.train() fails on 'fmikaelian/flaubert-base-uncased-squad' fine-tuning SQuAD
Author: BenoitDalFerro
Pull request: no
Labels: name: wontfix

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     

    transformers version: 3.0.2
    Platform: Windows 10 Home v2004, Conda 4.8.3, Anaconda CLI 1.7.2
    Python version: 3.7.1
    PyTorch version (GPU?): 1.6.0 CUDA GPU
    Tensorflow version (GPU?): none
    Using GPU in script?: torch.set_default_tensor_type('torch.cuda.FloatTensor')
    Using distributed or parallel set-up in script?: dunno

### Who can help
@fmikaelian @julien-c
 
 ## Information

Model I am using : FlauBERT

The problem arises when using the official example script.

The tasks I am working on is an official GLUE/SQUaD task: fine-tuning 'fmikaelian/flaubert-base-uncased-squad'

## To reproduce
```
french_squad_train = load_dataset('json', data_files="""./French-Squad/SQuAD-v1.1-train_fr_ss999_awstart2_net.json""", field='data')
french_squad_dev = load_dataset('json', data_files="""./French-Squad/SQuAD-v1.1-dev_fr_ss999_awstart2_net.json""", field='data')
print (french_squad_train)
```
> {'train': Dataset(features: {'title': Value(dtype='string', id=None), 'paragraphs': [{'context': Value(dtype='string', id=None), 'qas': [{'id': Value(dtype='string', id=None), 'question': Value(dtype='string', id=None), 'answers': [{'answer_start': Value(dtype='int64', id=None), 'text': Value(dtype='string', id=None)}]}]}]}, num_rows: 442)}
```
model_name = 'fmikaelian/flaubert-base-uncased-squad'
config = AutoConfig.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForQuestionAnswering.from_pretrained(model_name, config=config)
model.train()
model.to('cuda')
training_args = TrainingArguments(
    output_dir='./results',          # output directory
    num_train_epochs=1,              # total # of training epochs
    per_device_train_batch_size=16,  # batch size per device during training
    per_device_eval_batch_size=64,   # batch size for evaluation
    warmup_steps=500,                # number of warmup steps for learning rate scheduler
    weight_decay=0.01,               # strength of weight decay
    logging_dir='./logs',            # directory for storing logs
)
trainer = Trainer(
    model=model,                         # the instantiated 🤗 Transformers model to be trained
    args=training_args,                  # training arguments, defined above
    train_dataset=french_squad_train,          # training dataset
    eval_dataset=french_squad_dev            # evaluation dataset
)
trainer.train()
```

> HBox(children=(FloatProgress(value=0.0, description='Epoch', max=1.0, style=ProgressStyle(description_width='i…
> HBox(children=(FloatProgress(value=0.0, description='Iteration', max=1.0, style=ProgressStyle(description_widt…
> ---------------------------------------------------------------------------
> KeyError                                  Traceback (most recent call last)
> <ipython-input-13-3435b262f1ae> in <module>
> ----> 1 trainer.train()
> 
> C:\Anaconda3\envs\xxx\lib\site-packages\transformers\trainer.py in train(self, model_path)
>     490                 self._past = None
>     491 
> --> 492             for step, inputs in enumerate(epoch_iterator):
>     493 
>     494                 # Skip past any already trained steps if resuming training
> 
> C:\Anaconda3\envs\xxx\lib\site-packages\tqdm\notebook.py in __iter__(self, *args, **kwargs)
>     226     def __iter__(self, *args, **kwargs):
>     227         try:
> --> 228             for obj in super(tqdm_notebook, self).__iter__(*args, **kwargs):
>     229                 # return super(tqdm...) will not catch exception
>     230                 yield obj
> 
> C:\Anaconda3\envs\xxx\lib\site-packages\tqdm\std.py in __iter__(self)
>    1128 
>    1129         try:
> -> 1130             for obj in iterable:
>    1131                 yield obj
>    1132                 # Update and possibly print the progressbar.
> 
> C:\Anaconda3\envs\xxx\lib\site-packages\torch\utils\data\dataloader.py in __next__(self)
>     361 
>     362     def __next__(self):
> --> 363         data = self._next_data()
>     364         self._num_yielded += 1
>     365         if self._dataset_kind == _DatasetKind.Iterable and \
> 
> C:\Anaconda3\envs\xxx\lib\site-packages\torch\utils\data\dataloader.py in _next_data(self)
>     401     def _next_data(self):
>     402         index = self._next_index()  # may raise StopIteration
> --> 403         data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
>     404         if self._pin_memory:
>     405             data = _utils.pin_memory.pin_memory(data)
> 
> C:\Anaconda3\envs\xxx\lib\site-packages\torch\utils\data\_utils\fetch.py in fetch(self, possibly_batched_index)
>      42     def fetch(self, possibly_batched_index):
>      43         if self.auto_collation:
> ---> 44             data = [self.dataset[idx] for idx in possibly_batched_index]
>      45         else:
>      46             data = self.dataset[possibly_batched_index]
> 
> C:\Anaconda3\envs\xxx\lib\site-packages\torch\utils\data\_utils\fetch.py in <listcomp>(.0)
>      42     def fetch(self, possibly_batched_index):
>      43         if self.auto_collation:
> ---> 44             data = [self.dataset[idx] for idx in possibly_batched_index]
>      45         else:
>      46             data = self.dataset[possibly_batched_index]
> 
> KeyError: 0

## Expected behavior

Ongoing training


Comments:
I don't see how you preprocessed your dataset. From the code you pasted, there is no point where you actually tokenize the elements of `french_squad_train`  and `french_squad_dev` (assuming `load_dataset` comes from the nlp library?)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.