SQUAD Question Answering example:: RuntimeError: Could not infer dtype of NoneType
Author: paniabhisek
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.2.1
- Platform: Linux
- Python version: 3.6.12
- PyTorch version (GPU?): 1.7.1
- Tensorflow version (GPU?): 2.4.0
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help
- @sgugger, @patil-suraj
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models: 

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library: @sgugger

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples: @patil-suraj

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

Model I am using (Bert, XLNet ...): Bert

The problem arises when using:
* [X] the official example scripts: (give details below)

        mkdir squad
        wget https://rajpurkar.github.io/SQuAD-explorer/dataset/train-v2.0.json -O squad/train-v2.0.json
        wget https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v2.0.json -O squad/dev-v2.0.json

        import json
        from pathlib import Path

        def read_squad(path):
            path = Path(path)
            with open(path, 'rb') as f:
                squad_dict = json.load(f)

            contexts = []
            questions = []
            answers = []
            for group in squad_dict['data']:
                for passage in group['paragraphs']:
                    context = passage['context']
                    for qa in passage['qas']:
                        question = qa['question']
                        for answer in qa['answers']:
                            contexts.append(context)
                            questions.append(question)
                            answers.append(answer)

            return contexts, questions, answers

        train_contexts, train_questions, train_answers = read_squad('squad/train-v2.0.json')
        val_contexts, val_questions, val_answers = read_squad('squad/dev-v2.0.json')

        def add_end_idx(answers, contexts):
            for answer, context in zip(answers, contexts):
                gold_text = answer['text']
                start_idx = answer['answer_start']
                end_idx = start_idx + len(gold_text)

                # sometimes squad answers are off by a character or two – fix this
                if context[start_idx:end_idx] == gold_text:
                    answer['answer_end'] = end_idx
                elif context[start_idx-1:end_idx-1] == gold_text:
                    answer['answer_start'] = start_idx - 1
                    answer['answer_end'] = end_idx - 1     # When the gold label is off by one character
                elif context[start_idx-2:end_idx-2] == gold_text:
                    answer['answer_start'] = start_idx - 2
                    answer['answer_end'] = end_idx - 2     # When the gold label is off by two characters

        add_end_idx(train_answers, train_contexts)
        add_end_idx(val_answers, val_contexts)

        from transformers import DistilBertTokenizerFast
        tokenizer = DistilBertTokenizerFast.from_pretrained('distilbert-base-uncased')

        train_encodings = tokenizer(train_contexts, train_questions, truncation=True, padding=True)
        val_encodings = tokenizer(val_contexts, val_questions, truncation=True, padding=True)

        def add_token_positions(encodings, answers):
            start_positions = []
            end_positions = []
            for i in range(len(answers)):
                start_positions.append(encodings.char_to_token(i, answers[i]['answer_start']))
                end_positions.append(encodings.char_to_token(i, answers[i]['answer_end']))

                # if start position is None, the answer passage has been truncated
                if start_positions[-1] is None:
                    start_positions[-1] = tokenizer.model_max_length

                # if end position is None, the 'char_to_token' function points to the space before the correct token - > add + 1
                if end_positions[-1] is None:
                    end_positions[-1] = encodings.char_to_token(i, answers[i]['answer_end'] + 1)
            encodings.update({'start_positions': start_positions, 'end_positions': end_positions})

        add_token_positions(train_encodings, train_answers)
        add_token_positions(val_encodings, val_answers)

        import torch

        class SquadDataset(torch.utils.data.Dataset):
            def __init__(self, encodings):
                self.encodings = encodings

            def __getitem__(self, idx):
                return {key: torch.tensor(val[idx]) for key, val in self.encodings.items()}

            def __len__(self):
                return len(self.encodings.input_ids)

        train_dataset = SquadDataset(train_encodings)
        val_dataset = SquadDataset(val_encodings)

        from transformers import DistilBertForQuestionAnswering, Trainer, TrainingArguments

        training_args = TrainingArguments(
            output_dir='./results',          # output directory
            num_train_epochs=3,              # total number of training epochs
            per_device_train_batch_size=16,  # batch size per device during training
            per_device_eval_batch_size=64,   # batch size for evaluation
            warmup_steps=500,                # number of warmup steps for learning rate scheduler
            weight_decay=0.01,               # strength of weight decay
            logging_dir='./logs',            # directory for storing logs
            logging_steps=10,
        )

        model = DistilBertForQuestionAnswering.from_pretrained("distilbert-base-uncased")

        trainer = Trainer(
            model=model,                         # the instantiated 🤗 Transformers model to be trained
            args=training_args,                  # training arguments, defined above
            train_dataset=train_dataset,         # training dataset
            eval_dataset=val_dataset             # evaluation dataset
        )

        trainer.train()


* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [X] an official GLUE/SQUaD task: (give the name) SQUaD
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Ran the example from [squad question answering](https://huggingface.co/transformers/custom_datasets.html#question-answering-with-squad-2-0)
2. getting the following

        ---------------------------------------------------------------------------
        RuntimeError                              Traceback (most recent call last)
        <ipython-input-9-fe1badbb2679> in <module>
             21 )
             22 
        ---> 23 trainer.train()
    
        /media/data2/anaconda/envs/hr/lib/python3.6/site-packages/transformers/trainer.py in train(self, model_path, trial)
            871             self.control = self.callback_handler.on_epoch_begin(self.args, self.state, self.control)
            872 
        --> 873             for step, inputs in enumerate(epoch_iterator):
            874 
            875                 # Skip past any already trained steps if resuming training

        /media/data2/anaconda/envs/hr/lib/python3.6/site-packages/torch/utils/data/dataloader.py in __next__(self)
            433         if self._sampler_iter is None:
            434             self._reset()
        --> 435         data = self._next_data()
            436         self._num_yielded += 1
            437         if self._dataset_kind == _DatasetKind.Iterable and \

        /media/data2/anaconda/envs/hr/lib/python3.6/site-packages/torch/utils/data/dataloader.py in _next_data(self)
            473     def _next_data(self):
            474         index = self._next_index()  # may raise StopIteration
        --> 475         data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
            476         if self._pin_memory:
            477             data = _utils.pin_memory.pin_memory(data)

        /media/data2/anaconda/envs/hr/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py in fetch(self, possibly_batched_index)
             42     def fetch(self, possibly_batched_index):
             43         if self.auto_collation:
        ---> 44             data = [self.dataset[idx] for idx in possibly_batched_index]
             45         else:
             46             data = self.dataset[possibly_batched_index]

        /media/data2/anaconda/envs/hr/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py in <listcomp>(.0)
             42     def fetch(self, possibly_batched_index):
             43         if self.auto_collation:
        ---> 44             data = [self.dataset[idx] for idx in possibly_batched_index]
             45         else:
             46             data = self.dataset[possibly_batched_index]

        <ipython-input-8-a9d5c9a06902> in __getitem__(self, idx)
              6 
              7     def __getitem__(self, idx):
        ----> 8         return {key: torch.tensor(val[idx]) for key, val in self.encodings.items()}
              9 
             10     def __len__(self):

        <ipython-input-8-a9d5c9a06902> in <dictcomp>(.0)
              6 
              7     def __getitem__(self, idx):
        ----> 8         return {key: torch.tensor(val[idx]) for key, val in self.encodings.items()}
              9 
             10     def __len__(self):

        RuntimeError: Could not infer dtype of NoneType
3. 

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior: should have run without the error.

<!-- A clear and concise description of what you would expect to happen. -->


Comments:
I'm not able to reproduce the issue. I went to [this page](https://huggingface.co/transformers/custom_datasets.html), then clicked on "Open in colab" on the top right (chose PyTorch), and then run the question-answering tutorial, and it's working fine for me:

![squad_bis](https://user-images.githubusercontent.com/48327001/105996770-0f760900-60ab-11eb-99e6-89d971dbfcd5.PNG)




Hi @paniabhisek 

For QA you could use the official `run_qa.py ` example scripts which now supports `Trainer` and `datasets`. You can find it here 
https://github.com/huggingface/transformers/tree/master/examples/question-answering   



@NielsRogge I ran the code in colab, it's working for me too. But not in conda environment.

@patil-suraj [example-script](https://github.com/huggingface/transformers/tree/master/examples/question-answering) only supports squad 1.1 ? Does it support squad 2.0 ?

It supports squad V1 and V2. For V2, just add the flag `--version2_with_negative` (on top of `--dataset_nme squad_v2`)

If you try to call `train_dataset[137]`, it returns an error (`[136]` and `[138]` both work properly). It is because `end_positions.append(encodings.char_to_token(i, answers[i]['answer_end']))` and `end_positions.append(encodings.char_to_token(i, answers[i]['answer_end'] + 1))` do not find the correct token; the `end_position[-1]` is None. The code before #9378 should work.

[#9378-comment](https://github.com/huggingface/transformers/pull/9378#issuecomment-759717949) have worked for me. I was wondering how to use a snippet without an unfamiliar script so I can use my own language model. thanks @kevinthwu .

btw thanks @sgugger I can use the squad 2.0 with the option `--version2_with_negative`.

I'm not closing as the docs are not updated yet.

> It supports squad V1 and V2. For V2, just add the flag `--version2_with_negative` (on top of `--dataset_nme squad_v2`)

 the argument name is '**version_2_with_negative**'  (line 444 run_qa.py)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.