run_ner.py with xlm-roberta-base raises an IndexError in tokenize_and_align_labels
Author: thvitt
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
 `transformers` version: 4.0.0  (and the example scripts from git master aka 72d6c9c6)
- Platform: Linux-4.19.0-12-amd64-x86_64-with-glibc2.10
- Python version: 3.8.3
- PyTorch version (GPU?): 1.5.0 (True)
- Tensorflow version (GPU?): 2.2.0 (True)
- Using GPU in script?: True (but we don’ŧ get that far)
- Using distributed or parallel set-up in script?: False

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.
 
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->
git blame says @sgugger 

## Information

Model I am using (Bert, XLNet ...): xlm-roberta-base

The problem arises when using:
* [X] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. `python3 run_ner.py --model_name_or_path xlm-roberta-base --task_name ner --dataset_name conll2003 --label_all_tokens --do_train --do_eval --output_dir finetuning-output`

Crashes with the following stacktrace:

```
Traceback (most recent call last):
  File "run_ner.py", line 394, in <module>
    main()
  File "run_ner.py", line 292, in main
    tokenized_datasets = datasets.map(
  File "/home/vitt/.conda/envs/cuda/lib/python3.8/site-packages/datasets/dataset_dict.py", line 286, in map
    {
  File "/home/vitt/.conda/envs/cuda/lib/python3.8/site-packages/datasets/dataset_dict.py", line 287, in <dictcomp>
    k: dataset.map(
  File "/home/vitt/.conda/envs/cuda/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 1239, in map
    update_data = does_function_return_dict(test_inputs, test_indices)
  File "/home/vitt/.conda/envs/cuda/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 1210, in does_function_return_dict
    function(*fn_args, indices, **fn_kwargs) if with_indices else function(*fn_args, **fn_kwargs)
  File "run_ner.py", line 277, in tokenize_and_align_labels
    current_label = label_to_id[label[label_index]]
IndexError: list index out of range
```

From a little debugging, the problem seems to be that this code assumes there are only as many sequences with `offset[0] == 0 and offset[1] != 0` as there are words in the original input (and thus as there are labels):

https://github.com/huggingface/transformers/blob/72d6c9c68ba19b2e991b0d7a32989410399b33f5/examples/token-classification/run_ner.py#L276-L278

However, the SentencePiece tokenizer may split input words to sequences starting with a single `'▁'` token. Then, the offset mapping for '▁' will be `(0, 1)` and for the following token `(0, x)` (E.g. '.' in the CONLL data ⇒ `['▁', '.']` with offsets `[(0, 1), (0, 1)]` or ['NACCO'] ⇒ `('▁', (0, 1)), ('NAC', (0, 3)), ('CO', (3, 5))`.

(Could this use `tokenized_inputs.word_ids()` instead?)

Comments:
Thanks for flagging! Yes, using `word_ids` is probably a better idea in this case, I did that in the PR mentioned above. If you want to review it, I'd be happy to take your comments into account!