run_ner.py crashes when dev or test contain previously unseen labels
Author: AleksandrsBerdicevskis
Pull request: no

## Environment info

- `transformers` version: 4.2.0dev0
- Platform:  Linux-3.10.0-1127.19.1.el7.x86_64-x86_64-with-centos-7.8.2003-Core
- Python version: 3.6.8
- PyTorch version (GPU?): 1.7.1+cu110 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: don't know

### Who can help
@stefan-it

## Information

Model I am using (Bert, XLNet ...): KB/bert-base-swedish-cased

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

I am using run_ner.py to POS-tag a Swedish corpus (TalbankenSBX). If dev and/or test files contain a label that is not present in the train file, the script crashes. The same issue arises with any other corpus I try.

## To reproduce

Steps to reproduce the behavior:

1. Create two toy files with the following contents:

sbx-1-train.json:
```
{"words": ["Vem", "får", "rösta", "?"], "pos": ["HP.UTR.SIN.IND", "VB.PRS.AKT", "VB.INF.AKT", "MAD"]}
```

sbx-1-dev.json
```
{"words": ["är", "född", "1950", "eller", "tidigare", ","], "pos": ["VB.PRS.AKT", "PC.PRF.UTR.SIN.IND.NOM", "RG.NOM", "KN", "AB.KOM", "MID"]}
```

2. Run  python run_ner.py --model_name_or_path KB/bert-base-swedish-cased --train_file sbx-1-train.json --validation_file sbx-1-dev.json --output_dir sbx1 --do_train --do_eval

This results in: 
```
Traceback (most recent call last):
  File "run_ner.py", line 412, in <module>
    main()
  File "run_ner.py", line 303, in main
    load_from_cache_file=not data_args.overwrite_cache,
  File "/home/sasha/venvs/hugtrans/lib64/python3.6/site-packages/datasets/dataset_dict.py", line 303, in map
    for k, dataset in self.items()
  File "/home/sasha/venvs/hugtrans/lib64/python3.6/site-packages/datasets/dataset_dict.py", line 303, in <dictcomp>
    for k, dataset in self.items()
  File "/home/sasha/venvs/hugtrans/lib64/python3.6/site-packages/datasets/arrow_dataset.py", line 1240, in map
    update_data = does_function_return_dict(test_inputs, test_indices)
  File "/home/sasha/venvs/hugtrans/lib64/python3.6/site-packages/datasets/arrow_dataset.py", line 1211, in does_function_return_dict
    function(*fn_args, indices, **fn_kwargs) if with_indices else function(*fn_args, **fn_kwargs)
  File "run_ner.py", line 288, in tokenize_and_align_labels
    label_ids.append(label_to_id[label[word_idx]])
KeyError: 'PC.PRF.UTR.SIN.IND.NOM'
```

...where PC.PRF.UTR.SIN.IND.NOM is the tag which is not present in the train set. 

## Expected behavior
The script should not crash when encountering unseen tags.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.