pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:108: cunn_ClassNLLCriterion_updateOutput_kernel: block: [0,0,0], thread: [1,0,0] Assertion `t >= 0 && t < n_classes` failed.
Author: yigit353
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.3.3
- Platform: Linux-5.4.0-65-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.9
- PyTorch version (GPU?): 1.7.1 (False)
- Tensorflow version (GPU?): 2.5.0-dev20210225 (False)
- Using GPU in script?: 2 V100 32GB
- Using distributed or parallel set-up in script?: parallel

### Who can help
@LysandreJik @sgugger @n1t0
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @jplu

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- nlp datasets: [different repo](https://github.com/huggingface/nlp)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

## Information

Model I am using (Bert, XLNet ...): Bert

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

I used `run_ner.py` from examples.

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

PoS Tagging task with the datasets: https://github.com/yigit353/turkish-bert-itu/tree/main/imst

## To reproduce

Steps to reproduce the behavior:

1. Converted a BERT TensorFlow 1 checkpoint pre-trained from scratch using a custom corpus and vocabulary with the original Google's BERT run_pretraining.py via `transformers-cli convert`
2. Used the datasets in this repo (I just uploaded them there): https://github.com/yigit353/turkish-bert-itu/tree/main/imst
3. Used run_ner.py on the dataset with the following code:

```bash
python3 "$USER_ROOT/$LIB_DIR/run_ner.py" \
  --task_name=pos \
  --model_name_or_path "$USER_ROOT/$BERT_DIR/$TORCH_DIR" \
  --train_file "$USER_ROOT/$DATA_DIR/tr_imst-ud-train.conllu.json" \
  --validation_file "$USER_ROOT/$DATA_DIR/tr_imst-ud-dev.conllu.json" \
  --output_dir "$USER_ROOT/$DATA_DIR/$OUTPUT_DIR-$SEED" \
  --per_device_train_batch_size=$BATCH_SIZE \
  --num_train_epochs=$NUM_EPOCHS \
  --overwrite_cache=True \
  --do_train \
  --do_eval \
  --seed=$SEED \
  --fp16
```
4. It worked good with NER datasets (which is parallel to PoS dataset) here: https://github.com/yigit353/turkish-bert-itu/tree/main/datasets/ner
5. It also worked with the PyTorch model (both with PoS and NER without errors or warnings): https://huggingface.co/dbmdz/bert-base-turkish-cased

I also receive the following warning for NER and POS datasets:
`thread '<unnamed>' panicked at 'no entry found for key', /__w/tokenizers/tokenizers/tokenizers/src/models/mod.rs:36:66`

However, NER task worked nonetheless with this script:
```bash
python3 "$USER_ROOT/$LIB_DIR/run_ner.py" \
  --model_name_or_path "$USER_ROOT/$BERT_DIR/$OUT_DIR/$TORCH_OUT_DIR" \
  --train_file "$USER_ROOT/$DATA_DIR/tr-data3/train.json" \
  --validation_file "$USER_ROOT/$DATA_DIR/tr-data3/dev.json" \
  --output_dir "$USER_ROOT/$DATA_DIR/$OUTPUT_DIR-$SEED" \
  --per_device_train_batch_size=$BATCH_SIZE \
  --num_train_epochs=$NUM_EPOCHS \
  --do_train \
  --do_eval \
  --fp16`
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

```
[INFO|trainer.py:837] 2021-02-28 16:04:10,685 >> ***** Running training *****
[INFO|trainer.py:838] 2021-02-28 16:04:10,685 >>   Num examples = 3664
[INFO|trainer.py:839] 2021-02-28 16:04:10,685 >>   Num Epochs = 10
[INFO|trainer.py:840] 2021-02-28 16:04:10,685 >>   Instantaneous batch size per device = 16
[INFO|trainer.py:841] 2021-02-28 16:04:10,685 >>   Total train batch size (w. parallel, distributed & accumulation) = 32
[INFO|trainer.py:842] 2021-02-28 16:04:10,685 >>   Gradient Accumulation steps = 1
[INFO|trainer.py:843] 2021-02-28 16:04:10,685 >>   Total optimization steps = 1150

  0%|          | 0/1150 [00:00<?, ?it/s]/okyanus/users/ctantug/.conda/envs/py37/lib/python3.7/site-packages/torch/nn/parallel/_functions.py:64: UserWarning: Was asked to gather along dimension 0, but all input tensors were scalars; will instead unsqueeze and return a vector.
  warnings.warn('Was asked to gather along dimension 0, but all '
Traceback (most recent call last):
  File "/okyanus/users/ctantug/transformers/examples/token-classification/run_ner.py", line 466, in <module>
    main()
  File "/okyanus/users/ctantug/transformers/examples/token-classification/run_ner.py", line 400, in main
    train_result = trainer.train(resume_from_checkpoint=checkpoint)
  File "/okyanus/users/ctantug/.conda/envs/py37/lib/python3.7/site-packages/transformers/trainer.py", line 940, in train
    tr_loss += self.training_step(model, inputs)
  File "/okyanus/users/ctantug/.conda/envs/py37/lib/python3.7/site-packages/transformers/trainer.py", line 1302, in training_step
    loss = self.compute_loss(model, inputs)
  File "/okyanus/users/ctantug/.conda/envs/py37/lib/python3.7/site-packages/transformers/trainer.py", line 1334, in compute_loss
    outputs = model(**inputs)
  File "/okyanus/users/ctantug/.conda/envs/py37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/okyanus/users/ctantug/.conda/envs/py37/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 162, in forward
    return self.gather(outputs, self.output_device)
  File "/okyanus/users/ctantug/.conda/envs/py37/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 174, in gather
    return gather(outputs, output_device, dim=self.dim)
  File "/okyanus/users/ctantug/.conda/envs/py37/lib/python3.7/site-packages/torch/nn/parallel/scatter_gather.py", line 68, in gather
    res = gather_map(outputs)
  File "/okyanus/users/ctantug/.conda/envs/py37/lib/python3.7/site-packages/torch/nn/parallel/scatter_gather.py", line 62, in gather_map
    for k in out))
  File "<string>", line 7, in __init__
  File "/okyanus/users/ctantug/.conda/envs/py37/lib/python3.7/site-packages/transformers/file_utils.py", line 1413, in __post_init__
    for element in iterator:
  File "/okyanus/users/ctantug/.conda/envs/py37/lib/python3.7/site-packages/torch/nn/parallel/scatter_gather.py", line 62, in <genexpr>
    for k in out))
  File "/okyanus/users/ctantug/.conda/envs/py37/lib/python3.7/site-packages/torch/nn/parallel/scatter_gather.py", line 55, in gather_map
    return Gather.apply(target_device, dim, *outputs)
  File "/okyanus/users/ctantug/.conda/envs/py37/lib/python3.7/site-packages/torch/nn/parallel/_functions.py", line 71, in forward
    return comm.gather(inputs, ctx.dim, ctx.target_device)
  File "/okyanus/users/ctantug/.conda/envs/py37/lib/python3.7/site-packages/torch/nn/parallel/comm.py", line 230, in gather
    return torch._C._gather(tensors, dim, destination)
RuntimeError: CUDA error: device-side assert triggered
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:108: cunn_ClassNLLCriterion_updateOutput_kernel: block: [0,0,0], thread: [1,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:108: cunn_ClassNLLCriterion_updateOutput_kernel: block: [0,0,0], thread: [2,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:108: cunn_ClassNLLCriterion_updateOutput_kernel: block: [0,0,0], thread: [5,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:108: cunn_ClassNLLCriterion_updateOutput_kernel: block: [0,0,0], thread: [9,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:108: cunn_ClassNLLCriterion_updateOutput_kernel: block: [0,0,0], thread: [12,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:108: cunn_ClassNLLCriterion_updateOutput_kernel: block: [0,0,0], thread: [14,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:108: cunn_ClassNLLCriterion_updateOutput_kernel: block: [0,0,0], thread: [1,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:108: cunn_ClassNLLCriterion_updateOutput_kernel: block: [0,0,0], thread: [4,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:108: cunn_ClassNLLCriterion_updateOutput_kernel: block: [0,0,0], thread: [14,0,0] Assertion `t >= 0 && t < n_classes` failed.
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:108: cunn_ClassNLLCriterion_updateOutput_kernel: block: [0,0,0], thread: [16,0,0] Assertion `t >= 0 && t < n_classes` failed.
```

The stack trace always gives a different error location.

Comments:
I think this means you have a label outside of boundaries from the error message but I can't be sure: `CUDA error: device-side assert triggered` are very tricky since they are thrown not when they appear but when there is a synchronization between all the CUDA processes.

The very best way to debug those errors is to try to run a few batches on the CPU to get a better error message. I can try to look into this later on but your model is not public (in the command you give us to repro) and you said it worked for other models?

> I think this means you have a label outside of boundaries from the error message but I can't be sure: `CUDA error: device-side assert triggered` are very tricky since they are thrown not when they appear but when there is a synchronization between all the CUDA processes.

After running with a single CPU and no GPU I got this more explanatory error:
```
  0%|          | 0/229 [00:00<?, ?it/s]Traceback (most recent call last):
  File "/okyanus/users/ctantug/transformers/examples/token-classification/run_ner.py", line 471, in <module>
    main()
  File "/okyanus/users/ctantug/transformers/examples/token-classification/run_ner.py", line 405, in main
    train_result = trainer.train(resume_from_checkpoint=checkpoint)
  File "/okyanus/users/ctantug/.conda/envs/py37/lib/python3.7/site-packages/transformers/trainer.py", line 940, in train
    tr_loss += self.training_step(model, inputs)
  File "/okyanus/users/ctantug/.conda/envs/py37/lib/python3.7/site-packages/transformers/trainer.py", line 1304, in training_step
    loss = self.compute_loss(model, inputs)
  File "/okyanus/users/ctantug/.conda/envs/py37/lib/python3.7/site-packages/transformers/trainer.py", line 1334, in compute_loss
    outputs = model(**inputs)
  File "/okyanus/users/ctantug/.conda/envs/py37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/okyanus/users/ctantug/.conda/envs/py37/lib/python3.7/site-packages/transformers/models/bert/modeling_bert.py", line 1708, in forward
    loss = loss_fct(active_logits, active_labels)
  File "/okyanus/users/ctantug/.conda/envs/py37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/okyanus/users/ctantug/.conda/envs/py37/lib/python3.7/site-packages/torch/nn/modules/loss.py", line 962, in forward
    ignore_index=self.ignore_index, reduction=self.reduction)
  File "/okyanus/users/ctantug/.conda/envs/py37/lib/python3.7/site-packages/torch/nn/functional.py", line 2471, in cross_entropy
    return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
  File "/okyanus/users/ctantug/.conda/envs/py37/lib/python3.7/site-packages/torch/nn/functional.py", line 2267, in nll_loss
    ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
IndexError: Target 9 is out of bounds.
```
> The very best way to debug those errors is to try to run a few batches on the CPU to get a better error message. I can try to look into this later on but your model is not public (in the command you give us to repro) and you said it worked for other models?

The model is not public yet, indeed. I will make them public if necessary. The model successfully performed the NER task (which is nearly the same considering the data structure). However, failed at PoS tagging (which has only different labels).

So the problem is indeed with the labels -> here you have a label with an index of 9 and you should print the value of `num_labels` in the script, but it looks like it's less than this from the error. I think the datasets should be fixed somehow.

It may be that your validation dataset has a label the training dataset did not have, which then causes this issue. In that case you should either make sure that label is present in the training set too or remove the samples with that label in your evaluation dataset.

These are my active labels:
```
Activate labels: tensor([-100,    9,    4,    7,   12, -100, -100, -100,    7,    0,    8,    7,
        -100, -100,   12, -100,   11, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100,    0,    2, -100,    0,    7, -100,   11,
           7, -100,    7,   12, -100,   12,    2, -100,    7, -100,    1,    8,
           7,    3, -100,   11, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100,    7, -100, -100,
           1,    0,    7,   11, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100,   10, -100, -100, -100,   10, -100,    0,   12, -100, -100, -100,
          11,    9,    0, -100,    7, -100, -100,    7,   12, -100, -100,   11,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100,    7, -100,   11,    7, -100, -100, -100,
          11,    7, -100, -100, -100,   11,    7, -100, -100,   12,    8,    7,
          11, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100,    4,    9,    9,
           1,    2, -100,   12, -100, -100, -100,    0,    3,   11, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100,    2,    7,    7, -100,   12,   11,    4,   12, -100,   11, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100,    2,   12, -100,   12, -100,   11, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100,    8,    7, -100,
          12, -100, -100,   12, -100,    1,   12,   11,    8,    7,    2,   12,
        -100,    7,    3,    9,   11,    4,    7, -100,    9,    8,    7,   12,
        -100, -100, -100,   11,    2,    0, -100, -100,    0, -100,   11,   12,
        -100,   12, -100,    1,    0,   12,   12,    0, -100,    3,   11, -100,
        -100,   10, -100, -100,   11,    0,    7, -100,    7,    1,    2,    0,
        -100, -100,   12, -100,    5,    7,   11, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100,    7, -100,    7,    7,   12, -100,   11,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100,   10, -100,    4,
          11,    7, -100,   10, -100, -100,   10, -100, -100, -100, -100,    0,
           7, -100,   12,   12, -100, -100,    4,   12, -100,   12, -100,   11,
           7, -100, -100, -100,   12,   11, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100,    0,    7,   12, -100,   11,   12, -100, -100,   11,   12, -100,
          11, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100,   10,    7, -100, -100, -100,   12,   11,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100,    0, -100, -100,
           7,   12, -100,   11, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100,   10,    9,    7,   12,   11, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100])
```
And the targets:
```
target: tensor([-100,    9,    4,    7,   12, -100, -100, -100,    7,    0,    8,    7,
        -100, -100,   12, -100,   11, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100,    0,    2, -100,    0,    7, -100,   11,
           7, -100,    7,   12, -100,   12,    2, -100,    7, -100,    1,    8,
           7,    3, -100,   11, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100,    7, -100, -100,
           1,    0,    7,   11, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100,   10, -100, -100, -100,   10, -100,    0,   12, -100, -100, -100,
          11,    9,    0, -100,    7, -100, -100,    7,   12, -100, -100,   11,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100,    7, -100,   11,    7, -100, -100, -100,
          11,    7, -100, -100, -100,   11,    7, -100, -100,   12,    8,    7,
          11, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100,    4,    9,    9,
           1,    2, -100,   12, -100, -100, -100,    0,    3,   11, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100,    2,    7,    7, -100,   12,   11,    4,   12, -100,   11, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100,    2,   12, -100,   12, -100,   11, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100,    8,    7, -100,
          12, -100, -100,   12, -100,    1,   12,   11,    8,    7,    2,   12,
        -100,    7,    3,    9,   11,    4,    7, -100,    9,    8,    7,   12,
        -100, -100, -100,   11,    2,    0, -100, -100,    0, -100,   11,   12,
        -100,   12, -100,    1,    0,   12,   12,    0, -100,    3,   11, -100,
        -100,   10, -100, -100,   11,    0,    7, -100,    7,    1,    2,    0,
        -100, -100,   12, -100,    5,    7,   11, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100,    7, -100,    7,    7,   12, -100,   11,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100,   10, -100,    4,
          11,    7, -100,   10, -100, -100,   10, -100, -100, -100, -100,    0,
           7, -100,   12,   12, -100, -100,    4,   12, -100,   12, -100,   11,
           7, -100, -100, -100,   12,   11, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100,    0,    7,   12, -100,   11,   12, -100, -100,   11,   12, -100,
          11, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100,   10,    7, -100, -100, -100,   12,   11,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100,    0, -100, -100,
           7,   12, -100,   11, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100,   10,    9,    7,   12,   11, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100,
        -100, -100, -100, -100])

```

I have already checked the number of labels first thing. That's why it surprised me that it is not the problem. I also run the script without eval. 
```
03/01/2021 17:45:18 - INFO - __main__ -   Label list ['ADJ', 'ADP', 'ADV', 'AUX', 'CCONJ', 'DET', 'INTJ', 'NOUN', 'NUM', 'PRON', 'PROPN', 'PUNCT', 'VERB', 'X']
03/01/2021 17:45:18 - INFO - __main__ -   Label to id {'ADJ': 0, 'ADP': 1, 'ADV': 2, 'AUX': 3, 'CCONJ': 4, 'DET': 5, 'INTJ': 6, 'NOUN': 7, 'NUM': 8, 'PRON': 9, 'PROPN': 10, 'PUNCT': 11, 'VERB': 12, 'X': 13}
03/01/2021 17:45:18 - INFO - __main__ -   Num labels 14
```
What might be another cause of this?

Solved it! Turns out in my config.json (which is copied from another PyTorch checkpoint) should also change the `label2id` and `id2label`. That was totally unexpected. In order to match 14 labels I changed the config file as follows:
```
{
  ...
  "architectures": [
    "BertForTokenClassification"
  ],
  "id2label": {
    "0": "LABEL_0",
    "1": "LABEL_1",
    "2": "LABEL_2",
    "3": "LABEL_3",
    "4": "LABEL_4",
    "5": "LABEL_5",
    "6": "LABEL_6",
    "7": "LABEL_7",
    "8": "LABEL_8",
    "9": "LABEL_9",
    "10": "LABEL_10",
    "11": "LABEL_11",
    "12": "LABEL_12",
    "13": "LABEL_13"
  },
  
  "label2id": {
    "LABEL_0": 0,
    "LABEL_1": 1,
    "LABEL_2": 2,
    "LABEL_3": 3,
    "LABEL_4": 4,
    "LABEL_5": 5,
    "LABEL_6": 6,
    "LABEL_7": 7,
    "LABEL_8": 8,
    "LABEL_9": 9,
    "LABEL_10": 10, 
    "LABEL_11": 11, 
    "LABEL_12": 12, 
    "LABEL_13": 13
  },
  ...
}
```
Thank you anyways.