Repository: huggingface/transformers

Issue:
run_ner.py crashes with RoBERTa because of incorrect sequence length
# 🐛 Bug

## Information

I'm running `examples/token-classification/run_ner.py` with RoBERTa. An assert statement fails:

```
assert len(input_ids) == max_seq_length
AssertionError
```

Looks like the cause is a mismatch between the value of roberta's `tokenizer.num_special_tokens_to_add()` and the number of special tokens that is actually added to the sequence in `utils_ner.py::convert_examples_to_features()`.

Specifically, `tokenizer.num_special_tokens_to_add()` is 2 (presumably for `
` and `
`). However, `convert_examples_to_features()` adds an extra `
` token at line 331, in addition to the `
` token and first `
` token. So the result is that there are three special tokens, and the sequence ends with `
 
`.

`convert_examples_to_features()` relies on `num_special_tokens_to_add()` to determine how many content tokens from the sequence to use, but because of the mismatch above, you can end up with a sequence length of 129 even when the sequence length was set to a max of 128.

To reproduce this, just follow the instructions at `examples/token-classification/README.md` except use the flag `--model_name_or_path roberta-base`.

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)


## Environment info
- `transformers` version: 2.11.0
- Platform: Linux-4.15.0-101-generic-x86_64-with-debian-buster-sid
- Python version: 3.6.10
- PyTorch version (GPU?): 1.5.0 (True)
- Tensorflow version (GPU?): 2.2.0 (False)
- Using GPU in script?: fails in both cases
- Using distributed or parallel set-up in script?: no




Discussion:
getting same error. any solution yet
The quick temp fix would be to change the num_special_tokens_to_add() to be 3 instead OR not write the extra sep tag since it's single sequence tagging.

I'm guessing the second option is more appropriate because the code that adds the extra sep tag is in utils_ner.py, and as far as I'm aware NER should never involve more than one sep tag.
I will prepare a fix for that soon!

Resolving patch:
diff --git a/examples/token-classification/run_pl_ner.py b/examples/token-classification/run_pl_ner.py
--- a/examples/token-classification/run_pl_ner.py
+++ b/examples/token-classification/run_pl_ner.py
@@ -65,7 +65,7 @@ def prepare_data(self):
                     cls_token=self.tokenizer.cls_token,
                     cls_token_segment_id=2 if self.config.model_type in ["xlnet"] else 0,
                     sep_token=self.tokenizer.sep_token,
-                    sep_token_extra=bool(self.config.model_type in ["roberta"]),
+                    sep_token_extra=False,
                     pad_on_left=bool(self.config.model_type in ["xlnet"]),
                     pad_token=self.tokenizer.pad_token_id,
                     pad_token_segment_id=self.tokenizer.pad_token_type_id,
diff --git a/examples/token-classification/utils_ner.py b/examples/token-classification/utils_ner.py
--- a/examples/token-classification/utils_ner.py
+++ b/examples/token-classification/utils_ner.py
@@ -119,7 +119,7 @@ def __init__(
                         cls_token=tokenizer.cls_token,
                         cls_token_segment_id=2 if model_type in ["xlnet"] else 0,
                         sep_token=tokenizer.sep_token,
-                        sep_token_extra=bool(model_type in ["roberta"]),
+                        sep_token_extra=False,
                         # roberta uses an extra separator b/w pairs of sentences, cf. github.com/pytorch/fairseq/commit/1684e166e3da03f5b600dbb7855cb98ddfcd0805
                         pad_on_left=bool(tokenizer.padding_side == "left"),
                         pad_token=tokenizer.pad_token_id,
@@ -172,7 +172,7 @@ def __init__(
                 cls_token=tokenizer.cls_token,
                 cls_token_segment_id=2 if model_type in ["xlnet"] else 0,
                 sep_token=tokenizer.sep_token,
-                sep_token_extra=bool(model_type in ["roberta"]),
+                sep_token_extra=False,
                 # roberta uses an extra separator b/w pairs of sentences, cf. github.com/pytorch/fairseq/commit/1684e166e3da03f5b600dbb7855cb98ddfcd0805
                 pad_on_left=bool(tokenizer.padding_side == "left"),
                 pad_token=tokenizer.pad_token_id,


Test patch: