Repository: huggingface/transformers

Issue:
[run_ner.py]You need to instantiate RobertaTokenizerFast with add_prefix_space=True to use it with pretokenized inputs
## Environment info

- `transformers` version: 4.2.0
- Platform: Linux-5.4.0-53-generic-x86_64-with-debian-buster-sid
- Python version: 3.6.12
- PyTorch version (GPU?): 1.7.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Yes


### Who can help
 examples/token-classification: @stefan-it
 tokenizers: @mfuntowicz

## Information

Model I am using Roberta:

The problem arises when using:
* The official example scripts: `transformers/examples/token-classification/run_ner.py`

The tasks I am working on is:
* an official task: Named Entity Recognition on `CoNLL 2003`

## To reproduce

Steps to reproduce the behavior:

run this command: 

`python ./transformers/examples/token-classification/run_ner.py --model_name_or_path roberta-base  --dataset_name conll2003 --output_dir ./roberta_base_cased_conll2003 --do_train --do_eval`
I am using the `run_ner.py` of a very recent commit:  `126fd281`

```
$ md5sum run_ner.py  
cb6401e787266812f791a1e3052465d3  run_ner.py
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

I got this error: 
```
AssertionError: You need to instantiate RobertaTokenizerFast with add_prefix_space=True to use it with pretokenized inputs.
```

I tested other models, such as `bert-base-cased`, `bert-large-cased`, `xlm-roberta-base`, `xlnet-base-cased`. All of these worked. But  `roberta-base` and `roberta-large` have this error. 


This is the full output on screen: 

```
01/14/2021 20:34:28 - WARNING - __main__ -   Process rank: -1, device: cuda:0, n_gpu: 2distributed training: False, 16-bits training: False
01/14/2021 20:34:28 - INFO - __main__ -   Training/evaluation parameters TrainingArguments(output_dir=./roberta_base_cased_conll2003, overwrite_output_dir=False, do_train=True, do_eval=True, do_predict=False, evaluation_strategy=EvaluationStrategy.NO, prediction_loss_only=False, per_device_train_batch_size=8, per_device_eval_batch_size=8, gradient_accumulation_steps=1, eval_accumulation_steps=None, learning_rate=5e-05, weight_decay=0.0, adam_beta1=0.9, adam_beta2=0.999, adam_epsilon=1e-08, max_grad_norm=1.0, num_train_epochs=3.0, max_steps=-1, lr_scheduler_type=SchedulerType.LINEAR, warmup_steps=0, logging_dir=runs/Jan14_20-34-28_ubuntu18, logging_first_step=False, logging_steps=500, save_steps=500, save_total_limit=None, no_cuda=False, seed=42, fp16=False, fp16_opt_level=O1, fp16_backend=auto, local_rank=-1, tpu_num_cores=None, tpu_metrics_debug=False, debug=False, dataloader_drop_last=False, eval_steps=500, dataloader_num_workers=0, past_index=-1, run_name=./roberta_base_cased_conll2003, disable_tqdm=False, remove_unused_columns=True, label_names=None, load_best_model_at_end=False, metric_for_best_model=None, greater_is_better=None, ignore_data_skip=False, sharded_ddp=False, deepspeed=None, label_smoothing_factor=0.0, adafactor=False, _n_gpu=2)
Reusing dataset conll2003 (/home/fangli/.cache/huggingface/datasets/conll2003/conll2003/1.0.0/63ba56944e35c1943434322a07ceefd79864672041b7834583709af4a5de4664)
[INFO|configuration_utils.py:445] 2021-01-14 20:34:29,366 >> loading configuration file https://huggingface.co/roberta-base/resolve/main/config.json from cache at /home/fangli/.cache/huggingface/transformers/733bade19e5f0ce98e6531021dd5180994bb2f7b8bd7e80c7968805834ba351e.35205c6cfc956461d8515139f0f8dd5d207a2f336c0c3a83b4bc8dca3518e37b
[INFO|configuration_utils.py:481] 2021-01-14 20:34:29,366 >> Model config RobertaConfig {
  "architectures": [
    "RobertaForMaskedLM"
  ],
  "attention_probs_dropout_prob": 0.1,
  "bos_token_id": 0,
  "eos_token_id": 2,
  "finetuning_task": "ner",
  "gradient_checkpointing": false,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "id2label": {
    "0": "LABEL_0",
    "1": "LABEL_1",
    "2": "LABEL_2",
    "3": "LABEL_3",
    "4": "LABEL_4",
    "5": "LABEL_5",
    "6": "LABEL_6",
    "7": "LABEL_7",
    "8": "LABEL_8"
  },
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "label2id": {
    "LABEL_0": 0,
    "LABEL_1": 1,
    "LABEL_2": 2,
    "LABEL_3": 3,
    "LABEL_4": 4,
    "LABEL_5": 5,
    "LABEL_6": 6,
    "LABEL_7": 7,
    "LABEL_8": 8
  },
  "layer_norm_eps": 1e-05,
  "max_position_embeddings": 514,
  "model_type": "roberta",
  "num_attention_heads": 12,
  "num_hidden_layers": 12,
  "pad_token_id": 1,
  "position_embedding_type": "absolute",
  "transformers_version": "4.2.0",
  "type_vocab_size": 1,
  "use_cache": true,
  "vocab_size": 50265
}

[INFO|configuration_utils.py:445] 2021-01-14 20:34:29,405 >> loading configuration file https://huggingface.co/roberta-base/resolve/main/config.json from cache at /home/fangli/.cache/huggingface/transformers/733bade19e5f0ce98e6531021dd5180994bb2f7b8bd7e80c7968805834ba351e.35205c6cfc956461d8515139f0f8dd5d207a2f336c0c3a83b4bc8dca3518e37b
[INFO|configuration_utils.py:481] 2021-01-14 20:34:29,405 >> Model config RobertaConfig {
  "architectures": [
    "RobertaForMaskedLM"
  ],
  "attention_probs_dropout_prob": 0.1,
  "bos_token_id": 0,
  "eos_token_id": 2,
  "gradient_checkpointing": false,
  "hidden_act": "gelu",
  "hidden_dropout_prob": 0.1,
  "hidden_size": 768,
  "initializer_range": 0.02,
  "intermediate_size": 3072,
  "layer_norm_eps": 1e-05,
  "max_position_embeddings": 514,
  "model_type": "roberta",
  "num_attention_heads": 12,
  "num_hidden_layers": 12,
  "pad_token_id": 1,
  "position_embedding_type": "absolute",
  "transformers_version": "4.2.0",
  "type_vocab_size": 1,
  "use_cache": true,
  "vocab_size": 50265
}

[INFO|tokenization_utils_base.py:1760] 2021-01-14 20:34:29,584 >> loading file https://huggingface.co/roberta-base/resolve/main/vocab.json from cache at /home/fangli/.cache/huggingface/transformers/d3ccdbfeb9aaa747ef20432d4976c32ee3fa69663b379deb253ccfce2bb1fdc5.d67d6b367eb24ab43b08ad55e014cf254076934f71d832bbab9ad35644a375ab
[INFO|tokenization_utils_base.py:1760] 2021-01-14 20:34:29,585 >> loading file https://huggingface.co/roberta-base/resolve/main/merges.txt from cache at /home/fangli/.cache/huggingface/transformers/cafdecc90fcab17011e12ac813dd574b4b3fea39da6dd817813efa010262ff3f.5d12962c5ee615a4c803841266e9c3be9a691a924f72d395d3a6c6c81157788b
[INFO|tokenization_utils_base.py:1760] 2021-01-14 20:34:29,585 >> loading file https://huggingface.co/roberta-base/resolve/main/tokenizer.json from cache at /home/fangli/.cache/huggingface/transformers/d53fc0fa09b8342651efd4073d75e19617b3e51287c2a535becda5808a8db287.fc9576039592f026ad76a1c231b89aee8668488c671dfbe6616bab2ed298d730
[INFO|modeling_utils.py:1027] 2021-01-14 20:34:29,701 >> loading weights file https://huggingface.co/roberta-base/resolve/main/pytorch_model.bin from cache at /home/fangli/.cache/huggingface/transformers/51ba668f7ff34e7cdfa9561e8361747738113878850a7d717dbc69de8683aaad.c7efaa30a0d80b2958b876969faa180e485944a849deee4ad482332de65365a7
[WARNING|modeling_utils.py:1135] 2021-01-14 20:34:32,134 >> Some weights of the model checkpoint at roberta-base were not used when initializing RobertaForTokenClassification: ['lm_head.bias', 'lm_head.dense.weight', 'lm_head.dense.bias', 'lm_head.layer_norm.weight', 'lm_head.layer_norm.bias', 'lm_head.decoder.weight']
- This IS expected if you are initializing RobertaForTokenClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing RobertaForTokenClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
[WARNING|modeling_utils.py:1146] 2021-01-14 20:34:32,134 >> Some weights of RobertaForTokenClassification were not initialized from the model checkpoint at roberta-base and are newly initialized: ['classifier.weight', 'classifier.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Traceback (most recent call last):
  File "/home/fangli/github/transformers/examples/token-classification/run_ner.py", line 428, in <module>
    main()
  File "/home/fangli/github/transformers/examples/token-classification/run_ner.py", line 319, in main
    load_from_cache_file=not data_args.overwrite_cache,
  File "/home/fangli/anaconda3/envs/nlp/lib/python3.6/site-packages/datasets/dataset_dict.py", line 303, in map
    for k, dataset in self.items()
  File "/home/fangli/anaconda3/envs/nlp/lib/python3.6/site-packages/datasets/dataset_dict.py", line 303, in <dictcomp>
    for k, dataset in self.items()
  File "/home/fangli/anaconda3/envs/nlp/lib/python3.6/site-packages/datasets/arrow_dataset.py", line 1240, in map
    update_data = does_function_return_dict(test_inputs, test_indices)
  File "/home/fangli/anaconda3/envs/nlp/lib/python3.6/site-packages/datasets/arrow_dataset.py", line 1211, in does_function_return_dict
    function(*fn_args, indices, **fn_kwargs) if with_indices else function(*fn_args, **fn_kwargs)
  File "/home/fangli/github/transformers/examples/token-classification/run_ner.py", line 290, in tokenize_and_align_labels
    is_split_into_words=True,
  File "/home/fangli/anaconda3/envs/nlp/lib/python3.6/site-packages/transformers/tokenization_utils_base.py", line 2329, in __call__
    **kwargs,
  File "/home/fangli/anaconda3/envs/nlp/lib/python3.6/site-packages/transformers/tokenization_utils_base.py", line 2514, in batch_encode_plus
    **kwargs,
  File "/home/fangli/anaconda3/envs/nlp/lib/python3.6/site-packages/transformers/models/gpt2/tokenization_gpt2_fast.py", line 155, in _batch_encode_plus
    f"You need to instantiate {self.__class__.__name__} with add_prefix_space=True "
AssertionError: You need to instantiate RobertaTokenizerFast with add_prefix_space=True to use it with pretokenized inputs.

```

Thanks for help! 
Best, 
Li


Discussion:
Hi,

I would like to report the same problem. I see this problem only with RoBERTa base or large and I am also using transformers4.2.2.

Any suggestions or help would be appreciated. 
Thanks.
Hi, 
I had the same issue. I solved it by adding add_prefix_space=True to the tokenizer.

Best
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Hi,

I am have the same issue. 
I am loading from json - 

`python $SCRATCH/transformers/examples/token-classification/run_ner.py \
  --model_name_or_path roberta-base \
  --train_file dict_structure/trivia_training.json \
  --validation_file dict_structure/trivia_val.json \
  --output_dir roberta_base_on_MITMovieNER/ \
  --do_train \
  --do_eval \
  --per_device_train_batch_size 64 \
  --per_device_eval_batch_size 20 \
  --num_train_epochs 40 \
  --overwrite_output_dir \
  --evaluation_strategy steps \
  --save_steps 1000 \
  --eval_steps 500 \
  --logging_first_step \`

Sorry, not sure if this is an issue on my end. @stefan-it 
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
This remains an issue using the official example and official task; it would be great to see this addressed.

Resolving patch:
diff --git a/examples/pytorch/token-classification/run_ner.py b/examples/pytorch/token-classification/run_ner.py
--- a/examples/pytorch/token-classification/run_ner.py
+++ b/examples/pytorch/token-classification/run_ner.py
@@ -304,13 +304,26 @@ def get_label_list(labels):
         revision=model_args.model_revision,
         use_auth_token=True if model_args.use_auth_token else None,
     )
-    tokenizer = AutoTokenizer.from_pretrained(
-        model_args.tokenizer_name if model_args.tokenizer_name else model_args.model_name_or_path,
-        cache_dir=model_args.cache_dir,
-        use_fast=True,
-        revision=model_args.model_revision,
-        use_auth_token=True if model_args.use_auth_token else None,
-    )
+
+    tokenizer_name_or_path = model_args.tokenizer_name if model_args.tokenizer_name else model_args.model_name_or_path
+    if config.model_type in {"gpt2", "roberta"}:
+        tokenizer = AutoTokenizer.from_pretrained(
+            tokenizer_name_or_path,
+            cache_dir=model_args.cache_dir,
+            use_fast=True,
+            revision=model_args.model_revision,
+            use_auth_token=True if model_args.use_auth_token else None,
+            add_prefix_space=True,
+        )
+    else:
+        tokenizer = AutoTokenizer.from_pretrained(
+            tokenizer_name_or_path,
+            cache_dir=model_args.cache_dir,
+            use_fast=True,
+            revision=model_args.model_revision,
+            use_auth_token=True if model_args.use_auth_token else None,
+        )
+
     model = AutoModelForTokenClassification.from_pretrained(
         model_args.model_name_or_path,
         from_tf=bool(".ckpt" in model_args.model_name_or_path),
diff --git a/examples/pytorch/token-classification/run_ner_no_trainer.py b/examples/pytorch/token-classification/run_ner_no_trainer.py
--- a/examples/pytorch/token-classification/run_ner_no_trainer.py
+++ b/examples/pytorch/token-classification/run_ner_no_trainer.py
@@ -317,16 +317,18 @@ def get_label_list(labels):
         config = CONFIG_MAPPING[args.model_type]()
         logger.warning("You are instantiating a new config instance from scratch.")
 
-    if args.tokenizer_name:
-        tokenizer = AutoTokenizer.from_pretrained(args.tokenizer_name, use_fast=True)
-    elif args.model_name_or_path:
-        tokenizer = AutoTokenizer.from_pretrained(args.model_name_or_path, use_fast=True)
-    else:
+    tokenizer_name_or_path = args.tokenizer_name if args.tokenizer_name else args.model_name_or_path
+    if not tokenizer_name_or_path:
         raise ValueError(
             "You are instantiating a new tokenizer from scratch. This is not supported by this script."
             "You can do it from another script, save it, and load it from here, using --tokenizer_name."
         )
 
+    if config.model_type in {"gpt2", "roberta"}:
+        tokenizer = AutoTokenizer.from_pretrained(tokenizer_name_or_path, use_fast=True, add_prefix_space=True)
+    else:
+        tokenizer = AutoTokenizer.from_pretrained(tokenizer_name_or_path, use_fast=True)
+
     if args.model_name_or_path:
         model = AutoModelForTokenClassification.from_pretrained(
             args.model_name_or_path,


Test patch: