Repository: huggingface/transformers

Issue:
Issue in running run_glue.py in Roberta, XLNet, XLM in latest release
## 🐛 Bug

<!-- Important information -->

Model I am using (Bert, XLNet....): Roberta, XLM, XLNet

Language I am using the model on (English, Chinese....): English

The problem arise when using:
* [x] the official example scripts: run_glue.py

The tasks I am working on is:
* [x] my own task or dataset: binary classification on my own dataset

## To Reproduce

Steps to reproduce the behavior:

1. Upon running glue.py with the following : python ~/pytorch-transformers/examples/run_glue.py --task_name cola --do_train  --do_eval  --do_lower_case --data_dir ~/bert-data/ --model_type roberta --model_name_or_path  roberta-base  --max_seq_length 512  --learning_rate 2e-5 --num_train_epochs 1.0   --output_dir ~/data/roberta-1/

Getting the following error:
```
08/16/2019 14:18:21 - WARNING - pytorch_transformers.tokenization_utils -   Token indices sequence length is longer than the specified maximum sequence length for this model (513 > 512). Running this sequence through the model will result in indexing errors
Traceback (most recent call last):
  File "/home/pytorch-transformers/examples/run_glue.py", line 494, in <module>
    main()
  File "/home/pytorch-transformers/examples/run_glue.py", line 447, in main
    train_dataset = load_and_cache_examples(args, args.task_name, tokenizer, evaluate=False)
  File "/home/pytorch-transformers/examples/run_glue.py", line 283, in load_and_cache_examples
    pad_token_segment_id=4 if args.model_type in ['xlnet'] else 0,
  File "/home/pytorch-transformers/examples/utils_glue.py", line 485, in convert_examples_to_features
    assert len(input_ids) == max_seq_length
AssertionError
```
2. Trying the same as above with XLNet and XLM, gives the following error:

```
08/16/2019 14:26:59 - INFO - __main__ -   Creating features from dataset file at /home/new-bert-data/keyword_data/
Traceback (most recent call last):
  File "/home/pytorch-transformers/examples/run_glue.py", line 494, in <module>
    main()
  File "/home/pytorch-transformers/examples/run_glue.py", line 447, in main
    train_dataset = load_and_cache_examples(args, args.task_name, tokenizer, evaluate=False)
  File "/home/pytorch-transformers/examples/run_glue.py", line 282, in load_and_cache_examples
    pad_token=tokenizer.encoder[tokenizer.pad_token] if args.model_type in ['roberta'] else tokenizer.vocab[tokenizer.pad_token],
AttributeError: 'XLMTokenizer' object has no attribute 'vocab'
```

## Environment

* OS: Debian
* Python version: 3.6.9
* PyTorch version: 1.2.0
* PyTorch Transformers version (or branch): 1.1.0
* Using GPU: Yes
* Distributed of parallel setup: Multi-GPU 



Discussion:
Thanks for the report! I'm looking into it.

Resolving patch:
diff --git a/examples/run_glue.py b/examples/run_glue.py
--- a/examples/run_glue.py
+++ b/examples/run_glue.py
@@ -279,7 +279,7 @@ def load_and_cache_examples(args, task, tokenizer, evaluate=False):
             sep_token=tokenizer.sep_token,
             sep_token_extra=bool(args.model_type in ['roberta']),           # roberta uses an extra separator b/w pairs of sentences, cf. github.com/pytorch/fairseq/commit/1684e166e3da03f5b600dbb7855cb98ddfcd0805
             pad_on_left=bool(args.model_type in ['xlnet']),                 # pad on the left for xlnet
-            pad_token=tokenizer.encoder[tokenizer.pad_token] if args.model_type in ['roberta'] else tokenizer.vocab[tokenizer.pad_token],
+            pad_token=tokenizer.convert_tokens_to_ids([tokenizer.pad_token])[0],
             pad_token_segment_id=4 if args.model_type in ['xlnet'] else 0,
         )
         if args.local_rank in [-1, 0]:
diff --git a/examples/utils_glue.py b/examples/utils_glue.py
--- a/examples/utils_glue.py
+++ b/examples/utils_glue.py
@@ -425,9 +425,10 @@ def convert_examples_to_features(examples, label_list, max_seq_length,
             # Account for [CLS], [SEP], [SEP] with "- 3"
             _truncate_seq_pair(tokens_a, tokens_b, max_seq_length - 3)
         else:
-            # Account for [CLS] and [SEP] with "- 2"
-            if len(tokens_a) > max_seq_length - 2:
-                tokens_a = tokens_a[:(max_seq_length - 2)]
+            # Account for [CLS] and [SEP] with "- 2" and with "- 3" for RoBERTa.
+            special_tokens_count = 3 if sep_token_extra else 2
+            if len(tokens_a) > max_seq_length - special_tokens_count:
+                tokens_a = tokens_a[:(max_seq_length - special_tokens_count)]
 
         # The convention in BERT is:
         # (a) For sequence pairs:


Test patch: