AlbertTokenizer behavior doesn't match docs
Author: jswift24
Pull request: no
Labels: name: wontfix

## 🐛 Bug

<!-- Important information -->

Model I am using (Bert, XLNet....): AlbertForQuestionAnswering

Language I am using the model on (English, Chinese....): English

The problem arise when using:
* [x] the official example scripts: (give details)
* [ ] my own modified scripts: (give details)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details)

## To Reproduce

Steps to reproduce the behavior:

1. Run the code sample from the docs: https://huggingface.co/transformers/v2.2.0/model_doc/albert.html#albertforquestionanswering

```
tokenizer = AlbertTokenizer.from_pretrained('albert-base-v2')
model = AlbertForQuestionAnswering.from_pretrained('albert-base-v2')
question, text = "Who was Jim Henson?", "Jim Henson was a nice puppet"
input_text = "[CLS] " + question + " [SEP] " + text + " [SEP]"
input_ids = tokenizer.encode(input_text)
token_type_ids = [0 if i <= input_ids.index(102) else 1 for i in range(len(input_ids))]
start_scores, end_scores = model(torch.tensor([input_ids]), token_type_ids=torch.tensor([token_type_ids]))
all_tokens = tokenizer.convert_ids_to_tokens(input_ids)
print(' '.join(all_tokens[torch.argmax(start_scores) : torch.argmax(end_scores)+1]))
# a nice puppet
```

2. Getting the following error:
---------------------------------------------------------------------------
```
ValueError                                Traceback (most recent call last)
<ipython-input-16-2185be87fe39> in <module>
      5 input_text = "[CLS] " + question + " [SEP] " + text + " [SEP]"
      6 input_ids = tokenizer.encode(input_text)
----> 7 token_type_ids = [0 if i <= input_ids.index(102) else 1 for i in range(len(input_ids))]
      8 start_scores, end_scores = model(torch.tensor([input_ids]), token_type_ids=torch.tensor([token_type_ids]))
      9 all_tokens = tokenizer.convert_ids_to_tokens(input_ids)

<ipython-input-16-2185be87fe39> in <listcomp>(.0)
      5 input_text = "[CLS] " + question + " [SEP] " + text + " [SEP]"
      6 input_ids = tokenizer.encode(input_text)
----> 7 token_type_ids = [0 if i <= input_ids.index(102) else 1 for i in range(len(input_ids))]
      8 start_scores, end_scores = model(torch.tensor([input_ids]), token_type_ids=torch.tensor([token_type_ids]))
      9 all_tokens = tokenizer.convert_ids_to_tokens(input_ids)

ValueError: 102 is not in list
```


<!-- If you have a code sample, error messages, stack traces, please provide it here as well. -->

## Expected behavior
Code says expected output is "a nice puppet"
<!-- A clear and concise description of what you expected to happen. -->

## Environment

* OS: Kaggle kernel, no GPU
* Python version: 3.6.6
* PyTorch version: 1.3.0
* PyTorch Transformers version (or branch): 2.3.0
* Using GPU ? No
* Distributed of parallel setup ? No
* Any other relevant information:

Some debugging info:
```
print(input_ids)
[2, 2, 72, 23, 2170, 27674, 60, 3, 2170, 27674, 23, 21, 2210, 10956, 3, 3]

tokenizer.decode(input_ids)
'[CLS][CLS] who was jim henson?[SEP] jim henson was a nice puppet[SEP][SEP]'
```
## Additional context

<!-- Add any other context about the problem here. -->


Comments:
The encoder itself would automatically add the [CLS] and [SEP] tokens so if you've done that during preprocessing, you would need to change the `add_special_tokens` parameter to `False`. So your code should probably be like this:

```
input_ids = tokenizer.encode(input_text, add_special_tokens=False)
```

Thanks, but that does not fix the error. The problem is there is no [102] token in the list. Maybe because we're using AlbertTokenizer?

```
input_ids = tokenizer.encode(input_text, add_special_tokens=False)
print(input_ids)
[2, 72, 23, 2170, 27674, 60, 3, 2170, 27674, 23, 21, 2210, 10956, 3]
```

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.