AutoTokenizer | TypeError: an integer is required (got type NoneType)
Author: danielbellhv
Pull request: no

Wasn't sure if this was for `Transformers` or `Tokenizers` libraries.

Based on [SO post](https://stackoverflow.com/q/70699247/17840900).

Goal: Amend this [Notebook][1] to work with **distilbert-base-uncased** model

Error occurs in **Section 1.3**.

Kernel: `conda_pytorch_p36`. I did Restart & Run All, and refreshed file view in working directory.

---

Section 1.3:
```python
# define the tokenizer
tokenizer = AutoTokenizer.from_pretrained(
        configs.output_dir, do_lower_case=configs.do_lower_case)
```

Traceback:
```
Evaluating PyTorch full precision accuracy and performance:
/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/data/processors/glue.py:67: FutureWarning: This function will be removed from the library soon, preprocessing should be handled with the 🤗 Datasets library. You can have a look at this example script for pointers: https://github.com/huggingface/transformers/blob/master/examples/pytorch/text-classification/run_glue.py
  warnings.warn(DEPRECATION_WARNING.format("function"), FutureWarning)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-31-1f864e3046eb> in <module>
    144 # Evaluate the original FP32 BERT model
    145 print('Evaluating PyTorch full precision accuracy and performance:')
--> 146 time_model_evaluation(model, configs, tokenizer)
    147 
    148 # Evaluate the INT8 BERT model after the dynamic quantization

<ipython-input-31-1f864e3046eb> in time_model_evaluation(model, configs, tokenizer)
    132 def time_model_evaluation(model, configs, tokenizer):
    133     eval_start_time = time.time()
--> 134     result = evaluate(configs, model, tokenizer, prefix="")
    135     eval_end_time = time.time()
    136     eval_duration_time = eval_end_time - eval_start_time

<ipython-input-31-1f864e3046eb> in evaluate(args, model, tokenizer, prefix)
     22     results = {}
     23     for eval_task, eval_output_dir in zip(eval_task_names, eval_outputs_dirs):
---> 24         eval_dataset = load_and_cache_examples(args, eval_task, tokenizer, evaluate=True)
     25 
     26         if not os.path.exists(eval_output_dir) and args.local_rank in [-1, 0]:

<ipython-input-31-1f864e3046eb> in load_and_cache_examples(args, task, tokenizer, evaluate)
    121     all_input_ids = torch.tensor([f.input_ids for f in features], dtype=torch.long)
    122     all_attention_mask = torch.tensor([f.attention_mask for f in features], dtype=torch.long)
--> 123     all_token_type_ids = torch.tensor([f.token_type_ids for f in features], dtype=torch.long)
    124     if output_mode == "classification":
    125         all_labels = torch.tensor([f.label for f in features], dtype=torch.long)

TypeError: an integer is required (got type NoneType)
```

Please let me know if there's anything else I can add to post.

  [1]: https://github.com/microsoft/onnxruntime-inference-examples/blob/main/quantization/notebooks/bert/Bert-GLUE_OnnxRuntime_quantization.ipynb

Comments:
Hey @danielbellhv, you're switching from BERT, which has the concept of token type IDs, to DistilBERT, which doesn't. In most cases that isn't important as our API should be able to handle that behind the scenes.

In this notebook however, there's a very explicit conversion of all token IDs here:
```
all_token_type_ids = torch.tensor([f.token_type_ids for f in features], dtype=torch.long)
```
This prevents the switch from models which do leverage token type IDs to others which do not. I recommend you take a look at the [following example](https://github.com/huggingface/transformers/blob/master/examples/pytorch/text-classification/run_glue_no_trainer.py) to see how we use our tools to be agnostic to all models.

In your current situation here, I would remove all explicit mentions to `token_type_ids`. DistilBERT wasn't trained with them, so it's fine to remove the concept of this variable when switching to this model.

Thank you, @LysandreJik.

I'll try implementing suggested changes in a new copy of the Notebook.

---

Note for anyone else.

This isn't as simple as removing the aforementioned line:
> ```
> all_token_type_ids = torch.tensor([f.token_type_ids for f in features], dtype=torch.long)
> ```

That yields:
```
Evaluating PyTorch full precision accuracy and performance:
Evaluating PyTorch full precision accuracy and performance:
---------------------------------------------------------------------------
UnboundLocalError                         Traceback (most recent call last)
<ipython-input-19-2d6264ca04e9> in <module>
    146 # Evaluate the original FP32 BERT model
    147 print('Evaluating PyTorch full precision accuracy and performance:')
--> 148 time_model_evaluation(model, configs, tokenizer)
    149 
    150 # Evaluate the INT8 BERT model after the dynamic quantization

<ipython-input-19-2d6264ca04e9> in time_model_evaluation(model, configs, tokenizer)
    134 def time_model_evaluation(model, configs, tokenizer):
    135     eval_start_time = time.time()
--> 136     result = evaluate(configs, model, tokenizer, prefix="")
    137     eval_end_time = time.time()
    138     eval_duration_time = eval_end_time - eval_start_time

<ipython-input-19-2d6264ca04e9> in evaluate(args, model, tokenizer, prefix)
     22     results = {}
     23     for eval_task, eval_output_dir in zip(eval_task_names, eval_outputs_dirs):
---> 24         eval_dataset = load_and_cache_examples(args, eval_task, tokenizer, evaluate=True)
     25 
     26         if not os.path.exists(eval_output_dir) and args.local_rank in [-1, 0]:

<ipython-input-19-2d6264ca04e9> in load_and_cache_examples(args, task, tokenizer, evaluate)
    129         all_labels = torch.tensor([f.label for f in features], dtype=torch.float)
    130 
--> 131     dataset = TensorDataset(all_input_ids, all_attention_mask, all_token_type_ids, all_labels)
    132     return dataset
    133 

UnboundLocalError: local variable 'all_token_type_ids' referenced before assignment
Evaluating PyTorch full precision accuracy and performance:
---------------------------------------------------------------------------
UnboundLocalError                         Traceback (most recent call last)
<ipython-input-19-2d6264ca04e9> in <module>
    146 # Evaluate the original FP32 BERT model
    147 print('Evaluating PyTorch full precision accuracy and performance:')
--> 148 time_model_evaluation(model, configs, tokenizer)
    149 
    150 # Evaluate the INT8 BERT model after the dynamic quantization

<ipython-input-19-2d6264ca04e9> in time_model_evaluation(model, configs, tokenizer)
    134 def time_model_evaluation(model, configs, tokenizer):
    135     eval_start_time = time.time()
--> 136     result = evaluate(configs, model, tokenizer, prefix="")
    137     eval_end_time = time.time()
    138     eval_duration_time = eval_end_time - eval_start_time

<ipython-input-19-2d6264ca04e9> in evaluate(args, model, tokenizer, prefix)
     22     results = {}
     23     for eval_task, eval_output_dir in zip(eval_task_names, eval_outputs_dirs):
---> 24         eval_dataset = load_and_cache_examples(args, eval_task, tokenizer, evaluate=True)
     25 
     26         if not os.path.exists(eval_output_dir) and args.local_rank in [-1, 0]:

<ipython-input-19-2d6264ca04e9> in load_and_cache_examples(args, task, tokenizer, evaluate)
    129         all_labels = torch.tensor([f.label for f in features], dtype=torch.float)
    130 
--> 131     dataset = TensorDataset(all_input_ids, all_attention_mask, all_token_type_ids, all_labels)
    132     return dataset
    133 

UnboundLocalError: local variable 'all_token_type_ids' referenced before assignment
```


There's about 4-5 lines which should need to be removed AFAICT from the notebook, searching for `token_type_ids` should show all of them