SyntaxError when fine-tuning ALBERT on NER
Author: manueltonneau
Pull request: no
Labels: name: wontfix

name: Ex: Named Entity Recognition

name: Should Fix

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): ALBERT 

Language I am using the model on (English, Chinese ...):

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)


## To reproduce

Steps to reproduce the behavior:

Running on a GCP VM: 

1. 
```
python3 ${REPO_DIR}/src/transformers/convert_albert_original_tf_checkpoint_to_pytorch.py --tf_checkpoint_path ${BERT_CKPT_DIR} --albert_config_file ${BERT_CKPT_DIR}/config.json --pytorch_dump_path ${BERT_CKPT_DIR}/pytorch_model.bin
```
2.
```
python3 ${REPO_DIR}/examples/ner/run_ner.py --model_type albert --model_name_or_path ${BERT_CKPT_DIR} --do_train --do_eval --data_dir ${NER_DATA_DIR} --labels ${NER_DATA_DIR}/labels.txt --max_seq_length 128 --num_train_epochs 3 --per_gpu_train_batch_size 32 --output_dir ${NER_MODEL_CKPT_DIR} --seed 1 --do_predict --save_steps 750
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->
```
Traceback (most recent call last):
  File "transformers/src/transformers/convert_albert_original_tf_checkpoint_to_pytorch.py", line 23, in <module>
    from transformers import AlbertConfig, AlbertForMaskedLM, load_tf_weights_in_albert
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 634, in _load_backward_compatible
  File "/home/manuto/.local/lib/python3.5/site-packages/transformers-2.6.0-py3.5.egg/transformers/__init__.py", line 23, in <module>
    from .benchmark_utils import (
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 954, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 896, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1147, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1123, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1104, in _legacy_get_spec
  File "<frozen importlib._bootstrap>", line 444, in spec_from_loader
  File "<frozen importlib._bootstrap_external>", line 541, in spec_from_file_location
  File "/home/manuto/.local/lib/python3.5/site-packages/transformers-2.6.0-py3.5.egg/transformers/benchmark_utils.py", line 44
    filename: str
            ^
SyntaxError: invalid syntax
Exception ignored in: <function WeakValueDictionary.__init__.<locals>.remove at 0x7fc729d8a598>
Traceback (most recent call last):
  File "/usr/lib/python3.5/weakref.py", line 117, in remove
TypeError: 'NoneType' object is not callable
Traceback (most recent call last):
  File "transformers/examples/ner/run_ner.py", line 33, in <module>
    from transformers import (
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 634, in _load_backward_compatible
  File "/home/manuto/.local/lib/python3.5/site-packages/transformers-2.6.0-py3.5.egg/transformers/__init__.py", line 23, in <module>
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 954, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 896, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1147, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1123, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1104, in _legacy_get_spec
  File "<frozen importlib._bootstrap>", line 444, in spec_from_loader
  File "<frozen importlib._bootstrap_external>", line 541, in spec_from_file_location
  File "/home/manuto/.local/lib/python3.5/site-packages/transformers-2.6.0-py3.5.egg/transformers/benchmark_utils.py", line 44
    filename: str
            ^
SyntaxError: invalid syntax
Exception ignored in: <function WeakValueDictionary.__init__.<locals>.remove at 0x7f1bd863ed08>
Traceback (most recent call last):
  File "/usr/lib/python3.5/weakref.py", line 117, in remove
TypeError: 'NoneType' object is not callable
```


## Expected behavior

Convert ALBERT TF checkpoint to PyTorch `model.bin` (works in 2.5.1 version of transformers) and fine-tune the model on NER (error in 2.5.1 mentioned [here](https://github.com/huggingface/transformers/issues/3412) which is the reason why I switched to 2.6.0).
<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 2.6.0 (installed with setup.py)
- Platform: Linux Ubuntu 18.04
- Python version: 3.5.3
- PyTorch version (GPU?): 1.4.0
- Tensorflow version (GPU?): 1.14.0
- Using GPU in script?:
- Using distributed or parallel set-up in script?:


Comments:
`transformers` 2.6.0 has dropped support for Python 3.5: https://github.com/huggingface/transformers/releases/tag/v2.6.0

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.