Repository: huggingface/transformers

Issue:
pytest Errors
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
```
     ai) ubuntu@ip-10-0-1-82:~/transformers$ transformers-cli env
/home/ubuntu/anaconda2/envs/ai/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/ubuntu/anaconda2/envs/ai/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/ubuntu/anaconda2/envs/ai/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/ubuntu/anaconda2/envs/ai/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/ubuntu/anaconda2/envs/ai/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/ubuntu/anaconda2/envs/ai/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
/home/ubuntu/anaconda2/envs/ai/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/ubuntu/anaconda2/envs/ai/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/ubuntu/anaconda2/envs/ai/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/ubuntu/anaconda2/envs/ai/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/ubuntu/anaconda2/envs/ai/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/ubuntu/anaconda2/envs/ai/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
Traceback (most recent call last):
  File "/home/ubuntu/anaconda2/envs/ai/bin/transformers-cli", line 33, in <module>
    sys.exit(load_entry_point('transformers==3.4.0', 'console_scripts', 'transformers-cli')())
  File "/home/ubuntu/anaconda2/envs/ai/bin/transformers-cli", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/home/ubuntu/anaconda2/envs/ai/lib/python3.6/site-packages/importlib_metadata/__init__.py", line 105, in load
    module = import_module(match.group('module'))
  File "/home/ubuntu/anaconda2/envs/ai/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 941, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 941, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/ubuntu/anaconda2/envs/ai/lib/python3.6/site-packages/transformers-3.4.0-py3.6.egg/transformers/__init__.py", line 135, in <module>
    from .pipelines import (
  File "/home/ubuntu/anaconda2/envs/ai/lib/python3.6/site-packages/transformers-3.4.0-py3.6.egg/transformers/pipelines.py", line 38, in <module>
    from .tokenization_auto import AutoTokenizer
  File "/home/ubuntu/anaconda2/envs/ai/lib/python3.6/site-packages/transformers-3.4.0-py3.6.egg/transformers/tokenization_auto.py", line 210, in <module>
    (XLMProphetNetConfig, (XLMProphetNetTokenizer, None)),
NameError: name 'XLMProphetNetTokenizer' is not defined

- `transformers` version: 3.4.0
- Platform: Ubuntu 20.04 LTS
- Python version: 3.6.11
- PyTorch version (GPU?): 1.7.0 (no GPU)
- Tensorflow version (GPU?): 2.2.0 (no GPU)
- Using GPU in script?:
- Using distributed or parallel set-up in script?:
```

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.
 
 tokenizers: @mfuntowicz
 examples/distillation: @VictorSanh

 -->

## Information

## To reproduce

Steps to reproduce the behavior:

1. RUN_SLOW=1 pytest examples
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->



Discussion:
I got the same error while loading BERT tokeniser and model from torch hub
Hello! Do you mind pasting the result of `pip list` done in your environment? Thank you!
It’s an Anaconda virtual environment.
Python 3.6.11

$ pip list
Package                           Version             Location
--------------------------------- ------------------- ----------------------------------------------------------
absl-py                           0.11.0
aiohttp                           3.7.2
appdirs                           1.4.4
argon2-cffi                       20.1.0
astor                             0.8.1
astunparse                        1.6.3
async-generator                   1.10
async-timeout                     3.0.1
attrs                             20.2.0
Automat                           20.2.0
awscli                            1.18.169
Babel                             2.8.0
backcall                          0.2.0
backports.functools-lru-cache     1.6.1
bcrypt                            3.2.0
beautifulsoup4                    4.9.3
bertopic                          0.2.3
black                             20.8b1
bleach                            3.2.1
blinker                           1.4
bokeh                             2.2.3
boto                              2.49.0
boto3                             1.16.9
botocore                          1.19.9
brotlipy                          0.7.0
bz2file                           0.98
cachetools                        4.1.1
certifi                           2020.6.20
cffi                              1.14.3
chainer                           7.7.0
chardet                           3.0.4
click                             7.1.2
cloudpickle                       1.2.2
colorama                          0.4.3
constantly                        15.1.0
cryptography                      3.2.1
cssselect                         1.1.0
cycler                            0.10.0
cymem                             1.31.2
Cython                            0.29.21
dataclasses                       0.7
decorator                         4.4.2
deepdist                          0.1
defusedxml                        0.6.0
dill                              0.3.2
diskcache                         4.0.0
docutils                          0.15.2
entrypoints                       0.3
feynman                           2.0.0
filelock                          3.0.12
findspark                         1.3.0
Flask                             1.1.2
flatbuffers                       1.12
funcy                             1.15
future                            0.18.2
gast                              0.3.3
gensim                            3.8.3
google-auth                       1.23.0
google-auth-oauthlib              0.4.2
google-pasta                      0.2.0
googleapis-common-protos          1.52.0
grpcio                            1.33.2
h5py                              2.10.0
hdbscan                           0.8.26
html5lib                          1.1
hyperlink                         20.0.1
hypothesis                        5.41.0
idna                              2.10
idna-ssl                          1.1.0
importlib-metadata                2.0.0
incremental                       17.5.0
iniconfig                         1.1.1
ipykernel                         5.3.4
ipython                           7.12.0
ipython-genutils                  0.2.0
ipywidgets                        7.5.1
itemadapter                       0.1.1
itemloaders                       1.0.3
itsdangerous                      1.1.0
jedi                              0.17.2
Jinja2                            2.11.2
jmespath                          0.10.0
joblib                            0.17.0
json5                             0.9.5
jsonschema                        3.2.0
jupyter-client                    6.1.7
jupyter-console                   6.2.0
jupyter-contrib-core              0.3.3
jupyter-core                      4.6.3
jupyter-nbextensions-configurator 0.4.1
jupyterlab                        2.2.9
jupyterlab-pygments               0.1.2
jupyterlab-server                 1.2.0
Keras-Applications                1.0.8
Keras-Preprocessing               1.1.2
kiwisolver                        1.3.0
llvmlite                          0.34.0
lxml                              4.6.1
Markdown                          3.3.3
MarkupSafe                        1.1.1
matplotlib                        3.3.2
mistune                           0.8.4
mnist                             0.2.2
more-itertools                    8.6.0
mpmath                            1.1.0
MulticoreTSNE                     0.1
multidict                         4.7.5
murmurhash                        0.26.4
mypy-extensions                   0.4.3
nbclient                          0.5.1
nbconvert                         6.0.7
nbformat                          5.0.8
nest-asyncio                      1.4.1
nltk                              3.4.4
notebook                          6.1.4
numba                             0.51.2
numexpr                           2.7.1
numpy                             1.19.2
oauthlib                          3.0.1
olefile                           0.46
opt-einsum                        3.3.0
packaging                         20.4
pandas                            1.1.4
pandocfilters                     1.4.2
parameterized                     0.7.4
parsel                            1.6.0
parso                             0.7.1
pathspec                          0.8.0
patsy                             0.5.1
petastorm                         0.7.6               /home/ubuntu/petastorm
pexpect                           4.8.0
pickleshare                       0.7.5
Pillow                            8.0.1
pip                               20.2.4
plac                              1.0.0
pluggy                            0.13.1
preshed                           0.46.4
prometheus-client                 0.8.0
promise                           2.3
prompt-toolkit                    3.0.8
Protego                           0.1.16
protobuf                          3.13.0
psutil                            5.7.3
ptyprocess                        0.6.0
py                                1.9.0
py4j                              0.10.9
pyarrow                           2.0.0
pyasn1                            0.4.8
pyasn1-modules                    0.2.7
pycparser                         2.20
PyDispatcher                      2.0.5
pydot                             1.4.1
Pygments                          2.7.2
PyHamcrest                        2.0.2
PyJWT                             1.7.1
pyLDAvis                          2.1.2
pyOpenSSL                         19.1.0
pyparsing                         2.4.7
PyQt5                             5.12.3
PyQt5-sip                         4.19.18
PyQtChart                         5.12
PyQtWebEngine                     5.12.1
pyrsistent                        0.17.3
PySocks                           1.7.1
pyspark                           3.0.1
pytest                            6.1.2
python-dateutil                   2.8.1
pytz                              2020.1
PyWavelets                        1.1.1
PyYAML                            5.3.1
pyzmq                             19.0.2
qtconsole                         4.7.7
QtPy                              1.9.0
queuelib                          1.5.0
regex                             2020.10.28
requests                          2.24.0
requests-oauthlib                 1.3.0
rsa                               4.4.1
s3transfer                        0.3.3
sacremoses                        0.0.43
scapy                             2.4.4
scikit-learn                      0.23.2
scipy                             1.5.2
Scrapy                            2.4.0
seaborn                           0.11.0
semver                            2.8.1
Send2Trash                        1.5.0
sense2vec                         0.6.0
sentence-transformers             0.3.6
sentencepiece                     0.1.91
service-identity                  18.1.0
setuptools                        49.6.0.post20201009
six                               1.15.0
sklearn                           0.0
smart-open                        1.6.0
sortedcontainers                  2.2.2
soupsieve                         2.0.1
spacy                             0.101.0
sputnik                           0.9.3
statsmodels                       0.12.1
sympy                             1.6.2
tensorboard                       2.3.0
tensorboard-plugin-wit            1.7.0
tensorflow                        2.2.0
tensorflow-datasets               1.2.0
tensorflow-estimator              2.2.0
tensorflow-metadata               0.14.0
tensorflow-probability            0.6.0
tensorflowonspark                 1.4.1
termcolor                         1.1.0
terminado                         0.9.1
testpath                          0.4.4
tfp-nightly                       0.5.0.dev20190522
thinc                             5.0.8
threadpoolctl                     2.1.0
timeout-decorator                 0.4.1
tokenizers                        0.9.2
toml                              0.10.1
torch                             1.7.0
torchaudio                        0.7.0a0+ac17b64
torchvision                       0.8.1
tornado                           6.1
tqdm                              4.51.0
traitlets                         4.3.3
transformers                      3.1.0               /home/ubuntu/anaconda2/envs/ai/lib/python3.6/site-packages
Twisted                           20.3.0
twython                           3.8.2
typed-ast                         1.4.1
typing-extensions                 3.7.4.3
umap-learn                        0.4.6
urllib3                           1.25.11
w3lib                             1.22.0
wcwidth                           0.2.5
webencodings                      0.5.1
Werkzeug                          1.0.1
wheel                             0.35.1
widgetsnbextension                3.5.1
wordcloud                         1.8.0
wrapt                             1.12.1
yarl                              1.6.2
zipp                              3.4.0
zope.interface                    5.1.2
> On Nov 2, 2020, at 7:33 AM, Lysandre Debut <notifications@github.com> wrote:
> 
> 
> Hello! Do you mind pasting the result of pip list done in your environment? Thank you!
> 
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub <https://github.com/huggingface/transformers/issues/8196#issuecomment-720544945>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXWFW2Z4DMWNXRLVPMHSD3SN3GLNANCNFSM4TFJGMGQ>.
> 


It seems you have a conflict between your `transformers` version, as `transformers-cli env` returns v3.4.0, while your `pip list` returns v3.1.0?
Mea culpa!  I sent you the pip list from my Mac.
Here’s the Ubuntu 20.04 LTS results

$ conda list transformers
# packages in environment at /home/ubuntu/anaconda2/envs/ai:
#
# Name                    Version                   Build  Channel
sentence-transformers     0.3.6                    pypi_0    pypi
transformers              3.4.0                     dev_0    <develop>
(ai) ubuntu@ip-10-0-1-82:~/transformers$ 


$ pip list
Package                           Version             Location
--------------------------------- ------------------- ---------------------------------------------------------------------------------------
absl-py                           0.11.0
aiohttp                           3.7.2
appdirs                           1.4.4
argon2-cffi                       20.1.0
astor                             0.8.1
astunparse                        1.6.3
async-generator                   1.10
async-timeout                     3.0.1
attrs                             20.2.0
Automat                           20.2.0
awscli                            1.18.169
Babel                             2.8.0
backcall                          0.2.0
backports.functools-lru-cache     1.6.1
bcrypt                            3.2.0
beautifulsoup4                    4.9.3
bertopic                          0.2.3
black                             20.8b1
bleach                            3.2.1
blinker                           1.4
bokeh                             2.2.3
boto                              2.49.0
boto3                             1.16.9
botocore                          1.19.9
brotlipy                          0.7.0
bz2file                           0.98
cachetools                        4.1.1
certifi                           2020.6.20
cffi                              1.14.3
chainer                           7.7.0
chardet                           3.0.4
click                             7.1.2
cloudpickle                       1.2.2
colorama                          0.4.3
constantly                        15.1.0
cryptography                      3.2.1
cssselect                         1.1.0
cycler                            0.10.0
cymem                             1.31.2
Cython                            0.29.21
dataclasses                       0.7
decorator                         4.4.2
deepdist                          0.1
defusedxml                        0.6.0
dill                              0.3.2
diskcache                         4.0.0
docutils                          0.15.2
entrypoints                       0.3
feynman                           2.0.0
filelock                          3.0.12
findspark                         1.3.0
Flask                             1.1.2
flatbuffers                       1.12
funcy                             1.15
future                            0.18.2
gast                              0.3.3
gensim                            3.8.3
google-auth                       1.23.0
google-auth-oauthlib              0.4.2
google-pasta                      0.2.0
googleapis-common-protos          1.52.0
grpcio                            1.33.2
h5py                              2.10.0
hdbscan                           0.8.26
html5lib                          1.1
hyperlink                         20.0.1
hypothesis                        5.41.0
idna                              2.10
idna-ssl                          1.1.0
importlib-metadata                2.0.0
incremental                       17.5.0
iniconfig                         1.1.1
ipykernel                         5.3.4
ipython                           7.12.0
ipython-genutils                  0.2.0
ipywidgets                        7.5.1
itemadapter                       0.1.1
itemloaders                       1.0.3
itsdangerous                      1.1.0
jedi                              0.17.2
Jinja2                            2.11.2
jmespath                          0.10.0
joblib                            0.17.0
json5                             0.9.5
jsonschema                        3.2.0
jupyter-client                    6.1.7
jupyter-console                   6.2.0
jupyter-contrib-core              0.3.3
jupyter-core                      4.6.3
jupyter-nbextensions-configurator 0.4.1
jupyterlab                        2.2.9
jupyterlab-pygments               0.1.2
jupyterlab-server                 1.2.0
Keras-Applications                1.0.8
Keras-Preprocessing               1.1.2
kiwisolver                        1.3.0
llvmlite                          0.34.0
lxml                              4.6.1
Markdown                          3.3.3
MarkupSafe                        1.1.1
matplotlib                        3.3.2
mistune                           0.8.4
mnist                             0.2.2
more-itertools                    8.6.0
mpmath                            1.1.0
MulticoreTSNE                     0.1
multidict                         4.7.5
murmurhash                        0.26.4
mypy-extensions                   0.4.3
nbclient                          0.5.1
nbconvert                         6.0.7
nbformat                          5.0.8
nest-asyncio                      1.4.1
nltk                              3.4.4
notebook                          6.1.4
numba                             0.51.2
numexpr                           2.7.1
numpy                             1.19.2
oauthlib                          3.0.1
olefile                           0.46
opt-einsum                        3.3.0
packaging                         20.4
pandas                            1.1.4
pandocfilters                     1.4.2
parameterized                     0.7.4
parsel                            1.6.0
parso                             0.7.1
pathspec                          0.8.0
patsy                             0.5.1
petastorm                         0.7.6               /home/ubuntu/petastorm
pexpect                           4.8.0
pickleshare                       0.7.5
Pillow                            8.0.1
pip                               20.2.4
plac                              1.0.0
pluggy                            0.13.1
preshed                           0.46.4
prometheus-client                 0.8.0
promise                           2.3
prompt-toolkit                    3.0.8
Protego                           0.1.16
protobuf                          3.13.0
psutil                            5.7.3
ptyprocess                        0.6.0
py                                1.9.0
py4j                              0.10.9
pyarrow                           2.0.0
pyasn1                            0.4.8
pyasn1-modules                    0.2.7
pycparser                         2.20
PyDispatcher                      2.0.5
pydot                             1.4.1
Pygments                          2.7.2
PyHamcrest                        2.0.2
PyJWT                             1.7.1
pyLDAvis                          2.1.2
pyOpenSSL                         19.1.0
pyparsing                         2.4.7
PyQt5                             5.12.3
PyQt5-sip                         4.19.18
PyQtChart                         5.12
PyQtWebEngine                     5.12.1
pyrsistent                        0.17.3
PySocks                           1.7.1
pyspark                           3.0.1
pytest                            6.1.2
python-dateutil                   2.8.1
pytz                              2020.1
PyWavelets                        1.1.1
PyYAML                            5.3.1
pyzmq                             19.0.2
qtconsole                         4.7.7
QtPy                              1.9.0
queuelib                          1.5.0
regex                             2020.10.28
requests                          2.24.0
requests-oauthlib                 1.3.0
rsa                               4.4.1
s3transfer                        0.3.3
sacremoses                        0.0.43
scapy                             2.4.4
scikit-learn                      0.23.2
scipy                             1.5.2
Scrapy                            2.4.0
seaborn                           0.11.0
semver                            2.8.1
Send2Trash                        1.5.0
sense2vec                         0.6.0
sentence-transformers             0.3.6
sentencepiece                     0.1.91
service-identity                  18.1.0
setuptools                        49.6.0.post20201009
six                               1.15.0
sklearn                           0.0
smart-open                        1.6.0
sortedcontainers                  2.2.2
soupsieve                         2.0.1
spacy                             0.101.0
sputnik                           0.9.3
statsmodels                       0.12.1
sympy                             1.6.2
tensorboard                       2.3.0
tensorboard-plugin-wit            1.7.0
tensorflow                        2.2.0
tensorflow-datasets               1.2.0
tensorflow-estimator              2.2.0
tensorflow-metadata               0.14.0
tensorflow-probability            0.6.0
tensorflowonspark                 1.4.1
termcolor                         1.1.0
terminado                         0.9.1
testpath                          0.4.4
tfp-nightly                       0.5.0.dev20190522
thinc                             5.0.8
threadpoolctl                     2.1.0
timeout-decorator                 0.4.1
tokenizers                        0.9.2
toml                              0.10.1
torch                             1.7.0
torchaudio                        0.7.0a0+ac17b64
torchvision                       0.8.1
tornado                           6.1
tqdm                              4.51.0
traitlets                         4.3.3
transformers                      3.4.0               /home/ubuntu/anaconda2/envs/ai/lib/python3.6/site-packages/transformers-3.4.0-py3.6.egg
Twisted                           20.3.0
twython                           3.8.2
typed-ast                         1.4.1
typing-extensions                 3.7.4.3
umap-learn                        0.4.6
urllib3                           1.25.11
w3lib                             1.22.0
wcwidth                           0.2.5
webencodings                      0.5.1
Werkzeug                          1.0.1
wheel                             0.35.1
widgetsnbextension                3.5.1
wordcloud                         1.8.0
wrapt                             1.12.1
yarl                              1.6.2
zipp                              3.4.0
zope.interface                    5.1.2
> On Nov 2, 2020, at 9:15 AM, Lysandre Debut <notifications@github.com> wrote:
> 
> 
> It seems you have a conflict between your transformers version, as transformers-cli env returns v3.4.0, while your pip list returns v3.1.0?
> 
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub <https://github.com/huggingface/transformers/issues/8196#issuecomment-720607259>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXWFWYIWSYOAK3B7CD7PRTSN3SM7ANCNFSM4TFJGMGQ>.
> 



Resolving patch:
diff --git a/src/transformers/tokenization_auto.py b/src/transformers/tokenization_auto.py
--- a/src/transformers/tokenization_auto.py
+++ b/src/transformers/tokenization_auto.py
@@ -113,6 +113,7 @@
     T5Tokenizer = None
     XLMRobertaTokenizer = None
     XLNetTokenizer = None
+    XLMProphetNetTokenizer = None
 
 if is_tokenizers_available():
     from .tokenization_albert_fast import AlbertTokenizerFast


Test patch: