Autotokenizer.from_pretrained failing on NVIDIA V100 (aws ml.p3.2xlarge)
Author: pjbhaumik
Pull request: no
Labels: name: Core: Tokenization

name: bug

### System Info

transformers v4.45.0 
Autotokenizer.from_pretrained('path to file of saved tokenizer') hosted on AWS ml.p3.2xlarge (NVIDIA V100) produces the error below... The same transformer version, code and tokenizer do _NOT_ produce errors on AWS ml.g4dn series instances (NVIDIA T4)

setting transformers==4.44.2 in requirements.txt works as a quick fix... 


2024-09-26T06:58:55,208 [INFO ] W-9000-model_1.0-stdout MODEL_LOG -   File "/opt/ml/model/code/inference.py", line 24, in model_fn
2024-09-26T06:58:55,209 [INFO ] W-9000-model_1.0-stdout MODEL_LOG -     ce_tokenizer = AutoTokenizer.from_pretrained(model_dir+'/crossEncoder')
2024-09-26T06:58:55,209 [WARN ] W-9000-model_1.0-stderr MODEL_LOG -   File "/opt/ml/model/code/inference.py", line 24, in model_fn
2024-09-26T06:58:55,209 [INFO ] W-9000-model_1.0-stdout MODEL_LOG -   File "/opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 897, in from_pretrained
2024-09-26T06:58:55,209 [WARN ] W-9000-model_1.0-stderr MODEL_LOG -     ce_tokenizer = AutoTokenizer.from_pretrained(model_dir+'/crossEncoder')
2024-09-26T06:58:55,210 [INFO ] W-9000-model_1.0-stdout MODEL_LOG -     return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
2024-09-26T06:58:55,210 [WARN ] W-9000-model_1.0-stderr MODEL_LOG -   File "/opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 897, in from_pretrained
2024-09-26T06:58:55,210 [WARN ] W-9000-model_1.0-stderr MODEL_LOG -     return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
2024-09-26T06:58:55,210 [INFO ] W-9000-model_1.0-stdout MODEL_LOG -   File "/opt/conda/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2271, in from_pretrained
2024-09-26T06:58:55,211 [WARN ] W-9000-model_1.0-stderr MODEL_LOG -   File "/opt/conda/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2271, in from_pretrained
2024-09-26T06:58:55,211 [WARN ] W-9000-model_1.0-stderr MODEL_LOG -     return cls._from_pretrained(
2024-09-26T06:58:55,212 [INFO ] W-9000-model_1.0-stdout MODEL_LOG -     return cls._from_pretrained(
2024-09-26T06:58:55,212 [WARN ] W-9000-model_1.0-stderr MODEL_LOG -   File "/opt/conda/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2505, in _from_pretrained
2024-09-26T06:58:55,213 [WARN ] W-9000-model_1.0-stderr MODEL_LOG -     tokenizer = cls(*init_inputs, **init_kwargs)
2024-09-26T06:58:55,213 [INFO ] W-9000-model_1.0-stdout MODEL_LOG -   File "/opt/conda/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2505, in _from_pretrained
2024-09-26T06:58:55,213 [WARN ] W-9000-model_1.0-stderr MODEL_LOG -   File "/opt/conda/lib/python3.10/site-packages/transformers/models/roberta/tokenization_roberta_fast.py", line 140, in __init__
2024-09-26T06:58:55,213 [WARN ] W-9000-model_1.0-stderr MODEL_LOG -     super().__init__(
2024-09-26T06:58:55,213 [INFO ] W-9000-model_1.0-stdout MODEL_LOG -     tokenizer = cls(*init_inputs, **init_kwargs)
2024-09-26T06:58:55,214 [WARN ] W-9000-model_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: model, error: Worker died.
2024-09-26T06:58:55,214 [WARN ] W-9000-model_1.0-stderr MODEL_LOG -   File "/opt/conda/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 115, in __init__
2024-09-26T06:58:55,214 [INFO ] W-9000-model_1.0-stdout MODEL_LOG -   File "/opt/conda/lib/python3.10/site-packages/transformers/models/roberta/tokenization_roberta_fast.py", line 140, in __init__
2024-09-26T06:58:55,215 [WARN ] W-9000-model_1.0-stderr MODEL_LOG -     fast_tokenizer = TokenizerFast.from_file(fast_tokenizer_file)
2024-09-26T06:58:55,215 [INFO ] W-9000-model_1.0-stdout MODEL_LOG -     super().__init__(
2024-09-26T06:58:55,215 [INFO ] W-9000-model_1.0 org.pytorch.serve.wlm.WorkerThread - Auto recovery start timestamp: 1727333935215
2024-09-26T06:58:55,215 [WARN ] W-9000-model_1.0-stderr MODEL_LOG - Exception: data did not match any variant of untagged enum ModelWrapper at line 250356 column 3



### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Autotokenizer.from_pretrained('path to tokenizer') where the tokenizer was saved from cross-encoder/stsb-roberta-large to a local directory and then copied into the Sagemaker model artifact (artifact.tar.gz)


### Expected behavior

tokenizer should load using Autotokenizer.from_pretrained('path to tokenizer')

Comments:
Huh! I have not seen this error before, have you @ArthurZucker ?

Could you send me the link of the model you are trying to load? 🤗 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

I met the same problem too.  It happened when I trained model with tfms=4.45.2 while inference with tfms==4.44.2.

Save error