Error loading roberta-base encoder for trained Punctuation-Capitalization model during evaluation using example scripts
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

I am using the provided [punctuation_capitalization_train.py](https://github.com/NVIDIA/NeMo/blob/main/examples/nlp/token_classification/punctuation_capitalization_train.py) and [punctuation_capitalization_evaluate.py](https://github.com/NVIDIA/NeMo/blob/main/examples/nlp/token_classification/punctuation_capitalization_evaluate.py) scripts provided in examples to train and evaluate a Punctuation-Capitalization model with roberta-base encoder (which is supported by NeMo).

I am able to train the model successfully with [punctuation_capitalization_train.py](https://github.com/NVIDIA/NeMo/blob/main/examples/nlp/token_classification/punctuation_capitalization_train.py). However, when evaluating this model with [punctuation_capitalization_evaluate.py](https://github.com/NVIDIA/NeMo/blob/main/examples/nlp/token_classification/punctuation_capitalization_evaluate.py), there is an error reloading the model. I've tried retraining another model from scratch to confirm that this wasn't due to a problem that happened while saving the model to disk, but did not make a difference.

I've followed the exact same procedure but with using `model.language_model.pretrained_model_name=bert-base-uncased` and `model.language_model.pretrained_model_name=distilbert-base-uncased`, but am not able to reproduce the issue with these encoders. Decoding error seems to only occur with `roberta-base` models.

**Steps/Code to reproduce bug**

1. Train a model with a `roberta-base` language model: `python punctuation_capitalization_train.py model.dataset.data_dir=data/punc-cap/de/indomain model.language_model.pretrained_model_name=roberta-base`. 
-> Model trains successfully, `.nemo` model saved to disk without error


2. Evaluate this saved model: `python punctuation_capitalization_evaluate.py model.dataset.data_dir=data/punc-cap/combined-dataset/ pretrained_model=nemo_experiments/Punctuation_and_Capitalization/2021-09-02_08-32-18/checkpoints/Punctuation_and_Capitalization.nemo`
-> Model fails to load (error in loading the vocab file?). The error gets caught and reported as `roberta-base` not being supported, although it is.

Log:
```
GPU available: True, used: True                                                                                                                                                                                                                                                                                [27/1988]
TPU available: False, using: 0 TPU cores
Using native 16bit precision.
[NeMo I 2021-09-03 12:03:02 exp_manager:219] Experiments will be logged at /home/lhr/nemo_experiments/Punctuation_and_Capitalization/2021-09-03_12-03-02
[NeMo I 2021-09-03 12:03:02 exp_manager:568] TensorboardLogger has been set up
[NeMo I 2021-09-03 12:03:11 tokenizer_utils:100] Getting HuggingFace AutoTokenizer with pretrained_model_name: roberta-base, vocab_file: /tmp/tmpcfbw34f8/7a630e42c1974fc5999881de0c5a9871_tokenizer.vocab_file, special_tokens_dict: {}, and use_fast: False
[NeMo I 2021-09-03 12:03:15 tokenizer_utils:100] Getting HuggingFace AutoTokenizer with pretrained_model_name: roberta-base, vocab_file: /tmp/tmpcfbw34f8/7a630e42c1974fc5999881de0c5a9871_tokenizer.vocab_file, special_tokens_dict: {}, and use_fast: False
Error executing job with overrides: ['model.dataset.data_dir=data/punc-cap/combined-dataset/', 'pretrained_model=nemo_experiments/Punctuation_and_Capitalization/2021-09-02_08-32-18/checkpoints/Punctuation_and_Capitalization.nemo']
Traceback (most recent call last):
  File "/home/lhr/.conda/envs/nemo/lib/python3.8/site-packages/nemo/collections/common/tokenizers/huggingface/auto_tokenizer.py", line 104, in __init__
    self.tokenizer = AUTOTOKENIZER.from_pretrained(
  File "/home/lhr/.conda/envs/nemo/lib/python3.8/site-packages/transformers/models/auto/tokenization_auto.py", line 566, in from_pretrained
    return tokenizer_class_py.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
  File "/home/lhr/.conda/envs/nemo/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 1719, in from_pretrained
    return cls._from_pretrained(
  File "/home/lhr/.conda/envs/nemo/lib/python3.8/site-packages/transformers/tokenization_utils_base.py", line 1792, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
  File "/home/lhr/.conda/envs/nemo/lib/python3.8/site-packages/transformers/models/roberta/tokenization_roberta.py", line 159, in __init__
    super().__init__(
  File "/home/lhr/.conda/envs/nemo/lib/python3.8/site-packages/transformers/models/gpt2/tokenization_gpt2.py", line 180, in __init__
    self.encoder = json.load(vocab_handle)
  File "/home/lhr/.conda/envs/nemo/lib/python3.8/json/__init__.py", line 293, in load
    return loads(fp.read(),
  File "/home/lhr/.conda/envs/nemo/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/home/lhr/.conda/envs/nemo/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/lhr/.conda/envs/nemo/lib/python3.8/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tmp/textNormalization/text_normalization/punctuation_capitalization/punctuation_capitalization_evaluate.py", line 80, in main
    model = PunctuationCapitalizationModel.restore_from(cfg.pretrained_model)
  File "/home/lhr/.conda/envs/nemo/lib/python3.8/site-packages/nemo/collections/nlp/models/nlp_model.py", line 429, in restore_from
    return super().restore_from(restore_path, override_config_path, map_location, strict, return_config)
  File "/home/lhr/.conda/envs/nemo/lib/python3.8/site-packages/nemo/core/classes/modelPT.py", line 481, in restore_from
    return cls._default_restore_from(restore_path, override_config_path, map_location, strict, return_config)
  File "/home/lhr/.conda/envs/nemo/lib/python3.8/site-packages/nemo/core/classes/modelPT.py", line 434, in _default_restore_from
    instance = cls.from_config_dict(config=conf)
  File "/home/lhr/.conda/envs/nemo/lib/python3.8/site-packages/nemo/core/classes/common.py", line 472, in from_config_dict
    instance = cls(cfg=config)
  File "/home/lhr/.conda/envs/nemo/lib/python3.8/site-packages/nemo/collections/nlp/models/token_classification/punctuation_capitalization_model.py", line 57, in __init__
    self.setup_tokenizer(cfg.tokenizer)
  File "/home/lhr/.conda/envs/nemo/lib/python3.8/site-packages/nemo/collections/nlp/models/nlp_model.py", line 112, in setup_tokenizer
    tokenizer = get_tokenizer(
  File "/home/lhr/.conda/envs/nemo/lib/python3.8/site-packages/nemo/collections/nlp/modules/common/tokenizer_utils.py", line 103, in get_tokenizer
    return AutoTokenizer(
  File "/home/lhr/.conda/envs/nemo/lib/python3.8/site-packages/nemo/collections/common/tokenizers/huggingface/auto_tokenizer.py", line 112, in __init__
    raise ValueError(f'{pretrained_model_name} is not supported by HuggingFace. {e}')
ValueError: roberta-base is not supported by HuggingFace. Expecting value: line 1 column 1 (char 0)
```


**Expected behavior**

Should be able to load the model for inference.

**Environment overview (please complete the following information)**

 - Environment location: local
 - Method of NeMo install: pip (following instructions on main page, `apt-get update && apt-get install -y libsndfile1 ffmpeg
pip install Cython
pip install nemo_toolkit['all']`)


**Environment details**

- OS version: Ubuntu 18.04.5 LTS
- PyTorch version: 1.9.0
- Python version: 3.8.11
- nemo-toolkit version: 1.2.0

**Additional context**

Using 1 GPU, CUDA Version: 11.2