[BUG] restore_from() method for GPTQAModel 
State: closed | Pull request: no
Labels: name: bug

name: stale

I have tried to fine-tune my .nemo model for question answering task in my own dataset, but the process have many bugs that I can fix my self.
Here is my code
```
CUDA_VISIBLE_DEVICES=0,1 python ~/dat/NeMo/examples/nlp/question_answering/question_answering.py \
    --config-path=conf \
    --config-name=qa_conf \
    trainer.devices=1 \
    trainer.num_nodes=1 \
    trainer.max_epochs=20 \
    trainer.max_steps=-1 \
    trainer.accumulate_grad_batches=1 \
    trainer.precision=16 \
    model.library=megatron \
    model.language_model.pretrained_model_name=gpt2 \
    model.nemo_path=./nemo_experiments/megatron_gpt/checkpoints/megatron_gpt.nemo \
    model.dataset.use_cache=False \
    model.dataset.version_2_with_negative=True \
    model.train_ds.file=./data/MRC_VLSP/v2_train_ViQuAD_new.json \
    model.validation_ds.file=./data/MRC_VLSP/v2_dev_ViQuAD.json \
    model.test_ds.file=./data/MRC_VLSP/v2_test_ViQuAD.json \
    model.train_ds.batch_size=128 \
    model.validation_ds.batch_size=128 \
    model.test_ds.batch_size=128 \
    model.tokenizer.tokenizer_name=sentencepiece \
    model.tokenizer.tokenizer_model=./data/tokenizer/spm_100mb_wiki.model \
    model.optim.lr=1e-4 \
    exp_manager.exp_dir=./qna-experiments \
    exp_manager.name=QnA-VLSP \
    exp_manager.create_wandb_logger=False 
```
And this is response of terminal.
```
[NeMo I 2023-03-20 09:00:16 question_answering:51] Restoring model from ./nemo_experiments/megatron_gpt/checkpoints/megatron_gpt.nemo
[NeMo E 2023-03-20 09:00:16 common:503] Model instantiation failed!
    Target class:       nemo.collections.nlp.models.language_modeling.megatron_gpt_model.MegatronGPTModel
    Error(s):   Trainer cannot be None for Megatron-based models. Please provide a PTL trainer object.
    Traceback (most recent call last):
      File "/home/jovyan/dat/NeMo/nemo/core/classes/common.py", line 482, in from_config_dict
        instance = imported_cls(cfg=config, trainer=trainer)
      File "/home/jovyan/dat/NeMo/nemo/collections/nlp/models/language_modeling/megatron_gpt_model.py", line 78, in __init__
        super().__init__(cfg, trainer=trainer, no_lm_init=True)
      File "/home/jovyan/dat/NeMo/nemo/collections/nlp/models/language_modeling/megatron_base_model.py", line 65, in __init__
        raise ValueError(f"Trainer cannot be None for Megatron-based models. Please provide a PTL trainer object.")
    ValueError: Trainer cannot be None for Megatron-based models. Please provide a PTL trainer object.
    
Error executing job with overrides: ['trainer.devices=1', 'trainer.num_nodes=1', 'trainer.max_epochs=20', 'trainer.max_steps=-1', 'trainer.accumulate_grad_batches=1', 'trainer.precision=16', 'model.library=megatron', 'model.language_model.pretrained_model_name=gpt2', 'model.nemo_path=./nemo_experiments/megatron_gpt/checkpoints/megatron_gpt.nemo', 'model.dataset.use_cache=False', 'model.dataset.version_2_with_negative=True', 'model.train_ds.file=./data/MRC_VLSP/v2_train_ViQuAD_new.json', 'model.validation_ds.file=./data/MRC_VLSP/v2_dev_ViQuAD.json', 'model.test_ds.file=./data/MRC_VLSP/v2_test_ViQuAD.json', 'model.train_ds.batch_size=128', 'model.validation_ds.batch_size=128', 'model.test_ds.batch_size=128', 'model.tokenizer.tokenizer_name=sentencepiece', 'model.tokenizer.tokenizer_model=./data/tokenizer/spm_100mb_wiki.model', 'model.optim.lr=1e-4', 'exp_manager.exp_dir=./qna-experiments', 'exp_manager.name=QnA-VLSP', 'exp_manager.create_wandb_logger=False']
Traceback (most recent call last):
  File "/home/jovyan/dat/NeMo/examples/nlp/question_answering/question_answering.py", line 52, in main
    model = model_class.restore_from(cfg.model.nemo_path)
  File "/home/jovyan/dat/NeMo/nemo/core/classes/modelPT.py", line 311, in restore_from
    instance = cls._save_restore_connector.restore_from(
  File "/home/jovyan/dat/NeMo/nemo/core/connectors/save_restore_connector.py", line 235, in restore_from
    loaded_params = self.load_config_and_state_dict(
  File "/home/jovyan/dat/NeMo/nemo/core/connectors/save_restore_connector.py", line 158, in load_config_and_state_dict
    instance = calling_cls.from_config_dict(config=conf, trainer=trainer)
  File "/home/jovyan/dat/NeMo/nemo/core/classes/common.py", line 504, in from_config_dict
    raise e
  File "/home/jovyan/dat/NeMo/nemo/core/classes/common.py", line 496, in from_config_dict
    instance = cls(cfg=config, trainer=trainer)
  File "/home/jovyan/dat/NeMo/nemo/collections/nlp/models/question_answering/qa_gpt_model.py", line 35, in __init__
    self.setup_tokenizer(cfg.tokenizer)
  File "/home/jovyan/dat/NeMo/nemo/collections/nlp/models/nlp_model.py", line 200, in setup_tokenizer
    if cfg.tokenizer_name or vocab_file or cfg.tokenizer_model:
omegaconf.errors.ConfigAttributeError: Key 'tokenizer_name' is not in struct
    full_key: tokenizer.tokenizer_name
    object_type=dict
```

This is the whole error, and now I will go in to description. Firstly, to my knowledge, method restore_from() will extract all config from my `megatron_gpt.nemo` file, and build tokenizer and also model weight to training. But I have try to printed my config and it show me that
```
{'micro_batch_size': 128, 'global_batch_size': 512, 'tensor_model_parallel_size': 1, 'pipeline_model_parallel_size': 1, 'resume_from_checkpoint': None, 'encoder_seq_length': 1024, 'max_position_embeddings': 1024, 'num_layers': 12, 'hidden_size': 768, 'ffn_hidden_size': 3072, 'num_attention_heads': 12, 'init_method_std': 0.021, 'hidden_dropout': 0.1, 'kv_channels': None, 'apply_query_key_layer_scaling': True, 'layernorm_epsilon': 1e-05, 'make_vocab_size_divisible_by': 128, 'pre_process': True, 'post_process': True, 'persist_layer_norm': True, 'tokenizer': {'library': 'sentencepiece', 'type': 'GPT2BPETokenizer', 'model': 'nemo:a9c1fd3579454343baabf2926c114ac5_spm_100mb_wiki.model', 'vocab_file': None, 'merge_file': None, 'delimiter': None, 'sentencepiece_legacy': False}, 'native_amp_init_scale': 4294967296, 'native_amp_growth_interval': 1000, 'hysteresis': 2, 'fp32_residual_connection': False, 'fp16_lm_cross_entropy': False, 'megatron_amp_O2': False, 'grad_allreduce_chunk_size_mb': 125, 'grad_div_ar_fusion': True, 'seed': 1234, 'use_cpu_initialization': False, 'onnx_safe': False, 'apex_transformer_log_level': 30, 'gradient_as_bucket_view': True, 'gradient_accumulation_fusion': False, 'activations_checkpoint_granularity': None, 'activations_checkpoint_method': None, 'activations_checkpoint_num_layers': None, 'sequence_parallel': False, 'data': {'data_prefix': [1.0, './data/gpt_training_data_passage_content_document'], 'index_mapping_dir': None, 'data_impl': 'mmap', 'splits_string': '980,10,10', 'seq_length': 256, 'skip_warmup': True, 'num_workers': 2, 'dataloader_type': 'single', 'reset_position_ids': False, 'reset_attention_mask': False, 'eod_mask_loss': False}, 'nsys_profile': {'enabled': False, 'start_step': 10, 'end_step': 10, 'ranks': [0], 'gen_shape': False}, 'optim': {'name': 'fused_adam', 'lr': 0.0006, 'weight_decay': 0.1, 'betas': [0.9, 0.95], 'sched': {'name': 'CosineAnnealing', 'warmup_steps': 750, 'constant_steps': 80000, 'min_lr': 6e-05}}, 'precision': 16, 'target': 'nemo.collections.nlp.models.language_modeling.megatron_gpt_model.MegatronGPTModel', 'nemo_version': '1.11.0'}
```
This config then will be go in to self.setup_tokenizer(cfg.tokenizer), but it misses many argument eg: tokenizer_name, tokenizer_model, vocab_file,... I do not know why my nemo file missing the config but I have trained it completely and successfully inference. 
And if the tokenizer bulit-in use the .nemo config, this means the all model.tokenizer configs in the qa_conf.yaml file not to be used for the "megatron" type.