PunctuationCapitalizationModel Config: Mixed Precision Parameter not Ignored when AMP_LEVEL=O0
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**

I've encountered this using the [example config provided ](https://github.com/NVIDIA/NeMo/blob/ed2005eda9470cb2c7c42e7ca1bd6b2e5b7a8479/examples/nlp/token_classification/conf/punctuation_capitalization_config.yaml#L28)for the PunctuationCapitalizationModel. Currently, the default trainer params are:
```
trainer:
  gpus=1
  amp_level: O0 # O1/O2 for mixed precision
  precision: 16 # Should be set to 16 for O1 and O2, default is 16 as PT ignores it when am_level is O0
```
I.e. the value for the precision param should be ignored when using AMP.FP32 training. However, if you try to restore a model (e.g. `punctuation-bert-en`) by overriding `trainer.gpus=0` with the above configuration, the following error is triggered:

```
Traceback (most recent call last):
  File "/home/lhr/tmp/textNormalization/text_normalization/punctuation_capitalization/profiling.py", line 193, in <module>
    m1, m2 = init_nemo_model()
  File "/home/lhr/tmp/textNormalization/text_normalization/punctuation_capitalization/profiling.py", line 141, in init_nemo_model
    trainer = pl.Trainer(**cfg.trainer)
  File "/home/lhr/.conda/envs/nemo-source/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/env_vars_connector.py", line 40, in insert_env_defaults
    return fn(self, **kwargs)
  File "/home/lhr/.conda/envs/nemo-source/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 354, in __init__
    self.accelerator_connector = AcceleratorConnector(
  File "/home/lhr/.conda/envs/nemo-source/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/accelerator_connector.py", line 162, in __init__
    self.accelerator = self.select_accelerator()
  File "/home/lhr/.conda/envs/nemo-source/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/accelerator_connector.py", line 700, in select_accelerator
    accelerator = acc_cls(training_type_plugin=self.training_type_plugin, precision_plugin=self.precision_plugin)
  File "/home/lhr/.conda/envs/nemo-source/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/accelerator_connector.py", line 333, in precision_plugin
    self._precision_plugin = self.select_precision_plugin()
  File "/home/lhr/.conda/envs/nemo-source/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/accelerator_connector.py", line 555, in select_precision_plugin
    raise MisconfigurationException(
pytorch_lightning.utilities.exceptions.MisconfigurationException: You have asked for native AMP on CPU, but AMP is only available on GPU.
```
I needed to additionally override `trainer.precision=32` in order to restore the model, suggesting that the precision parameter is not actually ignored for `amp_level=O0`.

**Expected behavior**
That the value of `trainer.precision` is ignored when `amp_level=O0`.

**Steps/Code to reproduce bug**

```
from omegaconf import OmegaConf
import sys
import pytorch_lightning as pl
from nemo.collections.nlp.models import PunctuationCapitalizationModel

def main(cfg):
  trainer = pl.Trainer(**cfg.trainer)
  model = PunctuationCapitalizationModel(cfg.model, trainer)

if __name__ == '__main__':
  sys.argv = [f"profiling.py",
              f"model.language_model.pretrained_model_name=bert-base-uncased",
              "model.dataset.data_dir=PATH_TO_DATA",
              "trainer.gpus=0",
              # "trainer.precision=32"  # need to toggle this 
              ]
  cfg = OmegaConf.load('conf/punctuation_capitalization_config.yaml')
  cli_conf = OmegaConf.from_cli()
  cfg = OmegaConf.merge(cfg, cli_conf)
  main(cfg)

```

**Environment overview (please complete the following information)**

 - Environment location: venv on local machine
 - Method of NeMo install: from source

**Environment details**

- OS version - Ubuntu 18.04.5 LTS
- PyTorch version - 1.9.0
- Python version - 3.8.11