Exception: precision 16 is not supported. Float16Module (megatron_amp_O2) supports only fp16 and bf16.
State: closed | Pull request: no
Labels: name: bug

I'm trying to run [GPT-2B-001](https://huggingface.co/nvidia/GPT-2B-001) inside the NeMo Megatron docker container on Ubuntu 22.04 with RTX 4090

```
sudo docker pull nvcr.io/nvidia/nemo:23.02
sudo docker run  --runtime=nvidia -it --rm -v --shm-size=16g -p 5555:5555 -p 8889:8888 -p 6006:6006 --ulimit memlock=-1 --ulimit stack=67108864 nvcr.io/nvidia/nemo:23.02
```

Inside the containter

```
cd examples/nlp/language_modeling
wget https://huggingface.co/nvidia/GPT-2B-001/resolve/main/GPT-2B-001_bf16_tp1.nemo -O nemo_2b_bf16_tp1.nemo
HYDRA_FULL_ERROR=1 python megatron_gpt_eval.py gpt_model_file=nemo_2b_bf16_tp1.nemo server=True tensor_model_parallel_size=1 trainer.devices=1
```

The error
```
Error executing job with overrides: ['gpt_model_file=nemo_2b_bf16_tp1.nemo', 'server=True', 'tensor_model_parallel_size=1', 'trainer.devices=1']
Traceback (most recent call last):
  File "megatron_gpt_eval.py", line 279, in <module>
    main()  # noqa pylint: disable=no-value-for-parameter
  File "/usr/local/lib/python3.8/dist-packages/nemo/core/config/hydra_runner.py", line 105, in wrapper
    _run_hydra(
  File "/usr/local/lib/python3.8/dist-packages/hydra/_internal/utils.py", line 389, in _run_hydra
    _run_app(
  File "/usr/local/lib/python3.8/dist-packages/hydra/_internal/utils.py", line 452, in _run_app
    run_and_report(
  File "/usr/local/lib/python3.8/dist-packages/hydra/_internal/utils.py", line 216, in run_and_report
    raise ex
  File "/usr/local/lib/python3.8/dist-packages/hydra/_internal/utils.py", line 213, in run_and_report
    return func()
  File "/usr/local/lib/python3.8/dist-packages/hydra/_internal/utils.py", line 453, in <lambda>
    lambda: hydra.run(
  File "/usr/local/lib/python3.8/dist-packages/hydra/_internal/hydra.py", line 132, in run
    _ = ret.return_value
  File "/usr/local/lib/python3.8/dist-packages/hydra/core/utils.py", line 260, in return_value
    raise self._return_value
  File "/usr/local/lib/python3.8/dist-packages/hydra/core/utils.py", line 186, in run_job
    ret.return_value = task_function(task_cfg)
  File "megatron_gpt_eval.py", line 182, in main
    model = MegatronGPTModel.restore_from(
  File "/usr/local/lib/python3.8/dist-packages/nemo/core/classes/modelPT.py", line 436, in restore_from
    instance = cls._save_restore_connector.restore_from(
  File "/usr/local/lib/python3.8/dist-packages/nemo/collections/nlp/parts/nlp_overrides.py", line 366, in restore_from
    loaded_params = super().load_config_and_state_dict(
  File "/usr/local/lib/python3.8/dist-packages/nemo/core/connectors/save_restore_connector.py", line 162, in load_config_and_state_dict
    instance = calling_cls.from_config_dict(config=conf, trainer=trainer)
  File "/usr/local/lib/python3.8/dist-packages/nemo/core/classes/common.py", line 507, in from_config_dict
    raise e
  File "/usr/local/lib/python3.8/dist-packages/nemo/core/classes/common.py", line 499, in from_config_dict
    instance = cls(cfg=config, trainer=trainer)
  File "/usr/local/lib/python3.8/dist-packages/nemo/collections/nlp/models/language_modeling/megatron_gpt_model.py", line 128, in __init__
    self.model = Float16Module(module=self.model, precision=cfg.precision)
  File "/usr/local/lib/python3.8/dist-packages/nemo/collections/nlp/modules/common/megatron/module.py", line 278, in __init__
    raise Exception(
Exception: precision 16 is not supported. Float16Module (megatron_amp_O2) supports only fp16 and bf16.
```