jdemouth commited on
Commit
65c6843
1 Parent(s): 0871876

Suggestion to add Megatron to PYTHONPATH

Browse files

Explains how to add fix a ModuleNotFoundError by adding Megatron to the PYTHONPATH

Files changed (1) hide show
  1. README.md +16 -0
README.md CHANGED
@@ -118,6 +118,22 @@ wget --content-disposition https://api.ngc.nvidia.com/v2/models/nvidia/megatron_
118
  python src/transformers/models/megatron_gpt2/convert_megatron_gpt2_checkpoint.py megatron_lm_345m_v0.0.zip
119
  ```
120
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
121
  3. Fetch missing files
122
  ```
123
  git clone https://huggingface.co/nvidia/megatron-gpt2-345m/
118
  python src/transformers/models/megatron_gpt2/convert_megatron_gpt2_checkpoint.py megatron_lm_345m_v0.0.zip
119
  ```
120
 
121
+ As explained by Stas Bekman in [PR #14956](https://github.com/huggingface/transformers/pull/14956), if when running this conversion
122
+ script and you're getting an exception:
123
+ ```
124
+ ModuleNotFoundError: No module named 'megatron.model.enums'
125
+ ```
126
+ you need to tell python where to find the clone of Megatron-LM, e.g.:
127
+ ```
128
+ cd /tmp
129
+ git clone https://github.com/NVIDIA/Megatron-LM
130
+ PYTHONPATH=/tmp/Megatron-LM python src/transformers/models/megatron_bert/convert_megatron_bert_checkpoint.py ...
131
+ ```
132
+ Or, if you already have it cloned elsewhere, simply adjust the path to the existing path.
133
+
134
+ If the training was done using a Megatron-LM fork, e.g. [Megatron-DeepSpeed](https://github.com/microsoft/Megatron-DeepSpeed/) then
135
+ you may need to have that one in your path, i.e., /path/to/Megatron-DeepSpeed.
136
+
137
  3. Fetch missing files
138
  ```
139
  git clone https://huggingface.co/nvidia/megatron-gpt2-345m/