Update README.md

README.md

@@ -4,8 +4,10 @@ widget:
 - text: "[CLS] 万 叠 春 山 积 雨 晴 ，"
 - text: "[CLS] 大 漠"
 
+
 ---
 
+
 # Chinese Poem GPT2 Model
 
 ## Model description
@@ -52,7 +54,7 @@ The model is pre-trained by [UER-py](https://github.com/dbiir/UER-py/) on [Tence
 python3 preprocess.py --corpus_path corpora/poem.txt \
                       --vocab_path models/poem_zh_vocab.txt \
                       --dataset_path poem_dataset.pt --processes_num 16 \
-                      --seq_length 128 --target lm 
+                      --seq_length 128 --data_processor lm 
 ```
 
 ```
@@ -62,15 +64,12 @@ python3 pretrain.py --dataset_path poem_dataset.pt \
                     --output_model_path models/poem_gpt2_model.bin \
                     --world_size 8 --gpu_ranks 0 1 2 3 4 5 6 7 \
                     --total_steps 200000 --save_checkpoint_steps 50000 --report_steps 1000 \
-                    --learning_rate 5e-4 --batch_size 64 \
-                    --embedding word_pos --remove_embedding_layernorm \
-                    --encoder transformer --mask causal --layernorm_positioning pre \
-                    --target lm --tie_weights
+                    --learning_rate 5e-4 --batch_size 64
 ```
 
 Finally, we convert the pre-trained model into Huggingface's format:
 ```
-python3 scripts/convert_gpt2_from_uer_to_huggingface.py --input_model_path poem_gpt2_base_model.bin-200000 \
+python3 scripts/convert_gpt2_from_uer_to_huggingface.py --input_model_path poem_gpt2_model.bin-200000 \
                                                         --output_model_path pytorch_model.bin \
                                                         --layers_num 12
 ```