uer commited on
Commit
eb5d751
1 Parent(s): 1b55a5c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -4
README.md CHANGED
@@ -45,7 +45,7 @@ Training data contains 700,000 Chinese couplets which are collected by [couplet-
45
 
46
  ## Training procedure
47
 
48
- Models are pre-trained by [UER-py](https://github.com/dbiir/UER-py/) on [Tencent Cloud TI-ONE](https://cloud.tencent.com/product/tione/). We pre-train 25,000 steps with a sequence length of 64.
49
 
50
  ```
51
  python3 preprocess.py --corpus_path corpora/couplet.txt \
@@ -58,7 +58,7 @@ python3 preprocess.py --corpus_path corpora/couplet.txt \
58
  python3 pretrain.py --dataset_path couplet_dataset.pt \
59
  --vocab_path models/google_zh_vocab.txt \
60
  --output_model_path models/couplet_gpt2_base_model.bin \
61
- --config_path models/bert_base_config.json \
62
  --world_size 8 --gpu_ranks 0 1 2 3 4 5 6 7 \
63
  --total_steps 25000 --save_checkpoint_steps 5000 --report_steps 1000 \
64
  --learning_rate 5e-4 --batch_size 64 \
@@ -88,5 +88,4 @@ python3 scripts/convert_gpt2_from_uer_to_huggingface.py --input_model_path coupl
88
  }
89
  ```
90
 
91
- [couplet]: https://huggingface.co/uer/gpt2-chinese-couplet
92
-
 
45
 
46
  ## Training procedure
47
 
48
+ The model is pre-trained by [UER-py](https://github.com/dbiir/UER-py/) on [Tencent Cloud TI-ONE](https://cloud.tencent.com/product/tione/). We pre-train 25,000 steps with a sequence length of 64.
49
 
50
  ```
51
  python3 preprocess.py --corpus_path corpora/couplet.txt \
 
58
  python3 pretrain.py --dataset_path couplet_dataset.pt \
59
  --vocab_path models/google_zh_vocab.txt \
60
  --output_model_path models/couplet_gpt2_base_model.bin \
61
+ --config_path models/gpt2/config.json \
62
  --world_size 8 --gpu_ranks 0 1 2 3 4 5 6 7 \
63
  --total_steps 25000 --save_checkpoint_steps 5000 --report_steps 1000 \
64
  --learning_rate 5e-4 --batch_size 64 \
 
88
  }
89
  ```
90
 
91
+ [couplet]: https://huggingface.co/uer/gpt2-chinese-couplet