uer commited on
Commit
be18daa
1 Parent(s): 48a70f9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -147,7 +147,7 @@ python3 pretrain.py --dataset_path cluecorpussmall_seq512_dataset.pt \
147
  --world_size 8 --gpu_ranks 0 1 2 3 4 5 6 7 \
148
  --total_steps 250000 --save_checkpoint_steps 50000 --report_steps 10000 \
149
  --learning_rate 5e-5 --batch_size 16 \
150
- --tie_weights --embedding word_pos_seg --encoder transformer --mask fully_visible --target mlm
151
  ```
152
 
153
  Finally, we convert the pre-trained model into Huggingface's format:
 
147
  --world_size 8 --gpu_ranks 0 1 2 3 4 5 6 7 \
148
  --total_steps 250000 --save_checkpoint_steps 50000 --report_steps 10000 \
149
  --learning_rate 5e-5 --batch_size 16 \
150
+ --embedding word_pos_seg --encoder transformer --mask fully_visible --target mlm --tie_weights
151
  ```
152
 
153
  Finally, we convert the pre-trained model into Huggingface's format: