uer commited on
Commit
b1a5f1b
1 Parent(s): 50b5083

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -65,7 +65,7 @@ python3 pretrain.py --dataset_path poem_dataset.pt \
65
  --learning_rate 5e-4 --batch_size 64 \
66
  --embedding word_pos --remove_embedding_layernorm \
67
  --encoder transformer --mask causal --layernorm_positioning pre \
68
- --target lm --tie_weight
69
  ```
70
 
71
  Finally, we convert the pre-trained model into Huggingface's format:
 
65
  --learning_rate 5e-4 --batch_size 64 \
66
  --embedding word_pos --remove_embedding_layernorm \
67
  --encoder transformer --mask causal --layernorm_positioning pre \
68
+ --target lm --tie_weights
69
  ```
70
 
71
  Finally, we convert the pre-trained model into Huggingface's format: