hhou435 commited on
Commit
9736dc3
1 Parent(s): b9fef9a
Files changed (1) hide show
  1. README.md +3 -25
README.md CHANGED
@@ -35,34 +35,12 @@ You can use this model directly with a pipeline for text2text generation :
35
  The model is pre-trained by [UER-py](https://github.com/dbiir/UER-py/) on [Tencent Cloud](https://cloud.tencent.com/). We pre-train 1,000,000 steps with a sequence length of 512.
36
 
37
 
38
- ```
39
- python3 preprocess.py --corpus_path corpora/cluecorpussmall.txt \
40
- --vocab_path models/google_zh_vocab.txt \
41
- --dataset_path cluecorpussmall_bart_seq512_dataset.pt \
42
- --processes_num 32 --seq_length 512 \
43
- --target bart
44
- ```
45
-
46
- ```
47
- python3 pretrain.py --dataset_path cluecorpussmall_bart_seq512_dataset.pt \
48
- --vocab_path models/google_zh_vocab.txt \
49
- --config_path models/bart/base_config.json \
50
- --output_model_path models/cluecorpussmall_bart_base_seq512_model.bin \
51
- --world_size 8 --gpu_ranks 0 1 2 3 4 5 6 7 \
52
- --total_steps 1000000 --save_checkpoint_steps 100000 --report_steps 50000 \
53
- --learning_rate 1e-4 --batch_size 16 \
54
- --span_masking --span_max_length 3 \
55
- --embedding word_pos --tgt_embedding word_pos \
56
- --encoder transformer --mask fully_visible --decoder transformer \
57
- --target bart --tie_weights --has_lmtarget_bias
58
- ```
59
-
60
- Finally, we convert the pre-trained model into Huggingface's format:
61
 
62
  ```
63
- python3 scripts/convert_bart_from_uer_to_huggingface.py --input_model_path cluecorpussmall_bart_base_seq512_model.bin-250000 \
64
  --output_model_path pytorch_model.bin \
65
- --layers_num 6
66
  ```
67
 
68
 
35
  The model is pre-trained by [UER-py](https://github.com/dbiir/UER-py/) on [Tencent Cloud](https://cloud.tencent.com/). We pre-train 1,000,000 steps with a sequence length of 512.
36
 
37
 
38
+ we convert the pre-trained model into Huggingface's format:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
39
 
40
  ```
41
+ python3 scripts/convert_bart_from_uer_to_huggingface.py --input_model_path cluecorpussmall_bart_medium_seq512_model.bin-250000 \
42
  --output_model_path pytorch_model.bin \
43
+ --layers_num 4
44
  ```
45
 
46