Update README.md

README.md

@@ -12,10 +12,18 @@ widget:
 ## Model description
 
 This model is pre-trained by [UER-py](https://arxiv.org/abs/1909.05658).
+This model is pre-trained by [UER-py](https://github.com/dbiir/UER-py/), which is introduced in [this paper](https://arxiv.org/abs/1909.05658).
+
+You can download the set of Chinese BART models either from the [UER-py Modelzoo page](https://github.com/dbiir/UER-py/wiki/Modelzoo), or via HuggingFace from the links below:
+
+|                   |              Link              |
+| ----------------- | :----------------------------: |
+| **BART-Base** | [**L=6/H=768 (Base)**][base] |
+| **BART-Large**  | [**L=12/H=1024 (Large)**][large] |
 
 ## How to use
 
-You can use this model directly with a pipeline for text2text generation :
+You can use this model directly with a pipeline for text2text generation (take the case of BART-Base):
 
 ```python
 >>> from transformers import BertTokenizer, BartForConditionalGeneration, Text2TextGenerationPipeline
@@ -33,14 +41,14 @@ You can use this model directly with a pipeline for text2text generation :
 ## Training procedure
 
 The model is pre-trained by [UER-py](https://github.com/dbiir/UER-py/) on [Tencent Cloud](https://cloud.tencent.com/). We pre-train 1,000,000 steps with a sequence length of 512.
-
+Taking the case of BART-Base
 
 ```
 python3 preprocess.py --corpus_path corpora/cluecorpussmall.txt \
                       --vocab_path models/google_zh_vocab.txt \
                       --dataset_path cluecorpussmall_bart_seq512_dataset.pt \
                       --processes_num 32 --seq_length 512 \
-                      --data_processor bart 
+                      --data_processor bart
 ```
 
 ```
@@ -50,14 +58,14 @@ python3 pretrain.py --dataset_path cluecorpussmall_bart_seq512_dataset.pt \
                     --output_model_path models/cluecorpussmall_bart_base_seq512_model.bin \
                     --world_size 8 --gpu_ranks 0 1 2 3 4 5 6 7 \
                     --total_steps 1000000 --save_checkpoint_steps 100000 --report_steps 50000 \
-                    --learning_rate 1e-4 --batch_size 16 \
+                    --learning_rate 5e-5 --batch_size 8 \
                     --span_masking --span_max_length 3
 ```
 
 Finally, we convert the pre-trained model into Huggingface's format:
 
 ```
-python3 scripts/convert_bart_from_uer_to_huggingface.py --input_model_path cluecorpussmall_bart_base_seq512_model.bin-250000 \                                                                
+python3 scripts/convert_bart_from_uer_to_huggingface.py --input_model_path cluecorpussmall_bart_base_seq512_model.bin-1000000 \                                                                
                                                         --output_model_path pytorch_model.bin \                                                                                            
                                                         --layers_num 6
 ```
@@ -80,4 +88,7 @@ python3 scripts/convert_bart_from_uer_to_huggingface.py --input_model_path cluec
   pages={241},
   year={2019}
 }
-```
+```
+
+[base]:https://huggingface.co/uer/bart-base-chinese-cluecorpussmall
+[large]:https://huggingface.co/uer/bart-large-chinese-cluecorpussmall