Update README.md

README.md

@@ -5,7 +5,7 @@ language:
 license: apache-2.0
 
 tags:
-  - bert
+  - DeBERTa
 
 inference: true
 
@@ -13,7 +13,7 @@ widget:
 - text: "生活的真谛是[MASK]。"
 ---
 
-# Erlangshen-Deberta-97M-Chinese
+# Erlangshen-DeBERTa-v2-97M-Chinese
 
 - Github: [Fengshenbang-LM](https://github.com/IDEA-CCNL/Fengshenbang-LM)
 - Docs: [Fengshenbang-Docs](https://fengshenbang-doc.readthedocs.io/)
@@ -35,9 +35,9 @@ Good at solving NLU tasks, adopting Whole Word Masking, Chinese DeBERTa-v2 with
 
 参考论文：[Deberta](https://readpaper.com/paper/3033187248)
 
-为了得到一个中文版的DeBERTa-v2（97M），我们用悟道语料库(180G版本)进行预训练。具体地，我们在预训练阶段中使用了[封神框架](https://github.com/IDEA-CCNL/Fengshenbang-LM/tree/main/fengshen)大概花费了24张A100约7天。
+为了得到一个中文版的DeBERTa-v2（97M），我们用悟道语料库(180G版本)进行预训练。我们在MLM中使用了全词掩码(wwm)的方式。具体地，我们在预训练阶段中使用了[封神框架](https://github.com/IDEA-CCNL/Fengshenbang-LM/tree/main/fengshen)大概花费了24张A100约7天。
 
-To get a Chinese DeBERTa-v2 (97M), we use WuDao Corpora (180 GB version) for pre-training. Specifically, we use the [fengshen framework](https://github.com/IDEA-CCNL/Fengshenbang-LM/tree/main/fengshen) in the pre-training phase which cost about 7 days with 24 A100 GPUs.
+To get a Chinese DeBERTa-v2 (97M), we use WuDao Corpora (180 GB version) for pre-training. We employ the Whole Word Masking (wwm) in MLM. Specifically, we use the [fengshen framework](https://github.com/IDEA-CCNL/Fengshenbang-LM/tree/main/fengshen) in the pre-training phase which cost about 7 days with 24 A100 GPUs.
 
 ### 下游任务 Performance