suolyer commited on
Commit
6fc883c
1 Parent(s): 76af23e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +38 -0
README.md CHANGED
@@ -1,3 +1,41 @@
1
  ---
 
 
2
  license: apache-2.0
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language:
3
+ - zh
4
  license: apache-2.0
5
+ widget:
6
+ - text: "生活的真谛是[MASK]。"
7
  ---
8
+ # Zhouwenwang-110M model (Chinese),one model of [Fengshenbang-LM](https://github.com/IDEA-CCNL/Fengshenbang-LM).
9
+ We modify the original position code of longformer to rotational position coding,and on the basis of [chinese_roformer_L-12_H-768_A-12.zip](https://github.com/ZhuiyiTechnology/roformer), use 180G of data to continue training
10
+
11
+ ## Usage
12
+ There is no structure of Longformer-base in [Transformers](https://github.com/huggingface/transformers), you can run follow code to get structure of Zhouwenwang-110M from [Fengshenbang-LM](https://github.com/IDEA-CCNL/Fengshenbang-LM)
13
+
14
+ ```shell
15
+ git clone https://github.com/IDEA-CCNL/Fengshenbang-LM.git
16
+ ```
17
+
18
+ ### Load Model
19
+ ```python
20
+ from fengshen import LongformerModel
21
+ from fengshen import LongformerConfig
22
+ from transformers import BertTokenizer
23
+
24
+ tokenizer = BertTokenizer.from_pretrained("IDEA-CCNL/longformer_base")
25
+ config = LongformerConfig.from_pretrained("IDEA-CCNL/longformer_base")
26
+ model = LongformerModel.from_pretrained("IDEA-CCNL/longformer_base")
27
+ ```
28
+
29
+
30
+
31
+ ## Citation
32
+ If you find the resource is useful, please cite the following website in your paper.
33
+
34
+ ```
35
+ @misc{Fengshenbang-LM,
36
+ title={Fengshenbang-LM},
37
+ author={IDEA-CCNL},
38
+ year={2021},
39
+ howpublished={\url{https://github.com/IDEA-CCNL/Fengshenbang-LM}},
40
+ }
41
+ ```