suolyer commited on
Commit
a925679
1 Parent(s): 689f704

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +40 -40
README.md CHANGED
@@ -1,41 +1,41 @@
1
- ---
2
- language:
3
- - zh
4
- license: apache-2.0
5
- widget:
6
- - text: "生活的真谛是[MASK]。"
7
- ---
8
- # longformer model (Chinese),one model of [Fengshenbang-LM](https://github.com/IDEA-CCNL/Fengshenbang-LM).
9
- We modify the original position code of longformer to rotational position coding,and on the basis of [chinese_roformer_L-12_H-768_A-12.zip](https://github.com/ZhuiyiTechnology/roformer), use 180G of data to continue training
10
-
11
- ## Usage
12
- There is no structure of Longformer-base in [Transformers](https://github.com/huggingface/transformers), you can run follow code to get structure of longformer from [Fengshenbang-LM](https://github.com/IDEA-CCNL/Fengshenbang-LM)
13
-
14
- ```shell
15
- git clone https://github.com/IDEA-CCNL/Fengshenbang-LM.git
16
- ```
17
-
18
- ### Load Model
19
- ```python
20
- from fengshen import LongformerModel
21
- from fengshen import LongformerConfig
22
- from transformers import BertTokenizer
23
-
24
- tokenizer = BertTokenizer.from_pretrained("IDEA-CCNL/longformer_base")
25
- config = LongformerConfig.from_pretrained("IDEA-CCNL/longformer_base")
26
- model = LongformerModel.from_pretrained("IDEA-CCNL/longformer_base")
27
- ```
28
-
29
-
30
-
31
- ## Citation
32
- If you find the resource is useful, please cite the following website in your paper.
33
-
34
- ```
35
- @misc{Fengshenbang-LM,
36
- title={Fengshenbang-LM},
37
- author={IDEA-CCNL},
38
- year={2021},
39
- howpublished={\url{https://github.com/IDEA-CCNL/Fengshenbang-LM}},
40
- }
41
  ```
 
1
+ ---
2
+ language:
3
+ - zh
4
+ license: apache-2.0
5
+ widget:
6
+ - text: "生活的真谛是[MASK]。"
7
+ ---
8
+ # longformer model (Chinese),one model of [Fengshenbang-LM](https://github.com/IDEA-CCNL/Fengshenbang-LM).
9
+ We modify the original position code of longformer to rotational position coding,and on the basis of [chinese_roformer_L-12_H-768_A-12.zip](https://github.com/ZhuiyiTechnology/roformer), use 180G of data to continue training
10
+
11
+ ## Usage
12
+ There is no structure of Longformer-base in [Transformers](https://github.com/huggingface/transformers), you can run follow code to get structure of longformer from [Fengshenbang-LM](https://github.com/IDEA-CCNL/Fengshenbang-LM)
13
+
14
+ ```shell
15
+ git clone https://github.com/IDEA-CCNL/Fengshenbang-LM.git
16
+ ```
17
+
18
+ ### Load Model
19
+ ```python
20
+ from fengshen import LongformerModel
21
+ from fengshen import LongformerConfig
22
+ from transformers import BertTokenizer
23
+
24
+ tokenizer = BertTokenizer.from_pretrained("IDEA-CCNL/Erlangshen-Longformer-110M")
25
+ config = LongformerConfig.from_pretrained("IDEA-CCNL/Erlangshen-Longformer-110M")
26
+ model = LongformerModel.from_pretrained("IDEA-CCNL/Erlangshen-Longformer-110M")
27
+ ```
28
+
29
+
30
+
31
+ ## Citation
32
+ If you find the resource is useful, please cite the following website in your paper.
33
+
34
+ ```
35
+ @misc{Fengshenbang-LM,
36
+ title={Fengshenbang-LM},
37
+ author={IDEA-CCNL},
38
+ year={2021},
39
+ howpublished={\url{https://github.com/IDEA-CCNL/Fengshenbang-LM}},
40
+ }
41
  ```