yf commited on
Commit
e378f9b
1 Parent(s): 98b1945

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +36 -0
README.md ADDED
@@ -0,0 +1,36 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Chinese BART-Large
2
+
3
+ ## Model description
4
+
5
+ This is an implementation of CPT-Large.
6
+
7
+ [**CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation**](https://arxiv.org/pdf/2109.05729.pdf)
8
+
9
+ Yunfan Shao, Zhichao Geng, Yitao Liu, Junqi Dai, Fei Yang, Li Zhe, Hujun Bao, Xipeng Qiu
10
+
11
+ ## Usage
12
+
13
+ ```python
14
+ >>> from modeling_cpt import CPTForConditionalGeneration
15
+ >>> from transformers import BertTokenizer
16
+ >>> tokenizer = BertTokenizer.from_pretrained("fnlp/cpt-large")
17
+ >>> model = CPTForConditionalGeneration.from_pretrained("fnlp/cpt-large")
18
+ >>> inputs = tokenizer.encode("北京是[MASK]的首都", return_tensors='pt')
19
+ >>> pred_ids = model.generate(input_ids, num_beams=4, max_length=20)
20
+ >>> print(tokenizer.convert_ids_to_tokens(pred_ids[i]))
21
+ ['[SEP]', '[CLS]', '北', '京', '是', '中', '国', '的', '首', '都', '[SEP]']
22
+ ```
23
+
24
+ **Note: Please use BertTokenizer for the model vocabulary. DO NOT use original BartTokenizer.**
25
+
26
+ ## Citation
27
+
28
+ ```bibtex
29
+ @article{shao2021cpt,
30
+ title={CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation},
31
+ author={Yunfan Shao and Zhichao Geng and Yitao Liu and Junqi Dai and Fei Yang and Li Zhe and Hujun Bao and Xipeng Qiu},
32
+ journal={arXiv preprint arXiv:2109.05729},
33
+ year={2021}
34
+ }
35
+ ```
36
+