p208p2002 commited on
Commit
c1d0724
1 Parent(s): fddd627

add readme

Browse files
Files changed (1) hide show
  1. README.md +21 -0
README.md ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ## Usage
2
+ Please use BertTokenizerFast as tokenizer instead of AutoTokenizer.
3
+ 請使用 BertTokenizerFast 而非 AutoTokenizer。
4
+ ```
5
+ from transformers import (
6
+ BertTokenizerFast,
7
+ AutoModelForCausalLM,
8
+ )
9
+
10
+ tokenizer = BertTokenizerFast.from_pretrained('bert-base-chinese')
11
+ model = AutoModelForCausalLM.from_pretrained('ckiplab/gpt2-base-chinese')
12
+ ```
13
+ ### Input Format
14
+ ```
15
+ C' = [c1, c2, ..., [HL], a1, ..., a|A|, [HL], ..., c|C|]
16
+ ```
17
+ ### Input Example
18
+ ```
19
+ 哈利·波特是英國作家[HL]羅琳[HL]撰寫的七部幻想小說系列。
20
+ ```
21
+ > 誰撰寫哈利·波特?