julien-c HF staff commited on
Commit
fa175f6
β€’
1 Parent(s): 079ade4

Migrate model card from transformers-repo

Browse files

Read announcement at https://discuss.huggingface.co/t/announcement-all-model-cards-will-be-migrated-to-hf-co-model-repos/2755
Original file history: https://github.com/huggingface/transformers/commits/master/model_cards/wptoux/albert-chinese-large-qa/README.md

Files changed (1) hide show
  1. README.md +34 -0
README.md ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # albert-chinese-large-qa
2
+ Albert large QA model pretrained from baidu webqa and baidu dureader datasets.
3
+
4
+ ## Data source
5
+ + baidu webqa 1.0
6
+ + baidu dureader
7
+
8
+ ## Traing Method
9
+ We combined the two datasets together and created a new dataset in squad format, including 705139 samples for training and 69638 samples for validation.
10
+ We finetune the model based on the albert chinese large model.
11
+
12
+ ## Hyperparams
13
+ + learning_rate 1e-5
14
+ + max_seq_length 512
15
+ + max_query_length 50
16
+ + max_answer_length 300
17
+ + doc_stride 256
18
+ + num_train_epochs 2
19
+ + warmup_steps 1000
20
+ + per_gpu_train_batch_size 8
21
+ + gradient_accumulation_steps 3
22
+ + n_gpu 2 (Nvidia Tesla P100)
23
+
24
+ ## Usage
25
+ ```
26
+ from transformers import AutoModelForQuestionAnswering, BertTokenizer
27
+
28
+ model = AutoModelForQuestionAnswering.from_pretrained('wptoux/albert-chinese-large-qa')
29
+ tokenizer = BertTokenizer.from_pretrained('wptoux/albert-chinese-large-qa')
30
+ ```
31
+ ***Important: use BertTokenizer***
32
+
33
+ ## MoreInfo
34
+ Please visit https://github.com/wptoux/albert-chinese-large-webqa for details.