Albert large QA model pretrained from baidu webqa and baidu dureader datasets.

Data source

  • baidu webqa 1.0
  • baidu dureader

Traing Method

We combined the two datasets together and created a new dataset in squad format, including 705139 samples for training and 69638 samples for validation. We finetune the model based on the albert chinese large model.


  • learning_rate 1e-5
  • max_seq_length 512
  • max_query_length 50
  • max_answer_length 300
  • doc_stride 256
  • num_train_epochs 2
  • warmup_steps 1000
  • per_gpu_train_batch_size 8
  • gradient_accumulation_steps 3
  • n_gpu 2 (Nvidia Tesla P100)


from transformers import AutoModelForQuestionAnswering, BertTokenizer

model = AutoModelForQuestionAnswering.from_pretrained('wptoux/albert-chinese-large-qa')
tokenizer = BertTokenizer.from_pretrained('wptoux/albert-chinese-large-qa')

Important: use BertTokenizer


Please visit for details.


Select AutoNLP in the “Train” menu to fine-tune this model automatically.

Downloads last month
Hosted inference API
Question Answering
This model can be loaded on the Inference API on-demand.