Extract QA Model (SQuAD2.0)
Model Information
Pretrained model: google/bert_uncased_L-12_H-768_A-12
Training Hyperparameters
epochs = 2
batch_size = 24
learning_rate = 3e-5
max_seq_length = 384
doc_stride = 128
max_query_length = 256
Latest Result
// 11 Apr 2023, 11:08 (Staging Epoch 1, total epoch 4, Correct Tokenizer)
{
"exact": 74.43780005053483,
"f1": 77.49749091378419,
"total": 11873,
"HasAns_exact": 73.73481781376518,
"HasAns_f1": 79.86297395738177,
"HasAns_total": 5928,
"NoAns_exact": 75.13877207737595,
"NoAns_f1": 75.13877207737595,
"NoAns_total": 5945,
"best_exact": 74.43780005053483,
"best_exact_thresh": 0.0,
"best_f1": 77.49749091378418,
"best_f1_thresh": 0.0
}
// 11 Apr 2023, 09:38 (Staging Epoch 3, total epoch 4, Correct Tokenizer)
{
"exact": 73.99983155057694,
"f1": 77.23749498407376,
"total": 11873,
"HasAns_exact": 72.85762483130904,
"HasAns_f1": 79.3422364955984,
"HasAns_total": 5928,
"NoAns_exact": 75.13877207737595,
"NoAns_f1": 75.13877207737595,
"NoAns_total": 5945,
"best_exact": 73.99983155057694,
"best_exact_thresh": 0.0,
"best_f1": 77.23749498407373,
"best_f1_thresh": 0.0
}
// 24 Mar 2023, 19:57 (Invalid Tokenizer)
{
"exact": 74.12616861787248,
"f1": 77.34212395572948,
"total": 11873,
"HasAns_exact": 72.72267206477733,
"HasAns_f1": 79.16380528447645,
"HasAns_total": 5928,
"NoAns_exact": 75.52565180824222,
"NoAns_f1": 75.52565180824222,
"NoAns_total": 5945,
"best_exact": 74.12616861787248,
"best_exact_thresh": 0.0,
"best_f1": 77.34212395572948,
"best_f1_thresh": 0.0
}
// 23 Mar 2023, (Invalid Tokenizer)
{
"exact": 73.77242482944496,
"f1": 76.83752039897598,
"total": 11873,
"HasAns_exact": 71.72739541160594,
"HasAns_f1": 77.86637646711235,
"HasAns_total": 5928,
"NoAns_exact": 75.81160639192599,
"NoAns_f1": 75.81160639192599,
"NoAns_total": 5945,
"best_exact": 73.77242482944496,
"best_exact_thresh": 0.0,
"best_f1": 76.83752039897604,
"best_f1_thresh": 0.0
}
- Downloads last month
- 11
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.