Hugging Face's logo

jkgrad
/
xlnet-base-squadv2

Question Answering Transformers PyTorch xlnet Inference Endpoints
Model card Files Community
1
xlnet-base-squadv2 / README.md
Oh Joon Kwon
Updated README
36d0bd0
preview code
|
raw history blame contribute delete
No virus
1.19 kB

XLNet Fine-tuned on SQuAD 2.0 Dataset

XLNet jointly developed by Google and CMU and fine-tuned on SQuAD 2.0 for question answering down-stream task.

Training Results (Metrics) 

{
    "HasAns_exact": 74.7132253711201
    "HasAns_f1": 82.11971607032643
    "HasAns_total": 5928
    "NoAns_exact": 73.38940285954584
    "NoAns_f1": 73.38940285954584
    "NoAns_total": 5945
    "best_exact": 75.67590331003116
    "best_exact_thresh": -19.554906845092773
    "best_f1": 79.16215426779269
    "best_f1_thresh": -19.554906845092773
    "epoch": 4.0
    "exact": 74.05036637749515
    "f1": 77.74830934598614
    "total": 11873
}

Results Comparison

Metric Paper Model
EM 78.46 75.68 (-2.78)
F1 81.33 79.16 (-2.17)

Better fine-tuned models coming soon.

How to Use 

from transformers import XLNetForQuestionAnswering, XLNetTokenizerFast

model = XLNetForQuestionAnswering.from_pretrained('jkgrad/xlnet-base-squadv2)
tokenizer = XLNetTokenizerFast.from_pretrained('jkgrad/xlnet-base-squadv2')