jkgrad commited on
Commit
ab73fba
1 Parent(s): ea8688f

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +24 -0
README.md ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # XLNet Fine-tuned on SQuAD / Quoref Dataset
2
+ [XLNet](https://arxiv.org/abs/1906.08237) jointly developed by Google and CMU and fine-tuned on [SQuAD / SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) and [Quoref](https://leaderboard.allenai.org/quoref) for question answering down-stream task.
3
+
4
+ ## Evaluation Result on Quoref
5
+ ```
6
+ {
7
+ "exact_match": 73.65591397848462,
8
+ "f1": 77.9981532789881
9
+ }
10
+ ```
11
+
12
+ ## Results Comparison
13
+ | Metric | Base Line | Model FT on SQuAD |
14
+ | ------ | --------- | --------- |
15
+ | **EM** | **61.88** | **73.66** (-11.78) |
16
+ | **F1** | **70.51** | **78.00** (+7.49)|
17
+
18
+ ## How to Use
19
+ ```
20
+ from transformers import XLNetForQuestionAnswering, XLNetTokenizerFast
21
+
22
+ model = XLNetForQuestionAnswering.from_pretrained('jkgrad/xlnet-base-cased-squad-quoref)
23
+ tokenizer = XLNetTokenizerFast.from_pretrained('jkgrad/xlnet-base-cased-squad-quoref')
24
+ ```