Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,24 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# XLNet Fine-tuned on SQuAD / Quoref Dataset
|
2 |
+
[XLNet](https://arxiv.org/abs/1906.08237) jointly developed by Google and CMU and fine-tuned on [SQuAD / SQuAD 2.0](https://rajpurkar.github.io/SQuAD-explorer/) and [Quoref](https://leaderboard.allenai.org/quoref) for question answering down-stream task.
|
3 |
+
|
4 |
+
## Evaluation Result on Quoref
|
5 |
+
```
|
6 |
+
{
|
7 |
+
"exact_match": 73.65591397848462,
|
8 |
+
"f1": 77.9981532789881
|
9 |
+
}
|
10 |
+
```
|
11 |
+
|
12 |
+
## Results Comparison
|
13 |
+
| Metric | Base Line | Model FT on SQuAD |
|
14 |
+
| ------ | --------- | --------- |
|
15 |
+
| **EM** | **61.88** | **73.66** (-11.78) |
|
16 |
+
| **F1** | **70.51** | **78.00** (+7.49)|
|
17 |
+
|
18 |
+
## How to Use
|
19 |
+
```
|
20 |
+
from transformers import XLNetForQuestionAnswering, XLNetTokenizerFast
|
21 |
+
|
22 |
+
model = XLNetForQuestionAnswering.from_pretrained('jkgrad/xlnet-base-cased-squad-quoref)
|
23 |
+
tokenizer = XLNetTokenizerFast.from_pretrained('jkgrad/xlnet-base-cased-squad-quoref')
|
24 |
+
```
|