julien-c HF staff commited on
Commit
46f4d71
β€’
1 Parent(s): 3cc5c3b

Migrate model card from transformers-repo

Browse files

Read announcement at https://discuss.huggingface.co/t/announcement-all-model-cards-will-be-migrated-to-hf-co-model-repos/2755
Original file history: https://github.com/huggingface/transformers/commits/master/model_cards/mrm8488/spanbert-large-finetuned-squadv1/README.md

Files changed (1) hide show
  1. README.md +80 -0
README.md ADDED
@@ -0,0 +1,80 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: en
3
+ thumbnail:
4
+ ---
5
+
6
+ # SpanBERT large fine-tuned on SQuAD v1
7
+
8
+ [SpanBERT](https://github.com/facebookresearch/SpanBERT) created by [Facebook Research](https://github.com/facebookresearch) and fine-tuned on [SQuAD 1.1](https://rajpurkar.github.io/SQuAD-explorer/explore/1.1/dev/) for **Q&A** downstream task ([by them](https://github.com/facebookresearch/SpanBERT#finetuned-models-squad-1120-relation-extraction-coreference-resolution)).
9
+
10
+ ## Details of SpanBERT
11
+
12
+ [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://arxiv.org/abs/1907.10529)
13
+
14
+ ## Details of the downstream task (Q&A) - Dataset πŸ“š 🧐 ❓
15
+
16
+ [SQuAD1.1](https://rajpurkar.github.io/SQuAD-explorer/)
17
+
18
+ ## Model fine-tuning πŸ‹οΈβ€
19
+
20
+ You can get the fine-tuning script [here](https://github.com/facebookresearch/SpanBERT)
21
+
22
+ ```bash
23
+ python code/run_squad.py \
24
+ --do_train \
25
+ --do_eval \
26
+ --model spanbert-large-cased \
27
+ --train_file train-v1.1.json \
28
+ --dev_file dev-v1.1.json \
29
+ --train_batch_size 32 \
30
+ --eval_batch_size 32 \
31
+ --learning_rate 2e-5 \
32
+ --num_train_epochs 4 \
33
+ --max_seq_length 512 \
34
+ --doc_stride 128 \
35
+ --eval_metric f1 \
36
+ --output_dir squad_output \
37
+ --fp16
38
+ ```
39
+
40
+ ## Results Comparison πŸ“
41
+
42
+ | | SQuAD 1.1 | SQuAD 2.0 | Coref | TACRED |
43
+ | ---------------------- | ------------- | --------- | ------- | ------ |
44
+ | | F1 | F1 | avg. F1 | F1 |
45
+ | BERT (base) | 88.5* | 76.5* | 73.1 | 67.7 |
46
+ | SpanBERT (base) | [92.4*](https://huggingface.co/mrm8488/spanbert-base-finetuned-squadv1) | [83.6*](https://huggingface.co/mrm8488/spanbert-base-finetuned-squadv2) | 77.4 | [68.2](https://huggingface.co/mrm8488/spanbert-base-finetuned-tacred) |
47
+ | BERT (large) | 91.3 | 83.3 | 77.1 | 66.4 |
48
+ | SpanBERT (large) | **94.6** (this) | [88.7](https://huggingface.co/mrm8488/spanbert-large-finetuned-squadv2) | 79.6 | [70.8](https://huggingface.co/mrm8488/spanbert-large-finetuned-tacred) |
49
+
50
+
51
+ Note: The numbers marked as * are evaluated on the development sets because those models were not submitted to the official SQuAD leaderboard. All the other numbers are test numbers.
52
+
53
+ ## Model in action
54
+
55
+ Fast usage with **pipelines**:
56
+
57
+ ```python
58
+ from transformers import pipeline
59
+
60
+ qa_pipeline = pipeline(
61
+ "question-answering",
62
+ model="mrm8488/spanbert-large-finetuned-squadv1",
63
+ tokenizer="SpanBERT/spanbert-large-cased"
64
+ )
65
+
66
+ qa_pipeline({
67
+ 'context': "Manuel Romero has been working very hard in the repository hugginface/transformers lately",
68
+ 'question': "How has been working Manuel Romero lately?"
69
+
70
+ })
71
+
72
+ # Output: {'answer': 'very hard in the repository hugginface/transformers',
73
+ 'end': 82,
74
+ 'score': 0.327230326857725,
75
+ 'start': 31}
76
+ ```
77
+
78
+ > Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488)
79
+
80
+ > Made with <span style="color: #e25555;">&hearts;</span> in Spain