1
---
2
language: en
3
thumbnail:
4
---
5
6
# SpanBERT base fine-tuned on TACRED
7
8
[SpanBERT](https://github.com/facebookresearch/SpanBERT) created by [Facebook Research](https://github.com/facebookresearch) and fine-tuned on [TACRED](https://nlp.stanford.edu/projects/tacred/) dataset by [them](https://github.com/facebookresearch/SpanBERT#finetuned-models-squad-1120-relation-extraction-coreference-resolution)
9
10
## Details of SpanBERT
11
12
[SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://arxiv.org/abs/1907.10529)
13
14
## Dataset πŸ“š
15
16
[TACRED](https://nlp.stanford.edu/projects/tacred/) A large-scale relation extraction dataset with 106k+ examples over 42 TAC KBP relation types.
17
18
## Model fine-tuning πŸ‹οΈβ€
19
20
You can get the fine-tuning script [here](https://github.com/facebookresearch/SpanBERT)
21
22
```bash
23
python code/run_tacred.py \
24
  --do_train \
25
  --do_eval \
26
  --data_dir <TACRED_DATA_DIR> \
27
  --model spanbert-base-cased \
28
  --train_batch_size 32 \
29
  --eval_batch_size 32 \
30
  --learning_rate 2e-5 \
31
  --num_train_epochs 10 \
32
  --max_seq_length 128 \
33
  --output_dir tacred_dir \
34
  --fp16
35
```
36
37
## Results Comparison πŸ“
38
39
|                   | SQuAD 1.1     | SQuAD 2.0  | Coref   | TACRED |
40
| ----------------------  | ------------- | ---------  | ------- | ------ |
41
|                         | F1            | F1         | avg. F1 |  F1    |
42
| BERT (base)             | 88.5*         | 76.5*      | 73.1    |  67.7  |
43
| SpanBERT (base)         | [92.4*](https://huggingface.co/mrm8488/spanbert-base-finetuned-squadv1)         | [83.6*](https://huggingface.co/mrm8488/spanbert-base-finetuned-squadv2)      | 77.4    |  **68.2** (this one)  |
44
| BERT (large)            | 91.3          | 83.3       | 77.1    |  66.4  |
45
| SpanBERT (large)        | [94.6](https://huggingface.co/mrm8488/spanbert-large-finetuned-squadv1)        | [88.7](https://huggingface.co/mrm8488/spanbert-large-finetuned-squadv2)     | 79.6    |  [70.8](https://huggingface.co/mrm8488/spanbert-base-finetuned-tacred)   |
46
47
48
Note: The numbers marked as * are evaluated on the development sets because those models were not submitted to the official SQuAD leaderboard. All the other numbers are test numbers.
49
50
51
> Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488)
52
53
> Made with <span style="color: #e25555;">&hearts;</span> in Spain
54