# mrm8488 /spanbert-base-finetuned-tacred

2.28 kB
 1 --- 2 language: en 3 thumbnail: 4 --- 5 6 # SpanBERT base fine-tuned on TACRED 7 8 [SpanBERT](https://github.com/facebookresearch/SpanBERT) created by [Facebook Research](https://github.com/facebookresearch) and fine-tuned on [TACRED](https://nlp.stanford.edu/projects/tacred/) dataset by [them](https://github.com/facebookresearch/SpanBERT#finetuned-models-squad-1120-relation-extraction-coreference-resolution) 9 10 ## Details of SpanBERT 11 12 [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://arxiv.org/abs/1907.10529) 13 14 ## Dataset 📚 15 16 [TACRED](https://nlp.stanford.edu/projects/tacred/) A large-scale relation extraction dataset with 106k+ examples over 42 TAC KBP relation types. 17 18 ## Model fine-tuning 🏋️‍ 19 20 You can get the fine-tuning script [here](https://github.com/facebookresearch/SpanBERT) 21 22 bash 23 python code/run_tacred.py \ 24  --do_train \ 25  --do_eval \ 26  --data_dir \ 27  --model spanbert-base-cased \ 28  --train_batch_size 32 \ 29  --eval_batch_size 32 \ 30  --learning_rate 2e-5 \ 31  --num_train_epochs 10 \ 32  --max_seq_length 128 \ 33  --output_dir tacred_dir \ 34  --fp16 35  36 37 ## Results Comparison 📝 38 39 | | SQuAD 1.1 | SQuAD 2.0 | Coref | TACRED | 40 | ---------------------- | ------------- | --------- | ------- | ------ | 41 | | F1 | F1 | avg. F1 | F1 | 42 | BERT (base) | 88.5* | 76.5* | 73.1 | 67.7 | 43 | SpanBERT (base) | [92.4*](https://huggingface.co/mrm8488/spanbert-base-finetuned-squadv1) | [83.6*](https://huggingface.co/mrm8488/spanbert-base-finetuned-squadv2) | 77.4 | **68.2** (this one) | 44 | BERT (large) | 91.3 | 83.3 | 77.1 | 66.4 | 45 | SpanBERT (large) | [94.6](https://huggingface.co/mrm8488/spanbert-large-finetuned-squadv1) | [88.7](https://huggingface.co/mrm8488/spanbert-large-finetuned-squadv2) | 79.6 | [70.8](https://huggingface.co/mrm8488/spanbert-base-finetuned-tacred) | 46 47 48 Note: The numbers marked as * are evaluated on the development sets because those models were not submitted to the official SQuAD leaderboard. All the other numbers are test numbers. 49 50 51 > Created by [Manuel Romero/@mrm8488](https://twitter.com/mrm8488) 52 53 > Made with in Spain 54