Edit model card

PubMedBert-PubMed200kRCT

This model is a fine-tuned version of microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext on the PubMed200kRCT dataset. It achieves the following results on the evaluation set:

  • Loss: 0.2833
  • Accuracy: 0.8942

Model description

More information needed

Intended uses & limitations

The model can be used for text classification tasks of Randomized Controlled Trials that does not have any structure. The text can be classified as one of the following:

  • BACKGROUND
  • CONCLUSIONS
  • METHODS
  • OBJECTIVE
  • RESULTS

The model can be directly used like this:

from transformers import TextClassificationPipeline
from transformers import AutoTokenizer, AutoModelForSequenceClassification
model = AutoModelForSequenceClassification.from_pretrained("pritamdeka/PubMedBert-PubMed200kRCT")
tokenizer = AutoTokenizer.from_pretrained("pritamdeka/PubMedBert-PubMed200kRCT")
pipe = TextClassificationPipeline(model=model, tokenizer=tokenizer, return_all_scores=True)
pipe("Treatment of 12 healthy female subjects with CDCA for 2 days resulted in increased BAT activity.")

Results will be shown as follows:

[[{'label': 'BACKGROUND', 'score': 0.0028450002428144217},
  {'label': 'CONCLUSIONS', 'score': 0.2581048607826233},
  {'label': 'METHODS', 'score': 0.015086210332810879},
  {'label': 'OBJECTIVE', 'score': 0.0016815993003547192},
  {'label': 'RESULTS', 'score': 0.7222822904586792}]]

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 64
  • eval_batch_size: 64
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 2.0

Training results

Training Loss Epoch Step Validation Loss Accuracy
0.3604 0.14 5000 0.3162 0.8821
0.3326 0.29 10000 0.3112 0.8843
0.3293 0.43 15000 0.3044 0.8870
0.3246 0.58 20000 0.3040 0.8871
0.32 0.72 25000 0.2969 0.8888
0.3143 0.87 30000 0.2929 0.8903
0.3095 1.01 35000 0.2917 0.8899
0.2844 1.16 40000 0.2957 0.8886
0.2778 1.3 45000 0.2943 0.8906
0.2779 1.45 50000 0.2890 0.8935
0.2752 1.59 55000 0.2881 0.8919
0.2736 1.74 60000 0.2835 0.8944
0.2725 1.88 65000 0.2833 0.8942

Framework versions

  • Transformers 4.18.0.dev0
  • Pytorch 1.10.0+cu111
  • Datasets 1.18.3
  • Tokenizers 0.11.6

Citing & Authors

If you use the model kindly cite the following work

@inproceedings{deka2022evidence,
  title={Evidence Extraction to Validate Medical Claims in Fake News Detection},
  author={Deka, Pritam and Jurek-Loughrey, Anna and others},
  booktitle={International Conference on Health Information Science},
  pages={3--15},
  year={2022},
  organization={Springer}
}
Downloads last month
21
Safetensors
Model size
109M params
Tensor type
I64
·
F32
·

Finetuned from