BioBert-PubMed200kRCT

This model is a fine-tuned version of dmis-lab/biobert-base-cased-v1.1 on the PubMed200kRCT dataset. It achieves the following results on the evaluation set:

  • Loss: 0.2832
  • Accuracy: 0.8934

Model description

More information needed

Intended uses & limitations

The model can be used for text classification tasks of Randomized Controlled Trials that does not have any structure. The text can be classified as one of the following:

  • BACKGROUND
  • CONCLUSIONS
  • METHODS
  • OBJECTIVE
  • RESULTS

The model can be directly used like this:

from transformers import TextClassificationPipeline
from transformers import AutoTokenizer, AutoModelForSequenceClassification
model = AutoModelForSequenceClassification.from_pretrained("pritamdeka/BioBert-PubMed200kRCT")
tokenizer = AutoTokenizer.from_pretrained("pritamdeka/BioBert-PubMed200kRCT")
pipe = TextClassificationPipeline(model=model, tokenizer=tokenizer, return_all_scores=True)
pipe("Treatment of 12 healthy female subjects with CDCA for 2 days resulted in increased BAT activity.")

Results will be shown as follows:

[[{'label': 'BACKGROUND', 'score': 0.0027583304326981306},
  {'label': 'CONCLUSIONS', 'score': 0.044541116803884506},
  {'label': 'METHODS', 'score': 0.19493348896503448},
  {'label': 'OBJECTIVE', 'score': 0.003996663726866245},
  {'label': 'RESULTS', 'score': 0.7537703514099121}]]

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 64
  • eval_batch_size: 64
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 2.0
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Accuracy
0.3587 0.14 5000 0.3137 0.8834
0.3318 0.29 10000 0.3100 0.8831
0.3286 0.43 15000 0.3033 0.8864
0.3236 0.58 20000 0.3037 0.8862
0.3182 0.72 25000 0.2939 0.8876
0.3129 0.87 30000 0.2910 0.8885
0.3078 1.01 35000 0.2914 0.8887
0.2791 1.16 40000 0.2975 0.8874
0.2723 1.3 45000 0.2913 0.8906
0.2724 1.45 50000 0.2879 0.8904
0.27 1.59 55000 0.2874 0.8911
0.2681 1.74 60000 0.2848 0.8928
0.2672 1.88 65000 0.2832 0.8934

Framework versions

  • Transformers 4.18.0.dev0
  • Pytorch 1.10.0+cu111
  • Datasets 1.18.4
  • Tokenizers 0.11.6

Citing & Authors

If you use the model kindly cite the following work

@inproceedings{deka2022evidence,
  title={Evidence Extraction to Validate Medical Claims in Fake News Detection},
  author={Deka, Pritam and Jurek-Loughrey, Anna and others},
  booktitle={International Conference on Health Information Science},
  pages={3--15},
  year={2022},
  organization={Springer}
}
Downloads last month
255
Safetensors
Model size
108M params
Tensor type
I64
·
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for pritamdeka/BioBert-PubMed200kRCT

Finetuned
(11)
this model