Model save

README.md

@@ -1,65 +1,47 @@
 ---
-language:
-- en
-license: apache-2.0
-library_name: transformers
+base_model: dmis-lab/biobert-base-cased-v1.2
 tags:
-- medical
-- pharmacovigilance
-- vaccines
-datasets:
-- chrisvoncsefalvay/vaers-outcomes
-metrics:
-- accuracy
-- f1
-- precision
-- recall
-pipeline_tag: text-classification
-widget:
-- text: Patient is a 90 y.o. male with a PMH of IPF, HFpEF, AFib (Eliquis), Metastatic
-    Prostate Cancer who presented to Hospital 10/28/2023 following an unwitnessed
-    fall at his assisted living. He was found to have an AKI, pericardial effusion,
-    hypoxia, AMS, and COVID-19. His hospital course was complicated by delirium and
-    aspiration, leading to acute hypoxic respiratory failure requiring BiPAP and transfer
-    to the ICU. Palliative Care had been following, and after goals of care conversations
-    on 11/10/2023 the patient was transitioned to DNR-CC. Patient expired at 0107
-    11/12/23.
-  example_title: VAERS 2727645 (hospitalisation, death)
-- text: 'hospitalized for paralytic ileus a week after the vaccination; This serious
-    case was reported by a physician via call center representative and described
-    the occurrence of ileus paralytic in a patient who received Rota (Rotarix liquid
-    formulation) for prophylaxis. On an unknown date, the patient received the 1st
-    dose of Rotarix liquid formulation. On an unknown date, less than 2 weeks after
-    receiving Rotarix liquid formulation, the patient experienced ileus paralytic
-    (Verbatim: hospitalized for paralytic ileus a week after the vaccination) (serious
-    criteria hospitalization and GSK medically significant). The outcome of the ileus
-    paralytic was not reported. It was unknown if the reporter considered the ileus
-    paralytic to be related to Rotarix liquid formulation. It was unknown if the company
-    considered the ileus paralytic to be related to Rotarix liquid formulation. Additional
-    Information: GSK Receipt Date: 27-DEC-2023 Age at vaccination and lot number were
-    not reported. The patient of unknown age and gender was hospitalized for paralytic
-    ileus a week after the vaccination. The reporting physician was in charge of the
-    patient.'
-  example_title: VAERS 2728408 (hospitalisation)
-- text: Patient received Pfizer vaccine 7 days beyond BUD. According to Pfizer manufacturer
-    research data, vaccine is stable and effective up to 2 days after BUD. Waiting
-    for more stability data from PFIZER to determine if revaccination is necessary.
-  example_title: VAERS 2728394 (no event)
-- text: Fever of 106F rectally beginning 1 hr after immunizations and lasting <24
-    hrs. Seen at ER treated w/tylenol & cool baths.
-  example_title: VAERS 25042 (ER attendance)
-- text: I had the MMR shot last week, and I felt a little dizzy afterwards, but it
-    passed after a few minutes and I'm doing fine now.
-  example_title: 'Non-sample example: simulated informal patient narrative (no event)'
-- text: My niece had the COVID vaccine. A few weeks later, she was T-boned by a drunk
-    driver. She called me from the ER. She's fully recovered now, though.
-  example_title: 'Non-sample example: simulated informal patient narrative (ER attendance,
-    albeit unconnected)'
+- generated_from_trainer
+model-index:
+- name: daedra
+  results: []
 ---
 
-DAEDRA (Detecting Adverse Event Dispositions for Regulatory Affairs) is a pharmacovigilance language model intended to facilitate the rapid identification and extraction of high-consequence outcomes from passive pharmacovigilance reporting. It was trained on the VAERS data set, and focuses on three main outcomes:
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
 
-* mortality (VAERS `DIED` flag);
-* emergency room attendance (`ER_VISIT`); and
-* hospitalisation (`HOSPITAL`).
+# daedra
 
+This model is a fine-tuned version of [dmis-lab/biobert-base-cased-v1.2](https://huggingface.co/dmis-lab/biobert-base-cased-v1.2) on an unknown dataset.
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 64
+- eval_batch_size: 64
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 3
+
+### Framework versions
+
+- Transformers 4.37.2
+- Pytorch 2.1.2+cu121
+- Datasets 2.3.2
+- Tokenizers 0.15.1

config.json

@@ -1,13 +1,13 @@
 {
-  "_name_or_path": "distilbert-base-uncased",
-  "activation": "gelu",
+  "_name_or_path": "dmis-lab/biobert-base-cased-v1.2",
   "architectures": [
-    "DistilBertForSequenceClassification"
+    "BertForSequenceClassification"
   ],
-  "attention_dropout": 0.1,
-  "dim": 768,
-  "dropout": 0.1,
-  "hidden_dim": 3072,
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
   "id2label": {
     "0": "No event",
     "1": "ER_VISIT",
@@ -19,6 +19,7 @@
     "7": "ER_VISIT, DIED"
   },
   "initializer_range": 0.02,
+  "intermediate_size": 3072,
   "label2id": {
     "DIED": 3,
     "ER_VISIT": 1,
@@ -29,17 +30,17 @@
     "HOSPITAL, DIED": 6,
     "No event": 0
   },
+  "layer_norm_eps": 1e-12,
   "max_position_embeddings": 512,
-  "model_type": "distilbert",
-  "n_heads": 12,
-  "n_layers": 6,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
   "pad_token_id": 0,
+  "position_embedding_type": "absolute",
   "problem_type": "single_label_classification",
-  "qa_dropout": 0.1,
-  "seq_classif_dropout": 0.2,
-  "sinusoidal_pos_embds": false,
-  "tie_weights_": true,
   "torch_dtype": "float32",
-  "transformers_version": "4.37.1",
-  "vocab_size": 30522
+  "transformers_version": "4.37.2",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 28996
 }

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8cde70f8a4c453a3d63e0941158445d9795528724cd0f1daece7e02bcb3633c4
-size 267851024
+oid sha256:2c678a157697f94bad7925f694c152bc9817bb8309f75ecee49f5f72c8292b8e
+size 433289224

tokenizer_config.json

@@ -43,13 +43,15 @@
   },
   "clean_up_tokenization_spaces": true,
   "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
   "do_lower_case": true,
   "mask_token": "[MASK]",
-  "model_max_length": 512,
+  "model_max_length": 1000000000000000019884624838656,
+  "never_split": null,
   "pad_token": "[PAD]",
   "sep_token": "[SEP]",
   "strip_accents": null,
   "tokenize_chinese_chars": true,
-  "tokenizer_class": "DistilBertTokenizer",
+  "tokenizer_class": "BertTokenizer",
   "unk_token": "[UNK]"
 }

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:69d1d6788d827ca923a1d0cdbf90d22765c77a85e86fb761181c602b888bbcea
+size 4728