Upload 9 files

README.md

@@ -0,0 +1,64 @@
+---
+language: 
+- en
+tags:
+- Named Entity Recognition
+- SciBERT
+- Adverse Effect
+- Drug
+- Medical
+datasets:
+- ade_corpus_v2
+- tner/bc5cdr
+- jnlpba
+- bc2gm_corpus
+- drAbreu/bc4chemd_ner
+- linnaeus
+- ncbi_disease
+widget:
+- text: "Abortion, miscarriage or uterine hemorrhage associated with misoprostol (Cytotec), a labor-inducing drug."
+  example_title: "Abortion, miscarriage, ..."
+- text: "Addiction to many sedatives and analgesics, such as diazepam, morphine, etc."
+  example_title: "Addiction to many..."
+- text: "Birth defects associated with thalidomide"
+  example_title: "Birth defects associated..."
+- text: "Bleeding of the intestine associated with aspirin therapy"
+  example_title: "Bleeding of the intestine..."
+- text: "Cardiovascular disease associated with COX-2 inhibitors (i.e. Vioxx)"
+  example_title: "Cardiovascular disease..."
+
+---
+
+This is a SciBERT-based model fine-tuned to perform Named Entity Recognition for drug names and adverse drug effects. 
+![model image](https://raw.githubusercontent.com/jsylee/personal-projects/master/Hugging%20Face%20ADR%20Fine-Tuning/hf_adr.png)
+
+This model classifies input tokens into one of five classes:
+
+- `B-DRUG`: beginning of a drug entity
+- `I-DRUG`: within a drug entity
+- `B-EFFECT`: beginning of an AE entity
+- `I-EFFECT`: within an AE entity
+- `O`: outside either of the above entities 
+
+To get started using this model for inference, simply set up an NER `pipeline` like below: 
+
+```python
+from transformers import (AutoModelForTokenClassification, 
+                          AutoTokenizer, 
+                          pipeline,
+                          )
+
+model_checkpoint = "jsylee/scibert_scivocab_uncased-finetuned-ner"
+model = AutoModelForTokenClassification.from_pretrained(model_checkpoint, num_labels=5,
+                                                        id2label={0: 'O', 1: 'B-DRUG', 2: 'I-DRUG', 3: 'B-EFFECT', 4: 'I-EFFECT'} 
+                                                        )                                                        
+tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
+
+model_pipeline = pipeline(task="ner", model=model, tokenizer=tokenizer)
+
+print( model_pipeline ("Abortion, miscarriage or uterine hemorrhage associated with misoprostol (Cytotec), a labor-inducing drug."))
+``` 
+
+SciBERT: https://huggingface.co/allenai/scibert_scivocab_uncased
+
+Dataset: https://huggingface.co/datasets/ade_corpus_v2

config.json

@@ -0,0 +1,39 @@
+{
+  "_name_or_path": "allenai/scibert_scivocab_uncased",
+  "architectures": [
+    "BertForTokenClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "LABEL_0",
+    "1": "LABEL_1",
+    "2": "LABEL_2",
+    "3": "LABEL_3",
+    "4": "LABEL_4"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "LABEL_0": 0,
+    "LABEL_1": 1,
+    "LABEL_2": 2,
+    "LABEL_3": 3,
+    "LABEL_4": 4
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.11.3",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 31090
+}

index.gitignore

@@ -0,0 +1 @@
+checkpoint-*/

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:82f8254b4b4cbc8ad6da92d6550d2f25dd1d5b477f5f5ceb30277df548d0d482
+size 437380979

special_tokens_map.json

@@ -0,0 +1 @@
+{"unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]"}

tokenizer_config.json

@@ -0,0 +1 @@
+{"do_lower_case": true, "unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]", "tokenize_chinese_chars": true, "strip_accents": null, "special_tokens_map_file": null, "name_or_path": "allenai/scibert_scivocab_uncased", "do_basic_tokenize": true, "never_split": null, "tokenizer_class": "BertTokenizer"}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:222d40f49fd459c71272dc2ab29f1ef348c8ed931f0579e5856eced883daa36c
+size 2613