File size: 2,774 Bytes
e0d60ee 697680e e0d60ee 697680e e0d60ee 75ae352 e0d60ee 75ae352 20fa86d 75ae352 e0d60ee 75ae352 e0d60ee 75ae352 e0d60ee |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 |
---
base_model: microsoft/phi-2
datasets:
- medalpaca/medical_meadow_health_advice
- medalpaca/medical_meadow_mediqa
- medalpaca/medical_meadow_mmmlu
- medalpaca/medical_meadow_medical_flashcards
- medalpaca/medical_meadow_wikidoc_patient_information
- medalpaca/medical_meadow_wikidoc
- medalpaca/medical_meadow_pubmed_causal
- medalpaca/medical_meadow_medqa
- medalpaca/medical_meadow_cord19
language:
- en
license: mit
license_link: https://huggingface.co/microsoft/phi-2/resolve/main/LICENSE
pipeline_tag: text-generation
tags:
- nlp
- Medicine
---
## Model Summary
Phi2_med_seg is a fine-tuned version of the Phi-2 model, specifically optimized for medical applications. This model has been trained using the Trainer framework on several different datasets from the MedAlpaca collection, which focuses on medical question answering and conversational AI.
This model can answer information about different excplicit ideas in medicine
## How to Get Started with the Model
## Sample Code
```python
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
base_model_id = "microsoft/phi-2"
base_model = AutoModelForCausalLM.from_pretrained(
base_model_id, # Phi2, same as before
device_map="auto",
trust_remote_code=True,
load_in_8bit=True,
torch_dtype=torch.float16,
)
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
eval_tokenizer = AutoTokenizer.from_pretrained(base_model_id, add_bos_token=True, trust_remote_code=True, use_fast=False)
eval_tokenizer.pad_token = tokenizer.eos_token
from peft import PeftModel
adapter_model_id = "segestic/phi2_medical_seg"
ft_model = PeftModel.from_pretrained(base_model, adapter_model_id)
eval_prompt = "What is medicine?"
model_input = eval_tokenizer(eval_prompt, return_tensors="pt").to("cuda")
ft_model.eval()
with torch.no_grad():
print(eval_tokenizer.decode(ft_model.generate(**model_input, max_new_tokens=100, repetition_penalty=1.11)[0], skip_special_tokens=True))
```
## Training
The fine-tuning process involved leveraging various medical datasets to enhance the model's ability to understand and generate relevant medical information. This approach aims to improve the model's performance in medical contexts, making it a valuable tool for healthcare professionals and researchers alike. By utilizing the Trainer framework, Phi2_med_seg benefits from advanced training techniques that help refine its responses and accuracy in medical scenarios.
### Model
* Architecture: a Transformer-based model with next-word prediction objective
* Context length: 2048 tokens
### Software
* [PyTorch](https://github.com/pytorch/pytorch)
* [DeepSpeed](https://github.com/microsoft/DeepSpeed)
* [Flash-Attention](https://github.com/HazyResearch/flash-attention)
|