Hugging Face's logo

sethuiyer
/
Dr_Samantha-7b

Text Generation
Transformers
Safetensors
English
Chinese
llama
Merge
medical
Eval Results
text-generation-inference
Inference Endpoints
Model card Files Community
1
Dr_Samantha-7b / README.md
sethuiyer's picture
sethuiyer
Update README.md
7dbdaab
preview code
|
raw
history blame
No virus
3.61 kB
metadata
license: llama2
library_name: transformers
tags:
  - llama
  - merge
  - medical
datasets:
  - GBaker/MedQA-USMLE-4-options
  - cognitivecomputations/samantha-data
  - shibing624/medical
language:
  - en
  - zh
pipeline_tag: text-generation

Dr. Samantha

SynthIQ

Overview

Dr. Samantha is a language model made by merging Severus27/BeingWell_llama2_7b and ParthasarathyShanmugam/llama-2-7b-samantha using mergekit.

Has capabilities of a medical knowledge-focused model (trained on USMLE databases and doctor-patient interactions) with the philosophical, psychological, and relational understanding of the Samantha-7b model.

As both a medical consultant and personal counselor, Dr.Samantha could effectively support both physical and mental wellbeing - important for whole-person care.

Yaml Config 


slices:
  - sources:
      - model: Severus27/BeingWell_llama2_7b
        layer_range: [0, 32]
      - model: ParthasarathyShanmugam/llama-2-7b-samantha
        layer_range: [0, 32]

merge_method: slerp
base_model: TinyPixel/Llama-2-7B-bf16-sharded

parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5 # fallback for rest of tensors
tokenizer_source: union

dtype: bfloat16

Prompt Template 

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
What is your name?

### Response:
My name is Samantha.

OpenLLM Leaderboard Performance

T Model Average ARC Hellaswag MMLU TruthfulQA Winogrande GSM8K
1 sethuiyer/Dr_Samantha-7b 52.95 53.84 77.95 47.94 45.58 73.56 18.8
2 togethercomputer/LLaMA-2-7B-32K-Instruct 50.02 51.11 78.51 46.11 44.86 73.88 5.69
3 togethercomputer/LLaMA-2-7B-32K 47.07 47.53 76.14 43.33 39.23 71.9 4.32

Evaluation by GPT-4 across 25 random prompts from ChatDoctor-200k Dataset

Overall Rating: 83.5/100

Pros:

  • Demonstrates extensive medical knowledge through accurate identification of potential causes for various symptoms.
  • Responses consistently emphasize the importance of seeking professional diagnoses and treatments.
  • Advice to consult specialists for certain concerns is well-reasoned.
  • Practical interim measures provided for symptom management in several cases.
  • Consistent display of empathy, support, and reassurance for patients' well-being.
  • Clear and understandable explanations of conditions and treatment options.
  • Prompt responses addressing all aspects of medical inquiries.

Cons:

  • Could occasionally place stronger emphasis on urgency when symptoms indicate potential emergencies.
  • Discussion of differential diagnoses could explore a broader range of less common causes.
  • Details around less common symptoms and their implications need more depth at times.
  • Opportunities exist to gather clarifying details on symptom histories through follow-up questions.
  • Consider exploring full medical histories to improve diagnostic context where relevant.
  • Caution levels and risk factors associated with certain conditions could be underscored more.