alpaca-med-lora-7b / README.md
zirui3's picture
Upload README.md
a81b3d7

Model summary

  • instruction-tuning on medical data based on LLaMA

data

  • Common
    • alpaca-5.2k
    • unatural-instruct 80k
    • OIG-40M
  • Chinese
    • english/chinese translation data
    • zhihu QA
    • pCLUE
  • Medical Domain:
    • MedDialog-200k
    • Chinese-medical-dialogue-data
    • WebMedQA
  • code
    • alpaca_code-20k

training

Model

  • LLaMA-7B

Hardware

  • 6 x A100 40G using NVLink 4 inter-gpu connects

Software

  • tokenizers==0.12.1
  • sentencepiece==0.1.97
  • transformers==4.28
  • torch==2.0.0+cu117

How to use

import torch
from transformers import LlamaTokenizer, LlamaForCausalLM, GenerationConfig
from peft import PeftModel

base_model="llma-7b"
LORA_WEIGHTS = "llma-med-alpaca-7b"
LOAD_8BIT = False

tokenizer = LlamaTokenizer.from_pretrained(base_model)

model = LlamaForCausalLM.from_pretrained(
    base_model
    load_in_8bit=LOAD_8BIT,
    torch_dtype=torch.float16,
    device_map="auto",
)
model = PeftModel.from_pretrained(
    model,
    LORA_WEIGHTS,
    torch_dtype=torch.float16,
)

config = {
    "temperature": 0 ,
    "max_new_tokens": 1024,
    "top_p": 0.5
}

prompt = "Translate to English: Je t’aime."
input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(model.device)
outputs = model.generate(input_ids=input_ids, max_new_tokens=config["max_new_tokens"], temperature=config["temperature"])
decoded = tokenizer.decode(outputs[0], skip_special_tokens=True).strip()
print(decoded[len(prompt):])

Limitations

  • This model may output harmful, biased, toxic, and illusory things, and currently does not undergo RLHF training, so this model is only for research purposes

TODO

  • self-instruct data
  • english medical data
  • code data
  • chinese corpus/medical dialog data

Reference