kazcfz's picture
Update README.md
e6a61c2 verified

Llama-2-7b-chat-Calgary-Cambridge-multi-turn

NousResearch/Llama-2-7b-chat-hf fine-tuned on kazcfz/Medical-Calgary-Cambridge-multi-turn-llama2-37


QLoRA parameters

Parameter Value
lora_r 64
lora_alpha 16
lora_dropout 0.1

bitsandbytes parameters

Parameter Value
use_4bit True
bnb_4bit_compute_dtype "float16"
bnb_4bit_quant_type "nf4"
use_nested_quant False

TrainingArguments parameters

Parameter Value
num_train_epochs 10
fp16 False
bf16 False
per_device_train_batch_size 4
gradient_accumulation_steps 1
gradient_checkpointing True
max_grad_norm 0.3
learning_rate 2e-4
weight_decay 0.001
optim "paged_adamw_32bit"
lr_scheduler_type "cosine"
max_steps -1
warmup_ratio 0.03
group_by_length True
save_steps 0
logging_steps 25

SFT parameters

Parameter Value
max_seq_length None
packing False

Prompt template: Llama-2-Chat

<s>[INST] {prompt} [/INST]

Sample (actual) output (Max length: 200)

Base Model

Query: <s>[INST] What is the Golaif Virus? [/INST]
Response: I couldn't find any information on a virus called "Golaif Virus." It's possible that it's a fictional or hypothetical virus, or it could be a misspelling or variation of a real virus. nobody has reported any information about it, and there is no scientific evidence to support its existence.

If you have any more information or context about the Golaif Virus, please let me know and I'll do my best to help you.

Fine-tuned

Query: <s>[INST] What is the Golaif Virus? [/INST]
Response: I apologize, but there is no such thing as the "Golaif Virus." It'<s a fictional term that doesn't exist in medical literature. nobody has ever reported a virus by that name. It's essential to rely on credible sources when discussing health topics, and I'm here to help you with any questions you may have. If you have any specific questions or concerns about viruses or other health-related topics, feel free to ask!

Reload model + Merge LoRA weights?

Query: <s>[INST] What is the Golaif Virus? [/INST]
Response: I apologize, but there is no known virus called "Golaif Virus." It is possible that you may have misspelled the name or that it is a fictional term. Additionally, I couldn't find any information on a virus with this name through reputable medical sources. If you have any specific questions or concerns about a particular virus, feel free to ask, and I'll do my best to help.