Llama-2-7b-chat-Calgary-Cambridge-multi-turn

NousResearch/Llama-2-7b-chat-hf fine-tuned on kazcfz/Medical-Calgary-Cambridge-multi-turn-llama2-37

---

QLoRA parameters

Parameter	Value
lora_r	64
lora_alpha	16
lora_dropout	0.1

bitsandbytes parameters

Parameter	Value
use_4bit	True
bnb_4bit_compute_dtype	"float16"
bnb_4bit_quant_type	"nf4"
use_nested_quant	False

TrainingArguments parameters

Parameter	Value
num_train_epochs	10
fp16	False
bf16	False
per_device_train_batch_size	4
gradient_accumulation_steps	1
gradient_checkpointing	True
max_grad_norm	0.3
learning_rate	2e-4
weight_decay	0.001
optim	"paged_adamw_32bit"
lr_scheduler_type	"cosine"
max_steps	-1
warmup_ratio	0.03
group_by_length	True
save_steps	0
logging_steps	25

SFT parameters

Parameter	Value
max_seq_length	None
packing	False

Prompt template: Llama-2-Chat

<s>[INST] {prompt} [/INST]

Sample (actual) output (Max length: 200)

Base Model

Query: <s>[INST] What is the Golaif Virus? [/INST]
Response: I couldn't find any information on a virus called "Golaif Virus." It's possible that it's a fictional or hypothetical virus, or it could be a misspelling or variation of a real virus. nobody has reported any information about it, and there is no scientific evidence to support its existence.

If you have any more information or context about the Golaif Virus, please let me know and I'll do my best to help you.

Fine-tuned

Query: <s>[INST] What is the Golaif Virus? [/INST]
Response: I apologize, but there is no such thing as the "Golaif Virus." It'<s a fictional term that doesn't exist in medical literature. nobody has ever reported a virus by that name. It's essential to rely on credible sources when discussing health topics, and I'm here to help you with any questions you may have. If you have any specific questions or concerns about viruses or other health-related topics, feel free to ask!

Reload model + Merge LoRA weights?

Query: <s>[INST] What is the Golaif Virus? [/INST]
Response: I apologize, but there is no known virus called "Golaif Virus." It is possible that you may have misspelled the name or that it is a fictional term. Additionally, I couldn't find any information on a virus with this name through reputable medical sources. If you have any specific questions or concerns about a particular virus, feel free to ask, and I'll do my best to help.