# Llama-2-7b-chat-Calgary-Cambridge-multi-turn
[`NousResearch/Llama-2-7b-chat-hf`](https://huggingface.co/NousResearch/Llama-2-7b-chat-hf) fine-tuned on [`kazcfz/Medical-Calgary-Cambridge-multi-turn-llama2-37`](https://huggingface.co/datasets/kazcfz/Medical-Calgary-Cambridge-multi-turn-llama2-37)

<hr style="margin-top: 1.0em; margin-bottom: 1.0em;">

**QLoRA parameters**
Parameter | Value
--- | ---
lora_r | 64
lora_alpha | 16
lora_dropout | 0.1

**bitsandbytes parameters**
Parameter | Value
--- | ---
use_4bit | True
bnb_4bit_compute_dtype | "float16"
bnb_4bit_quant_type | "nf4"
use_nested_quant | False

**TrainingArguments parameters**
Parameter | Value
--- | ---
num_train_epochs | 10
fp16 | False
bf16 | False
per_device_train_batch_size | 4
gradient_accumulation_steps | 1
gradient_checkpointing | True
max_grad_norm | 0.3
learning_rate | 2e-4
weight_decay | 0.001
optim | "paged_adamw_32bit"
lr_scheduler_type | "cosine"
max_steps | -1
warmup_ratio | 0.03
group_by_length | True
save_steps | 0
logging_steps | 25

**SFT parameters**
Parameter | Value
--- | ---
max_seq_length | None
packing | False

<br>

## Prompt template: Llama-2-Chat
```
<s>[INST] {prompt} [/INST]
```

<br>

## Sample (actual) output (Max length: 200)
**Base Model**
```
Query: <s>[INST] What is the Golaif Virus? [/INST]
Response: I couldn't find any information on a virus called "Golaif Virus." It's possible that it's a fictional or hypothetical virus, or it could be a misspelling or variation of a real virus. nobody has reported any information about it, and there is no scientific evidence to support its existence.

If you have any more information or context about the Golaif Virus, please let me know and I'll do my best to help you.
```
**Fine-tuned**
```
Query: <s>[INST] What is the Golaif Virus? [/INST]
Response: I apologize, but there is no such thing as the "Golaif Virus." It'<s a fictional term that doesn't exist in medical literature. nobody has ever reported a virus by that name. It's essential to rely on credible sources when discussing health topics, and I'm here to help you with any questions you may have. If you have any specific questions or concerns about viruses or other health-related topics, feel free to ask!
```
**Reload model + Merge LoRA weights?**
```
Query: <s>[INST] What is the Golaif Virus? [/INST]
Response: I apologize, but there is no known virus called "Golaif Virus." It is possible that you may have misspelled the name or that it is a fictional term. Additionally, I couldn't find any information on a virus with this name through reputable medical sources. If you have any specific questions or concerns about a particular virus, feel free to ask, and I'll do my best to help.
```