---
license: apache-2.0
datasets:
- databricks/databricks-dolly-15k
---

# Instruct_Phi2_Dolly15K
Fine-tuned from phi2，used Dolly15k for the dataset. 90% for training, 10% validation.  Trained for 2.0 epochs using QLora.  Trained with 1024 context window.

# Model Details
* **Trained by**: trained by HenryJJ.
* **Model type:**  **Instruct_Phi2_Dolly15K** is an auto-regressive language model based on the phi 2 transformer architecture.
* **Language(s)**: English
* **License for HenryJJ/Instruct_Phi2_Dolly15K**: apache-2.0 license


# Prompting

## Prompt Template With Context
chatml format

```
<|im_start|>system
{instruction}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
```

## Prompt Template Without Context
```
<|im_start|>system
{instruction}<|im_end|>
<|im_start|>assistant
```

# Training script:
Fully opensourced at: https://github.com/hengjiUSTC/learn-llm/blob/main/trl_finetune.py. Run on 1 A10G instance for 4 hours.

```
python3 trl_finetune.py --config configs/phi2-dolly.yml
```