Update README.md
Browse files
README.md
CHANGED
|
@@ -31,6 +31,8 @@ You are responsible for any content you create using this model. Enjoy responsi
|
|
| 31 |
## Training
|
| 32 |
It took 3 days to train 3 epochs on 8x H100s using qLoRA, deepspeed zero-2, and Axolotl. learning rate 2e-4.
|
| 33 |
|
|
|
|
|
|
|
| 34 |
## Prompt format
|
| 35 |
This model uses ChatML prompt format.
|
| 36 |
```
|
|
|
|
| 31 |
## Training
|
| 32 |
It took 3 days to train 3 epochs on 8x H100s using qLoRA, deepspeed zero-2, and Axolotl. learning rate 2e-4.
|
| 33 |
|
| 34 |
+
Liberated was trained with [Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl), using this [config](https://huggingface.co/abacusai/Liberated-Qwen1.5-72B/blob/main/configs/Liberated-Qwen-1.5-72b.qlora.yml)
|
| 35 |
+
|
| 36 |
## Prompt format
|
| 37 |
This model uses ChatML prompt format.
|
| 38 |
```
|