File size: 581 Bytes
4c34514 52f5e13 4c34514 c999be6 1cf2454 c999be6 5b1b3e6 fb8c16b 3587495 be0b964 fb8c16b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
---
license: other
datasets:
- Photolens/MedText-DoctorLLaMa-OpenOrca-formatted
- shibing624/medical
language:
- en
tags:
- medicine
- doctor
---
# This model is the DeciLM-6b-Instruct model, trained specifically for medicine
Galen uses the
```
### User: {prompt}
### Response:
```
or
```
{prompt}
```
Prompt templates
# Galen Training Recipe:
- target_modules = ["q_proj", "v_proj", "gate_proj", "down_proj", "up_proj", "k_proj", "o_proj"]
- Learning Rate: 4e-4
- LR Scheduler: constant
- 250 Steps
<img src="Loss.png" alt="Loss" width="600" height="400" />
## T3: 1 Hour |