| license: other | |
| datasets: | |
| - Photolens/MedText-DoctorLLaMa-OpenOrca-formatted | |
| - shibing624/medical | |
| language: | |
| - en | |
| tags: | |
| - medicine | |
| - doctor | |
| # This model is the DeciLM-6b-Instruct model, trained specifically for medicine | |
| Galen uses the | |
| ``` | |
| ### User: {prompt} | |
| ### Response: | |
| ``` | |
| or | |
| ``` | |
| {prompt} | |
| ``` | |
| Prompt templates | |
| # Galen Training Recipe: | |
| - target_modules = ["q_proj", "v_proj", "gate_proj", "down_proj", "up_proj", "k_proj", "o_proj"] | |
| - Learning Rate: 4e-4 | |
| - LR Scheduler: constant | |
| - 250 Steps | |
| <img src="Loss.png" alt="Loss" width="600" height="400" /> | |
| ## T3: 1 Hour |