macadeliccc/distil-gemma-2-2b

Distil Gemma 2 2b

This model is a gemma 2 2b model distilled from google/gemma-2-9b-it and finetuned on the tome.

Prompt Template

ChatML

<|im_start|>system
{system}<|im_end|>
<|im_start|>user
{user}<|im_end|>
<|im_start|>assistant

Training Information

This model trained on 8x Nvidia H100 NVL for the equivalent of 120 GPU hours.

• Loss Achieved: 0.27
• Epochs: 3

Checkpoints are available in the repo to continue training

Evals

IN PROGRESS