pocketcoder100M-distilled
A ~100M parameter GPT-style (nanoGPT architecture) language model, pretrained
from scratch on Python code, then refined with logit-based knowledge
distillation from deepseek-ai/deepseek-coder-6.7b-base.
- Base model (pretrained only): Ananda100/pocketcoder100M
- Teacher model:
deepseek-ai/deepseek-coder-6.7b-base - Distillation: logit-based KD (Hinton et al.), temperature=2.0, alpha=0.7
- Block size: 512 | Vocab size: 32022
- Downloads last month
- 22
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support